Patch to support Coral distribution network

Rasmus

Joined: 2003-11-25
Posts: 46
Posted: Sat, 2004-09-11 08:09

Coral is pretty cool. Read about it here: http://www.scs.cs.nyu.edu/coral/

This patch to Gallery1 will adjust outbound links when a Coral request comes in. This means that all further clicks will be coral'ed if the user came in via Coral.

Index: init.php
===================================================================
RCS file: /cvsroot/gallery/gallery/init.php,v
retrieving revision 1.109
diff -u -r1.109 init.php
--- init.php    7 Sep 2004 21:41:02 -0000       1.109
+++ init.php    11 Sep 2004 08:01:02 -0000
@@ -123,6 +123,13 @@
                        eregi_replace("^https:", "http:", $gallery->app->photoAlbumURL);
                $gallery->app->albumDirURL =
                        eregi_replace("^https:", "http:", $gallery->app->albumDirURL);
+               /*
+                * We have a Coral request coming in, adjust outbound links
+                */
+               if(strstr($_SERVER['HTTP_USER_AGENT'],'CoralWebPrx')) {
+                       $gallery->app->photoAlbumURL = preg_replace("=(http://[^/]+)(.*)$=",'\1.nyud.net:8090\2',$gallery->app->photoAlbumURL);
+                       $gallery->app->albumDirURL = preg_replace("=(http://[^/]+)(.*)$=",'\1.nyud.net:8090\2',$gallery->app->albumDirURL);
+               }
        }
 }

Note that this patch obviously doesn't address the issue of links that shouldn't be coralized. Login screen, comments, admin stuff, etc. To do this right these links would have to be separated out and not be modified. But this quick hack is a good way to quickly make a read-only distributed way to see your photos.

 
signe
signe's picture

Joined: 2003-07-27
Posts: 2322
Posted: Sat, 2004-09-11 09:03

That's a very interesting tool... this is what I'm actually going to check in, though. A couple changes were necessary to make it Gallery-compatible.

	/*
	 * We have a Coral (http://www.scs.cs.nyu.edu/coral/) request coming in, adjust outbound links
	 */
	if(strstr($_SERVER['HTTP_USER_AGENT'], 'CoralWebPrx')) {
		// If the URL contains a port number, map it to the correct format for Coral ("www.host.com.80") before converting the URL
		if (ereg("^(http://[^:]+):(\d+)(.*)$", $gallery->app->photoAlbumURL)) {
			$gallery->app->photoAlbumURL = ereg_replace("^(http://[^:]+):(\d+)(.*)$", "\1.\2\3", $galllery->app->photoAlbumURL);
		}
		$gallery->app->photoAlbumURL = ereg_replace("^(http://[^/]+)(.*)$", '\1.nyud.net:8090\2',$gallery->app->photoAlbumURL);

		// If the URL contains a port number, map it to the correct format for Coral ("www.host.com.80") before converting the URL
		if (ereg("^(http://[^:]+):(\d+)(.*)$", $gallery->app->albumDirURL)) {
			$gallery->app->albumDirURL = ereg_replace("^(http://[^:]+):(\d+)(.*)$", "\1.\2\3", $galllery->app->albumDirURL);
		}
		$gallery->app->albumDirURL = ereg_replace("^(http://[^/]+)(.*)$", '\1.nyud.net:8090\2',$gallery->app->albumDirURL);
	}

First, Gallery uses ereg commands, instead of preg. Not everyone has the PCRE module loaded.

Second, Gallery has lots of users who add port numbers to their URLs because they're behind firewalls, or use restrictive ISPs, so I had to add a check and conversion for that case.

 
Rasmus

Joined: 2003-11-25
Posts: 46
Posted: Sat, 2004-09-11 09:50

I suppose that makes sense, although I really have to wonder how many people don't have PCRE these days.

Related to that, I have noticed that you like to use ereg_replace() in many places where the much faster str_replace() would do the job.

eg.

ereg_replace("_", "-", $gallery->app->default_language)

 
signe
signe's picture

Joined: 2003-07-27
Posts: 2322
Posted: Sat, 2004-09-11 16:16

Well, since there are still people who refuse to upgrade PHP, and are running 4.0.4 or 4.0.6....

Yeah. strtr() would probably be faster, too.

 
Rasmus

Joined: 2003-11-25
Posts: 46
Posted: Sat, 2004-09-11 16:46

All of these functions have been available since PHP3 and the string ones since well before that. But it also doesn't matter all that much. It would be a micro-optimization. A much more effective optimization would be to optionally cache the $gallery class from config.php in shared memory. I'll provide a patch for that sometime soon.

 
signe
signe's picture

Joined: 2003-07-27
Posts: 2322
Posted: Sun, 2004-09-12 21:51

$gallery is already a global variable, so I have no idea what you would intend to do by caching it.

What I meant was that since there are people who refuse to upgrade away from a version of PHP that's over three years old, a lot of people still don't have, or won't install out of sheer stubbornness, PCRE.

 
Rasmus

Joined: 2003-11-25
Posts: 46
Posted: Thu, 2004-09-16 05:54

I meant caching it in shared memory. I did this on my site and got a 10% overall speedup. Creating this $gallery class full of data that rarely changes on every request is quite expensive. But the way I did it is in no way appropriate for the public distribution of Gallery.

 
signe
signe's picture

Joined: 2003-07-27
Posts: 2322
Posted: Thu, 2004-09-16 06:01

Maybe you could give an example?

PHP is a run-time language... it has no persistant memory cache.

 
Rasmus

Joined: 2003-11-25
Posts: 46
Posted: Thu, 2004-09-16 06:56

I know my way around PHP ;)

PHP has shared memory functions. You could use shm_put_var() and shm_get_var() to do this, but then you have to also write code to manage your shared memory segment. I used the APC opcode cache's apc_store() and apc_fetch() calls to do this. I haven't released a new version of APC that includes these functions yet though, so you would have to get it from CVS (pecl/apc) if you wanted to try it.

I rolled Version.php into config.php and called it apc_config.php. Then I modified init.php like this:

Index: init.php
===================================================================
RCS file: /cvsroot/gallery/gallery/init.php,v
retrieving revision 1.112
diff -u -r1.112 init.php
--- init.php    11 Sep 2004 09:23:43 -0000  1.112
+++ init.php    16 Sep 2004 05:46:36 -0000
@@ -58,7 +58,7 @@
 mt_srand((double) microtime() * 1000000);

 global $gallery;
-require(dirname(__FILE__) . "/Version.php");
+#require(dirname(__FILE__) . "/Version.php");
 require(dirname(__FILE__) . "/util.php");

 /* Load bootstrap code */
@@ -69,7 +69,7 @@
 }

 if (fs_file_exists(dirname(__FILE__) . "/config.php")) {
-   include(dirname(__FILE__) . "/config.php");
+   if(!($gallery=apc_fetch('gallery'))) include(dirname(__FILE__) . "/apc_config.php");

    /* Here we set a default execution time limit for the entire Gallery script
     * the value is defined by the user during setup, so we want it inside the
@@ -169,7 +169,7 @@
 }
 $gallerySanity = gallerySanityCheck();
 initLanguage();

And at the end of my apc_config.php I have:

apc_store('gallery',$gallery);

It's not that hard to do, but both solutions would rely on users having either the shm extension or a very recent APC installed. Still, it could be a config option.

 
signe
signe's picture

Joined: 2003-07-27
Posts: 2322
Posted: Thu, 2004-09-16 08:46

The major drawback here is the very large

"Note:
This function does not work on Windows systems" for all of the shm var functions.

 
GirishKalele

Joined: 2004-09-17
Posts: 2
Posted: Fri, 2004-09-17 21:58

You don't need any code...all you have to do is use the MirrorURL feature already present in Gallery. Point the mirror URL to the coral url of your albums directory...Gallery will try to check the lock/dat file there, find that it is identical to it's local copy, decide that it is a valid mirror and point the IMG tags to the Coral Web Cache URL. Done ! No problems with any links/cookies/etc not working. And anyway, you will keep serving the HTML pages from your server, and the bandwidth-hogging photos from the Coral Cache (I've observed that the CDN cache is getting thrashed so much that things fall out of the LRU cache pretty quickly nowadays). Try it out, you'll eliminate all your cookie/link problems...moreover, your click counters will be accurate.

http://kalele.homelinux.com:8080

Rasmus wrote:
Coral is pretty cool. Read about it here: http://www.scs.cs.nyu.edu/coral/

This patch to Gallery1 will adjust outbound links when a Coral request comes in. This means that all further clicks will be coral'ed if the user came in via Coral.

Index: init.php
===================================================================
RCS file: /cvsroot/gallery/gallery/init.php,v
retrieving revision 1.109
diff -u -r1.109 init.php
--- init.php    7 Sep 2004 21:41:02 -0000       1.109
+++ init.php    11 Sep 2004 08:01:02 -0000
@@ -123,6 +123,13 @@
                        eregi_replace("^https:", "http:", $gallery->app->photoAlbumURL);
                $gallery->app->albumDirURL =
                        eregi_replace("^https:", "http:", $gallery->app->albumDirURL);
+               /*
+                * We have a Coral request coming in, adjust outbound links
+                */
+               if(strstr($_SERVER['HTTP_USER_AGENT'],'CoralWebPrx')) {
+                       $gallery->app->photoAlbumURL = preg_replace("=(http://[^/]+)(.*)$=",'\1.nyud.net:8090\2',$gallery->app->photoAlbumURL);
+                       $gallery->app->albumDirURL = preg_replace("=(http://[^/]+)(.*)$=",'\1.nyud.net:8090\2',$gallery->app->albumDirURL);
+               }
        }
 }

Note that this patch obviously doesn't address the issue of links that shouldn't be coralized. Login screen, comments, admin stuff, etc. To do this right these links would have to be separated out and not be modified. But this quick hack is a good way to quickly make a read-only distributed way to see your photos.

[url]FAQ Gallery:c.35[/url]

 
Rasmus

Joined: 2003-11-25
Posts: 46
Posted: Sun, 2004-09-19 01:50

But you missed the main point here. I am only using CDN if the user accessed my content via CDN. As you observed, CDN isn't necessarily faster unless you happen to be near a CDN server that isn't being spanked. But if someone has chosen to use CDN, who am I to argue and subsequent album links shouldn't kick the user off of CDN. As far as I can tell you can't really make this distinction with MirrorURL. Perhaps it would be better to move this patch to the MirrorURL functionality and have it do this check.

 
GirishKalele

Joined: 2004-09-17
Posts: 2
Posted: Sun, 2004-09-19 19:11

Yeah, I kind of lost track of the main reason you wrote the patch. I was thinking of always pointing people at a CDN version, regardless of how they came in...but you're right, sometimes it may just be better for some servers not to use any Coral links at all ! And if people come in via CDN links, we give them more CDN links...sounds like a good idea.
:wink:

It's not a big b/w gain to point all the HTML links to the CDN version of them though...also, I kind of like to keep track of all the hits to the albums.
Maybe only the img tags should be changed ?
:wink:

 
Rasmus

Joined: 2003-11-25
Posts: 46
Posted: Thu, 2004-09-23 03:21
GirishKalele wrote:
It's not a big b/w gain to point all the HTML links to the CDN version of them though...also, I kind of like to keep track of all the hits to the albums.
Maybe only the img tags should be changed ?
:wink:

Yup, that was what I initially had a look at, but it wasn't easily done with a simple 2-line patch.