[Tip] How to run G1 over NFS

The following information is required to get an answer:
Get this information from the PHP diagnostic (in the configuration wizard).
Gallery URL (optional but very useful): http://solid77.doodle22.com:8080/nature?newlang=en_US
Gallery version: 1.5.2-pl2
Apache version: Apache/2.2.3 (Ubuntu) PHP/5.2.1
PHP version (don't just say PHP 4, please): PHP/5.2.1
Graphics Toolkit: NetPBM
Operating system: Linux 2.6.20-15-server (Ubuntu 7.04 server)
Web browser/version (if applicable): N/A

Why NFS? I wanted to migrate my existing gallery to a new server, with newer OS, newer Apache, newer PHP and many other unknowns. Copying 50 GB of data would be a waste if the new environment doesn't work. (Don't have a hot swap tray or anything like that.) So exporting albums for testing before the real move would be valuable.

But this may yet be valuable for production purposes. For example, one of G1's performance bottlenecks is CPU usage during upload. (I always load pictures to server disk first. So "upload" doesn't involve wire transfer.) I haven't tried large scale upload in G2, but this may very well be true in G2, too, because the CPU load is mainly due to resizing images. Meta data handling takes considerably longer in G1 than in G2, but only represents a small percentage of total time. By using a load balancer and shared storage, multiple simultaneous uploads can be performed with different CPU's.

The challenge: When first exported albums from the old server to the new server, G1 gave me all kind of errors, all related to permissions.

[0] I started by using exportfs for temporary set up.
-# exportfs newserver:/usr/local/apache2/gallery/albums
The first error "server is down" came when I attempted to mount on the new server. The cause? Host firewall. The old server is a RedHat box. By default, RedHat later than 6.x shuts down any port it deems unnecessary. (To me, this is mere marketing gimick and causes more hassle than the perceived out-of-the-box security.)

[1] So I opened NFS port in ipchains on the RedHat. (Edit /etc/sysconfig/ipchains and restart.) But somehow it still refuses to use. (Even if nmap is able to "see through" the port.

[2] So I had no choice but disable ipchains all together. (The LAN is firewalled, and this would be for a short period of time.) Now I am alble to mount NFS. Yet G1 told me it "Cannot create /var/gallery/albums/.users." NFS might be read-only?
$ exportfs -v
/usr/local/apache2/gallery/albums
newserver(ro,async,wdelay,root_squash)

Indeed, exportfs defaults to ro. O, by the way, by default, NFS uses nobody to "squash" root on the other side.

[3] So I added -o rw.
-# exportfs -o rw newwserver:/usr/local/apache2/gallery/albums
$ exportfs -v
/usr/local/apache2/gallery/albums
newserver(rw,async,wdelay,root_squash)

Still "Cannot create /var/gallery/albums/.users." Now, what? My old albums/ directory has always been owned by nobody, as required by most common Apache configurations. Time to read the fine prints.
$ man exports # 2nd time in the day

It says that NFS actually maps to "a uid and gid of -2 (i.e. 65534)", the "anonymous" user often a.k.a. "nobody". Does this bear any significance?
(RedHat)$ id nobody
uid=99(nobody) gid=99(nobody) groups=99(nobody)

No wonder! Nobody is NOT anonymous on RedHat, even though Apache apparently got cozy with the impostor of anonymous.

[4] So I added -o anonuid=99.
-# exportfs -o rw,anonuid=99 newwserver:/usr/local/apache2/gallery/albums
$ exportfs -v
/usr/local/apache2/gallery/albums
newserver(rw,async,wdelay,root_squash,anonuid=99)

Still "Cannot create /var/gallery/albums/.users." UID mismatch again? I had to bring my attention to the new server, an out-of-the-box Ubuntu 7.04 with packaged LAMP solution.
(Debian)$ id nobody
uid=65534(nobody) gid=65534(nogroup) groups=65534(nogroup)

Evidently, nobody on the new server is anonymous. O what can the matter be? Again, fine print. This time on the new server. Peruse of httpd.conf - no, apache2conf in Ubuntu's LAMP bundle, gives this clue:
User www-data
Group www-data

Huh? That's no nobody.
(Ubuntu/LAMP)$ id www-data
uid=33(www-data) gid=33(www-data) groups=33(www-data)

That is to say, Apache is run by uid 33, in "America" or the New World, yet in "Europe", or the "Old World", uid 99 owns Apache. No wonder they can't talk.

[4] The solution? -o all_squash
# exportfs -o rw,all_squash,anonuid=99 newwserver:/usr/local/apache2/gallery/albums
$ exportfs -v
/usr/local/apache2/gallery/albums
newserver(rw,async,wdelay,root_squash,all_squash,anonuid=99)

With this, every uid in Ubuntu is "squashed" into uid 99 in RedHat, not just the root. Therefore, uid 33 in new server can now write to files owned by uid 99 on the old server.

(Further tweaks were necessitated by issues such as http://gallery.menalto.com/node/67706 and http://gallery.menalto.com/node/67723 to make this Gallery work. But they are unrelated to NFS.)

So what have we learned? Know thy systems. Anyone remember my system specs posted above?

1. Use similar systems on both sides if possible. (If you use dedicated NFS devices such as NetApp, this would not be a big problem but you still have to know NetApp well.)

2. All nobodies are not made equal. Inspect actual systems.

3. All out-of-the-box Apaches are not made equal. Know who runs Apache. Read the "fine prints" in httpd.conf - or apache2.conf if the system so pleases.

4. RedHat is notorious in default ipchains settings. Get ipchains out of the way is advisable upon first sign of trouble. You can always add it back and tune it later.

5. Play some squash. After all, all work and no play, makes Jack a dull boy!

(Gone squashing ...