Can't recover G2 site from XML backup after clean FC10 upgrade!

jbwarren

Joined: 2008-09-28
Posts: 10
Posted: Sun, 2009-03-08 17:43

I have been running Gallery2 for a while on Fedora Core 8, and recently upgraded my machine to Fedora Core 10 with a clean install (I had to reinstall MySQL, Gallery2, etc.). After getting Gallery2 installed, I tried to recover from my XML backup and it didn't work - no albums are visible, and there is only 1 record in g2_Item. Please help!

My old system was already running Gallery 2.3, so I had done both a backup from the Admin/Maintenance UI and a backup of the entire g2data directory. The XML backup file is ~7MB, and the g2data.tgz is ~10GB, and both appear ok (I can find the list of image filenames in the XML file, etc.).

I did a clean install of everything Gallery2, going through the web-based install without problems. I then mounted my backed-up g2data directory in place of the new (clean) one and attempted to run the db recovery on the /lib/support page, accessing the XML file in the /backup directory. (I did have to make some SELinux-related mods for the new install, as my FC10 SELinux now requires 'httpd_sys_content_rw_t' type on writable files instead of just 'httpd_sys_content_t' as before.) Afterwards, the import webpage claims success, but my db is still essentially empty, and thus no albums appear on the site (even though the /albums directory is fully populated).

Is there some special process I must follow to make this work? Do I have to run the import before running the install process? Do I need to have the new/clean g2data directory in place when I run the XML db import, then copy over the /albums directory? I have tried a number of different combinations, to no avail.. And since I relied on the Gallery2 XML db backup (which is full of XML encoding mess) instead of the MySQL backup, I can't easily just recover directly in MySQL. What do I do?

-Jeremy

Gallery URL = http://www.jbwarren.net:8080/gallery2/main.php
Gallery version = 2.3 core 1.3.0
API = Core 7.54, Module 3.9, Theme 2.6, Embed 1.5
PHP version = 5.2.6 apache2handler
Webserver = Apache/2.2.10 (Fedora)
Database = mysqli 5.0.67, lock.system=flock
Toolkits =
Acceleration = none, none
Operating system = Linux urban.jbwarren.net 2.6.27.19-170.2.35.fc10.i686 #1 SMP Mon Feb 23 13:21:22 EST 2009 i686
Default theme = matrix
gettext = enabled
Locale = en_US
Browser = Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; DOJ3jx7bf; New Value #1; DI60SP1001; SLCC1; .NET CLR 2.0.50727; Tablet PC 2.0; InfoPath.2; .NET CLR 3.5.30729; OfficeLiveConnector.1.3; OfficeLivePatch.0.0; .NET CLR 3.0.30618)
Rows in GalleryAccessMap table = 6
Rows in GalleryAccessSubscriberMap table = 1
Rows in GalleryUser table = 2
Rows in GalleryItem table = 1
Rows in GalleryAlbumItem table = 1
Rows in GalleryCacheMap table = 0

 
jbwarren

Joined: 2008-09-28
Posts: 10
Posted: Tue, 2009-03-10 05:25

I also noticed this thread: http://gallery.menalto.com/node/85821

I appear to be having the same problem.. It sounds like a bug with the backup/import process.. I even ran it in SELinux "PERMISSIVE" mode to make sure that wasn't the issue.

I would really appreciate it if someone knowledgeable with the G2.3 XML import capability would respond to either posting. Otherwise, I'm going to have to try to write a bunch of Python code to parse the backup file myself and generate a SQL script to load the database..

 
alecmyers

Joined: 2006-08-01
Posts: 4342
Posted: Tue, 2009-03-10 13:54
Quote:
I would really appreciate it if someone knowledgeable with the G2.3 XML import capability would respond to either posting.

I suspect a response is unlikely, as all the developers are now working on Gallery3.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2009-03-15 09:22

this really should be a documented process. :/

either way, it looks like you're doing it the right way.

backup process:
1. archive your g2data folder. e.g. with tar -czf g2data.2009-mar-7.tar.gz g2data/
2. generate a backup of the database. i'd recommend a mysql dump. the built-in g3 database backup feature is fine too. it's just that with database backups, you'd usually go with an established solution.
3. assuming you go with g3's db backup file (e.g. from site admin -> maintenance -> database backup), make sure to move the .xml backup file to a secure location.
ideally, include the gallery2.zip file (either an archive of your current gallery2 folder or a fresh gallery2.zip download from the official download site) to have a backup of the application of the exact version you're using.
(4. verify that the backup works: try to restore it in a new location / with a separate database. a backup is only worth something if you can restore from it, if it's not broken.)

restore process:
1. if you're replacing an existing installation: create a new / separate backup of your current installation. ever wiped out an existing installation just to find out that the backup you tried to replace it with didn't work? well, some of us did. so, don't make matters worse - create a backup of what you have right now.
if all goes well, you can throw away this backup after the restore process. if the restore process fails, you should be able to at least go back to this new backup.
2. if you're replacing an existing installation, wipe it out. delete the g2data/ folder, delete the gallery2/ folder, drop the database.
3. install gallery2, a fresh installation, using the gallery2.zip that you included in your backup. that is, it should be the version of gallery2 that you backed up.
it doesn't have to be the exact same version, but it should be the same or a newer version. things are more likely to work if it's the same.
we perform an installation, just to generate a proper config.php file which contains the database connection information that the importer needs.
keep config.php writeable for the import process.
4. put the database backup file (.xml) into the g2data/backup/ folder.
5. extract your g2data/ backup over the fresh g2data/ folder.
6. browse to gallery2/lib/support/index.php -> database import. pick the backup file and start the process.
7. it should report success and you should see your restored gallery when browsing to gallery2/index.php.

--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2009-03-16 03:28

also, please check your apache error logs. if you reinstall / upgrade your system, it's likely that the PHP configuration changed.
e.g. the php memory limit could be too low or necessary php extensions could be missing (simplexml?).

 
jbwarren

Joined: 2008-09-28
Posts: 10
Posted: Thu, 2009-03-19 14:23

I have been out of town for the last few days - I'll try again and check the apache error logs tonight.

Note that the process you lay out is essentially the one that I have tried to no avail. The lib/support/import script says "success" but nothing has changed in the db. I have also tried upping the php memory limit to 75M (in both php.ini and the gallery2/.htaccess file that limits it to 24M - verified through the lib/support php params script) to no avail. Wouldn't the script give errors if php ran out of memory or the simplexml module wasn't present?

Regardless, I'll retry and take a closer look for errors.

Thanks,
-Jeremy

 
jbwarren

Joined: 2008-09-28
Posts: 10
Posted: Thu, 2009-03-19 15:29
valiant wrote:
2. generate a backup of the database. i'd recommend a mysql dump. the built-in g3 database backup feature is fine too. it's just that with database backups, you'd usually go with an established solution.

Of course, this is the part that I left out, trusting the XML backup.. :/

 
jbwarren

Joined: 2008-09-28
Posts: 10
Posted: Sat, 2009-03-21 18:37

I (re-) tried the restore process above from scratch. I dropped and recreated the gallery2 and g2data folders; installed gallery2 v2.3 (the same as my backup) from the full install tarball; ran the gallery2/install script to create the config.php file; copied my g2data directory over the fresh one (including the xml db backup); and ran the gallery2/lib/support import script. Same results as always: it said "success," but there were no records added to the database, and no albums were visible.

A few extra notes:
* The apache error_log file had no errors listed from the import process
* The import script seemed to run too quickly (< 1 min), and never showed a progress bar even though I have php output buffering off
* SELinux was in PERMISSIVE mode during the import, just in case - but no warnings anyway
* The PHP script memory limit was set to 128MB

It appears that the import script is happily ignoring everything in my xml backup file. Any ideas? This is really quite frustrating...

On a side note - how much of the installation am I supposed to run before overwriting the g2data directory and attempting the import? I tried running it only up to the creation of the config.php, but if I don't at least install the core module then the lib/support/import script gives errors (ERROR_STORAGE_FAILURE) when it's just trying to initialize the G2 PHP classes - before I even select a backup file to import. So I ran the install process through the core module installation, and it at least let me run the import script then. Do I also need to install the modules? (Not that it really matters - I'm pretty sure that's how I've tried to do it in the past.) Do I need to have the exact same set of modules installed as was installed before the backup? (The backup appears to contain the module installation info from the db anyway.)

 
bobob

Joined: 2003-07-12
Posts: 110
Posted: Tue, 2010-11-09 20:36

jbwarren, did you get the xml import tool working? I only get errors. See http://gallery.menalto.com/node/99037#comment-359402 for more details.