I am trying to restore an old, XML style backup of my database. I have the full g2data directory, but the database became corrupt and is much older than the g2data directory, and is in XML format, v1.0. Here is the core module version information from the very top of the XML backup file:
<?xml version="1.0"?>
<export schema-version="1.0" tables="45" core-version="1.2.21" gallery-version="2.3-svn">
I have a brand new Gallery2 v2.3.1 (core 1.3.0.1) installation in to which I want to restore this XML backup.
Executing the gallery2/lib/support/index.php?import function, selecting the .xml backup file, I get these errors:
Errors Occurred
The exported file is not the correct version. Expecting 1.1 received 1.0.
The available version of plugin "albumselect" may not be compatible with the imported version.
The available version of plugin "archiveupload" may not be compatible with the imported version.
The available version of plugin "comment" may not be compatible with the imported version.
The available version of plugin "publishxp" may not be compatible with the imported version.
The available version of plugin "webdav" may not be compatible with the imported version.
Is it possible to import this XML file in to this Gallery2 version? .. or must I install an appropriately old Gallery2 version, import, then upgrade Gallery2?
Which version of Gallery2 will best support this XML file, based on that odd "2.3-svn" version string in the XML backup?
Thanks,
Jay
Here's the rest of the system information of the new 2.3.1 installation:
Gallery URL = http://gator1173.hostgator.com/~libove/gallery2/main.php
Gallery version = 2.3.1 core 1.3.0.1
API = Core 7.54, Module 3.9, Theme 2.6, Embed 1.5
PHP version = 5.2.12 cgi-fcgi
Webserver = Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8i DAV/2 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635
Database = mysqli 5.1.30, lock.system=flock
Toolkits = ArchiveUpload, Exif, Getid3, ImageMagick, LinkItemToolkit, SquareThumb, Thumbnail, Gd
Acceleration = none, none
Operating system = Linux gator1173.hostgator.com 2.6.28.9 #51 SMP Fri Jan 15 22:59:44 CST 2010 i686
Default theme = matrix
gettext = enabled
Locale = en_US
Browser = Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; FDM; MS-RTC LM 8)
Rows in GalleryAccessMap table = 32
Rows in GalleryAccessSubscriberMap table = 3
Rows in GalleryUser table = 2
Rows in GalleryItem table = 1
Rows in GalleryAlbumItem table = 1
Rows in GalleryCacheMap table = 0
Posted: Fri, 2010-01-29 08:54
Posts: 4342
That probably means whatever you do will never achieve a satisfactory result. The two need to match exactly for a backup to be any use.
I would install from scratch and reimport your images; you will have to recreate all the rest of the data.
Posts: 8339
I wrote a script for libove that compares filenames and size from the old xml backup to items in the new gallery.
It then copies the old title, summary, description and keywords over to the matched gallery item.
All in about ~120 lines of code. Not bad.
If anyone is interested, lemme know.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 69
The amount of time and help which suprsidr has been giving me is so far above and beyond, I must have accumulated a huge karmic debt
I sure hope he's having fun with this!
I hope I can give back somehow. I'm not a PHP coder (my last coding was more than a decade ago in "C", yeah, I'm an old fart). I'm well qualified in security and privacy governance, in case the project ever needs some advice on how to structure things to make it easier for a user of the gallery to comply with laws and regulations relating to infosec, privacy or data protection anywhere in the world.
-Jay
Posts: 110
suprsidr, I think I have the samen problem you faced. After a upgrade from G2.2.4 to G.2.3.1 I screwed up the database. I have however a xml export of the database that was made during the upgrade wizard. I now want to import / recover this database. Do you know how? I described the problem more in detail in post http://gallery.menalto.com/node/99037#comment-359402
Posts: 8339
yeah, I never faced said problem.
But as my post above states, I wrote a script to help fix this issue - if you're interested.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
I am definitly intrested! Almost 10 years of photo's are about to be lost.
So this script may help my situation? I thus have an xml export of my 2.2.4 database and the g2data folder. I also have a corrupted installed gallery 2.3.1 (because I accidently deleted the database). Ofcourse I can re-install a fresh copy of G2.3.1 or, if needed 2.2.4 (I have the original zip)
Posts: 8339
You'll need to install G2.3, and do a server import to get all your images back into gallery.
Then follow the directions in the top of the script.
This script compares images by filename and size and copies the relevant data like title, description ... it won't copy comments and such as that's a separate module w/ its own terms.
Running on such a larger gallery may need some tweaking.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
thanks. I am going to give it a try!
Currently the g2data is on my local windows server. Should I do the import on this local server only, or is it possible to install G2.3 on a remote linux hosting server, transfer the g2data and xml export there and then do an import? I prefer the latter, because my local server is going to have a though time making for thousands of pics thumbnails. I however don't know if this will give problems since your tool depends on filename and size (perhaps the size is different on linux filesystem because of minimum chunk size?).
What kind of tweaking will be needed? I believe the amount of imported pictures are somewhere between 5000 - 10.000.
Posts: 8339
http://codex.gallery2.org/Gallery2:How_to_Add_Items#How_to_Add_Items_from_the_Local_Server_.28or_with_FTP.29
Gallery will recreate albums based on directory structure of your old albums directory
then rum my script to transfer title, description...
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
hmmm....I ran the Add Item as in the URL you provided, but get the following error (everything is chmod 755).
Posts: 8339
Add from local server requires 777 recursively on your g2data and I believe the old album directory as well.
Luckily you can delete the few that may have been created, and start again until you get it right.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
the FAQ said that 755 should be sufficient, but I will do a recursive 777 just to be sure.
I have about 9000 pictures to add (17 Gb). I read in some post my host might be killing processes that take up too much time / memory and that creating thumbnails can be shutt of, and to use of symbolic links. Will your script work if I will do this?
<edit>my intension is, if I get this working, I want to upgrade to g3. So the symbolic links should mess up the possibility to later to a g2 import into g3</edit>
And, while importing the server files, I have the option to copy the filebase to the title (or description or summary). I assume this isn't a problem for your script? (it will override were it is needed)
Posts: 110
I was able to add the pictures by chmodding everything with 777, using symbolink links and deferring the creation of thumbnails. One or two times the script aborted, so I just retried adding a smaller amount of albums.
After this I used your tool. It did not give any errors and finished by a "done!" statment. However, I do not notice any changes. Many albums are still called "album23" or something and the decription (or summary?) I gave those albums did not get imported. The description can be found in the XML file however. Here is an example of text I can find in the xml file, but did not get imported
Did something go wrong, or am I missing the point?
Posts: 8339
My script does nothing to albums themselves, just to images.
It does filename and filesize comparisons can't exactly do that with a directory.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
ah, ok. I understand.
BTW, when the script was running it outputted lines like
The difference between two items ID's was always 3. Is this standard behaviour (nothing was skipped)?
Posts: 8339
it only updates the item, the numbers in between are likely derivatives or albums...
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
ah, I thought so. Just wanted to check.
In case you ever want to tweak your script, it would be nice if it could update the album info also. Although it doesn't have a size itself, perhaps the comparison could be based on the size of it's nested objects (or perhaps more easier, just only check for identical names and leave the size out of the equation? g2 auto-numbers albums ("album76"). changes are small that there are several albums with identical name).
In anyway, thanks a lot! Having to re-edit about 100 album title/descriptions beats editting 10.000 photo's! Now at least with some evenings of going through everything, I can still repair the damage (can't believe G2 had a stupid xml export that even the most knowledgable users did not get working)
Posts: 8339
phpmyadmin 3.x can handle xml DB exports, but I've never actually tried. I believe php 5.3 is required for it too.
Glad my script could help.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 4342
The xml export/import was one of the very last "big" features added to G2 before development attention was switched to G3, so it didn't get the issues sorted out.
Posts: 110
ah, ok. Generally I find Gallery an impressive product (unbelievable how so many people deliver such great work). I can understand that with the many features this specific feature didn't get developped well enough. However, in such a case, I think it was better not to implement the feature at all, untill it was fully functioning. I didn't make a backup because the gallery upgrader suggested it would do it for me. If I knew it was buggy, I would have looked into it better before hand.
Anyway, otherwise it is a great platform (and ofcourse I should have made a full system backup in anyway. Lesson well learned ;-) )
BTW, do you think this feature still will be debugged in the future, or has all the development on G2 stopped since G3 is there?
Posts: 110
suprsidr, should I give the phpmyadmin a try (with your help/instructions)? I have myphpmyadmin 3.2.4. However, PHP is only version 5.2.14?
Posts: 8339
for the 100 album titles?
make a backup of the current working db first.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
yes, to retrieve the album titles and album description/summary (my personal description of each event/album is quite valuable to me).
I just now made a back-up from the G2 database according to the instructions on http://codex.gallery2.org/Gallery2:How_do_I_Make_Backups_of_My_Database (I had one extra field in myphpadmin, comments, that I left checked. See screenshot).
BTW, I moved (again) to a new server, installed G2.3.1 fresh and did an Add Server import. I have not run your script yet (ofcourse, I can do so if this is needed before proceeding).
Posts: 8339
your db backup will not work w/ your new gallery. the correct procedure:
FAQ: How can I move my gallery installation from one server to another?
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
I am confused. I cannot do all the steps since I did not make a backup when I needed to do it. To reiterate:
a) I had a Gallery 2.2 running on a windows machine. During a upgrade to 2.3 an automated backup XML was made. By accident, I deleted the database of the functioning 2.3 afterwards, only leaving me with a non-functioning G2.3 and an intact g2data folder.
b) I installed a fresh copy of G2.3 on a remote unix shared server. I FTP the intact g2data to the server and added the pictures succesfullly via the Add Server module.
c) 2.3 has now all the pictures in it, but not any meta information. I now made a backup of the g.2.3 via myphpadmin just for the security. (I thought you meant this with
.)
Do you mean, I should install G2.2 on the unix server (not G2.3), just FTP the original g2data to new server and not do an import?
My aim is to get the meta-information from the XML backup from 2.2 into the G2.3 . If I am complicating things by the fact that 2.2. was installed on my home (windows) server and 2.3 on a remote (unix) server, then I can ofcourse instead try the rescue operation on the home server by re-installing 2.2 fresh on the windows server.
Posts: 8339
you should be working with the gallery you restored the other day. and move that.
Your gallery that was broken is gone.
Your item ids will never match up with your old xml backup again.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
I was never able to fully restore it. I only achieved re-installing a fresh copy and doing an Add Server import. I thus now have a working 2.3 on the new server, including the imported pictures, but excluding stuff like album information, user accounts, etc.
does this mean my XML file is useless / has no use anymore (besides that your script could still retrieve picture titles etc)?
I thought that you meant with
there was a chance for me to import the meta information (album descriptions) from the XML backup, but may be I misunderstood you.
Posts: 8339
your db and g2data must match
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 8339
you can still hand edit those ~100 album titles.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
suprsidr, all right. I will have to give up on that and do the album info by hand. However, if I want to use your script for it's intented purpose to import titles of pictures, etc. I now get the following error when I run the script:
In responce the helpdesk altered the PHP settings to
This did not however resolve the issue. After this the helpdesk increased the settings twice, without any effect. Finally, they concluded this script appearantly needs many GB of memory and hours of executing time, and obviously refused this for a shared hosting account.
How is this possible? I do have about 10.000 pictures in the database, but got this import tool working without any problem at my previous hosting company (which I believe were much more strict with their php settings; they did not allow php_exec).
Posts: 8339
So now you've lost the previously restored version too?
I've updated the script above to do less actual interaction w/ gallery.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
Yes. I moved to a new server, so thought I would re-do the steps freshly (install g2.3, import pictures and run your script). Could you attach the updated script? Then i will give it a try.
Posts: 8339
http://gallery.menalto.com/node/94122#comment-359451
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
same result/error
(I downloaded the zip, but the date mentioned in the php file was still 9/19/2010 so I hope I had the correct version)
Posts: 8339
that is the correct version, but it should be from 11/19 make sure you're overwriting the old one.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
yes, I overwritten the original file with the updated version. Still get the same error:
The system admin recovered the following errors from the log file
Posts: 8339
my suggestion. when you've successfully recovered a "lost forever" gallery, don't throw it away.
I've never lost any of my galleries. I've never needed to restore from xml backup - but yet I spent my time to write a script to help others.
I don't get paid for this, and I feel you've already wasted plenty of my time.
That said.
The above error is obvious and in plain English but let me break it down:
make sure that path is correct and the file is readable.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 110
sorry for asking. I double checked the path and permissions (perhaps the error retrieved from the log was from a prior incident). That is not causing the memory allocate problem. Thanks for your help anyway. Over and out.
Posts: 8339
try installing WAMP or similar on your local machine and import/repair your gallery then move it to a live server.
Otherwise I'm out of suggestions.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2
Posts: 8339
17GB no way.
-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2