Halt on some albums, no error.
|
heisters
Joined: 2006-03-31
Posts: 13 |
Posted: Fri, 2006-03-31 17:05
|
|
I followed the migration howto, and it worked for about 200 albums. However, there are still several albums, all chronologically adjacent, that won't import. There is no error message. When I try to import one of these albums I simply get the orange bar in its minimal state, and the page loading icon is stopped, the browser reporting "Done.", wheras other imports the browser kept loading until the import was done. There is no "Continue..." link. The php memlimit reads "Memory used: 11558128, total: 64M" and there are no errors in the apache log. There are no non-standard/UTF-8 characters anywhere in the albums. I thought I might download the albums as zip files and add them that way, but at 350M a piece, it's a bit impractical, and I'd like to keep the metadata intact. However, if there is some workaround (I do have root access to the server) that would keep the metadata (author, creation date, etc.) intact, and avoid downloading and then re-uploading several GBs I'd be glad to hear it. Thank you for any help you can provide. I'll try to move Gallery2 into debug mode and get more info that way shortly.
|
|
Posts: 13
Well, I got some debugging output... it's really long, so I've attached a file.
Posts: 974
You didn't really follow the migration howto or you would have shut off gd and exif. Try that next and let me know how it turns out.
Posts: 13
Indeed, I guess I didn't read it as closely as I should have. However, disabling GD and EXIF did not change the import app's behavior at all.
Posts: 974
From the howto:
# If the migration simply stops at any time, copy the name of the album and/or item that was being processed when it stopped. Paste it somewhere.
# If the import fails during a particular album, please try importing that album by itself first.
# If the import fails on a particular image or movie, please make sure that the original version of the image or movie exists in your g1 first.
# If the image or movie exists, please download the original image or movie to your computer and upload it to g2 outside of the import process. If it works or does not work, let us know.
Posts: 13
Uh, okay: "Angels"
The debugging information I posted further shows that I was trying to import one album at a time.
not applicable. Nonetheless, I have ensured as much. All images in the album exist and are viewable in G1.
furthermore, the download of the archive failed due to PHP out of memory errors, making it not just impractical, but impossible without further troubleshooting.
And yet furthermore, the status bar is not stopping on a specific image; the dialog above it says the album's name, rather than an image's name. It is in an absolutely minimal state, ie. it did not succeed on one image, and fail on the next, it didn't even start processing the first image.
However, I tried downloading the first image in the album in full res and adding it manually to a new album. Adding via the "From Web Browser" option results in an error message like this: (FF 1.5)
Adding it from the Upload Applet fails with no error. I just click upload, things blink a bit, and nothing changes.
So it does seem like it may be the image. I just find it highly weird that images from various cameras posted to the G1 installation between two dates are all somehow borked. Nonetheless, I've attached the image to this message in hopes it might shed some light on this problem.
Thanks for your advice.
Umm, nevermind... trying to attach the image resulted in the same error on this page as I got from G2 when trying to manually add it there. The image is 2.4MB. I'd be glad to email it to you if you can provide an address.
Posts: 974
Thank you for taking the time to go step by step diagnosing your problems.
Does your apache error log show anything when the new upload fails?
I got an image out of your "Angels" album and uploaded it to my own server. It took a while but appeared to work fine. It might be worthwhile for you to try turning off Imagemagick or netpbm and see if one particular toolkit is gumming up the works. You might also keep an eye on the output of "top" on your server -- maybe you are running out of physical memory or CPU.
Posts: 13
Unfortunately no.
top showed the CPUs maxing out at 5.7% usage, with the load at 0.05, 0.08, 0.07. Physical memory (1GB) is about used up (19MB free), but swap still has more than 3GB free. free shows 654MB free in buffers/cache, and this did not fluctuate significantly during the import operation. Overall, it looks like the system is not being swamped. Do you think it might be the near-exhaustion of physical memory? If so, do you have any recommendations for remedeying such a situation?
deactivating all image modules (gd, imagemagick, netpbm, + exif and ffmpeg for good measure) did not observably change behavior.
Thank you for the tips so far.
edit: Just to clarify, all of the above was done trying to add the first image of the album in question from my workstation to a new album. I have not tried using the import app again. Additionally, FWIW, the server reports that it is running on 2 Pentium 4 Xeon (Foster) CPUs, each clocked at an estimated 2.8GHz and each with 16KB L1 cache and 1MB L2 cache. ie., it's not an underpowered machine.
Posts: 974
Can you provide debug output for when you try to upload just the one image?
Posts: 13
Your wish is my command.
The following results from this: go to "add items", select image in first file field, click "add items", error message described in my previous post pops up, click "stop script", page reloads with this debug output.
Posts: 974
Err... yes, there is not really an error there, like you said. I'm kind of stumped now. Have you tried each image toolkit in turn -- turn off netpbm and try it, then turn off imagemagick and try it?
Posts: 13
Tried turning off each of the image processing modules one at a time, but observed no change in the behavior of the "add item" app. Also tried unchecking "Create thumbnails now", which also had no observable affect.
But (aha!?) I uploaded the image to another server, and used the "From Web Page" application, and it worked fine. Thumbnails were even created successfully.
Posts: 974
That doesn't make any sense -- it shouldn't have worked for "From Web Page" and not during migration.
Posts: 13
Well... maybe I'm completely stupid. Wouldn't be the first time. If you send me an email address, I can give you an admin account for a while so you can look around. Maybe setup an IRC chat or something.. you can reach me at ismithhe {at} marlboro {dot} edu
Posts: 13
tried uploading the one image with other browsers.. (had been using FF 1.5 on OS X 10.4.6). They all simply reload the add items page without any of the forms displayed, but give no error message. The image has not been added to the album.
Posts: 13
Ummmm... hello? Any thoughts on how I might get these albums imported (preferrably with metadata intact)? I guess if nothing else, I could put all the pictures in a web folder and use the add from webpage feature. But that doesn't really solve the real issue here.
Posts: 974
Ummmm... If you want faster support you could always pay me.
Try the advice in this FAQ and try to upload the image again.
FAQ: Why can't I upload big files (over a megabyte or two)?
If you can upload it after the change, try the import again.
Posts: 13
My apologies, no offense intended, I just wasn't sure if the problem was considered closed or not. Thanks for bearing with me. At this point I feel like it's as much for the project as myself, since I have a viable workaround.
The plot thickens... I added the following to my Apache2 config:
php_value post_max_size 10M php_value upload_max_filesize 10M LimitRequestBody 0...and restarted apache. When I tried to add the image individually, I got the same unresponsive script error as above. However, when I clicked "Stop Script" the page reloaded, and the image had been successfully added! Unfortunately, the behavior of the import app was still unchanged. It may be important to note that there are several (as in hundreds, if not thousands) of other images that are well over 2MB in size, which all imported successfully. It seems to me that there was a compounded problem here, part of which was just solved.
Edit: NB that the other browsers, consistent with previous behavior, did not give the unresponsive script error, and did add the image correctly. It appears that the unresponsive script error is a Firefox error, and independent of the failure to add images.
Another update: trying another browser to import the problem album failed just as it had before the apache config changes.
Posts: 974
Do you have exif on or off? Which image toolkit are you using? Are you running out of disk space? The first problem that you have to deal with is being able to upload those images through the web form.
Posts: 13
The HDD has 35G available.
Web form image upload works, except that there is an error message in FF 1.5 only. The error message seems superfluous, since the image is still added successfully. This behavior does not seem to be affected by enabling/disabling exif, gd, imagemagick, or netpbm. Again, Safari, IE, Opera, and even Camino all can add images (even the problem image) using the web form without any observable errors or warnings. Furthermore, I have had no reports of problems using the web form from any of my 700 users, many of whom use FF. I would posit that the error message is an artifact of one of the many extensions I have installed in my copy of FF, some of which deal with JavaScript debugging.
The import app still works as I described in my original post, regardless of disabling exif, gd, imagemagick, and/or netpbm.
Posts: 2
@heisters: i had the same problem:
I got it solved by deleting the folder of the partly imported album in the g2 datastore (e.g. via ftp). importing it again after doing that should work fine.
Posts: 13
That seems to have done the trick! I kiss you! I think I had tried this before, but since the web form image uploading wasn't working, when I went back it didn't work. It must've been the combination of getting the web form working, then going to delete the directory.
Thanks very much, both of you.