Import freezes on ~140th picture

sbyam

Joined: 2007-01-13
Posts: 9

Posted: Wed, 2007-01-17 00:49

Gallery version = 2.1.2 core 1.1.0.2
PHP version = 5.1.5 cgi
Webserver = Apache
Database = mysql 4.1.18-standard-log, lock.system=flock
Toolkits = ArchiveUpload, Exif, Gd, ImageMagick
Acceleration = none, none
Operating system = Linux cgi1507.int.bizland.net 2.4.32-grsec #1 Fri Mar 17 12:22:27 EST 2006 i686
Default theme = matrix
Locale = en_US
Browser = Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322)

I am in the process of moving my Gallery site from one server to another. The master plan is to upgrade to Gallery 2. Gallery 2 is installed next to Gallery 1 (http://sbyam.net/G2) the problem comes when I try to import the G1 gallery. I can only load about 140 pictures at a time. Seeing so I have albums with more than 600 pictures and the gallery itself has about 31000 pictures this will take a long time to move over if it is even possible.

I have been chatting with one of the Gallery 1 guys about this issue so we should probably start there:

http://gallery.menalto.com/node/59455

As you can see there it looks like I am having a memory issue even though I have bumped up my memory all the way up to 160M. The import still freezes. I am on the Powweb server so I have also set up my temp directory correctly as well. Any thoughts?

Thanks,
Steve

ivan_bev

Joined: 2006-04-10
Posts: 14

Posted: Wed, 2007-01-17 16:17

Aside from memory, the only other things I can think of to try are:
* do not create thumbnails on import (saves time and RAM use)
* check whether there is an EXIF header in the problematic image

If there is no EXIF header, gallery2 import appears to have issues if you have the exif module enabled. Try disabling the exif module when importing and re-enable afterwards. For more info on this see:
http://gallery.menalto.com/node/44743

Ivan

sbyam

Joined: 2007-01-13
Posts: 9

Posted: Thu, 2007-01-18 01:32

Not sure if I know how to fix the problem but I think I figured out what is causing it. It looks like Powweb has a CGI "stop" that kills any script that runs for more than 60 seconds. There are all kinds of writeups about this problem in thier forums. Here is one of them:

http://forum.powweb.com/showthread.php?t=68409&highlight=Gallery

I'm working with thier tech support to see if there is something they can do. If they can't fix the problem it looks like I will be forced to move to yet another webhost.

sbyam

Joined: 2007-01-13
Posts: 9

Posted: Thu, 2007-01-18 18:11

I got the following E-mail back from Powweb confirming that the 60 second timeout was the problem:

Quote:

We have a number of PHP/CGI/Web/Load timeouts that occur. At any level, when the script reaches 60 seconds and hasn't returned any data, either the web server or load balancer will kill the process thinking it has stalled/gone off to infinity. The way around this is to return some data periodically. Maybe every 10-15 seconds return some data back so that the server knows that the script is still alive and working.That should get around the 60 second timeout. Unfortunately, there's not a whole lot we can do. We can't extend the timeout, as it would reduce the performance of everyone using that CGI server

Now I noticed the list of webhosts that gallery suggests. Can anyone suggest one over the rest?

valiant

Joined: 2003-01-04
Posts: 32509

Posted: Thu, 2007-01-18 20:05

Quote:

At any level, when the script reaches 60 seconds and hasn't returned any data, either the web server or load balancer will kill the process thinking it has stalled/gone off to infinity. The way around this is to return some data periodically.

that's what we do. the gallery import script has a progress bar. it sends data to the browser periodically.

when it freezes for you, doesn't the progress bar update every (few) second(s) until it freezes? how long does it take from the last progress-bar update until it freezes?

--------------
Enter the Gallery 2 Theme Contest today!

sbyam

Joined: 2007-01-13
Posts: 9

Posted: Thu, 2007-01-18 22:28

I don't even see the progress bar until it has frozen. Until then it stays on the screen before that where you verify everything.

I had the Powweb guys try it on their computer and the progress bar works but it still freezes after about 60 seconds. They did manage to load more pictures than I did though. I suppose because they are closer to the server and on a faster network. While I was poking around the powweb forums I noticed lots of people tried work around scripts but none seemed to really work. I think it is something wrong on their end that they are not willing to fess up too. Maybe I'm wrong.

sbyam

Joined: 2007-01-13
Posts: 9

Posted: Wed, 2007-01-24 00:55

OK so the problem is not Powweb. I moved my site to Bluehost.com and I'm getting roughly the same problem. The only difference is that in Bluehost the memory bar in the upper right says 0, total: 0M rather than the 13572480, total: 160M that I used to get. The only good news is that I can upload closer to 500 pictures at a time with bluehost rather than the 140 that I was able to before.

I also found that the reason that the import progress bar was not showing up is because I was using internet explorer 7. When I use Internet Explorer 6 the progress bar shows up as it should. To answer your question the progress bar move very quickly until it freezes.

Any thoughts as to why the progress bar is freezing?

sbyam Joined: 2007-01-13 Posts: 9	Posted: Wed, 2007-01-17 00:49
	Gallery version = 2.1.2 core 1.1.0.2 PHP version = 5.1.5 cgi Webserver = Apache Database = mysql 4.1.18-standard-log, lock.system=flock Toolkits = ArchiveUpload, Exif, Gd, ImageMagick Acceleration = none, none Operating system = Linux cgi1507.int.bizland.net 2.4.32-grsec #1 Fri Mar 17 12:22:27 EST 2006 i686 Default theme = matrix Locale = en_US Browser = Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322) I am in the process of moving my Gallery site from one server to another. The master plan is to upgrade to Gallery 2. Gallery 2 is installed next to Gallery 1 (http://sbyam.net/G2) the problem comes when I try to import the G1 gallery. I can only load about 140 pictures at a time. Seeing so I have albums with more than 600 pictures and the gallery itself has about 31000 pictures this will take a long time to move over if it is even possible. I have been chatting with one of the Gallery 1 guys about this issue so we should probably start there: http://gallery.menalto.com/node/59455 As you can see there it looks like I am having a memory issue even though I have bumped up my memory all the way up to 160M. The import still freezes. I am on the Powweb server so I have also set up my temp directory correctly as well. Any thoughts? Thanks, Steve
	XUser login Username: * Password: * Create new account Request new password
ivan_bev Joined: 2006-04-10 Posts: 14	Posted: Wed, 2007-01-17 16:17
	Aside from memory, the only other things I can think of to try are: * do not create thumbnails on import (saves time and RAM use) * check whether there is an EXIF header in the problematic image If there is no EXIF header, gallery2 import appears to have issues if you have the exif module enabled. Try disabling the exif module when importing and re-enable afterwards. For more info on this see: http://gallery.menalto.com/node/44743 Ivan

sbyam Joined: 2007-01-13 Posts: 9	Posted: Thu, 2007-01-18 01:32
	Not sure if I know how to fix the problem but I think I figured out what is causing it. It looks like Powweb has a CGI "stop" that kills any script that runs for more than 60 seconds. There are all kinds of writeups about this problem in thier forums. Here is one of them: http://forum.powweb.com/showthread.php?t=68409&highlight=Gallery I'm working with thier tech support to see if there is something they can do. If they can't fix the problem it looks like I will be forced to move to yet another webhost.

sbyam Joined: 2007-01-13 Posts: 9	Posted: Thu, 2007-01-18 18:11
	I got the following E-mail back from Powweb confirming that the 60 second timeout was the problem: Quote: We have a number of PHP/CGI/Web/Load timeouts that occur. At any level, when the script reaches 60 seconds and hasn't returned any data, either the web server or load balancer will kill the process thinking it has stalled/gone off to infinity. The way around this is to return some data periodically. Maybe every 10-15 seconds return some data back so that the server knows that the script is still alive and working.That should get around the 60 second timeout. Unfortunately, there's not a whole lot we can do. We can't extend the timeout, as it would reduce the performance of everyone using that CGI server Now I noticed the list of webhosts that gallery suggests. Can anyone suggest one over the rest?

valiant Joined: 2003-01-04 Posts: 32509	Posted: Thu, 2007-01-18 20:05
	Quote: At any level, when the script reaches 60 seconds and hasn't returned any data, either the web server or load balancer will kill the process thinking it has stalled/gone off to infinity. The way around this is to return some data periodically. that's what we do. the gallery import script has a progress bar. it sends data to the browser periodically. when it freezes for you, doesn't the progress bar update every (few) second(s) until it freezes? how long does it take from the last progress-bar update until it freezes? -------------- Enter the Gallery 2 Theme Contest today!

sbyam Joined: 2007-01-13 Posts: 9	Posted: Thu, 2007-01-18 22:28
	I don't even see the progress bar until it has frozen. Until then it stays on the screen before that where you verify everything. I had the Powweb guys try it on their computer and the progress bar works but it still freezes after about 60 seconds. They did manage to load more pictures than I did though. I suppose because they are closer to the server and on a faster network. While I was poking around the powweb forums I noticed lots of people tried work around scripts but none seemed to really work. I think it is something wrong on their end that they are not willing to fess up too. Maybe I'm wrong.

sbyam Joined: 2007-01-13 Posts: 9	Posted: Wed, 2007-01-24 00:55
	OK so the problem is not Powweb. I moved my site to Bluehost.com and I'm getting roughly the same problem. The only difference is that in Bluehost the memory bar in the upper right says 0, total: 0M rather than the 13572480, total: 160M that I used to get. The only good news is that I can upload closer to 500 pictures at a time with bluehost rather than the 140 that I was able to before. I also found that the reason that the import progress bar was not showing up is because I was using internet explorer 7. When I use Internet Explorer 6 the progress bar shows up as it should. To answer your question the progress bar move very quickly until it freezes. Any thoughts as to why the progress bar is freezing?

Import freezes on ~140th picture

XUser login