G 1.5 > G2 Multisite, Orange Bar Stops During Import of Large Album, No Error Shown
|
darley
Joined: 2003-04-05
Posts: 28 |
Posted: Fri, 2005-11-04 20:49
|
|
On the "Importing Photos" page during an import of a previous 1.5 album containing 577 photos, the orange bar will freeze around 142-145 photos. I have successfully imported smaller albums from my previous Gallery install.(100 images and under) In the top right corner it says, "Memory used: 11423944, total: 40M" I contacted my Host about their memory limit for PHP. It is defaultly set to 40M. I have read some other posts and have: 1) disabled the EXIF module Trying each one of these things singularly and together has produced the same result. This install is also a multisite install. I have setup my other multisites but have not uploaded any images to them. This is the first site I tried to upload images to. Here is the information from my system: Gallery version = 2.0.1 core 1.0.0.1 |
|
Posts: 32509
it's obviosuly not a php memory limit.
take a look at your webserver error log. if it just stops, there must be an error.
Posts: 28
i called my host, 1 and 1, and they do not have viewable error logs for me. the tech guy said i would have to write a script to report the errors specifically for the php that i am running.
ideas on that?
btw, i did turn on debugging mode in gallery but since i just freezes it does not load that debugging info on the page i am having the problem with.
Posts: 32509
well, you can use "logged" debug mode in g2 (see the instructions in config.php) which logs to a file.
but that still won't help. we need the apache errors, if there are any.
Posts: 28
okay...
my host gave me a script to use for logging errors. i just have no clue where to put the code or which file from gallery to put it into.
-------------------------------------------------------------------------CODE
error_reporting(0);
$old_error_handler = set_error_handler("userErrorHandler");
function userErrorHandler ($errno, $errmsg, $filename, $linenum, $vars)
{
$time=date("d M Y H:i:s");
// Get the error type from the error number
$errortype = array (1 => "Error",
2 => "Warning",
4 => "Parsing Error",
8 => "Notice",
16 => "Core Error",
32 => "Core Warning",
64 => "Compile Error",
128 => "Compile Warning",
256 => "User Error",
512 => "User Warning",
1024 => "User Notice");
$errlevel=$errortype[$errno];
//Write error to log file (CSV format)
$errfile=fopen("errors.csv","a");
fputs($errfile,"\"$time\",\"$filename:
$linenum\",\"($errlevel) $errmsg\"\r\n");
fclose($errfile);
if($errno!=2 && $errno!=8) {
//Terminate script if fatal errror
die("A fatal error has occured. Script execution has been aborted");
}
}
-------------------------------------------------------------------------END CODE
ideas?
Posts: 32509
well, you can set
error_reporting(E_ALL);
right after
<?php
in main.php and it will show all php errors directly in the browser.
the above script won't catch all errors. the php apache module can still crash / die and then we see the message only in the apache error log.
Posts: 28
okay...
so i added the above code to my config.php file in my multisite directory, not the root codebase dir.
it generated a CSV file for me to look at the errors. Here is what it said.
BEGIN ERROR ----------
11/4/2005 19:01
"/htdocs/dphotos/smarty/templates_c/%%2308901455/%%55^55E^55E14245%%SystemLinks.tpl.php:5"
(Notice) Undefined index: separator
END ERROR ------------
The 'dphotos' dir is my albums directory for this site that is ABOVE the root dir for the site itself.
Meaning... I have in the root of the server i have 3 dirs, one called 'dphotos', one called 'dsite' and one called 'gallery'.
GALLERY dir contains the codebase. DSITE dir contains a dir called 'photos' that hold the config.php, main.php, embed.php and index.php for the gallery multisite for this site. then the DPHOTOS dir contains the actual photos that are uploaded via gallery.
Posts: 28
i added what you showed above:
error_reporting(E_ALL);
in the main.php file in the codebase, nothing showed. then removed it and added it to the main.php in the multisite dir and nothing showed as well.
went and deleted the existing dir that i had tried to upload and tried reupload and this time it got to 212 photos before it tanked.
Posts: 32509
that's just a notice. it wouldn't stop because of this notice. and again, this error handler script from your webhost won't help here.
use g2's "logged" debug mode (see config.php for instructions) and if that doesn't help, we absolutely need to see the apache error logs, else we can't solve your problem.
Posts: 28
Well, I added turned on the "logged" mode and it put out a whole file of errors.
I have the text file. It is about 2MB.
Here are the last few lines of the file.
BEGIN CODE -------------------------------------------------
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] <hr>
(mysql): UPDATE dar_Item SET dar_viewedSinceTimestamp=1107362733,dar_originationTimestamp=1107463479 WHERE dar_id=9907 <code></code>
<hr>
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] file_exists(/homepages/34/d115060170/htdocs/dphotos/cache/entity/9/9)
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] is_dir(/homepages/34/d115060170/htdocs/dphotos/cache/entity/9/9)
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] rename(/homepages/34/d115060170/htdocs/dphotos/cache/entity/9/9/9907.incypsGyN, /homepages/34/d115060170/htdocs/dphotos/cache/entity/9/9/9907.inc)
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] fclose(Resource id #13899)
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] file_exists(/homepages/34/d115060170/htdocs/dphotos/locks/9/9/9907)
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] unlink(/homepages/34/d115060170/htdocs/dphotos/locks/9/9/9907)
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] [1131155875] can't guarantee 30 -- extending!
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] fwrite(Resource id #812, ..., )
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] fflush(Resource id #812)
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] file_exists(/homepages/34/d115060170/htdocs/dsite/pics/albums/honeymoon/honeymoon154.jpg)
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] is_uploaded_file(/homepages/34/d115060170/htdocs/dsite/pics/albums/honeymoon/honeymoon154.jpg)
2005-11-04 20:57:55 [eb0e5fc6fdf0755a79293d41b65f1269] file_exists(/homepages/34/d115060170/htdocs/dsite/pics/albums/honeymoon/honeymoon154.jpg)
END CODE -------------------------------------------------
This is also what was happening when the orange bar stopped moving. So I figure this is where the problems start.
Thanks!
Posts: 32509
well, i see nothing here. maybe if you post the last x hundred lines or the whole log, it would help. do not post the log here, you may zip it or upload it somewhere else and point to it.
and it looks more and more that the apache error log would really help here.
Posts: 32509
also, i assume you've run the find_orphans tool of g1 before starting the migration, right?
see the migraton docs on codex.gallery2.org
Posts: 28
I have not done that, that I am aware of. I did speak with my ISP and they informed me that they will have to have a higher level tech call me back about the error logs. I will give the orphans tool a shot!
BTW I sent you directly the link to my log from before.
Thanks!
Posts: 28
Yes there were 224 orphaned files found in my previous version of gallery. It says I need to delete them. My guess is they are gone for good once I do this yes? Or is deleting them from the dat file or whatever holds the master list of images?
Posts: 28
I ran it and deleted the files. Then I ran the validate albums in the old gallery as well. everything checked out.
went back to G2 and it hung up again. bummer. ideas?
Posts: 32509
i'm not a g1 expert. ask the g1 guys about the orphan tool but my guess is: yes, delete the orphans. you're free to readd them with g1 or g2, but right now g1 is broken.
@debug:
it stops at /homepages/34/d115060170/htdocs/darleys/pics/albums/honeymoon/honeymoon154.jpg . does that file exist? is it one of the orphans?
you didn't yet post a phpinfo link!! please post a link to a phpinfo file that is in your gallery2 folder:
info.php
<?php phpinfo(); ?>
it could be a plain timeout, not sure yet. if set_time_limit is in the list of disable_functions in your phpinfo, that would explain everything.
so please clean your g1 according to the orphans tool and post a phpinfo link.
Posts: 28
I am attaching my PHPinfo now...
Posts: 32509
so it's not a timelimit issue.
still:
- did you solve the orphans issue?
- else the apache error log is the last thing that could help
Posts: 28
Okay...
I ran the orphan tool on my old gallery. It came back, as I said before with 224 orphaned files. I proceeded to let the tool fix the problems and once it was done I ran the 'validate albums' tool from the admin page. Everything was shown as fine from there. That is where I stand. I have heard nothing else from my host at this point.
Posts: 32509
did you retry the import with a fresh G2 installation now that the g1 issues are resolved?
Posts: 28
I am going to freshly install G2 now. Will let you know how it works. Thanks again!
Posts: 28
Alright, I reinstalled G2 fresh and clean, bummer, still no go. Will go back to working on getting the Apache logs.
Posts: 27
Just FYI, while playing with conversion, I get this "error" quite often (the orange bar stopping part way through with no warnings, errors, etc). If I go back to album admin, and delete the partially completed album, then "re-import", the exact same import works perfectly.
I see this about 20% of the imports that I perform.
And I always get 0/0 for memory usage in the upper right corner.
Steve
Posts: 28
i tried your idea sparker, went about 10 images further and then stopped.
the problem it seems only occurs on larger albums. anything 150 and under works fine. but i have 2 that are over 300 and 500 each. those are the ones it freezes on. oddly enough, before i ran the orphan finder on my existing G1, i had successfully imported the G1 albums (that wound up having orphans) into G2 without error. the 2 G1 albums (the ones with 300 and 500 plus) that DID NOT have any orphans are ones that give the error while running the import.
my hosting company's tech support is based out of country (surprise) and i called them again today about the Apache logs. they informed me that they will move me to a higher level of tech support ( in the U.S.) that will hopefully help me further with the log issue.
i have also been reading about '.htaccess' that needs to be put in the G1 and G2 galleries for URL redirect. is this ONLY if you want to redirect? i want all the photos moved from the old dir to the new one.
ideas?
Posts: 32509
ah, that is new information.
open modules/migrate/ConfirmImport.inc and change all guaranteeTimeLimit(..) calls to guaranteeTimeLimit(180); or so. this will increase the time limit to 3 minutes.
Posts: 28
i followed your word here, and changed them all. it still timed out and the orange bar stopped. i deactivated the 'migrate' module and reactivated it, thinking it may force the change to be accepted. no avail. do i need to have the server rebooted? or is there an easier way?
Posts: 28
seems this guy has a VERY similar problem:
http://gallery.menalto.com/node/36956?highlight=guaranteeTimeLimit%28..%29
alas no solution, yet.
Posts: 28
anyone have any other ideas? my host STILL has not gotten back to me.still working on the log files.
Posts: 1
FYI, im also having this issue with 1&1 hosting, im not going to start a new thread just yet, i will keep my eye on this one and see what happens
Posts: 81
FYI, I having the same problem. I'm not with 1&1 hosting, but with Pair. Mine times out in small albums, at the same place every time. (60 images) No memory used and total = 0.
I ran the find_orphans tool of g1 with no errors. Can't get it to work either. Some small albums work fine, my biggest album is 370 images, haven't tried it yet????
Posts: 81
My biggest album times out also, orange bar stops with no error or nothing. Same place every time???
Posts: 62
If you are using 1 and 1. You are out of luck.
The problem with them is that they crowds the server a little too much for their shared hosting package. Once I moves the installation to local machine for testing. I got everything converated very smoothly.
That is just how they are, sometime then a process excute a file too much it will just time out. Very frustrating but there isn't much you can do.
Try to only import one or two album at a time and delete those that stop in the middle. Beside that. I have found no solution to make the converation smooth.
Luckily, it only happens at migrate time. After that, generally, the server is fine if you are not too overly loaded.
In my experience, you can only convert about 50 images at one go before it stops.
Posts: 28
Well, I finally got some response from 1&1.
"Only our server administrators have access to the Apache error logs, we unfortunately cannot monitor these or make portions of them available. You may want to look at whether the script is timing out; if it exceeds the set CGI limits for your package this may be the problem."
I have the Developer package, which is their highest shared program.
I am so frustrated now. I wonder if I can ask them about their support for Gallery itself. They have their own "Photo Gallery" software in the control panels, but it is no where as good as Gallery.
Back to the drawing board I guess.
Posts: 8
Well this thread is certainly a bummer. Another 1&1 user here (also using the developer package) and getting the same problem.
Their tech support really sucks, and you'll find lots of similar comments on many hosting forums.
I was really hoping to avoid migrating G1 to a local test machine, doing the converstion locally, and then moving G2 to the server.
id01: when you say 'they crowd the server' do you mean in terms of hard drive space? That doesn't seem like that's the cause of the problem if converting small albums one at a time will work, but not large ones all at once... but if it is really the problem, maybe one could ask 1&1 to move their site to a more suitable server.
Posts: 28
After hearing from 1&1 and seeing the CGI memory comment, the area in the top right became much more interesting to me.
The "Memory used: 11423944, total: 40M" area changes while the process is in progress. But, when I first hit the page, which is after I press the submit button on the previous page, the Memory used says "9482192". Shouldn't it start at 0?
IS this related to the CGI memory limits?
Posts: 32509
nope, the framework uses about 9mb in your case, that's normal.