Step 2 hang - memory limitations
|
OldJacques
Joined: 2009-04-29
Posts: 22 |
Posted: Wed, 2009-04-29 14:30
|
|
The installation of Gallery 2.3 is getting stuck, hung at step 2. I've tried the solutions found here in the forum but they don't help, and don't seem directly applicable in any case. In the /install/error_log file are the following lines: since I tried to increase my memory_limit in PHP as 'ini_set("memory_limit","64M");' as per other posts in the forums. The .htaccess solution didn't work either. The memory_limit value is set to 32M in any case (according to phpinfo), so it should be enough. The page just stops loading after a few seconds, without any other error message. The message is referring to roughly 16MB, which doesn't seem to make sense. I also modified config.php-template trying both $gallery->setDebug('logged'); and $gallery->setDebug('immediate'); (loaded both as config.php-template and config.php, I was guessing the -template needed to be removed even if I didn't see it written in the file to include). The logged never updated the file (writeable permissions) and immediate never showed any errors on the screen. I have Gallery1 working, but am not attempting an upgrade. The problem existed trying to install also BEFORE I had Gallery1 present and functioning as well, so it would seem NOT a conflict with other installation.
|
|
Posts: 4342
You're running out of memory, and it looks like ("PHP Warning: ini_set() has been disabled for security reasons") your hosting provider is deliberately preventing you from allocating more.
Posts: 22
I got that far in the troubleshooting, and from the other posts, but don't understand why phpinfo.php says I have a 32M memory_limit, but the file has allocated only 15466496 (<16M).
It would seem like it were a setting from somewhere inside a Gallery2 include that were setting the "15466496" allocation, not the server.
The testscript "<?php print system('ulimit -a'); ?>" doesn't return any value either.
using ini_get_all() in the gallery directory I get:
[global_value] => 32M
[local_value] => 32M
which matches the phpinfo.
Any ideas why Gallery says a a limit half what PHP indicates?
Posts: 4342
I would try some test scripts to see how much memory you can allocate using your php setup, outside of Gallery. I can't think of a reason why the Gallery code should barf especially, and if you can write a two line script that shows the same discrepancy you have something to take back to your hosting provider to query them about.
http://us3.php.net/manual/en/function.memory-get-usage.php might help.
Please report what you find.
Posts: 22
On a page which has a lot of images loaded dynamically from directories (recursive listing) I get a value of 505256 for memory_get_usage().
Inserting it at the end of of /install/index.php, I get values of 3394720 (3,394,720) for authentication, while on the Step 2 "System Checks", it doesn't appear (obviously blocked by the fatal error).
Inserting it as line 1259 into /public_html/gallery2/modules/core/classes/GalleryUtilities.class (function readIndividualManifest) it repeats 69 times, the first of which with a value of 4633304 (4,633,304) and the last of which with a value of 14120808 (14,120,808).
I still don't know how that changes anything since it should have 32M available.
If there were a testscript to test your hypothesis (some loop?) I am more than willing to try, but don't know how to stresstest the PHP and get useful information.
Also, are you sure we are dealing with the same "memory" in all the measurements?
Posts: 4342
I mean a test script that doesn't run gallery at all. Look at the link I posted, modify the script there to allocate progressively more and more memory and see where it fails.
Posts: 4342
I'm not sure of anything, but the php Fatal Error Out of Memory is a php error, not a Gallery error. If you're worried that there's a problem that's causing it to occur when only 15M of memory is used instead of 32M the first thing to do is to try to recreate the error without the multi-megabyte headache of the G2 code getting in the way. If you can allocate 32M of memory successfully in a test script then you can consider further what the problem in Gallery might be. If you get the same out-of-memory condition after allocating only 15M of memory in a four-line test script that has nothing to do with any of the G2 code when php_info still says 32M then you can ask your hosting provider what the problem is. Divide and conquer.
Posts: 22
I know it is a PHP error, but thought it could be how Gallery is dealing with an anomoly.
I looped with increasingly larger strings created like in the example, and arrived to ** 16694960 (16,694,960) which is suspiciously close to 16M binary (16,777,216) before it got interrupted. It doesn't explain the 15466496 + 91 bytes, though.
Even specifying error_reporting(E_ALL); it doesn't display the error, and I can't set 'display_errors' in htaccess, so I'm not positive of the exact error.
But now I think I need to contact the host, as it seems almost for sure some problem on the hosting side.
* UPDATE 8 May: the way the loop worked, the last step was giant. See the later post, modifying the loop to stop one less than error and then stepping by little baby steps it arrives to 32MB like it is supposed to...
Posts: 22
UPDATE: 7.05.2009
I managed to install by using the 2.2.6 version.
But I still get memory errors, even though the installer says "Memory Limit (32Mb) Success".
The latest are like: "[07-May-2009 14:55:17] PHP Fatal error: Out of memory (allocated 12582912) (tried to allocate 30720 bytes) in /home/.../gallery2/lib/smarty/plugins/function.html_options.php on line 63" or "[07-May-2009 14:55:45] PHP Fatal error: Out of memory (allocated 13107200) (tried to allocate 7680 bytes) in /home/.../gallery2/lib/smarty/Smarty_Compiler.class.php on line 924".
Can anyone suggest a way to find some clue as to what the problem is?
Posts: 16504
Simply put, you're host.
You already have 32MB of memory available to PHP, yet you keep getting these error messages. That issue is at your host with what ever they have going on, probably the restriction on ini_set.
Unless your host also doesn't allow or make available Perl and some other scripting languages, it's also silly to restrict functions like exec, since you could use Perl to work around that.
There are better and more reliable ways to secure a system than to tie a user's hands.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here
Posts: 22
the only errors I found registered in error_log for ini_set were easily solved by commenting them out in "init.inc":
// ini_set('magic_quotes_runtime', 0);
// ini_set('magic_quotes_sybase', 0);
and as far as I can tell, they didn't have any impact in any case, because PHPinfo already reads:
Directive Local Value Master Value
magic_quotes_runtime Off Off
magic_quotes_sybase Off Off
so the two lines were re-setting the same values.
Laying all the blame on the host without any idea what the specific problem the host is causing isn't, unfortunately, very helpful. Might as well lay the blame on my bad karma.
Do previous versions of 2.2 or 2.1 use less memory, that they might help better diagnose the problem?
Gallery version: 2.2.6 (2.3 didn't get past step 2 of installation)
PHP Version: 5.2.6
PHPInfo Link: http://www.angelofeletto.com/test/phpinfo.php
Webserver: Apache 2.2.9 (Unix)
Database:MySQL version 5.0.67-community
Activated toolkits: GD, ImageMagick (works with Gallery1)
Operating system:Linux
Browser: Seamonkey 1.1.15, Chrome 1.0.154.59, IE7, Firefox 3.05,
Posts: 4342
Actually it's very much more helpful. $6 a month will see you installed on a new host - pick any one from dozens - that doesn't impose any of these issues on you deliberately to make your life harder. A karma upgrade is much more pricey.
Posts: 22
And being certain that there won't be other issues, either with Gallery, or any of the other scripts already functioning perfectly correctly on the site, besides the time to transfer and reconfigure everything (scripts, database, paths, ecc.), some inherited from the original programmer and not even mine... sure, I'll take a blind shot just to get Gallery working (wouldn't you?)
Seriously, does anyone have suggestions of diagnostics or possible problem spots, seeing that the original diagnostics of the install all worked and gave a string of "Success" messages without any possible or probable problems indicated?
Thanks for any constructive information that anyone can suggest.
Posts: 4342
I ran G2 for 18 months flawlessly on Hostmonster, and I found their technical support first-rate.
Posts: 22
Unfortunately, that isn't what I meant, I thought I was clearer.
Since I am not looking for a host that runs Gallery2, but a host that is capable of running the entire site, of which the Gallery2 may become the latest addition (if I can get it running correctly), I have no real way of knowing that all the other scripts and personalized PHP apps already present in the site, some of which were there before I started, and some of which aren't public (site admin stuff) will work on Hostmonster without problems. Nor do I know, as I didn't code it all, if moving will require tracking down some/any/much path information which is specific to the current host. Idem for MySQL authentication information. Ecc. Ecc. Ecc. as they used to say in ancient Rome when they had similar problems with their PHPhosting.
So moving isn't really an option at this point in time.
If someone has an idea why Gallery2 is giving memory errors on a Server that shows as having all the prerequisites listed for installation, it would be helpful.
Suggestions to move the domain to another unknown host (for other aspects not tied to Gallery2) are not feasible at this point, as it is risking opening another (different) can of worms which hasn't been asked and which isn't going to be paid for by the client.
I reran the memory test, and realized that fine tuning the last step to add a very small amount at a time, I do arrive with a pure memory test to 32M before running out ( http://www.angelofeletto.com/test/memload2.php ). So it seems less a problem of the host and more of a slight misinformation of the installation requirements of G2.
Does anyone know of how the 32M is divided by PHP in case of includes?
Posts: 4342
I honestly think you're letting your client in for a world of pain and misery if your/their host (it's not clear which) is sufficiently stingy with memory that 2.2.6 installs and 2.3 doesn't. You are aware of the security problems in 2.2.6 that are only fixed in 2.3?
Posts: 22
The host has 32M of memory, which is standard for PHP 5.2.6, and is supposed to be sufficient for G2.
It still isn't clear why the memory errors are popping up way before the effective memory_limit, which would seem as likely some problem with an incompatibility of G2 as any problem with "stinginess" with the host of the site.
Posts: 16504
Well even though you already have 32MB of memory available to Gallery, which should be sufficient, looks like there is probably something else running on your server that's interfering with stuff or trying to consume more memory than Gallery does while doing the same task on most other servers.
You can try allowing more memory for PHP if your host allows it.
FAQ: PHP memory_limit - Why do I get the error Allowed memory size of Xxx bytes exhausted?
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here
Posts: 16504
And perhaps some of the info suprsidr has posted in this thread will help:
http://gallery.menalto.com/node/87635
Although you're getting this issue during the install process, I think you'd make those changes in /install/index.php instead of config.php
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here
Posts: 22
Actually I have now managed to get past the install (picking few to no plugins during step 10), but still keep getting blocked on the configuration pages for themes and plugins under siteadmin. Themes shows the header and left menu, while the plugins configuration page shows up completely blank white empty.
The @ini_set('memory_limit', 67108864); doesn't seem to do anything, either with the 48M suggested nor higher. The error.log lines remain the same, with the same values.
Posts: 16504
I don't know what else to suggest. There is some odd-ball configuration issue on your server that is making PHP choke when it actually tries to do something more than load some page. I looked at your site, unless there is a bunch of backend stuff that's not publicly visible, I seriously doubt you've have any problems getting it to run at a host who has a properly configured server.
My only other suggestion is to see if you can work with the host. Gallery is installed on literally 10 of thousands of servers around the world and many 100s or 1000s of hosts without problems. This is why we keep pointing to your host, there is something they are doing, or something they have setup that is preventing a process that requires any moderate amount of resources from actually running. There is something on their server that is killing processes.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here
Posts: 22
There is a bunch of private stuff that is tied to workflow for the studio, but not linked to or from the public site.
The PHPinfo seems pretty straightforward, without any special throttling or limitations. One thing I noticed trying an install, is that it claims that the check for exec() was a "Success", but that is disabled (along with ini_set).
The host IS trying to work with us, but unless I can provide some suggestion as to what is causing the choking, they claim they aren't doing anything strange, and I don't have reason to doubt that what they say.
Posts: 27300
I don't know what to suggest either. Both alecmyers and nivekiam are in the same camp as me.
From your symptoms
1. 'PHP Fatal error: Out of memory'
2. 'I managed to install by using the 2.2.6 version.
But I still get memory errors, even though the installer says "Memory Limit (32Mb) Success".'
3. 'picking few to no plugins during step 10'
4. 'plugins configuration page shows up completely blank white empty.'
Don't know what to say other than: it seems to be a memory/resource/configuration issue.
No, I don't know how to fix other than the suggestions made previously.
Sorry I would not know where to begin to have 'suggestions of diagnostics or possible problem spots'
Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team
Posts: 16504
o.k. Let's try one more thing. Can you zip and attach your install log file? It's located in your g2data directory.
-nivekiam
Posts: 22
I appreciate the effort. This is one of the first times I've had any problems with this host/server, and they have been helpful in the past, so I just don't understand.
The install log is attached.
Posts: 16504
Unfortunately I don't see anything odd in there. Looks like the install went nice and smooth.
And it's that behavior that is leading me to believe that they are in fact doing something strange. If phpinfo is telling you those functions are disabled they should be disabled, period and that's what the install.log is telling me that exec ini_set are in fact disabled.
phpinfo isn't going to tell you anything about throttling by Apache or some other resource limitation applied to the server that's not running through PHP.
I see you are running an older version of PHP in CGI mode. Do they have either that version or the latest version, 5.2.9, available as an Apache module VS a CGI? If so, try that. Though that may not be an option for you if you are running PHP as your user instead of as the user the webserver (Apache) runs as.
If that doesn't work I'm done here. I can't think of anything else to try. I can only think they do have something guarding their resources and killing processes that go over some low threshold. My host does the same thing and not to just PHP processes, if something goes over the threshold, they kill it. They also get notified if other thresholds are exceeded and investigate promptly. They keep their servers running lean, mean and tight and I've never had a problem running Gallery there, ever.
If trying PHP as an apache module VS CGI doesn't work, then try another gallery then or try G3 when it comes closer to release and just forget about G2 for the time being.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here
Posts: 22
Well, the host removed the limitations to ini_set and exec, and bumped the predefined memory_limit for the site to 64M, but to no avail. At least this way I can have errors displayed... but still get the same error: "Fatal error: Out of memory (allocated 15466496) (tried to allocate 91 bytes) in /home/.../gallery2/modules/core/classes/GalleryUtilities.class on line 1262".
I don't think that an upgrade for this problem is possible, and switching servers at this point in time won't work.
Since I can now run exec, does anyone have a suggestion as to where to find some diagnostics that can lead to finding the bottleneck, which seems in Apache or some other server module/program/??? at this point?
P.S. I am now trying 2.3 again, which is still blocked on Step 2. I have to re-upload 2.1 and 2.2 to try again with the new higher memory_limit, but I'm not too hopeful unless *I* can find out what the problem is to help out the tech-support of my host.
P.P.S.S. I just noticed that the message received within G2 is different than the one that PHP publishes if I intentionally looped through to run out of memory storage to test. In that case it returns the error: "Allowed memory size of 67108864 bytes exhausted (tried to allocate 4097 bytes)..." which is much more similar to the other messages found in Internet (specifically the term "exhausted" instead of "Out of memory (allocated 15466496)"). And setting the memory_limit above 64M doesn't do anything, it still stops at "67108864".
Posts: 22
I managed to get the ulimit command running with exec() in PHP, and got the following values, about which, I unfortunately understand nothing as I am getting out of my league in Apache details:
Can this help to understand anything (it is listed as a possible check in the FAQ under "PHP memory_limit - Why do I get the error Allowed memory size of Xxx bytes exhausted?", but with the "system" command, which is still disabled). But the exact message I find in the logs, or with debugging errors displayed, is actually "Out of memory", and not "Allowed memory ... exhausted".
Does anyone see what the problem might be from this information? I did manage to get debugging going, but can't tell what to look for. Can anyone point me to information on the debugging or is that just general developer stuff?
Posts: 16504
In the debug output I search for "error" to see if anything is popping up.
As for the ulimit output, I have no idea.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here
Posts: 22
I didn't find anything that spoke of "error" or "fatal", which would have been relatively clear signals. When the error happens, it blocks the execution of the script, including the debugging. It still gets logged to the error_log file in the gallery2 directory, but that is the PHP error message "Out of memory" already cited.
As to the ulimit -a (and other parameters) output, it is in the "Memory exhausted" FAQ so I'm hoping someone who knows why, and how to interpret it, can either offer explanations or provide other diagnostics to run with "exec()" that can point to a problem that I can ask the host to modify or reset or resolve.
Posts: 4342
From the ulimit manpage: (http://compute.cnr.berkeley.edu/cgi-bin/man-cgi?ulimit+2)
-a Lists all of the current resource limits. -c The number of 512-byte blocks on the size of core dumps. -d The number of K-bytes on the size of the data area. -f The number of 512-byte blocks on files written by child processes (files of any size may be read). -n The number of file descriptors plus 1. -s The number of K-bytes on the size of the stack area. -t The number of seconds (CPU time) to be used by each process.Prefixed by H for hard, S for soft. So you have a hard limit on file descriptors of 16383, for instance, it seems.
Posts: 22
Thanks, but that was where I got all the parameters to run the tests. Since the FAQ refers to "ulimit -a", and that returns "file locks (-x) unlimited" in both the maximum Hard Limit and the (possibly lower, but in this case identical Soft Limit), but I don't have any idea if that is a valid response of "ulimit -a" for G2. According to the same Man page, the "File Descriptors" categories is one of two which are straight numbers, and which are not multiplied by 512 or 1024 to get the value. So it wouldn't match the error messages which already have "15466496" allocated, which is well beyond the "16383" File Descriptors allocated, so I am guessing that can't be part of the problem, unless each file descriptor has a maximum dimension of 1Kb, which would seem absurd.
In the meanwhile, another support technician just weighed in with the bomb that they have always had something called "suphp/suexec" which increases effective memory usage by a large factor (5x-10x depending on the specific functions called) as a means of protecting the system from overloads from "bad" scripts. This after more than two weeks of discussion with a different (and otherwise very nice and helpful) tech. NOW they are looking into it...
Posts: 4342
Oh, I didn't mean it was the file descriptors that were the problem - but you asked where to find out what the information meant, so I was suggesting how to decode it.
Maybe it's the same tech from a different email address, and he's playing good-tech-bad-tech to wear you down ;-)
Posts: 22
But, unfortunately, it doesn't help me decode anything. I still don't understand what impact would the values returned mean, regarding the error messages received?
Besides the fact that the recent suphp/suexec revelation would seem to explain the problem almost completely (except that I don't understand that either, as the explanation provided by the host indicated that the memory "multiplier" or "divisor", depending on one's point of view, varied according to the type of code and specific function calls involved).
Now I just have to discover if there is a workaround or not, and if not, find out if we can push for the costs required for transferring the site to another host. If anyone were to have direct experience dealing with the limitations of suphp/suexec and have good resources (beyond googling "suphp/suexec", which I've already tried, without much directly applicable information, it seems) I would love to hear about it.
Posts: 1
Hello,
I've recently been having a similar problem which appeared once I switched the way PHP was running from the 'dso' handler to 'suphp'. Switching back from suphp to dso (at the host level) seemed to do the trick and stop the php out of memory errors. I'm not sure why this interferes with the gallery software, I can only assume that a seperate, more restrictive set of memory management is employed by this handler.
Thanks
Craig
Posts: 22
After a bit of experimentation by the ISP/host, it seems that the suPHP/suExec handler, for certain types of functions, increases memory consumption in a disproportionate manner. Some functions it increases (as overhead) by little (maybe 10%?) while others much, much more (even 200% increase?). Sometimes it seemed tied to the speed of execution too: faster memory peaks (faster code?) got accelerated/amplified more than slower gradual increases.
What my host says to have done, which seems to permit the correct functioning of Gallery (I need to try a reinstall with the most recent version, since I used an older, less "hungry" version to troubleshoot, to be more sure) was to recalibrate in the Apache configuration the values for RLimitCPU and RLimitMEM and the way the were tied from a "server average" to a "script average" (unfortunately the host descripbed everything in Italian, so I hope the terms make sense in English). Unfortunately they haven't supplied the values before or after ("security reasons"?) but maybe that will help you in the right direction.
If I manage to have further (more specific) information, I will try to pass it on so, maybe, it can be of help to others.
Posts: 16504
Actually 2.3 is (or should be) less "hungry" resource wise, at least when in use. It has better caching features, so it should be hitting the disk and the database less as well as using fewer cycles to process pages since they are cached and they can just be served up.
Security reasons, what ever. That's why robust hosts like, www.pair.com, don't and won't use suPHP or suExec, those solutions don't scale well and use more resources. Hosts do that to keep everyone in their own sandbox so all the little kiddies on the servers can't fight. On a properly configured server you shouldn't need to lock each user in a jail to prevent them from messing up the server, they shouldn't be able to mess up the server to begin with, locked in a jail or not.
I prefer to use hosts that don't use suExec or suPHP, but instead has setup cgiwrap so I can setup my php and perl scripts can run under my user account and I can lock down my files so only I (and PHP which is running under my user account) can access them and if something happens then they can only damage my files and not those in another user's account on the same server. It achieves the same goal and uses less resources.
But I've also grown tired of sharing a server, so I've been looking at a VPS, playing with and setting up a server at Linode
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here
Posts: 22
The biggest problem was that 2.3 didn't install with the limitations, because the install script peaked and was killed by the RLimitCPU and RLimitMEM settings. Even installing previous versions required installing very few plugins, and those one or two at a time.
Add to that the fact that the first two techs to respond either didn't know of suPHP or didn't consider its impact in limiting memory, and cited the same PHPInfoe memory_limit values that I could see, and you understand the frustration.
I still haven't discovered how it is possible to tell if suPHP/suExec is being used by any site-side diagnostics, except for asking directly. I actually asked the first two about some element on the server which might be throttling memory and both said: "no you have 32M" (and then "64M" after they raised the memory_limit in an attempt truly to be helpful and resolve the problem). Since they don't seem to be involved though, with day to day server maintanence, they didn't realize the other factors involved.
I'm up, now, with Gallery 2, and everything is working, finally.
Posts: 1
I think it's clear that something in the upgrade process has changed.
If I install a new gallery everything is fine.
Try to upgrade , doesn't work
Posts: 16504
clearstr, we need information to go on to help you. Please start a new thread, filling out the support template and including a link to phpinfo.
FAQ: What information is required when I ask for help in the forums?
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here