Performance is painful
|
BobbyCarp
Joined: 2005-06-30
Posts: 18 |
Posted: Mon, 2005-07-18 15:10
|
|
Functionality is impressive, but G2 is way TOOOOO slow. Uploading pictures is dreadfully slow and retrieval is not great either. Is the team going to work on performance before 2.0 goes final? I sure hope so. |
|
Posts: 32509
what are your system specs' got a phpinfo link?
Posts: 18
Here is the output from System Maintenance/System Information:
Gallery version = 2.0-beta-3+ core 0.9.31
PHP version = 4.4.0 cgi
Webserver = Apache/1.3.33 (Unix) FrontPage/5.0.2.2635 mod_ssl/2.8.22 OpenSSL/0.9.7d PowWeb/1.1
Database = mysql 4.1.9-log
Toolkits = ArchiveUpload, Exif, Gd, ImageMagick, NetPBM, Thumbnail
Operating system = FreeBSD clust06-www03.powweb.com 4.11-RELEASE FreeBSD 4.11-RELEASE #0: Thu Ma i386
Browser = Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4
I am configured as multi-site, so I don't know how to give you the link to the phpinfo.php file. I tried using gallery2/lib/tools/phpinfo.php and I got the following error:
You must create a config.php file in the Gallery directory before you can continue configuring the application. Use the installer to create one.
Thanks for helping with this,
Bobby
Posts: 32509
you're reporting that g2 is too slow. this is a fairly relative statement and G2 is surely slow on an old systems.
what system specs does your server have? cpu / RAM / php accelerators / ... ?
Posts: 18
My web site and gallery is hosted by PowWeb which I understand has a fair number of users who use Gallery. I would have to contact them for the specs of the server that my account runs on. I don't have time to do that right now.
I did do a comparison between Gv1.5 and Gv2 uploading photos:
Test is 5 photos from 2.1 megapixel camera, total size 4.5MB.
G1.5 took 7 minutes to upload
G2 to 9 minutes to upload and I unchecked the create thumbnails on upload so it would be faster.
9 minutes for 5 photos from a very small megapixel camera is not good. My Nikon D70 pictures are much larger than these Canon S110 pics.
Any thoughts about these numbers? Could you try an upload of 5 similar pics to your host and measure the time just to see if we're in the same ballpark?
Thanks,
Bobby
Posts: 68
I just uploaded 5 4megapixel photos to my server in 30 seconds. It's a pretty slow computer (PIII 550MHz, 384Mb RAM) but then again, it's in the next room. Also I don't have thumbnails working yet
So the comparison with other people doesn't really work because that's an internet speed thing, either you or your host. I really don't get the 2 minute difference though, especially if you have thumbnails off. That seems a little strange, maybe run that test again. G2 uses MySql, is a bit bigger and more robust, and will end up being slower for sure, but 2 minutes, man.
Posts: 32509
BobbyCarp, an uploading benchmark? that's interesting, but not too important. because you view much more often than you upload new pictures.
and RalphTheWonderLlama is right, there are a lot of variables in this uploading. you'd have to make sure that your connection and the connection of powweb is in the exact same state (bandwidth, response times). etc. you'd have to run this "benchmark 20+ times and maybe run also a upload only 1 file and upload 50 files etc. cases.
having said that, you are not the only user who is reporting that g2 is slower than g1. that's true for small galleries. as soon as your gallery gets a little larger, g2 will be faster than g1 because g2 won't suffer from the size of your gallery or only minimally.
and we're planning to make g2 faster. we want to introduce more caching (we cache already a lot on different levels).
Posts: 16504
I'd like to note, to rule out G2, try uploading those same 5 images to your server via FTP how long does that take?
What's the advertised upload speeds from your ISP? This is a connection related issue or maybe poor performance on the server, but NOT G2. If others were having the problem, yeah, maybe G2 would be a factor.
Maybe the connection between you and PowWeb just goes through some bad "pipes". Try also doing a traceroute between you and the server you're uploading to.
Posts: 5
I have a question.
I would like to benchmark gallery performance with apache 2.0.56 mysql 4.1.10 and PHP5 on fedora core 4 without any specific compiler optmizations. I have a PII-450 with 128Mb ram with 3g harddrive.
Then I would like to add optimizations such as march=pentium2 etc. to see how will this change the performance. Plus, this machine is mine and connected to LAN. I think that this way there are not many things that can alter the consistance of the result.
Thank you
Posts: 32509
aseac, and what is the question?
What I can tell you without benchmarking is that in webhosting, the MHz is what counts. And RAM is also important. You machine won't perform that good. It's a good system to serve static pages, but not for a G2.
(I develop and test on a P3 600 MHz / 512MB RAM, and it's ok, but not fast on it).
And if your question was how to benchmark: Maybe try ab/ab2 (apache benchmark).
Posts: 5
valiant, well, my question was how to benchmark g2.
?
I do not expect my setup to be particulary fast. I would like to see some numbers so if I get myself a faster machine (or more memory) I can compare not just by feel but by some number.
I had Mandrake 10 before with similar setup and I feel that FC 4 is slower. Maybe its just my feeling
Posts: 26
aseac, I run g2 on a FC4 box, so I may give you some advice. By default FC4 is slow because it turns on a billion services that you're probably not using. Check your process list and see what you can diable. This will free up some cpu time and also a bunch of memory.
Also, looking at your system spec, you need lot more memory. I bet your system is swapping out to your HD even when idleing. (for shits & giggles, would you paste the output from free -h to confirm this)
Also, install an accelerator (This will again require additional memory) I gained about 3x speed improvement from eaccelerator on PIII-1ghz with 512 MB
By the way, I recently upgraded the server to a P4 -2ghz with 512MB ram (from PIII-1ghz with 512 MB ram) and didn't really gain a whole lot of performance on serving out pages. On the other hand, creating thumbnails/resizes is now ultra quick...
Posts: 43
I have a little file on my webiste who give me lots of info about my web hoster,
It need some changes for each Webhoster, but i t should at least give som basic info (Server related info)
Together with phpinfo, you should get pretty good ide about the server your site is running on (Linux)
See attachment info.php
Very simple php system commands
it display this info: (and some more) http://gallery.nenni.net/info.php
Uptime:
8:25pm up 1 day, 1:56, 0 users, load average: 2.86, 2.67, 2.78
System Information:
Linux node10.c1 2.4.20-28.8smp #1 SMP Thu Dec 18 12:25:21 EST 2003 i686 i686 i386 GNU/Linux
CPU Information:
processor : 0
model name : Intel(R) Pentium(R) 4 CPU 3.00GHz
processor : 1
model name : Intel(R) Pentium(R) 4 CPU 3.00GHz
Memory Usage (MB):
total used free shared buffers cached
Mem: 1999 1984 15 0 20 1041
-/+ buffers/cache: 922 1077
Swap: 1027 92 934
More memory info (vmstat):
>> Ask google, he will tell...
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
1 3 0 94916 15608 21288 1066536 5 44 7 53 166 126 6 6 89
Disk Usage:
Filesystem Size Used Avail Use% Mounted on
/dev/hda3 35G 1.4G 31G 4% /
/dev/hda1 197M 15M 173M 8% /boot
none 1000M 0 999M 0% /dev/shm
super.c1:/exports 106G 91G 9.8G 91% /imports
storage.c1:/home 458G 302G 133G 70% /mounted-storage/home
storage2.c1:/home 223G 200G 12G 95% /mounted-storage/home2
storage3.c1:/home 362G 330G 14G 96% /mounted-storage/home3
storage4.c1:/home 360G 143G 200G 42% /mounted-storage/home4
storage5.c1:/home 367G 135G 214G 39% /mounted-storage/home5
Ip address:
192.168.10.10
Process info:
>> Change this as you like
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 Jul19 ? 00:00:05 init [3]
root 2 1 0 Jul19 ? 00:00:00 [keventd]
Posts: 5
johann@nenni.net,
. I tried installing eaccellerator, but it failed.
Wow you have a superb PC! Dual Xeon (?) 3Ghz! with 2Gb of ram w/SCSI drivces. amazing. I cant afford this much power. Plus I dont think I need it. THis is a home pc that is dedicated to giving pics to my family members. I dont even have enough upload speed to bring this machine down.
Having said that, I recompiled MySQL with some optimizations (-O3 -march=pentium2) and installed a 256 ECC memory. This is one of my fastest machines besides my working PC. I cant afford anything faster - I am a student. This stuff I got either from a friend of mine or bough certain components for knock off prices (10-20 bucks).
All of that combined I have what I have (I do have a dell pe 4300 with dual p2 450 and 512 ram, yet this pc is so noisy that it just takes space).
I got some more performance out of my rig. I disabled some services, installed more memory... I think I'll recompile apache
Posts: 22
Where is the attached file? I only see a link, and then I cant see the source.
Posts: 16504
johann@nenni.net, the uploader probably filtered out the php file. Try zipping it or changing it to .php.txt with instructions to remove the .txt extension.
Here's the list of allowed file extensions: http://gallery.menalto.com/index.php?name=PNphpBB2&file=posting&mode=rules&f=22
Posts: 43
I can be course I did outsource my private home servers to a Web hosting company, and pay only 7,50€ a month... :D I did have Quad CPU Server at home, but the Electricity was to expensive... I save many many $$$$ by outsource this... :D And it does look good at this Webhost, at least my little script show at the CPU and memory usage is very low in average...
Posts: 43
I do not know where the Attachment did go...
So here is the script..
I do not take any credit for this, it is just from the manual and google search on Linux commands..
----- cut -----
<html>
<head>
<title><?php echo $SERVER_NAME; ?> - Webhost info</title>
</head>
<STYLE type=text/css>
BODY { FONT-SIZE: 8pt; COLOR: black; FONT-FAMILY: Verdana,arial, helvetica, serif; margin : 25 25 25 25;}
</STYLE>
</head>
<body>
<pre>
<b>Uptime:</b>
<?php system("uptime"); ?>
<b>System Information:</b>
<?php system("uname -a"); ?>
<b>CPU Information:</b>
<?php system("cat /proc/cpuinfo | grep \"model name\\|processor\""); ?>
<b>Memory Usage (MB):</b>
<?php system("free -m"); ?>
<?php system("free -h"); ?>
<b>More memory info (vmstat):</b>
>> Ask google, he will tell...
<?php system("vmstat"); ?>
<b>Disk Usage:</b>
<?php system("df -h"); ?>
<b>Disk Usage:</b>
<?php system("du -h $PWD"); ?>
<b>Ip address:</b>
<?php system("hostname -i"); ?>
<b>Process info:</b>
>> Change this as you like
<?php system("ps -ef"); ?>
<b>Environment info:</b>
>> Change this as you like
<?php system("set"); ?>
<?php system("locate wget"); ?>
<b>Apache info:</b>
>> Change this to the path of httpd if it is not in path
<?php system("/usr/local/apache/bin/httpd -V"); ?>
<b>Apache error log last 20 lines:</b>
>> Change this to the path of according to info from Apache Info
<?php system("tail -n 20 /usr/local/apache/logs/error_log"); ?>
<br>
</body>
</html>
----- cut --------
Posts: 32509
don't wonder if it doesn't work on your (freebsd) host.
i've googled a lot a few months ago and the
cat /proc/cpuinfo
or a few others commands / files used here are not available on all systems. it will work on most linux distributions though.
Posts: 7
I love the G2 ... its faster than G1.
Posts: 18
I have many years of experience with performance tuning on software, so I understand the need to benchmark similar test cases over different times of day, etc. with the fluctuations in Internet traffic & other issues.
That being said, I ran 3 tests this morning which I think highlights the performance problem with gallery and isolates it to a specific functional area and removes the concern about my PowWeb web host server performance.
All 3 tests involved the upload of the same 5 pictures which total 4.4MB.
Test 1: Upload 5 pictures using the "From Web Browser" tab in the Add Items page. Turned off the create thumbnails now check box.
Time: 7 min 30 secs
Test 2: Upload 5 pictures to my web server using FTP.
Time 1 min 58 secs
Test 3: Upload 5 pictures using the "Upload Applet" tab in the Add Items page.
Time: 2 min 36 secs
As you can see, the FTP and Upload Applet are very similar in performance, I would expect the applet to take longer as it involves server logic that has to run in addition to the transfer. During the applet upload, the speed shown in the dialog was approximately 38kb/s.
Something is clearly wrong with the upload from web browser functionality. Can a gallery developer help me instrument the code to figure out where the time is being spent?
Thanks,
Bobby
Posts: 32509
modules/core/ItemAddFromBrowser.inc
could you also test ItemAddFromServer.inc (Add item from local server).
purpose: if you first upload the items via FTP and then use this method, you can measure the time that G2 actually spends on adding the items.
If you test ItemAddFromBrowser, you also benchmark how long your browser has to send form data (files as POST data) to the server.
Posts: 18
I just did add items from server for the 5 files and left the create thumbnails now on. It took 22 seconds.
For more info, I include my DSL connection speed info below:
I also ran broadbandrepors test on my connection. I get 1.277 Mbps upload and 319 kbps download. 319 kbps is 31 kps which is around what the upload applet told me.
The 4.4MB for 5 files is 4296K / 31 kps = 138 seconds. This is 2 min 18 secs which really validates my connection working good for FTP and the upload applet.
I have to leave for the day, so I can pick this up tomorrow.
Posts: 32509
then, i guess, it's your browser / apache that take so long.
in the add from web browser method, g2 receives the form variables from apache.
there's not much overhead involved.
i'd like you to investigate. e.g. you could open a webbrowser, prepare the add items from web browser with the same files and then before initiating the upload, please edit main.php and write exit; right after <?php on the next line
then initiate the upload in the browser and wait until the page finishes loading. stop this time with a clock. the time you measured is the time your browser/webserver combination requires to transfer and give G2 the data.
Posts: 63
It also would help to gain some performance when g2 wouldnt create thumbnails parallel when viewing fresh added pictures without creating thumbnails first. When i do so, G2 starts 12x times "convert" to do the thumbnails and resizes at ones. This really sucks much power and mainly freeze my server. (amd1700+, 256mb ram, turckmmcache 8mb) Would be much nicer to get the thumbs generated in a "serial" way. Maybe this is possible? (Maintenance-Module do it serial not parallel).
cheers
doc
Posts: 7994
doctormord, if you select the "create thumbnails" checkbox at upload time we create them in serial. Or if you do Site Admin -> Maintenance and select the "Build thumbnails/resizes" task it will also do them in serial.
Posts: 63
Thats right, but when i use Gallery Remote it wont create the thumbs for me. Does it also create the resizes at upload time? I dont think so at all.
Posts: 32509
no, when you use GR, it doesn't create thumbs at upload time. and resizes are not generated at upload time in any case.
isn't site admin -> maintenance -> build thumbs/resizes ok?
Posts: 43
Is it possible to get info from the Developers about what we should expect in performance?
Response time, or some info how we should test (like the info from valiant)
Or some simple performance test we can use to test our G2 install, so we do not need to complain so much... I did try ab from apache project, but I am not so sure if that give correct results, we need some methods we can use as "standard"...
When I say at Gallery is slow, it is when I Display the Gallery, or access for the first time... (My ADSL is 3700/600, but at work I have BIG connection to Internet) and I get very similar results when testing from home and from work.
Things like 5 standard photos in diffrent size, or standard configuration on one test album, so we can test this photos against standard album (using same toolkit, same thumb size, same anything...)
Posts: 32509
how to benchmark etc. is an often discussed topic. we already thought about a little how a G2 benchmark could look like.
it's good to have something like a standardized benchmark, but it's not that important. in the end, you can browse a few of the G2s that are linked in the demo topic and compare the speed to your own G2. if the conclusion is that your G2 is much slower, then take a look at the FAQ to see our recommendations on how to speed it up.
for now, ab must be enough. the most important requests are showitem requests for albums and for single items as well as download item requests.
from time to time we use APD or other tools to profile G2, that is, to find out how many times a function gets called per request and how much we lose where etc.
Posts: 26
doctormord, you get 12 convert running because your browser requests all 12 images at once. apache will spawn a new thread or instance to serve each request via g2. g2 realizes that it needs to convert the image first so spawns convert. The threads/instances are not aware of the fact that 11 other ones are already running. Your browser wants the image now, so apache/g2 tries to give it to it.
Technically you could configure your apache to not run multiple instances or threads, but that would just kill your performance for everything and potentially cause your browser to time out...
Instead, I would recommend you run site admin -> maintenance -> build thumbs/resizes like valiant suggested above. I usually do this once a day if I have uploaded new pictures on that day.
For people who complain about performance. Think about what you really need and choose the correct Gallery version based on that:
If you just want a gallery with less than 5-10,000 pictures. Gallery 1 may be your choice. It is not database driven and is quick.
If you want more than 10,000 pictures G2 is your choice. G2 scales very nicely to 100,000+ pictures. But there is an overhead for this.
So just because it's version 2 it doesn't mean it's better and it's what you need. I think Gallery 1 & 2 are complimentary tools of each other addressing different needs.
So, if you have an older linux box and want to share your pictures with your family use Gallery1. G2 will have an added db overhead regardless if you're sharing 100 pictures or 100,000.
Posts: 32509
Maybe I'd agree that G2 doesn't serve the ultra low-end market at the moment and G1 may be a better choice for this kind of application.
But G2 has barely reached its first release. Of course you'll see a lot of improvements during the lifecycle of G2, improvements that will address performance, requirements and resources consumption.
Posts: 18
Valiant,
I have spent a lot of time isolating and further understanding this issue (too much time!). I did what you requested putting an exit; right in main.php after starting the upload and found that all of the time is in the upload. In order to understand if this was a PHP vs. Perl thing, I downloaded and tested both a PHP and a Perl CGI upload script and that they were both as slow as the gallery upload.
I then went up to my notebook upstairs to run the test, which it turns out I should have done first to compare performance with my desktop! Gallery upload tests using Firefox were 2:01 for "From Web Browser" form and 2:10 "Upload Applet" which is dead on with FTP times.
It turns out that the CPU on the notebook is around 2-5% utilization during the upload. On the desktop the CPU is 100% for the first 3 minutes and then 2-5% for the final 1 minute. I turned off ZoneAlarm load on startup and rebooted my desktop to make sure that ZA was not causing this CPU usage and it isn't the cause. iexplorer process in the Task Manager shows to be the one using all the CPU, but now the question is why? What is configured on my TCP/IP on the desktop that is causing IE to peg the CPU? Any ideas for further digging into this problem would be appreciated.
The setup is a DSL modem to a Linksys router. The desktop has a cable connect to the router and the notebook is using wireless.
I hope when we figure this out, others can benefit from our findings...
Cheers,
Bobby
Posts: 32509
good to know that it's not G2, but a weird windows/IE issue.
i'd turn off any unneeded programs, firewalls, windows firewall, unneeded services in general etc.
then i'd check if it's for all browsers on the desktop the same (firefox vs. IE).
i'd deactivate all IE bloat (have you installed any toolbars, e.g. netcraft or yahoo), etc.
try to isolate the case by deactivating everything that is not absolutely needed. only then you have a chance to find out what is causing it.