Any suggestions for performance tweaks?
|
bkreid
Joined: 2006-08-18
Posts: 8 |
Posted: Tue, 2007-03-13 00:22
|
|
My Gallery has 22,000 photographs in 400 top-level folders. Performance is unbelievably bad, even though I have a moderately beefy machine supporting it. Gallery version = 2.1.2 core 1.1.0.2 :/usr/obj/usr/src/sys/SMP i386 The computer is a dual-processor 2.2GHz Athlon with 3 gigs of RAM and hardware RAID 1. I figure I can get maybe 3 times the speed by switching to the fastest new server I can afford, which is about 3 times faster than this. But I need it to be 100 times as fast. It takes about 5 minutes to log in, 3-5 minutes to "Return to gallery" after editing a photo, and 5-10 minutes to upload a photograph. It's not a bandwidth issue; I have world-class connectivity at 100MB/sec. There must be some database parameter I can tweak, or some array size I can change, or some caching option somewhere, to get it to use more resources but run faster. I was hoping that 2.2 would have performance improvements, but the release notes don't mention any. Anybody have any suggestions? I'm even willing to write code if it would help, though I don't think I'm a good enough PHP programmer to be able to feed my code back into the source pool. here is the gallery, if you care. Brian Reid |
|
Posts: 32509
5 minutes per request?
your server must be misocnfigured badly.
i can't confirm that though.
i just browsed your site. pages load in about 1-3 seconds images stop loading after about 10 seconds.
a dedicated server should perform better.
anyhow, did you already install a php opcode cache (eaccelerator, apc)?
apache2 performs better than apache 1.
etc etc.
please read the performance tips. documentation -> admin & maintenance -> performance tips.
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 8
The server is configured properly. It is a dedicated server. I am an expert sysadmin and I am 100% confident that this is a performance issue in Gallery somewhere. I have read all of the documentation, all of the performance tips, and so forth, and I am running all of the modules that I decided, after reading the documentation, were wise to run.
This is not an apache problem either. You can see that by clicking around http://server3.reid.org/manual/, which is pure HTML pages on that server.
I interpret your answer as saying, somewhat indirectly, that you do not know of any performance tweaks that are available and that the Gallery code is just slow. I was pretty sure that would be the answer, but I thought I'd ask anyhow.
Posts: 32509
No, that's not what I'm saying.
- You claimed that normal requests like browing around in your gallery take up to 5 minutes. I don't see that when I browse around in your Gallery.
- I pointed you to php opcode caches and other software suggestions like using apache2 and php5 instead of apache 1 and php 5.
- and finally, the performance tips page in the gallery codex has some more tips.
The performance you're reporting is not to be expected.
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 114
I am also not experiencing 5 min load times. The load times seem ok.
http://www.trueppc.com
http://www.trueppc.com/gallery
Posts: 8
I didn't say 5 minute load times, I said 5 minutes to log in. During the past hour I've only seen it take 3 minutes to log in, and perhaps 2 minutes to do "Return to album" from the end of "Add items".
The major difference I see bewteen my site and yours is that you have 21 top-level albums while I have 375 top-level albums. And I'm guessing that you have 21 members; I have 557 members.
I have more members than top-level albums because my gallery was created by migration from G1, and about 200 members haven't logged in since the migration.
Since people are responding to my request for help by saying that they aren't experiencing this problem or that I'm an idiot, I'm starting to think about ways in which my large gallery might be different from other large galleries, and the first thing that comes to mind is that I have 200 members who have never logged in and about 150 albums that aren't part of any member's login gallery. Maybe I should try to contact these 200 members and remove their accounts if they don't respond; maybe I should take the 150 albums that are not part of any member's login gallery and put them someplace besides the top level.
Posts: 129
I would back up your gallery files and database before deleting those 150 albums and see if that makes a difference. If not, restore them, otherwise you will at least know that Gallery2 can't handle that many albums and it's out of your hands.
my gallery
Posts: 32509
- the parts of g2 that don't scale too well yet are mentioned on the performance tips page. including: random image block, too diverse permissions (e.g. excessive use of user-albums) and a few other things.
> Since people are responding to my request for help by saying that they aren't experiencing this problem or that I'm an idiot,
noone says the latter. i just stated that you misinterpreted my reply. you said things in your 2nd post about the replies you've got that simply aren't true.
- you say it takes about 2 minutes to "return to album" which is the same operation as just browsing albums. when i browse YOUR gallery, i can't confirm these times (as i said in my previous comment). your album pages load for me in 5-10 seconds (including all thumbnails). it's not the fastest g2 i've seen, but it's not that slow.
did you test your gallery browsing it from other computers / locations as well? maybe your connection at home/work is slow.
anyhow, i see some potential for improvement:
- turn off the random block for now.
- install a php opcode cache if you haven't already.
- check whether your permissions are too heterogenous (the number of ACL ids explodes).
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 8
I tried all of the performance tips before I ever opened my mouth here. I reinstated the random image because it had no effect on performance, and I don't have any explicit user permissions at all.
You claim that the "return to album" is the same thing as just browsing, but simple measurement shows that is not true. Browsing is not terribly slow (5 to 10 seconds, as you say) but "return to album" takes several minutes.
As I said previously, the server and connection are not the problem, which you can verify by looking at the Apache manual pages on that server.
> did you test your gallery browsing it from other computers / locations as well? maybe your connection at home/work is slow.
This kind of statement is what I am referring to when I say you are not helping with my question but assuming I am an idiot. I've already told you I have a 100MB/Sec connection to the Internet, and I've already told you I am an expert sysadmin. Of course I've tried these things. I actually have a gigabit LAN connection from my office computer to the server, and our building has a gigabit connection to the Palo Alto Internet Exchange (from which we have transit from Cogent, XO, and UUnet), but there's a 100MB/Sec rate limiting filter between the /24 that my server is on and the PAIX link itself.
I've spent another couple of weeks studying this performance problem, and what I've learned is that Gallery 2 is simply making too many references to the database when performing simple operations such as "return to album" from image upload. My packet trace software doesn't know how to look inside these packets (which are on a Unix-domain socket and not a network socket) so I don't know what sort of database references are being made.
My best lead is the one provided by Jelly who showed me a gallery that is the same size as mine that does not have performance issues for viewing, and probably over the weekend I'm going to try moving or removing those extra 150 top-level albums.
Posts: 32509
@Back to Album:
If you take a look at the URL of "Back to Album", it should be the same as the one when you are in the parent album and click on the album thumbnail.
the &g2_navId might be there or not, but that doesn't make a real difference. The navigation history data is loaded anyway.
Do you see any redirects or so when you click "Back to album"? (livehttpheaders is a convenient tool to diagnose this, ethereal is a little overkill here)
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 8
Yes, the URL is the same. But when pages are active (e.g. generated with PHP) the server side can do whatever computing it wants in response to any given URL.
When you are returning to an album after making changes to it, the page cache is invalid and the page image will have to be reconstructed from the database. So the time that it takes to "return to album" is pretty much the time that it takes to construct an un-cached page image. So in my mind this pretty much narrows the problem to being in the code that constructs a page image. In order to do this, it must of course make database references, and my packet tracing shows me that it is making several hundred such references, which seems extreme.
I'm working on instrumenting the MySQL server so I can find out what those database references are.
Posts: 32509
FYI: I wrote a considerable part of G2. Thus I know what is usually going on on the server side.
You're right in that the first request for an album page after changing the album is heavier than subsequent requests if page caching is enabled.
You can check your theory by turning off page caching in "site admin -> performance." if normal album browsing takes suddenly 2 minutes, then it looks like page caching is indeed doing a good job for you.
But even without page caching, G2 should be much faster.
BTW: You don't have to instrumet the MySQL server to see what queries G2 is doing.
- You can enable MySQL query logging (mysql config)
- You can enable database profiling in G2 (config.php setProfile).
You'll see that G2 does a lot of DB queries per request. And it does a lot of file I/O. We know about this and it's and minimizing I/O is on our roadmap.
But there must be some reasons why your G2 is that slow. I'm just reiterating that while there are inherent problems with G2, it's not normal that it is as slow as you describe.
--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage