How to Improve System Performance And CPU Load

Gallery version = 2.2.5 core 1.2.0.7
PHP version = 4.4.8 cgi
Webserver = Apache/2.2.8 (Unix) mod_ssl/2.2.8 OpenSSL/0.9.8b DAV/2 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635
Database = mysqlt 4.1.22-standard, lock.system=flock
Toolkits = ArchiveUpload, Exif, Gd, Thumbnail, Getid3, LinkItemToolkit, SquareThumb
Acceleration = none, none
Operating system = Linux
Default theme = matrix
gettext = enabled
Locale = en_US
Browser = Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0
Rows in GalleryAccessMap table = 51
Rows in GalleryAccessSubscriberMap table = 8300
Rows in GalleryUser table = 2
Rows in GalleryItem table = 8284
Rows in GalleryAlbumItem table = 257
Rows in GalleryCacheMap table = 0

Hi ,

I am very new in webhosting as well as these technical stuffs so please consider this and help me.

recently i got mail from hosting providers that my account is using more cpu load and usage and told me to switch to vps or any other plan ,

now i have only 300 visitors per day .

pleas tell me all possible steps or ways through which i can minimize CPU load or database load or server load .

i am not able to understand Admin option > performance option > Performance Tuning >> Acceleration

i want to minimize server load and dynamic data is not at all important for me... so which settings do i need to keep No acceleration ??? or High acceleration ???

apart from this please let me know what else i should do at Admin option > Maintenance to make less CPU load and usage.

Please reply me soon i am waiitng for some help

regards

You want High Acceleration

Then, tell users to avoid downloading static stuff over and over (effect should be tiny, but it's easy). Edit your .htaccess file in gallery2's root adding this:

ExpiresActive On
ExpiresDefault "access plus 1 week"

Now, in each of

/modules/core/data/ ; /modules/icons/iconpacks/KSIcons/ (and other packs) ; 
/modules/search/ ; /modules/rating/images/ ; /themes/matrix/images/ ; /themes/matrix/ ; 
/images/ ; /modules/comment/ 

create a .htaccess file with:

ExpiresActive On
ExpiresDefault "access plus 1 year"

Now, let's avoid wasting CPU cycles for stuff that you can live without.

Uninstall plugins that you don't use. EXIF can be your main problem.

Avoid imageframes, random picture, newest pictures, RSS, dynamic album select, sitemap, useralbums. If you have a set of known image uploaders and your visitors don't need to login, consider hiding the login/register links, as they can cause a lot of hits from dumb bots. I believe that URL rewriting can be good for you (keep in mind the perf tips), but it's a guess. Try to have all images and albums public (everyone can see)

Remove (or password protect) /install/ and /upgrade/.

If you can convince your providers to install a opcode cache (like eaccelerator), you'd be lighter on the CPU.

Last, the deal breaker. If you're lucky enough to convince your hosting to enable mod_cache and mod_mem_cache (mod_disk_cache would work too) in Apache, you'll never hear about CPU issues and your site will be blazing fast. This one should live in http://codex.gallery2.org/Gallery2:Performance_Tips and your dream config would be something like this (in your vhost or apache2.conf, not settable in .htaccess):

<IfModule mod_mem_cache.c>
        CacheEnable mem /
        MCacheSize 64000
        MCacheMaxObjectCount 1000
        MCacheMinObjectSize 1
        MCacheMaxObjectSize 6400000
</IfModule>

I can benchmark mod_mem_cache versus no cache if you want numbers, the difference is astonishing.

If you want to know whether mod_expires and mod_cache/mem_cache are enabled, create a .php file containing this:

<?php
phpinfo();
?>

, upload it and visit it.
Then look for Loaded modules. You can paste the modules list here (not all of your php info!) if you're still unsure.

HTH,
Daniel

Dave,
Great to hear there's interest, I hope I can help more than disturb

I'm trying to get more meaningful benchmarks involving representative load, trying some tools from http://www.opensourcetesting.org/performance.php . In the mean time, here are some results of using mod_disk_cache under ab.

without mod_disk_cache

ajaksu@Belkar:~/$ ab -c 5 -n 1000 http://127.0.0.1/gallery2/main.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
[...]
Server Software:        Apache/2.2.4
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /gallery2/main.php
Document Length:        11509 bytes

Concurrency Level:      5
Time taken for tests:   122.574364 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      11877000 bytes
HTML transferred:       11509000 bytes
Requests per second:    8.16 [#/sec] (mean)
Time per request:       612.872 [ms] (mean)
Time per request:       122.574 [ms] (mean, across all concurrent requests)
Transfer rate:          94.62 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   115  611 416.4    582    5989
Waiting:      108  535 313.7    516    4774
Total:        115  611 416.4    582    5989

Percentage of the requests served within a certain time (ms)
  50%    582
  66%    685
  75%    754
  80%    814
  90%    965
  95%   1139
  98%   1393
  99%   1710
 100%   5989 (longest request)

with mod_disk_cache

ajaksu@Belkar:~/$ ab -c 5 -n 1000 http://127.0.0.1/gallery2/main.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Server Software:        Apache/2.2.4
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /gallery2/main.php
Document Length:        11509 bytes

Concurrency Level:      5
Time taken for tests:   1.868203 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      11910000 bytes
HTML transferred:       11509000 bytes
Requests per second:    535.27 [#/sec] (mean)
Time per request:       9.341 [ms] (mean)
Time per request:       1.868 [ms] (mean, across all concurrent requests)
Transfer rate:          6225.23 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     1    8  95.9      1    1349
Waiting:        0    8  95.8      1    1349
Total:          1    8  95.9      1    1349

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      8
  98%     15
  99%     34
 100%   1349 (longest request)

See that 1349 (longest request) above? Cache miss

Totally unreliable as measure of real impact on Gallery2 sites (this test doesn't touch more than a single HTTP request for main.php) ... but aint that still impressive?

The bad news: mod_disk_cache is supported for Apache 2.2 only (2.0 has it as 'experimental'), it needs Server or Virtual Host config (i.e., .htaccess isn't enough) to load mod_cache and mod_disk_cache, the file cache needs to be cleaned in a semi-manual way (cronjob).

@ajaksu, this is great info. I typically do benchmarking for new G2 features the same way that you do it. http_load with a distribution of urls is good for overall benchmarking for systemic stuff, and then I typically do a breakdown over several types of pages. These tests were performed with a concurrency of 3 on a fairly fast machine with, eaccelerator, mod_disk_cache and G2 performance turned on to demonstrate high level performance. It will vary considerably on loaded machines when you start hitting memory limits, but under reasonable load if you can keep most of the important parts of your Gallery in memory the caches really perform well.

1. Static assets (css, gif, etc) to get an idea of the "speed of light" on the system under test. I typically use something like http://www.menalto.com/gallery/images/gallery.gif. Expected: median 0ms, 90%ile 2ms.

2. Simple PHP files to get the minimum time to do any PHP work. I typically use something like http://www.menalto.com/gallery/config.php for that. Expected: median 1ms, 90%ile 3ms

3. Individual thumbnail views, eg: http://www.menalto.com/gallery/d/2564-27/family.jpg Expected: median 2ms, 90%ile < 5ms

4. Individual photo view, eg: http://www.menalto.com/gallery/family/NewYork2007/IMG_6564.JPG.html Expected: median <50ms, 90%ile <80ms

5. Album views with a small window (9 images per page, this is a common setting). I typically use something like http://www.menalto.com/gallery/family/ Expected: median <50ms, 90%ile < 80ms

6. Album views with a large window (100 images per page, this is uncommon but useful to find SQL bottlenecks). I don't have a public URL for the what I use for this, but I'm sure you can find one. Expected: median <60ms, 90%ile <100ms

Some important things to note here:
a) #3 should be as close as possible to #2 so that it's the "speed of light" for a PHP response. Since we're employing an image firewall, it can't be any faster than the PHP engine itself.
b) #5 should be very close to #4. If it's not, then our album queries are not as efficient as individual lookups
c) #6 should be very close to #5, else our code for processing album entities is inefficient

For development, I typically benchmark on much slower machines so that inefficiencies are much more visible.

That is pretty interesting stuff. It's interesting to me that apachebench would be affected by mod_expires, since I would expect it to not be adding If-Modified-Since to its requests (ie: I wouldn't expect it to obey cache headers). However, I think that it makes sense to add a snippet like this to the .htaccess file. You need to guard it, though for installs that don't have mod_expires active, so it'd look like:

<IfModule mod_expires.c>
  ExpiresActive On
  ExpiresDefault "access plus 5 minutes"
</IfModule>

Would you like to take a shot at adding this information about mod_disk_cache and mod_expires to the http://codex.gallery2.org/Gallery2:Performance_Tips page? That would be very useful for all! I'll investigate getting this into the default .htaccess file for our distribution, too.

Just a note: I've got Gallery2 working with memcached via adodb's support. You'd have to download adodb to get some support files, add some configuration to adodb.inc.php and change some Execute commands in GalleryStorage.class to ExecuteCached.

Since I'm still figuring how things work, expect better instructions soon

If someone with knowledge on the many G2 caches, maps and storages wants to suggest/ask/explain something, I'll all ears

Regards,
Daniel

Article about mod_deflate settings like on Amazon EC2 AMI
http://railsgeek.com/2008/12/16/apache2-httpd-improving-performance-mod_deflate-gzip