Multiple 'Server Add'

griffinmt
griffinmt's picture

Joined: 2009-09-06
Posts: 128
Posted: Tue, 2009-09-29 14:49

I was just wondering whether there are any inherent issues with starting a Server Add task running from more than one browser at the same time. This would be for updating/adding discrete albums for each.
I suppose this would end up being logically identical to several stations doing a Web upload, each pointing to one or more folders on there own system all at the same time.

Martyn T. Griffin
 
nivekiam
nivekiam's picture

Joined: 2002-12-10
Posts: 16503
Posted: Tue, 2009-09-29 17:43

Probably be fine as long as you kept things separate, import from different paths into different albums.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here
 
CaCtus491

Joined: 2003-10-18
Posts: 19
Posted: Tue, 2009-11-10 21:51

I'd be a little careful.

I have a large number of photos in multiple directories that I'm trying to upload using server add.
I found uploading to be slow (~10 seconds per photo), and noticed that a single core was being used on the server so I attempted to upload a different folder to a different album (as nivekiam suggests above) from a different PC.
Whilst it appears to work at first glance, more often than not ~2-3 photos out of ~50 wouldn't be added to the album. Going back and adding them again worked fine (after working out which were missing!), so it wasn't due to corrupt images.
I haven't seen this issue with only a single server add running at a time. This is using G3 B3.

Cheers,
-Kent
 
nivekiam
nivekiam's picture

Joined: 2002-12-10
Posts: 16503
Posted: Tue, 2009-11-10 22:47
Quote:
I found uploading to be slow (~10 seconds per photo)

I've been able to add much faster than that. I didn't time it but at least 1, maybe 2 per second. I'd double check right now, but serveradd doesn't appear to work with the current code.

http://sourceforge.net/apps/trac/gallery/ticket/875

You don't mention how large your photos are or the specs on the hardware.

Adding photos from multiple workstations at the same time should only be impacted if running on low-end hardware, a VPS where you don't have a lot of resouces or an oversold shared host that's already performing poorly. Since G3 hasn't been around long and is still in Beta we haven't seen really large sites using it, but G2 is used on some very large hosts where many 1000s of people are connected at any given point adding and viewing photos
www.care2.com (a heavily modified, custom version of G2, but it's G2 at the core of it for their photo hosting)
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here
 
CaCtus491

Joined: 2003-10-18
Posts: 19
Posted: Wed, 2009-11-11 05:42

I'll wait until that ticket is fixed, update, and try again, paying more attention to the timing.
The machine is a dedicated core2 duo running debian, with about 70k+ photos (only a dozen or so users though). It was definitely adding photos much slower than it should have, and from memory it was the database which seemed to be taking up most of the time.

Anyways, I'll report back when #875 is fixed.

Edit: Photos are in the order of ~3mb each
 
CaCtus491

Joined: 2003-10-18
Posts: 19
Posted: Mon, 2009-11-30 10:36

So, after updating to the latest nightly build a few days ago, it appears that server add is still running very slowly. I confirmed that is it is ~7 per photo (at ~3mb per photo).

Running 'top' on the server indicates that the database appears to be the culprit (rather than Apache or image conversion).