/var/resizes/ dir is huge!

Ok, here's my 'problem'. I've got a VPS with 5 gig of webspace. Since I have 2.37 gig of photos I have to upload to G3 you would figure that should fit easily.

But, last week, after uploading 1,2 gig of those 2.37 gig of photos, I got this email from my hosting company which said I am almost running out of space. It turns out I just have 300 mb of my 5 gig left, because the /var/thumbs/ directory from G3 is 3,6 gigabyte large... So the /var/thumbs/ directory is three times te size of the /var/albums/ directory, in which the originally sized pictures are stored.

That seems a bit strange to me and I'm not sure that's the way it is supposed to be. Is there something I can do about this, aside from moving to a 10 gig VPS???

It sounds like if thumbs is larger than albums something is going wrong in the conversion/resizing operation. Are these JPEG images? Do they have EXIF data attached? Are you using GD or ImageMagick? And get the information Dave asked for as well about the file counts.

personal comments below...

A VPS with only 5GB of space? Dang that's low for any VPS I've seen.

www.linode.com
www.slicehost.com

Resize your stuff before uploading, so your originals are smaller in filesize. You can get away with having most (if not all) 1600x1200 size images (if you even need them that large) reduced in quality to make the file size about 256KB without any issues you can see on the screen and at that, 2370 MB would be about 9,500 images.

But this is just a work around for such a limitation on space.

Out of 5GB, depending on your distribution and installed apps, you could be consuming 500MB-2GB just with the OS alone.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here

What is the compression factor in the original pictures you uploaded. Not sure of the value that is used in Gallery3 when creating the resize versions, but perhaps this is the ultimate cause of the photo size differences.

Martyn T. Griffin

Ok, so now I have installed ImageMagick using SSH (check attached screenshot of my phpinfo), but Gallery still says "ImageMagick could not be found on your system". What am I to do?

And even if this is working eventually, will I be able to regenerate the (huge) resized images Gallery has made so far and not only the new ones I would like to upload?

Just to be complete, this is my platform info:

# OS: Linux 2.6.18-028stab064.4
# Apache: Apache/2.2.8 (Ubuntu) mod_python/3.3.1 Python/2.5.2 PHP/5.2.4-2ubuntu5.6 with Suhosin-Patch mod_ssl/2.2.8 OpenSSL/0.9.8g mod_perl/2.0.3 Perl/v5.8.8
# PHP: 5.2.4-2ubuntu5.6
# MySQL: 5.0.51a-3ubuntu5.4

And my stats:

# Version: 3.0 git (pre-RC1)
# Albums: 215
# Foto's: 12522

Thanks again for all the help!

Ok nivekiam, so I have tried this twice in SSH, but it gives me no result. After I give that command all I get back is
convert:
And that's it!

Anymore suggestions, or would you like to take a look?

By the way, I just upgraded to the latest GIT of last night (gallery-gallery3-fbc5517), and now I can't seem to be able to login anymore. Not as a user, and not as an Admin. I get an Internal Server Error 500 on the page /login/auth_html.

Thanks again!

Well, I just updated to the latest GIT (gallery-gallery3-66e1e32), ran the upgrader as stated above, and unfortunately, it gives me the same result: unable to login, internal server error 500 and the 'Dang!-page'.

I have attached two of my logfiles from yesterday and today, so maybe those will be of any help. Hopefully this is something that can be fixed...

In the meantime I will ask my hosting company about ImageMagick. Maybe they can install it so I can start using that toolkit once my Gallery is working again...

Thanks!

I'm sorry for the confusion, I always update using http://github.com/gallery/gallery3/zipball/master and FTP, I just thought that's called 'the latest GIT'

But just to make things clear, I have to delete all the current Gallery-files on my server (except of course the var-dir) and then upload the latest install (gallery-gallery3-66e1e32), so that no old files exist on my server anymore.

Is that correct?

Thanks again for all the help!

Sweet! It's working again My Gallery that is. Now I can get back on that ImageMagick-issue!

Thx nivekiam!

You need to change the open_basedir setting. You need to edit your php.ini and either remove all the paths for open_basedir or add /usr/bin to the list of paths.

That's the only way Gallery will be able to see ImageMagick or any binary listed in /usr/bin

Once you do that you should be able to activate ImageMagick under Admin > Settings > Graphics
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here

Ok. I've remembered more since posting before. In your SSH sessions, type which convert and let's see if the which command is available to you. That's what Gallery uses to find the binaries. We copied that from the Kohana system.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here

It seems the which-command is available for me and gives me the following result:

which convert
/usr/bin/convert

I upgraded succesfully to the latest experimental version (gallery-gallery3-91c5a9a), so which path exactly do I put in graphics_toolkit_path in my Advanced Settings?

That /usr/bin/convert or just /usr/bin?? Or should it be more specific to my gallery install dir. something like /var/www/vhosts/mydomain.nl/subdomains/fotos/httpdocs ?

I tried /usr/bin/convert as well as /usr/bin, but in both ways, I couldn't select ImageMagick under the graphic settings.

Or should I also change that manually in the Advanced Settings, where it says graphics_toolkit from GD to IM?

Sorry for all the questions, and thanks again for helping out!

Well, now I have done exactly as stated above. Check attached screenshots gallery_advanced_settings.jpg and gallery_graphics.jpg.

But there still seems to be a problem, aside from ImageMagick still not being recognised on the graphics-page. I had hoped it would work anyway, since I read something about that in another thread (http://gallery.menalto.com/node/90052), but it doesn't. The Simple Uploader gives me an error when using ImageMagick, and switching back to GD gives me no problem at all... But then again, that's the toolkit that gave me all the big resizes...

I have attached the logfile from today as well, it says something about "The ImageMagick directory specified does not contain a required program, convert."

Ok nivekiam, typing convert on the command line, gives me a whole list of results (ie. Image Sequence Operators and Image Stack Operators), so it seems I can execute it.

The command ls -al /usr/bin/convert gives me the following:

ls -al /usr/bin/convert
-rwxr-xr-x 1 root root 7440 Jun 5 00:47 /usr/bin/convert

Hope that is helpfull!

I will PM you the link to my phpinfo, if that's allright?

Thanks!

EDIT: The link is in your inbox!

I truly couldn't believe my eyes, but it's working!!

First that test-script came up with: "Version return code is 127", which is not good apparently. Then I asked my hosting company to remove all the entries from open_basedir AND to turn off safe mode voor PHP (I found that on another forum about return value 127), and now ImageMagick is being recognised by Gallery and working like a charm!!

I have uploaded a couple of photos to a testalbum and checked that dir in the /var/albums/ folder as well as the /var/resizes/ folder, and the resized images are way smaller! Just like they're supposed to!!!!

So now I can come to the question this whole thread was actually started about: is there a way to regenerate all the thumbs and resizes with ImageMagick for the more then 12000 pictures I uploaded using GD???

Thank you!

Well, something went terribly wrong there

Since my test was succesfull (resizes of new images were a lot smaller), I started to regenerate all the existing images according to the above method. Of course I don't mind the 1 pixel difference, so I set the resize-size tot 799 instead of 800 and started regenerating the photos.

Doing that of course takes a while for 12.500 pictures, but when it finished in the morning, I quickly checked the size of the dir in SSH, and to my surprise, instead of being less then 2,2 gig, the resizes-folder now was 2.6 gig (screenshot)!

With those 400 mb I exceeded my disk quota, so within a couple of minutes SSH and FTP were automatically blocked, my server was shutdown and my sites were down Luckily I don't have that much visitors...

For now, my hosting company has given me some extra disk space temporarily to fix this, but I'm curious if one of the following three ways to do that will work:

- Apparantly, regenerating didn't the trick, but what will happen if I manually delete all the .jpgs in the resize-dir? Will Gallery also recognise that and then as well would want to renew the images? I just would like to start the resize-dir clean, if you know what I mean.

- What would be even more perfect, is if I could tell Gallery to look somewhere else for the resizes. I mean, since all of my full images are the same size as the resizes (800x600) it would be great if I could just somewhere tell Gallery to use the albums-folder instead of the resizes-folder for both the resizes and full photos. Can I change that somewhere in the template or is that locked down in the core??

- Eventually, if both the above options don't work, I can always set the resize-size to an even smaller one. I noticed the 799 px resized-images are being resized again in the browser to a width of 696px, so I could always set 799 to 696, to see if the resizes will get a lot smaller then.

Hopefully this all makes a little bit sense...

Ok, so what I have tried now was actually the second option. Since none of my full images exceeds the resolution of 800x600, for me there really wasn't any need to have a resize-folder at all.

So in /modules/gallery/models/item.php I changed the location of where the resized image should be found from /var/resizes/ to /var/albums/:

And that works just great! The resized images are being grabbed from the albums-folder instead of the resizes-folder, so I tried deleting one particular resized image to see if everything is still working properly, and it is!

So if this isn't going to give my Gallery any trouble, I just want to skip the resize-dir and delete all of the huge-sized-resizes. Do you guys think it's safe to do that?

Yeah, well, I would like to know that as well. Because allthough the problem is fixed for my resizes, it still exists for the albums-dir.

Though I am using ImageMagick right now, photos still are being stored way bigger then they originally are.

For example, an image that is 131 kb on my harddrive, is 256 kb in my albums-dir. Almost twice the size it should be.

I switched back to GD to test what would happen, but the same image got even bigger in size using that toolkit.

I really don't understand why my Gallery is unable to simply just store the full (800x600px) images, instead of generating way bigger ones.

If there's anything I can do for test purposes to figure this thing out, I am more then willing to help!

Will test on my 3 hosts as well.

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team

Well when I upload that image, everything gets smaller in filesize as expected.

/var/albums/pande/pande_image.jpg = 109322 bytes (109 KB)
/var/resizes/pande/pande_image.jpg = 31183 bytes (31 KB)

We'll see what Dave's findings are but, I'm not expecting much different than mine. I think this is going to take a Linux geek to review what libraries are installed and used to process jpeg files. I just did aptitude search jpeg on my Debian 5.0 (Lenny) system and got this:

Don't know what the v, p or A mean, but I know that i means installed so I have libjpeg-progs and libjpeg62 installed
http://packages.debian.org/lenny/libjpeg-progs
http://packages.debian.org/lenny/libjpeg62

I think you may have some library on your system that's way out of date or something. But that's just a guess, and I still think you should consider a different VPS:
http://gallery.menalto.com/node/90641#comment-321550
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here

Dave's so much more thorough than me

I should have mentioned my resized version is 480 (created an image 480x360)

I didn't test GD.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here