A Question about Cache
|
jeffo
Joined: 2005-02-28
Posts: 9 |
Posted: Mon, 2005-02-28 21:07
|
|
Hi! I am new to using the Gallery. The company I am working for is looking at putting Gallery 2 in production. There will be over 70,000 pictures taking up almost 6 TB. We have everything installed and are looking at the filesystem usage. The cache folder seems to have 512MB of "stuff" in it. Looking down the different paths, and I see many many .inc files that seem to contain the same data as what is in the database. What exactly is all of this stuff? Is there any way to manage these files? |
|
Posts: 32509
- thumbnails
- caches that speed up the application (caches of permissions, caches of mappings, ...)
what would you like to manage? given the size of your gallery (wow!), 512mb don't seem to be that much. or are these 512mb excluding the thumbnails and resized versions of the images? if so, maybe it's not really 512mb, perhaps it's a lot of very tiny files and because of the cluster size of the filesystem, each tiny file allocates 4kb or something like that.
Posts: 9
I knew I was off when I first wrote this... The finger is faster than the eye, it is 600GB not 6TB! OOPS!
This is my data folder...
577708072 albums
6423720 cache
48724 locks
16 sessions
728 smarty
68 thumbnail
1858036 tmp
4 versions.dat
24 watermark
And an example of one of the many files that is under the cache folder
<?php GalleryCoreApi::relativeRequireOnce("modules/core/classes/GalleryPhotoItem.class"); $data = unserialize('O:16:"galleryphotoitem":22:{s:6:"_width";i:2175;s:7:"_height";i:1590;s:19:"_canContainChildren";i:0;s:12:"_description";s:14:"ANTICIPATION10";s:9:"_keywords";N;s:8:"_ownerId";i:5;s:8:"_summary";s:14:"ANTICIPATION10";s:6:"_title";s:14:"ANTICIPATION10";s:21:"_viewedSinceTimestamp";i:1101330653;s:14:"_pathComponent";s:18:"ANTICIPATION10.tif";s:3:"_id";i:11;s:18:"_creationTimestamp";i:1101330653;s:11:"_isLinkable";i:1;s:7:"_linkId";N;s:13:"_linkedEntity";N;s:22:"_modificationTimestamp";i:1101330654;s:13:"_serialNumber";i:1;s:11:"_entityType";s:16:"GalleryPhotoItem";s:17:"_persistentStatus";O:8:"stdClass":3:{s:5:"flags";i:0;s:8:"modified";a:0:{}s:13:"originalValue";a:0:{}}s:9:"_parentId";i:10;s:9:"_mimeType";s:10:"image/tiff";s:5:"_size";i:10383710;}'); ?>
There seems to be more than 200000 files under the entity directory... These are the ones that I am confused by.
Posts: 32509
I assume this is the result of a "du" command and that the numbers reflect the disk usage in kilobytes. i.e. you have there not 512mb cache, but 6GB cache, 1.8GB tmp, etc. And you don't even have thumbnails at the moment, lol.
Well, that's all fine. G2 manages locking in the filesystem and it caches a whole lot of entities (objects) in this filesystem cache.
What do you want to manage? Why does it bother you? Is diskspace sparse?
Of course it will be very interesting to see, how well G2 manages so many items. Note that G2 isn't "stable" as in "stable for business production environments", not yet. It's going beta within a week and some changes to speed up the application are in the making.
Posts: 9
Yes, the whole cache directory is 6GB and the entity directory is 512MB (those 200000 files). And, yes that was a du -sk * in the data directory.
We are looking for a way of linking to the database, and we may have to make a leap of adding data into the Gallery via batch. I have found that I can not update the database and have it immediately appear on the site. It seems that every record has a matching file under the entity directory. To update the data, do I need to update the database, and find its matching file and modify/delete it?
Posts: 32509
You should definitiely use the G2 API to add/alter/delete data that is maintained by G2.
However, if you know what you are doing...
browse to your gallery2/lib/tools/test/
FlushCache and FlushTemplates will empty the cache folder. After working directly on the database without using G2 API, just hitting FlushCache (and FlushTemplates) may fix the problems you experience.
I'd still encourage the use of the G2 API though.
Posts: 974
du -h is kind of nice because it makes things easier to read.
have you tried using the "add from server" functionality?
Or, have you tried g2add.pl?
Either one of these things might streamline your process.
Posts: 9
Thank you for the extra du option choice! You get used to using something, and sometimes you never go back to look at the "other" options.
I am looking to accomplish the following task...
Images will get dropped in a root folder.
A program will calculate the images appropriate album based on the image filename (from another database - Not gallery related)
This program will then need to insert the image into the gallery based on its findings.
Valiant - When you say G2 API, where can I get more info on this?
Posts: 32509
http://gallery.sourceforge.net/wiki.php?page=Development%20Documents
but the documentation isn't exhaustive.
you can use embed.php (GalleryEmbed::init) to initialize G2, then you can call any GalleryCoreApi:: method, load other modules etc.
You don't need to use GalleryEmbed, alternatively you can also start with GalleryInitFirstPass/SecondPass, see how it's done in main.php.