Thumbnail image maintenance module failure

the problem is corrupted referential integrity in your database or something related.
how that could happen, i do not know. it's a very bad thing and i see missing object errors here in the forums from time to time. so it would be great if we knew whether it is a bug or what is causing the problem.

how to find the item (here a derivative) that is causing the problem?
we're talking here about 13'000 derivatives i guess, and one of them is causing the problem.
it isn't necessarily item 4275 of 13373, because the progress bar makes steps of 5 not 1.

first, clean your db cache in site admin maintenance -> delete db cache.

then try again? is it fixed?
if not:

activate buffered debug mode and edit modules/core/classes/GalleryStorage/DatabaseStorage.class
directly before the return statement on line 378 add:

		    $gallery->debug($query);
		    $gallery->debug('ids: ' . implode(', ', $targetIds));

and run the build task again. in the debug output, at the end, you should see the output we're looking for.

ah i see. the additional debug output we added wasn't necessary, it already had such an output on another level.

now that you know that 8877 is the problem, take a look at the g2_entity table and search for the g_entityId 8877, could you post the row?

basically you have to search all of the involved db tables for the id 8877 and report if the id is missing or not:

SELECT g2_Entity.g_id, g2_Entity.g_creationTimestamp, g2_Entity.g_isLinkable, g2_Entity.g_linkId, g2_Entity.g_modificationTimestamp, g2_Entity.g_serialNumber, g2_Entity.g_entityType, g2_Entity.g_onLoadHandlers, g2_ChildEntity.g_parentId, g2_FileSystemEntity.g_pathComponent, g2_Item.g_canContainChildren, g2_Item.g_description, g2_Item.g_keywords, g2_Item.g_ownerId, g2_Item.g_summary, g2_Item.g_title, g2_Item.g_viewedSinceTimestamp, g2_Item.g_originationTimestamp, g2_DataItem.g_mimeType, g2_DataItem.g_size, g2_PhotoItem.g_width, g2_PhotoItem.g_height 
FROM g2_Entity, g2_ChildEntity, g2_FileSystemEntity, g2_Item, g2_DataItem, g2_PhotoItem 
WHERE g2_Entity.g_id IN (8877) AND g2_ChildEntity.g_id=g2_Entity.g_id AND g2_FileSystemEntity.g_id=g2_Entity.g_id AND g2_Item.g_id=g2_Entity.g_id AND g2_DataItem.g_id=g2_Entity.g_id AND g2_PhotoItem.g_id=g2_Entity.g_id

tables:
g2_Entity,
g2_ChildEntity,
g2_FileSystemEntity,
g2_Item,
g2_DataItem,
g2_PhotoItem

backup your database now. (e.g.
mysqldump --opt --compress --user=foo --password=bar gallery2 > g2_dump.sql
)

have you ever had a g2 error before?
what operations have you tried before in g2? link item? delete item?

see:
http://gallery.menalto.com/modules/GalleryAPI/apidoc/GalleryCore/Classes/GalleryPhotoItem_core.html

there should be an entry in g2_ChildEntity, g2_FileSystemEntity , g2_Item , g2_DataItem, with g_id = 8877
and also in g2_PhotoItem.

can you double check that there's really just an entry in the entity and in the dataItem table?

again, i can't explain how this happened.
your problem is that a specific resize/thumbnail says "photo item 8877 is my original, if you want to generate me, you need 8877".
let's try to fix this.

search the table g2_Derivative for rows with g_derivativeSourceId = 8877:
SELECT * from g2_Derivative WHERE g_derivativeSourceId = 8877;

then we know which derivatives have an non existant source / original.
the question is then how do we fix this?
the best thing would be to delete these derivatives. until now, there exists only one kind of derivatives, derivativeImages. see:
http://gallery.menalto.com/modules/GalleryAPI/apidoc/GalleryCore/Classes/GalleryDerivativeImage.html

now we have to remove the entries of these corrupt derivatives (derivatives with a non existing source id) from all db tables involved:
g2_derivativeImage
g2_derivative
g2_childEntity
g2_entity

good luck.

hmm, i don't see any problem.

on http://gallery.light-speed.de/v/damato/2005/birthday_party/
i see 9 thumbs in the album (plus the imageblocks)

and on http://gallery.light-speed.de/v/damato/ ... 2.JPG.html
i see the image.

exactly like it should be.

maybe a browser caching issue?
close this album page, then clear your browser cache.

i'd say it's a browser cache issue. browse to another page (don't have your browser showing this album page) and then clear your browser cache.

no idea.

i guess it's impossible to find out how you ended up with corrupted referential integrity in the database. so this case is closed.

Yes I also don't know. So thank you once again for your superb help!

And to prevent future corruptions: Isn't it possible in MySQL to use primary key mechanism to link all table to eachother so that the referntial integrity is always persistent? I know that from other database systems. So with this mechanism the database would always be valid and if the software tries to modify the database such that the integrity would be violated the database denies these operations?!?

And if not, then I guess it is time to write another maintenance module which tries to analyze the data integrity of the database and report the results with some short suggestions to the user. Would that be possible/reasonable?

i've created 2 tasks for these 2 ideas. i'm not sure if we will ever implement them, but they would be very nice to have.

florre, somewhere in your database there is a reference to an object with g_id / sourceId / ...Id = 4260. probably in the g2_Derivative / g2_DerivativeImage table.

you have the choice between deleting this id from all tables or trying to fix the existing entries / adding some rows if necessary to fix the data corruption.
see:
http://gallery.menalto.com/modules.php?full=1&set_albumName=miscellaneous&id=aaf&op=modload&name=gallery&file=index&include=view_photo.php

if there's an entry in g2_DerivativeImage, there should also be an entry in g2_Derivative and in some other tables and in g2_Entity. It's complicating yes, but it was never meant to be done manually.

I got something similar, for me it is not trigered by a bug, but rather by messing up tables. I used to have some troubles which needed manual editing, though I didn't do well obviously. This is the ERROR:

**
Error (ERROR_MISSING_OBJECT) : Missing object for 1657

* in modules/core/classes/GalleryStorage/DatabaseStorage.class at line 2124 (gallerystatus::error)
* in modules/core/classes/GalleryStorage/DatabaseStorage.class at line 305 (mysqldatabasestorage::_identifyentities)
* in modules/core/classes/GalleryStorage.class at line 118 (mysqldatabasestorage::loadentities)
* in modules/core/classes/helpers/GalleryEntityHelper_simple.class at line 82 (gallerystorage::loadentities)
* in modules/core/classes/GalleryCoreApi.class at line 2186 (galleryentityhelper_simple::loadentitiesbyid)
* in modules/core/classes/helpers/GalleryDerivativeHelper_advanced.class at line 673 (gallerycoreapi::loadentitiesbyid)
* in modules/core/classes/GalleryCoreApi.class at line 843 (galleryderivativehelper_advanced::rebuildcache)
* in modules/core/classes/helpers/GalleryDerivativeHelper_simple.class at line 66 (gallerycoreapi::rebuildderivativecache)
* in modules/core/classes/GalleryCoreApi.class at line 830 (galleryderivativehelper_simple::rebuildcacheifnotcurrent)
* in modules/core/classes/BuildDerivativesTask.class at line 92 (gallerycoreapi::rebuildderivativecacheifnotcurrent)
* in modules/core/AdminMaintenance.inc at line 104 (buildderivativestask::run)
* in ??? at line 0 (adminmaintenancecontroller::runtask)
* in modules/core/classes/GalleryTemplateAdapter.class at line 758
* in g2data/smarty/templates_c/%%3965159830/%%C7^C79^C790B080%%theme.tpl.php at line 75 (gallerytemplateadapter::trailer)
* in lib/smarty/Smarty.class.php at line 1251
* in modules/core/classes/GallerySmarty.class at line 90 (smarty::fetch)
* in modules/core/classes/GalleryTemplate.class at line 228 (gallerysmarty::fetch)
* in main.php at line 318 (gallerytemplate::display)
* in main.php at line 87
* in main.php at line 80

So far I figured out the error is trigered at line 2124, I checked out the code trying to find what tables are having issues, but so far i'm failing in that.

Thanks for your time.
gOochi

I'm having a similar problem with my Gallery 2.0.2 release version. Visit http://www.drewvogel.com/gallery2/ to see it for yourself.

Here's the error I'm seeing (this is different from other errors because my "(ERROR_MISSING_OBJECT)" doesn't have anything after the last colon (for example, Goochi's error says, "Error (ERROR_MISSING_OBJECT) : Missing object for 1657"):
Error (ERROR_MISSING_OBJECT) :
in modules/core/classes/helpers/GalleryItemAttributesHelper_medium.class at line 58 (MatrixTheme::error)
in modules/core/classes/GalleryCoreApi.class at line 1178 (MatrixTheme::fetchViewCounts)
in modules/core/classes/GalleryTheme.class at line 1203 (MatrixTheme::fetchItemViewCounts)
in themes/matrix/theme.inc at line 75 (MatrixTheme::loadCommonTemplateData)
in modules/core/classes/GalleryTheme.class at line 900 (MatrixTheme::showAlbumPage)
in modules/core/classes/GalleryView.class at line 285 (MatrixTheme::loadTemplate)
in main.php at line 287 (ShowItemView::doLoadTemplate)
in main.php at line 87
in main.php at line 80

The only line that looks suspicious to my (non-technical) eyes in the debug output is:

(mysql): SELECT g2_ChildEntity.g_parentId, COUNT(g2_ChildEntity.g_id) FROM g2_ChildEntity, g2_AccessSubscriberMap WHERE g2_ChildEntity.g_parentId IN (7,4831,4954,4500,19,7879,5786,6501,6546,6904,7036,7551,4383,7622,203,7745) AND g2_AccessSubscriberMap.g_itemId = g2_ChildEntity.g_id AND g2_AccessSubscriberMap.g_accessListId IN (4388,8912,7749,8911,7751,7880) GROUP BY g2_ChildEntity.g_parentId

I have NO idea how to mend this error. I've tried completely re-installing the latest Gallery (2.0.2), I've cleared the cache on my browser, my server, and the Gallery cache (via lib/support/cache.php). No joy.

I'm running Gallery 2.0.2 on a fully-up-to-date Fedora Core 4, PHP 5.04, Apache 2.0.54, MySQL 4.1.16.

Someone please help!

-drew
www.drewvogel.com

Valiant, thanks! Managed to resolve my issues with corrupted database based on the information in this topic (even figured out how to write some very basic scripts to sae time searching (PHPMYADMIN takes a long time!).

Anyhow, i think i know why my database problems occured and thought i would post to let you know!In my case the images that i was having problems with where all uploaded at the same sort of time using REMOTE. When i uploaded the files i had to stop half way (using wireless from a bus, not a clever option), and while i had time to 'cancel' the upload in the remote program, i never came back and finished uploading the files. I suspect that this loss of connection while remote was uploading the files is what has caused my problem, though i don't know how remote works so it might be totally unrelated!

Anyways, thanks again, thus far thumbnails are creating normally!

Pildiga 10780 / 10801 'st tegelemine

Eeldatult aega järgi: 0:00

Mälu kasutatud: 11490992, kokku: 64M

Error (ERROR_MISSING_OBJECT) : Missing object for 20128

* in modules/core/classes/GalleryStorage/GalleryStorageExtras.class at line 2054 (GalleryCoreApi::error)
* in modules/core/classes/GalleryStorage/GalleryStorageExtras.class at line 98 (GalleryStorageExtras::_identifyEntities)
* in modules/core/classes/GalleryStorage.class at line 313 (GalleryStorageExtras::loadEntities)
* in modules/core/classes/helpers/GalleryEntityHelper_simple.class at line 71 (GalleryStorage::loadEntities)
* in modules/core/classes/GalleryCoreApi.class at line 2361 (GalleryEntityHelper_simple::loadEntitiesById)
* in modules/core/classes/BuildDerivativesTask.class at line 133 (GalleryCoreApi::loadEntitiesById)
* in modules/core/AdminMaintenance.inc at line 128 (BuildDerivativesTask::run)
* in ??? at line 0 (AdminMaintenanceController::runTask)
* in modules/core/classes/GalleryTemplateAdapter.class at line 1092
* in g2data/smarty/templates_c/%%339145069/matrix/%%3A^3A8^3A818B59%%theme.tpl.php at line 75 (GalleryTemplateAdapter::trailer)
* in lib/smarty/Smarty.class.php at line 1255
* in modules/core/classes/GallerySmarty.class at line 61 (Smarty::fetch)
* in modules/core/classes/GalleryTemplate.class at line 241 (GallerySmarty::fetch)
* in main.php at line 506 (GalleryTemplate::display)
* in main.php at line 104
* in main.php at line 88

Gallery URL = http://www.funholiday.org/gallery2/main.php
Galerii versioon = 2.3 tuum 1.3.0
API = Core 7.54, Module 3.9, Theme 2.6, Embed 1.5
PHP versioon = 5.2.6 apache2handler
Veebiserver = Apache/2
Andmebaas = mysqli 5.0.67-log, lock.system=flock
Tööriistad = Thumbnail, Gd
Kiirendus = none, none
Operatsioonisüsteem = Linux np-43-166.netpoint.ee 2.6.26.5 #1 SMP Tue Sep 9 21:50:20 EEST 2008 x86_64
Üldine teema = matrix
gettext = enabled
Locale = et_EE
Lehitseja = Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5
Rows in GalleryAccessMap table = 16
Rows in GalleryAccessSubscriberMap table = 9296
Rows in GalleryUser table = 2
Rows in GalleryItem table = 9294
Rows in GalleryAlbumItem table = 92
Rows in GalleryCacheMap table = 0

Delete corrupt data in database from a "broken photo"
I had a similar Error (ERROR_MISSING_OBJECT) but non of the above post was a solution for me.

(ERROR_MISSING_OBJECT)in
modules/core/classes/GalleryStorage/GalleryStorageExtras.class at line 167 (GalleryCoreApi::error) 
modules/core/classes/GalleryStorage.class at line 313 (GalleryStorageExtras::loadEntities)
modules/core/classes/helpers/GalleryEntityHelper_simple.class <b>at line</b> 71 (GalleryStorage::loadEntities)
modules/core/classes/GalleryCoreApi.class at line 2361 (GalleryEntityHelper_simple::loadEntitiesById)
modules/core/classes/GalleryView.class at line 356 (GalleryCoreApi::loadEntitiesById) modules/core/ShowItem.inc at line 106 (GalleryView::getItem)
modules/core/ShowItem.inc at line 61 (ShowItemView::getItem)
modules/core/classes/GalleryView.class at line 293 (ShowItemView::loadTemplate)
main.php at line 465 (GalleryView::doLoadTemplate)
main.php at line 104
main.php at line 88

Request variables: Array  [path] => SOME_ALBUM_2009/SOME_PHOTO.JPG.html\n    [itemId] => 12345
    [view] => core.ShowItem

The effect was that one broken picture (damaged data in mysql database) was the source of the problem that resulted in the error while the page where the broken image was on crashed. This happened with the normal album view and also with dynamic album view. Like:

New Photos : /index.php?q=gallery&g2_view=dynamicalbum.UpdatesAlbum
Most viewed Photos : /index.php?q=gallery&g2_view=dynamicalbum.PopularAlbum
Random Photos : /index.php?q=gallery&g2_view=dynamicalbum.PopularAlbum

The effect was that I could view any page except the one with the broken image on it. I had a small album with 5 pages and the broke image was on page 4. I could not view page 4. All other pager where OK.

I searched more than a day in my database for the problem. I couldn't find it.
I checked all tables that valiant named and all others:

g2_Entity,
g2_ChildEntity,
g2_FileSystemEntity,
g2_Item,
g2_DataItem,
g2_PhotoItem

I checked the integrity of the record by compare the complete record from the broken g_id with a the first good g_id before the broken g_id. I didn't find any different that could cause the problem.

This was my solution that SOLVED my problem.

Step 1 Find out with Photo causes the problem.
I did that by changing the layout of the album:
- Go to the album with the problem.
- Edit Album
- Tab Theme
- change 12 (or whatever) pictures per row to 1
- and change 3 (or whatever) pictures per colum to 1
- Click Save These Setting

Now every picture has his one page. By stepping trough the pages I finally got the broken Photo. In my case is was photo SOME_PHOTO.JPG

Step 2 Find the g_id from the album
In the MYSQL databse you can find the g_id of this picture in table g2_FileSystemEntity.
Please notice that it's possible that a photo with name SOME_PHOTO.JPG is used more than once. You can use the same filename in every album again. So the correct SOME_PHOTO.JPG is the first one under a record that has the album name (album path name or path name on filesystem) in column g_pathComponent. You can find this by getting the album path name in the frond end from gallery:

- go to the album with the problem
-> Edit Album
-> tab General. The first field in the album path name (ALBUM_PATH_NAME)

Now execute this query in your MYSQL Client:

SELECT g_id FROM g2_FileSystemEntity WHERE g_pathComponent = 'ALBUM_PATH_NAME';

Suppose the album ALBUM_PATH_NAME is SOME_ALBUM_2009 than the query looks like this:

SELECT g_id FROM g2_FileSystemEntity WHERE g_pathComponent = 'SOME_ALBUM_2009';

This query gives you one line with one value. This is the g_id of the album. ALBUM_G_ID

Step 3 Find the g_id from the photo

Now execute this query:

SELECT * FROM g2_FileSystemEntity WHERE g_id >= ALBUM_G_ID AND g_pathComponent = 'SOME_PHOTO.JPG'

We suppose the found g_id is 12345 the the query look like this:

SELECT * FROM g2_FileSystemEntity WHERE g_id >= 123456 AND g_pathComponent = 'SOME_PHOTO.JPG'

Now you got the g_id of the broken picture.
Suppose the photo g_id = 12380

Step 4 Delete the broken photo
Because every method of displaying the broken image end up in a crash I could not delete the photo using the Gallery2 Front end.
To delete the image I made a index.html file on my desktop with a form to delete the broken photo.

First visit the album with the front end, login and get a cookie.
Select Delete Items from the menu <h4>IMPORTANT: NOT FROM THE SELECT-LIST UNDER THE PHOTO !!!</h4>
Open the source from that page and copy the following lines to the index.html file on your desktop.
Where:
- The domain name example.com has to be substituted by you domain name
- The value of g2_authToken has a other value than printer here.
- The value of g2_itemId has to be changed to the value of your album (ALBUM_G_ID in our example or 12345)
- The id and the name of the checkbox must be changed to the g_id from the broken photo (12380 in this example)

The HTML file looks like this:

<form action="http://www.example.com/gallery2/main.php" method="post" enctype="application/x-www-form-urlencoded" id="itemAdminForm">

<input type="hidden" name="g2_return" value="http://www.example.com/gallery2/gallery/v/zeilen/mannenweekend_juli_2009/?g2_page=5"/>
<input type="hidden" name="g2_formUrl" value="http://www.example.com/gallery2/main.php?g2_view=core.ItemAdmin&amp;g2_subView=core.ItemDelete&amp;g2_itemId=85372&amp;g2_anchorId=86470"/>
<input type="hidden" name="g2_authToken" value="6ef99825c278"/>

<input type="hidden" name="g2_controller" value="core.ItemDelete"/>
<input type="hidden" name="g2_form[formName]" value="ItemDelete"/>
<input type="hidden" name="g2_itemId" value="12345"/>

<input type="checkbox" id="cb_12380" name="g2_form[selectedIds][12380]"/>
<input type="submit" class="inputTypeSubmit" name="g2_form[action][delete]" value="Delete"/>
</form>

- Save the file on your desktop
- Open it with a browser.
- Check the checkbox
- Click on delete

Now the broken image (destructed data) is deleted.
You can upload the image again if you like. The problem in/was not in the image itself but somewhere in the database. Don't forget to change the album properties rows and columns to the correct values (you changed them in step 1).

OK, I've uninstalled and reinstalled the Dynamic Albums Plugin, have also deleted and added it back. After I reinstalled it, while the options for it appears on the Site Admin page, on the gallery itself it is not accessible: there is no link for it under the Albums drop down menu.

Looks like the database is screwed up big time huh. And I don't even know what caused it in the first place.

~fhung~

Thanks, but yes, I did. I was pointed to this post from my post here (http://gallery.menalto.com/node/90561) - and I noticed that the last post which gives the instructions for a solution in this thread is very recent Aug. 11, 2009 so I tried his solution which unfortunately didn't work for me. I am using Gallery 2.3 myself, the latest stable release.

~fhung~