Analysis and Diagnostics module

Hello,

I've written an analysis module to list things that need attention:

Now installable from the community repository:

http://codex.gallery2.org/Gallery2:Modules:mranalysis

Currently it adds two views to the Administration section:

'Broken Images' - Lists photos which have broken derivatives so these can be investigated.
'Missing Keywords' - Lists photos which have not got any keywords.

If you have any ideas for features, design or improvements on the current code, please post here and let me know - Thanks!

Currently I've only tested this on MS SQL Server 2005, but in theory it should work with other platforms - let me know either way if it works for you if you are using a different database.

Full version history and screenshots.
_________
Mark

I've corrected the case and removed the markup bar.

I didn't do anything with the derivatives other than bring up the edit page - my broken derivatives was down to a bad transfer of files to my new server - and just needed a new image to be uploaded. - There is already a plugin (customderivatives) which does a lot with derivatives on the item edit menu.

_________
Mark

Well I can get it to the right person, but the best bet is to pop into IRC ( http://codex.gallery2.org/IRC ) and see if bharat or ckdake are about.
Would be nice if it was available via downloadable plugins.

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team

Thanks for this great idea and your work! This plugin makes my work much more easier ;-)

I've found a few issues and workarounds/fixes for ver.0.0.4:

1) DefaultFileNames:

- Does not bring a result for me. By changing line 89 in 'DefaultFileNames.inc'
from:
$items = ItemList::readItemsFromQuery2( $query, array( "DefaultFileNamesView", "IsDefaultFileName" ) );
to:
$items = ItemList::readItemsFromQuery( $query, array( "DefaultFileNamesView", "IsDefaultFileName" ) );
seems to fix the error.

- Typo in DefaultFileNames.tpl
I think line 51 should be:
There are no items that have default (Camera generated) filenames.

2) MissingTitles:

- The same issue with ItemList::readItemsFromQuery2 in line 96 of MissingTitles.inc.

3) Classification:

As far as I understand for both you are using the classification to filter the results. Only items with numbers after the given prefix should be listed.
This is an really good enhancement, but seems not to work for me.

I have a entities beginning with 'Petrasch' (filename and title). These items are listed under DefaultFileNames/DefaultTitles - but they shoul not.
By now I have not found a workaround. Tried e.g. to alter manually the prefix in Classification.inc (to 'P_' and/or the search string in the SQL-request to 'P_%'. Nothing happens. The results are still the same...

Suggestions

Now your prefixes/search-strings are located in 3 files. What about storing them in an separated file? Think this would be easier to maintain. Better would be to give the user the possibility to alter these values via an option page at the admin interface. There are many different default prefixes so I would give the possibility to maintain it for personal needs.

As I am far away from giving all items individual names and titles I have a list of nearly 600 items. I think it would be a good idea to give the user the option to activate a pagination and the adjustment how many items per page should be shown.

Did you thought about a limitation of the query? I think the plugin could run into trouble if you start the query on a huge gallery with a lot of default items.

Keep on tis good work!

Cheers,
Jens

___________________________________
http://jekophoto.eu | http://jekophoto.de

WOW, that was really fast!

Thank you for the update. All works fine now, including the filtering-feature by classification.

Cheers,
Jens
___________________________________
http://jekophoto.eu | http://jekophoto.de

Thanks! I've added them and they're in v0.0.6.

They've already found some problems in my gallery that I need to fix ;-).

Does the tags module need to be installed\configured for the 'Missing Tags' view to work? - I haven't added a check for this yet.
_________
Mark

I've just installed the tags module, and it claimed all my items had tags! I've corrected the SQL so it now uses a left join rather than a product:

	    SELECT DISTINCT
	      [GalleryPhotoItem::id]
	    FROM
	      [GalleryPhotoItem]
	      LEFT JOIN [GalleryTagItemMap] on [GalleryPhotoItem::id] = [GalleryTagItemMap::itemid]
            WHERE
              [GalleryTagItemMap::itemid] is NULL

	    ORDER BY [GalleryPhotoItem::id]

I've submitted this along with the changes to enable\disable the tags checks if the tags module is not installed or not enabled.

Have also added a 'duplicate titles' test, but this is still experimental so let me know if it gets anything wrong. - have bumped the version to v0.0.7 for this.

_________
Mark

in terms of base and core - I just coppied these from another working module - can't remember which.

Having one entry in the sidebar Is something I'm undecided about - it was something I considered when doing the initial version, but eventually decided against it. Obviously if this list gets ridiculously long I'll need to reconsider this. If I do go down this route, I'll have two entries on the sidebar - one for settings, one for te tests.

I deliberately put the zero sized images in and treated them as broken as I've had oddities where even after viewing the resizes and rebuilding from the maintenance, they stayed as 0x0. I haven't manages to track down what. It may be an idea to split these up.

The settings page seems to be becoming more important now - I'll have a look at implementing that next.

I've added support for missing Google map entries for albums only in v0.0.8 if assuming that is installed and active.
_________
Mark

Thanks for the update.

I have two questions:
You have defined $query in 'DuplicateTitles.inc' twice. As far as I understand the first one is obsolete. Did I understood right?

In 'DuplicateItemChecker.inc' you are using the second if-condition for the check if the possible duplicated titles are in the same album. For testing purposes I have deleted this condition. My results are still the same. Only items within the same albums are found. Did I make a mistake or could this condition be eliminated?
I wanted to test if this test could found duplicate titles in different albums too...

Cheers,
Jens
___________________________________
http://jekophoto.eu | http://jekophoto.de

Have added a setting to 'Expand' the duplicate titles check across albums. Now in V0.0.10.

The check for duplicate items across albums is more than just changing\removing the line you found - the items come in pre-sorted by the database within album groups so removing the line you removed would only match items across albums where the last item in album A was the same as the first item in album B.

Let me know if the changes work for you.
_________
Mark

I'm interested - I'll need to modify the tpl files to use the g->text rather than just the English text. I'll let you know when I've produced the strings.raw - the original strings.raw was somewhat out of date\incorrect having being copied from the module I used as a base to start coding.

I'm not sure what needs doing after the strings.raw is produced - IIRC there's something to do with .mo and .po files, but I'll need to read up on them.
_________
Mark

From the strings.raw you can create a so called .pot file as a template with all translatable strings in it.
On this base you could create a .po (portable) file for a language - this is the human readable file for a specific language with the original and the translated strings.
The last step is to create the compiled .mo file.
For the English language you do not need to do these steps, because there is nothing to translate ;-)

Further information you can find: at http://codex.gallery2.org/index.php/Gallery2:Localization and Wikipedia

Cheers,
Jens

___________________________________
http://jekophoto.eu | http://jekophoto.de

//EDIT: you are faster - I will work tomorrow on it and will give you the files

I'm really interested to hear how the translations work when you get the compiled .mo files into the -contrib repository.

I've had a lot of problems reported with the checkout series of modules in respect of language packages not being downloaded correctly (and therefore translations remaining stubbornly unavailable). I can't find any fault with the way the language files are included in the checkout module and I have a (totally unsubstantiated) theory that the problems stem from the rewrite of the language system in G2.3. So it would be really good to get a green/red light for another recently updated module which has translations in -contrib.

Thanks!

I've put the translations in as v0.0.11 and switching language to German has correctly switched the language. Let me know if it works when downloaded from the -contrib repository and any changes I need to make to persuade it to if it doesn't.

_________
Mark

There are update cycles, but I would have thought that it would have passed through the update cycle - I can't remember when they go off but there are several a day. I've reupdated and submitted the manifest seeing it claims it has changed - so it may be I missed that.

_________
Mark

The push to the repository was run.
For Gallery Plugin-update there are no changes for me with version 2.3:
mranalysis: v 0.0.10 with the same error-message
checkout: version 0.4.3 (no folder ./locale and no .mo files in the folder .po)
checkoutemail: version 0.4 (empty folder ./locale and no .mo files in the folder ./po)

Tried to uninstall/deltete items, cleared the cache, etc. nothing brings me to a positive result.
Wanted to check with an pre-2.3 version too (api versions 7.18 / 3.4) - but without success. Seems to me that the required api-versions are to high for this test (7.4 and 3.0 for checkout / 7.41 and 3.9 for mranalysis).

There are two options I could imagine. There is something wrong with the repository or the error is sitting in front of my pc ;-)
I think the second is the true one. So it would be helpful to get test-results from somebody else.

Cheers,
Jens
____________________________________
http://jekophoto.eu | http://jekophoto.de

I've just killed mine and got the same issues with the error platform failure.

Not sure what I'm doing wrong - I follow the following steps after checking in any changes:

1. Update Manifest
php.exe lib\tools\bin\makemanifest.php -p modules/mranalysis

2. Submmit manifest
svn commit --username=mridgwel -m"update Manifest"

and somehow it has managed to do something that its unhappy about.

If I kill the manifest and start again, I get:

svn diff MANIFEST
Index: MANIFEST
===================================================================
--- MANIFEST    (revision 1793)
+++ MANIFEST    (working copy)
@@ -9,7 +9,7 @@
 modules/mranalysis/ExtensionsInTitles.inc      1244324745      1404480735 2801    2901
 modules/mranalysis/ItemList.inc        2919650555      2828949983      3551 3693
 modules/mranalysis/KeywordAnalysis.inc 4028582285      3674449520      10377 10749
-modules/mranalysis/MANIFEST    2586718966      2936534315      2809    2850
+modules/mranalysis/MANIFEST    2681771964      1298735901      68      70
 modules/mranalysis/MissingKeywords.inc 1322638131      4260730576      1971 2043
 modules/mranalysis/MissingTags.inc     762972354       1980065384      1936 2006
 modules/mranalysis/MissingTitles.inc   3350614406      1642479403      2624 2725
@@ -36,6 +36,3 @@
 modules/mranalysis/templates/MissingTitles.tpl 1245126716      551782051 1502    1560
 modules/mranalysis/templates/NoExtensionsInFileNames.tpl     629317698 4070875245      1493    1551
 modules/mranalysis/templates/Settings.tpl      1996074658      3673915720 1470    1513
-R      modules/mranalysis/AdminController.inc
-R      modules/mranalysis/BrokenImages.txt
-R      modules/mranalysis/MissingKeywords.txt

_________
Mark

Patience... I haven't (hadn't) done checkoutemail, just checkout. I'm doing the other checkout modules now.

According to the developers the old file structure (module/locale/...) is (should be) used as a fallback to copy the .mo files to the g2data/locale/... directory if the .mo files aren't found in the module/po directory. That seems to be where things were falling down.

I've updated all the checkout modules (*email, *paypal, *download etc) removing the locale directory and adding the .mo files in the po/ directory, so they should be good to go when the next repository push goes through. I've also updated with the latest translations that I can lay my hands on.

Its almost as if its decided to ignore the last update. I've spent a bit of time looking at the files and noticed that the ones which appear to get marked as changed didn't have the same svn auto-props and line ending tags as others. I've corrected those, and am waiting on the 12pm push. Could do with a big button that says 'push now!' so I can make a change and see what happens. The debug log didn't seem to give me much help when looking at the platform failure.

_________
Mark

I did have it set then I updated subversion and it seems to have stomped on my config file. Am hoping re-tagging the files with the line ending stuff and re-submitting will help.

_________
Mark

This might sound silly, but a MANIFEST file shouldn't contain a line for the MANIFEST file should it? Others don't appear to, but mine strangely insists on putting it in.
_________
Mark

Sounds strange to me - nice to read that you solved this.

Now the download and installation of version 0.0.11 works as expected for the plugin.
The language files were not downloaded. Updating the German language via the filemanager seems to change nothing for me. Are they included yet?

___________________________________
http://jekophoto.eu | http://jekophoto.de

Mark,

The de.po isn't downloading for me either, even though (as you point out) it shows as available. It's not downloading in initial install (with de enabled) or after an update of the de language in the language manager.

I've tried packaging it myself (discovered today you can do that) - and the packaging works ok except I get a php warning

I've looked at that file, but a cause doesn't jump out at me. I know not if this is significant somewhere.

Also can I suggest changed the de_DE descriptor to de to match the other de translations? I think the country code should be added by default, but just in case...

Thanks Mark - now all works great!
___________________________________
http://jekophoto.eu | http://jekophoto.de

The Add missing keywords is a feature I've not really finished - was intended to look at other keywords on an item and suggest keywords that are related from a configuration file. e.g. if you had a keyword that mentioned a specific type of bird, you could configure it so that it would add\suggest the keyword 'Bird' to items.

I tend to maintain the keywords in G2 and then extract them out of the DB to tag the original images, so have a different workflow to you.

If you can write PHP, and want to add your mechanism then feel free to update the source of the module, or if you would prefer, create your own to do this. I believe there are other modules which read information from the image e.g. the google maps module reads GPS info from an image.

_________
Mark
ACCU - www.accu.org - Professionalism in Programming.