TagFaces Module

Attached is the current revision of my "TagFaces" module. This module will allow you to assign a tag to a region on a photo, and will then make that region into a link that leads to other photo's with the same tag.

I've set up a wiki page for it here:
http://codex.gallery2.org/Gallery3:Modules:tagfaces
And committed it to my github here:
https://github.com/rWatcher/gallery3-contrib/tree/master/3.0/modules/tagfaces

This module has been tested against the version of Gallery currently in github, and probably will not work with Beta 2.

Comments/Suggestions?

Don't worry, it run's on top of the existing Tags module, so you don't have to re-tag anything

Interesting. If I'm reading the error log correctly, it's crashing when trying to determine if the current user has view/edit privileges on pages that don't belong to an actual item (album/photo/video). In which case this should probably fix the problem.

It's weird that I'm not seeing the error here though. I wonder if we're using different versions of git or something?

The reset password link doesn't work here either. I get the "Dang... Something went wrong!" message, and then it goes on to list the error as

So csrf() is not the issue. I did a gut pull and not the same error.... Let me scratch everything and start over.

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team

Cool, I'm glad it's working now

The focus of this module is mapping existing tags (created using the Tags module) to a region of a photo. You can tag whatever part of the photo you want. The module does not have any kind of AI component, and thus cannot distinguish a face from a car (or anything else).

That link looks more like it's assigning notes to a photo instead of just a tag. That's beyond the scope of what this module was designed for, although it's certainly possible to make a module to do that.

Thanks for the link, I'll take a look at it.

Cool. Is this update doing what fotonotes does or just updating your previous module?

Hi rWatcher,
First I wanted to thank you again for all your hard work you've done on creating modules for G3. I really appreciate it.

I'm interested in the more "note-like" capability because I find my users like to write little "clues" and "snippets" to photos for more information about a photo. I believe FLICKR does this, as well. As a matter of fact, I heard Flickr uses the fotonotes code.

if you plan to do a note-like feature in the future, I'd love to help you test it out and even give you a donation because it would be great for my site. Either way, thanks again for all your hard work.

Wow! I will test it out in the next couple of days. thanks!!

I currently have no plans to implement fotonotes in TagFaces.

Fotonotes definitely looks like it has some interesting potential. However the fact that it hasn't been updated in over three years makes me hesitant to invest the time to implement it, especially when there are plenty of other issues with this module (like deleting notes) that I still have to deal with. If anyone else wants to work on it, please feel free to set up a fork on github and make whatever changes you want to the TagFaces module, but it's not something that I plan on working on right now.

Hmm, this isn't working for me. I just installed the latest experimental version, and when I go to drawfaces it gives me a page without any content. The header, but no side bar and no way to indicate where the face is.

Very nice plugin! Great functionality. I haven't looked into how the information is stored or anything. But may I suggest making it "compatible" with the way Picasa3.5's face recognition and tagging functionality?
Face information are stored in a file called ".picasa.ini" (hidden on my windows computer). The file contains information like this;
A single picture with two persons:

I haven't worked out how this information should be interpreted. But I guess there is a lot of clever guys out there!

Hello, again

I've commented the functions

and put a simple string on their place (eg "TEST" ) in the file drawfaces.html.php and the tagging page loaded.
I add a zone and mark a tag and it saves fine.

However, when I return to the photo view and point to the area the tag doesn't show. On the lower corner of the browser I see that the link changes when I'm over the zone, just the text of the tag doesn't show.

EDIT:
In IE the following pops out at the bottom of the page

I guess the p::clean() function messed it up again.

Thank you. The new version works great.
I had to change all id's that were "g-photo" to "gPhoto", I guess this are the css Git changes you was talking about. When I upload the latest galley I'll overwrite the view files.

on beta 3 doesnt work

I just installed Gallery 3 and the Tag Faces plugin from Git. The basic Gallery setup if functioning well, and the Tag Faces plugin let me tag the faces, but when I go back to view the image, the marked spots have been shifted up a few pixels. Is there a fix for this problem?

Tried both Chromium (4.0.253.0) and Firefox (3.5.5) on Fedora 12. I'm using the default theme, I haven't had time to try out any of the other themes. Going back it looks like the highlight is shifted up to display the tag at the bottom.

After some tests/trials:

1. TagFaces doesn't add a real Gallery tag associating it to the face when you tag a face on a photo. As a result, if you tag a face writing its label in the 'title' field, you'll get something functionally like the 'description', which means that that face will be not click-able until you tag another one and associated to *existed* Gallery tag. That makes the module less useful IMO, since it limits the ability of browsing all photos tagged with that face too, unless you have already tags for all possible faces in your Gallery (i.e tags for all your family members) and use TagFaces just to associate the faces to those existed ones. I don't understand what is the feature that the new 'descriptions' provide, as the adding of a face without choosing an existed tag is functionally equal to this.

2. When you click on a click-able face, you'll see the dang page, as the module directs you to /tags/X instead of /tags/show/X. (maybe that has changed in Gallery just recently? anyway, I could simply fix this by replacing the two 'tags/$oneFace->tag_id' to 'tags/show/$oneFace->tag_id' in the file 'tagfaces/views/drawfaces_highlight_block.html.php')

3. I can confirm the issue jcollie told about. I'm using the latest development version with the default theme. I tried that in Safari, Firefox, Opera and Chrome, all under Mac OS X.

4. The tags system should be done in a way that TagFaces automatically added a Gallery tag then associated to the face you're tagging.

5. What about adding a mini TagFaces admin page to the above menu-bar, in a way anyone can see/edit/delete any of the face tags if he was authorized (so even visitors can access it) with the ability to browse the photos of any of these faces... I don't know if it is possible, but if it was and if you could do it then it will be really great!

6. Go ahead with this wonderful one

Not working for me with the latest checkouts of g3 and tagfaces, it says "Please select a face" after drawing the rectangle, typing in a name and clicking Save Face. If drawing is supposed to fill in coordinates in the boxes, it is not doing so.

However, this is enough for me to make some hopefully helpful but probably just annoying suggestions about the Ui. (Why can't me type a capital i in this editor without it putting in italic tags?)

I just went through face tagging experiments using Picasa desktop and Picasa Web Albums, Polar Rose and Facebook. Have not tried iPhoto. One thing that is clear after tagging many thousands of faces is that user interface is really, really important here. While you want an easy interface for the newcomer, you will have many people doing it hundreds of times which means you need the absolute minimum of keystrokes and clicks, minimal switching back and forth between keyboard and mouse etc.

As you know, in Facebook you enter face-tag mode, then click in the center of the face and type. That's a pretty good UI, especially for something that has not pre-found faces like Picasa. Picasa has found faces, so you just move the mouse near them, or click on the list of faces on the right to fill in names. In these cases, they do various fancy things to help you with autocompletion of the name, which is a huge plus if you are doing a lot of this. Gallery is able to use web browser autocomplete, which is a start, but a smart autocomplete would understand things like which names you type the most, globally or in the given album, and which names are seen the most with the other people named in the photo.

When Picasa doesn't find a face, you click on the "manual add" which is down in the bottom right (not a big fan of that, having to move the mouse far every time) and it draws a standard size box you can move and drag. Flickr as you know just lets you click and drag in the photo to tag at any time. However, they recently made it worse by forcing you to now choose between adding a note or a person, more moving and clicking. At least then you can type right there without having to move the mouse again.

It is a toss up between Flickr's older click-drag-type-ENTER interface and facebook's (enter tag mode) click-once-type-ENTER However, they both win heavily over having to go to photo options, open a menu, click tagfaces, then click and drag, then move mouse to the box to enter the name, then move box over to "save face" and click.

Not knowing the architecture of g3, I presume that it does not make it easy for you to be always on (ie. being the action for click and drag by default) or to have a facetag mode where click and drag or single click belong to you, and you can then go through the photos with next, tagging as you go?

However, the closer you can get to being able to tag by drawing/clicking, typing and then hitting enter, the more of a win it would be in tagging lots and lots of faces.

Of course, importing picasa's boxes and associations of the same face, if not the names themselves, would also be a big win. Polar Rose import is in the works I see but Polar Rose is very, very slow. Slow is even worse than lots of keystrokes and mouse moves.

In the future, I expect face taggers to get clever and if they have not gone so far as to identify the faces for you as the face-rec systems do, to at least figure out the typical size of faces, so that if you want to do "single click creates the box, and then you resize if needed" as some systems do, you get decent internal guess for the size. However, there is a lot to be said for facebook's approach of not having to have a box at all, and just a center. It serves the needs of face tagging just fine, and allows a much simpler interface with no dragging needed. You get the two things you want -- you can put your mouse on a person's face and see who it is (though you must be smart enough to move around to assure you are not just on an unknown face) and you can move to the name and have it show you in the photo who that is. Not as perfect as a box but much simpler in UI.

If one has to get to the level that flickr now has, where a box has two meanings (note or person) it should would be nice to have a way to streamline that. Examples would include a special character to type at the start of different types, like "@Person Name" for people or "/note text" for notes etc. Another, less intuitive interface would be to default to one or the other depending on how you draw the box (down and right vs. up and left etc.) but have buttons you don't need to click on to change the choice. Of course, another approach the 2nd time you use a name is to just see that it is a name because of that.

All of this is just dancing around the main goal - streamlining what you will do a lot of, repetitively. Every click, mouse move, or move of hands from keyboard to mouse and back that isn't needed should be eliminatable if it can be.

Thanks. I am new to GIT. What are the right GIT URLs for the version of G3 that works with the module, and the module itself?

In general, what's the right way to check out a GIT copy, and then check out a module into the modules directory from your contrib repository?
I like to run checkouts because that's often easier to upgrade on PHP programs than trying to dump a new tarball on top of an old one.

And I also am curious how you feel about the UI streamlining comments I was making. Not trying to demand code but just reviewing what seems to be going on out there, and what I'm looking for when a lot of tagging has to be done.

Here's the lasted TagFaces code:
- Fixed issue were selecting a region of a photo did not populate the x1y1x2y2 boxes.
- Fixed issue with tag links not working.
- Fixed access::verify_csrf() issue.
- Fixed issue were face selection boxes don't show up in the right spot.
- Fixed bug that caused the entire faces table to get deleted whenever a user tried to delete one or more tagged faces.
- Fixed a similar bug that caused the entire faces table to get deleted whenever a face-tagged photo is deleted.
- Added code for deleting notes from existing photos.
- Tested everything against Gallery 3 Git (as of commit 38f2784fbbb0661dc57627d2878cb640bbffe271).

It is attached to this post, and on my github.

Ah, I now see you are the author of the fine displaytags module, so I will presume that having it also display the facetags is probably on your todo list already. The version of displaytags I got a few days ago has a bug with the current git of g3 in that if you click on the tag it throws a Kohana_404_Exception [ Page Not Found ]. Just FYI, as I realize the codebase is in flux. Curiously it works if you click on a face associated with a tag, but throws the exception if you click on the tag as displayed by displaytags. With displaytags the tag is linked by the string of the tag, while the link from the face is done with an internal gallery3 tag number.

I also see that the interface for face tagging offers you the option of connecting the box you draw to an existing tag on the photo, so that it is both the face tag and a regular tag, which makes sense, though it means the interface then is that first you should create a tag with the name, and then you should do the face tag. For that then, I would suggest that the "Tag Face" button next to the "Add Tag" button makes a lot of sense, in that what it would do is create a tag with that name on the photo, and put you into the face tag interface with that tag already selected, and then you would just draw the box and confirm, no need to type anything. I would prefer to not have to have two modes where I must "return to photo" after making each tag as well.

Tags imported from Polar Rose would presumably also be created in this manner.

Does the Tag title have a meaning if you associate the box with a regular tag in this manner?

It is a matter of taste on click vs. drag. Drag is obviously more accurate, click is faster. Of course, one could do both. Drag draws a box, click creates a standard size box (which possibly can be resized, but that's obviously extra work.) I admire to some degree the simplicity of Facebook's approach but it does mean that it's better for mousing over the names than it is for mousing over the faces.

What I was asking though was in Gallery, can you possibly get a hook into click and drag even when the user has not clicked to enter facetag mode? Obviously once in the mode you can do what you like to the UI. But I'm not picky, both of these work if you can also navigate, add captions and other tags while in face tag mode. But that's the rub, you don't want to duplicate everything, which is I guess why I lead towards modeless if it can be done. Perhaps the ideal would be that somehow Gallery itself goes into face tag mode, so that clicking on the picture, instead of zooming to full size (which already has a button to do that) gets owned by the module whose mode is turned on, ie. tagfaces. Ditto drag, though right now drag doesn't seem to do anything in normal mode, does it?

Having a save button is fine, but to me the normal UI is that you type and don't hit enter if you want to work on other fields -- or you hit tab. If you hit enter it closes the form. That's how the major of web forms seem to operate these days and is why I suggest it as a UI.

I was a bit confused on the role of the modules here. I wasn't realizing that the existing "Add tag" button was a result of the (to my mind, not very pretty) popular tags module. An "add tag" button is a must in my view. Now that I see how it works, what makes sense to me is that you always want to add an official "tag" but sometimes you want to assign it to a rectangle (face). So in that case one UI is that you have an text box (under the existing tags) and you can type a tag or name into it, and there would be two choices -- to add a plain tag, or to add a tag with a rectangle.

The problem is how to deal with the two choices. You can have two buttons "Add Tag" and "Add Face" that say what to do with the name you just entered, and the latter one puts you in a mode where you can click or click and drag. This is the opposite of the way most systems work where you draw the box first and then put the name on it but it could make sense. The problem again is the two choices. I would probably solve that by having two boxes, one for regular and one for faces. Type a word in the regular box and hit enter -- get a plain tag. Type something in the faces box and hit enter, and perhaps that gives you a sample rectangle on the photo which you can then move or resize, or perhaps it just puts you in a mode where you can click and drag a rectangle. In this way, entering "tag face mode" is accomplished by typing in the box, so no menu item is required to enter it, and there is no real need for a mode. If you want a form that helps you tweak the tag after drawing it, that complicates matters as you now must have a way to finalize the tag. Though clicking on it might be enough. This needs more thought, but it goes towards the goal of streamlining, and few moves from keyboard to mouse and back for the ordinary tagging operation.

Flickr, as you say, has two modes of tagging, but it does it with a different UI. It actually has 2x2 types of tagging now so the UI is not as clear. On the right it has "Add a Tag" and "Add a Person" and it also allows you to click and drag without being in a special mode, and then asks you if you want a person or note. I hate that last part it adds a whole extra mouse move and click to every operation.

Here's another UI idea: Type in the names of people and things, creating tags, in a box for this. Then click on the resulting name (this requires displaytags) and drag the name onto the picture, onto the person's face. There it would draw a standarized box on the face. You could then leave it there, or grab its edges or corners and resize it. To make it easy, when the name is dragged onto the picture, it would get the box around it so you could drop not on the face exactly, but to put a corner of the box on the corner of the face. Then go pick up the other corner.

What I got confused about was I was creating face tags with no "tag" and just a "note title" and that works -- but it's not what you actually want to do.

Tagging "offline" first is one approach but for collaborative tagging projects (what I have in mind is a group of people collectively labeling their group photo collection) everybody is going to be online. In my case, I don't even know the names of the people in many of the photos, I count on others to add that.

I downloadted the tgz you pointed to, but I will try again with the git version to be sure the bug is gone.

As for Polar Rose, I thought you also wrote the polar rose module? I now see it uses Polar Rose's deprecated beta, so I guess it would need a revamp in any event. It does not seem to import the faces back from Polar Rose onto the photos as they are now doing (halfway) with flickr.

I'll think about this a bit more and see if I can some up with an even better (and easy to implement) UI.

But here's my main question: Right now if you click and drag on a photo in the normal view, the photo goes blue and you can drag a little icon away but this seems to serve no purpose. You can't drag it anywhere or do anything with it I can see. If this is true, perhaps you can be (without having entered a mode) the thing that happens on click and drag, namely drawing a box -- this is how flickr works. If not, if you need a user action to give you control of click/drag events on the photo, then the UI has to start with something, like entering tag mode, or typing a tag into a box.

I can confirm that this is still an issue under Windows / Google Chrome, although I have absolutely no idea what's causing it.

I did notice that (under Windows/Chrome) if you left-click in the black box, and then drag right into the actual photo, you'll cause a selection box to appear in the black area, with the selected region of the photo visible underneath it. You can then click in the center of the selection box to move it around, or click and drag the edges of it to resize it in order to select the proper region of the picture.