[digiKam GSoC] Face Tags progress

The GSoC deadline is coming soon, so here’s what I’ve done so far:

Face detection is now almost fully integrated into digiKam, you can scan photographs in all of your albums by clicking the “Scan for faces” button in the new People Sidebar on the left.

Scanning your albums for people

Scanning your albums for people

The scanner uses the same progress dialog as the already-existing image fingerprint generator for the fuzzy searches uses. It has resume support, of course.

You can configure your detection and recognition accuracy :

Tune speed, accuracy, etc.

Tune speed, accuracy, etc.

The new People Sidebar has a rescan button and a filtered tag tree view that only shows tags below the “People” tag. So yes, we’re using normal digiKam tags to organize people, and not some separate marking.

By default, images with detected faces go into the “Unknown” tag. Once you tag all people in the an image, the “Unknown” tag is unassigned for it.

In a recent change, People Tagging now supports grouping people into categories. You could, for example, make a new sub tag of “People” named “Family”, and drag-move all family members names to it. Later, whenever a family member is tagged, the tag below “Family” will be assigned to him/her, instead of creating a new tag below the “People” tag. No, you can’t have two people with the same name (as of now).

People Sidebar

People Sidebar

Clicking on a person name in the tag tree will show you all photos that have that person in them. We also have a normal tag sidebar on the right (not shown here), using which you can select any two or more persons to see photos with all of these in them. Of course, this is standard tag behavior.

And here’s the region tagging in the image preview:

Face region tagging

Face region tagging

Most of the region tagging widget’s code is lifted straight from the nepomuk-peopletag widget and the rectangle-region tagging class is now exported by libkface. It isn’t awesome, has a few glitches, but it works. Will be improved later.

Dismiss a face (or non-face ;-) ) by clicking on the red cross, and approve a name by pressing the tick mark, upon which the tag is immediately applied. If a face is not detected (happens), you can add your own face region tag.

The iconview-based tagging is to be done later this week . There were some deep changes required in digiKam model-view code which my co-mentor Marcel is working on. Once these changes are done, it should be easy to have faces as icons in the iconview, with a text entry widget below each face thumbnail, etc. The face recognition code is already there in the FaceIface interface I’ve written in digiKam, but will be used only with the iconview.

I probably could have done things faster, but real life gets in the way. My new semester started half a month ago, so my work pace isn’t exactly what I’d like. I’ll blog about work as soon as I finish the recognition.

Also, we’re releasing digiKam 2.0 on Christmas, this December. The current release is 1.3.0. The plan is to release 1.4.0 and probably 1.5.0 as bugfix/incremental releases. There will be a 2.0 beta about a month before it’s release. 2.0 will have all the 3 huge GSoC projects merged in. So that gives me a lot of time to fix things, improve/change the recognition algorithm, etc.

Anyway,

I’m going to : :D

Gilles is organizing the event in his home city, Aix en Provence, near the famous city of Marseille in France, so kudos to him.

I’ll be discussing parts of my project like usability issues, fixing my crappy GUI code, and will do some bugfixes. And finishing off possibly unfinished parts of my project.

Looking forward to meeting with the great team, and a big thank-you to Claudia and the e.V for sponsoring everything!

36 thoughts on “[digiKam GSoC] Face Tags progress

  1. Very nice work (if it works as advertised :)

    Regarding your second screen shot, please don’t educate users in the user interface. The long texts that explain the options make the interface cluttered, are never needed again once you know what the options do, and should be in Tool Tips and/or “What’s This For” help.

  2. Please, don’t provide that configuration interface.
    The two sidebars are impossible to understand, really…
    Perhaps could you show an example, colours or just find something that would make it possible to understand it.
    Perhaps a configuration wizard ?
    But please, do not release these two sidebars in a stable release, people would say again that KDE has bad UI…

      • Yes, I meant slidebars, sorry, I’m tired (it’s 00:50 AM here…)
        I don’t know how exactly you plan to make it understandable with just markings and/or numbers.
        “Recognize 10% of the faces, forget about it if your friends have skin disease” vs “Recognize 100% of the faces, but forget about it if you have more than two friends in the database” ? :)

        P.S : I’m joking, don’t take that seriously, and I hope I’m not offending anyone (I myself had a skin disease a few years ago)

      • Haha :)

        I mean, there is a certain threshold value – if the candidate matches (faces) are too “far apart” in appearance, more than the threshold value, it won’t show them in the suggestions (is this guy XXX?). The slider allows you to adjust that threshold.

        Low threshold = lots of suggested matches, both good and false.
        High threshold = only a few suggested matches, usually good.

        I should have been more clear :)

    • There is a widget in Krita that is a slider and spinbox at the same time (allows you to click in the bar, shows the number, and has arrows at the same time). Perhaps you could use that here.

    • Why not use the hierarchical tags? That is the best way for that. But you could always place all person tags under the “persons” tag or have them in the root.

  3. It would be nice to be able to change the Accuracy vs Faster slider while the recognition is running. You don’t know how long it’s going to take until you run it, and by then you’re too committed.

  4. I forgot to mention that this looks really sweet!

    Also, one idea I have (in case you ever find yourself with too much free time :) is that Facebook could be used as a seed for the faces database.

  5. That looks awesome! I just have one question: I’ve tagged my pictures in digikam quite a while ago already with the names of people in the picture. This was, of course, before face recognition.

    Will digikam use the already available tags and consider them in the recognition process? E.g. when it is not sure whether it’s Peter’s face and normally wouldn’t suggest anything, but since there is already a Peter tag for the pic, it will accurately guess it is indeed Peter.

    And I hope it will reuse the old tags and assign them to the faces so I don’t have to delete the old name tags.

    • “And I hope it will reuse the old tags and assign them to the faces so I don’t have to delete the old name tags.”

      You do not need to remove the old one first. I hope that the facetagging works like normal tagging that you can transfer tags to other by first searching by a tag, apply a new tag (facetag) and then remove the old (non-facetag) if not otherwise possible to set it use specific tag.

  6. Great work. Really great work. Can’t wait till Christmas. And I’ve a question (or more?;-):
    – Are the people tags normal tags or are they connected (or can be) to the Akonadi/Nepomuk persons?

    • Currently they are exported as normal tags, but I want to make them into Pimo persons.

      Maybe after gsoc, when I get some time.

  7. Hello,
    Thanks a lot for your great work
    Perhaps you could find some very interesting informations on this site
    http://www.anvo-it.de/wiki/avpicfacexmptagger:main

    “This program was written because of the new Google Picasa 3.5 face recognition feature. Due to the fact that Picasa 3.5 doesn’t store the face recognition data inside JPEG pictures, this program fetches that data and stores it as XMP-MP meta tags inside the JPEG pictures. ”

    an interesting part … :
    “There is already a method available how to store face recognition data inside JPEG pictures. This method uses XMP tag system created by Adobe with the addition of Microsoft XMP namespace Microsoft Photo Region Schema and is used by Windows Live Photo Gallery. “

    • Yes, we’re going to use metadata, but we’re not sure if the Microsoft spec is the best thing to choose. Probably it isn’t if we want a tweak in it. It’s better to use digiKam’s namespace, gives us maximum freedom. We’ll probably support importing face tags from the Microsoft namespace, but I can’t say for sure.

      This is something to be discussed in the sprint.

      • There is already a standard XMP namespace for that :
        check on this document (search for it on google) — Guidelines for Handling Image Metadata, Version 2 —-
        why not use it ? It is always better to inforce standards.
        regards

  8. I would be very interested to test this against my photos. I have a stable running backup system so no data would be lost even if it screws up the entire system bad.

    Any howto to start with to compile my own digikam with your face detection lib integrated?

    Regards,

    liquidat

  9. Hi,

    I just installed 1.5.0 via svn alongside 1.4.0, but there is no people sidebar on the left.
    When will it be available via svn again?

    Thanks,

    Chris

    • People tags and the other two GSoC projects shall arrive in digiKam 2.0, which will be released on Christmas.

      1.5.0 will probably be a bug-fix release, just like 1.4.0.
      The GSoC work is going on in a separate branch, not trunk,

  10. I only want to thank you for your effort to bring face recognition technology to DigiKam, it’s something I really was waiting for and, most important, it will make me abandon the ugly crappy-ported Linux version of Picasa :D

    Thanks again :D

  11. Thank you!

    I tried face recognition in Picasa (via wine), but it is somewhat clunky and has no people grouping option. I am planning to migrate to ditch Picasa completely in favor of Digikam, which is native Linux app with open design and if your work gets integrated also with more features.

    Awesome work!

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 46 other followers

%d bloggers like this: