digiKam GSoC progress: libkface is almost ready

Ah well, today marks a month since the official GSoC coding period started.

Brief intro: I’m working on implementing automatic tagging of faces for digiKam, with face detection and recognition. For that, I’ve been working on a library named libface that does the detection and training/recognition.

At some point, it was deemed necessary to make a Qt wrapper for libface, so that Qt/KDE programs could easily use it. This has been now (almost) done, and the resulting library is named libkface. libkface will be put somewhere in kdegraphics/libs as soon as the API freeze is over (I think), and will be usable not only by digiKam, but also by other programs like KPhotoAlbum.

We’re also going to keep the lib(k)face database in a shared directory, so that all programs that may use it can share the same training data. That way, a user migrating from digiKam to KPhotoAlbum (hypothetical, with face recog) would not have to retrain libkface with his entire collection.

In branches/extragear/graphics/digikam/libkface/test/gui/, I have made (with help from Alex) a simple GUI for playing with libkface:

Two-Face Guy

Two-Face Guy

It has a few buttons for the usual operations. Mind you, this is NOT going to be the way digiKam will present it’s interface to you, so don’t be afraid :D. With my last commit, I have added support in libface for “tunable accuracy” for face detection. Look at the slider above the photo – you can set the accuracy of detection on a Five-point scale. 1 gives you very low accuracy, but insanely high speed. Setting the accuracy to 5 gives you very high accuracy, but very low speed.

From what I’ve observed :

Accuracy of 1 : ~0.8 seconds.

Accuracy of 2 : ~2.1 seconds

Accuracy of 3 : ~4 seconds

Accuracy of 4 : ~7 seconds

Accuracy of 5 : ~13-15 seconds

The default accuracy is kept as 3.

This is a slight improvement from when I blogged about it last time. A nice use-case for this is – if you have a weak/slow computer, you can set the accuracy to 2. 1 is waaay too less, only for waaay slow computers, but in that case you probably shouldn’t be using face detection anyway. My laptop isn’t exactly powerful, and 3 works best. For those of you with newer laptops, you can go for 4. 5 is for those who have obscenely large amounts of processing power.

In the screenshot above, the red frame and the nice little black rectangle you can see is the embryo of the tagging widget I’m making. You can edit the name and all. In the next few days, I’ll be adding Tick/Cross buttons near the text, and making the face rectangle resizable.

In all, this week was quite productive, and I must also thank Marcel (patches, code review), Gilles (Porting libkface from qmake to CMake), and Alex (CMake porting and some code) for their help.

40 thoughts on “digiKam GSoC progress: libkface is almost ready

  1. Pingback: Tweets that mention digiKam GSoC progress: libkface is almost ready « Aditya Bhatt's Blog -- Topsy.com

  2. Hi!

    Great work!

    Is the recognition multi-threaded to make use of Multi-Core-CPUs?

    If not, will digikam be able to process more than one image of the same time?

  3. What about when most users eventually have a fast computer? The slider will be irrelevant. When 5 detects faces almost instantly for most people then why even have a slider? I know this is not the case today, but I wonder if this could be placed in a config gui somewhere and the default setting be 3 and one day we could move the value to 4 or 5 in future releases of KDE?

    • Well then, in the future, when computers do get faster, I can always upgrade the performance requirements of each scale division :)
      Of course, we’ll have a slider somewhere in digiKam’s settings.

  4. Awesome! This is one feature that could save me a tonne of time when tagging family photos — especially group photos.

  5. Great Amazing feature. When will it arrive in Digikam or KDE when it arrives it will be time for me to say goodbye to picasa. I only run picasa 3.6 in wine for the face detection.

  6. Awesome! cant wait for this to be integrated into digiKam. BTW, do you find the min/max/close buttons more useful on the right or on the left?

  7. Pingback: Digikam et Libkface | Ktoekoms

  8. Thanks so much for working on this. Do you have any examples of how to compile this on Ubuntu as a standalone program to play with generating tags of peoples faces?

    I’ve looked at the sourceforge page and checked out the latest SVN revision, but it’s complaining about not being able to find the OpenCVConfig.cmake file.

    At this time I’m using Ubuntu 10.04 (Lucid) with their packages opencv libs. Which looks to be:

    libcv4 2.0.0-3ubuntu2 lucid

    Basically, I’m just looking for a why to trawl my photos and tag them so that gqview (geeqie now I think…) can use those tags.

    Thanks for all your hard work!
    John

    • You need to install OpenCV 2.1 (lucid gives you an older version). Get it here : https://launchpad.net/~gijzelaar/+archive/opencv2
      (Install opencv after that, not libcv).

      Well, it doesn’t “tag” them at the moment, just gives you the coordinates of faces. Although it shouldn’t be a big deal to write some code that will write said coordinates to the metadata.
      The actual tagging will be done in digiKam.

      Thanks

  9. Great news! It seems quite a good work!
    Are you planning to integrate this face database with Nepomuk?
    It would be great if I could ask nepomuk to display to me every file related to my friend Joe, and get in the list also the photos where there is Joe :-)

    • I don’t exactly know how to work with nepomuk right now, but digiKam already synchronizes with nepomuk, I don’t know to what extent. If that’s not enough, I’ll work on that after the project.

      Thanks

    • AFAIK the digikam tags can be synced with nepomuk. In this case the thing you mentioned is working exactly as you described it (out of the box :) ).

      You detected the face of your friend joe and tag it approprietly (friend/joe). Nepomuk will then have the tag associated with the jpeg/picture file and when you ask it for all files with the tag joe you’ll get your picture ;)

  10. Hi,
    great work can’t wait for it. Currently using Picasa and avpicfacexmptagger (http://www.anvo-it.de/wiki/doku.php?id=avpicfacexmptagger:main), on windows to do the job.

    are you planning to store the faces information in the metadata? like avpicfacexmptagger does for Picasa faces, or like microsoft live gallery does?
    I believe that would be a “must have” so that you can send / migrate your photos anywhere without worrying for the database.
    It would be great to store this info in what already exist too.
    Regards
    Eric

  11. The current path to the tool is /branches/extragear/graphics/libkface/test/gui/

    Looking forward to test it :)

  12. Hi.
    Overjoyed at the news.
    Some time ago, I conducted research with a disappointing conclusion that now there is no software able to work with medium and large number of photos. I’ve tried a lot of managers, despite the fact that I most use DigiKam, and found significant deficiencies, which are not visible in small collections, but which become quite unusable in my collection 100GB, about 100k files (this is just my photos for the 6-7 years, no foto from my friends or collegues)

    I want to say about people.
    In the first place – it should not be just a tag! (OK, it could be a tag, but not only tag) This should be a separate entity “person”, which further can be linked to a person in my email/im program (such as evolution, gmail or exchange).
    And secondly “people” should be able to group. (Yes, I tried to observe people in Picaso, after first two-three hundreds a linear list of “people” is completly unusable!)

    And the last one – sometimes i need to find all albums with some person. Not single photos, but albums.

    Please note my opinion, let digikam will be the first program that can work with a lot of photos.

  13. Hi, I’ve installed the libkface on a test box, but I cannot understand how to use it… :(

    Platform Kubuntu 10.10
    branches/extragear/graphics/libkface/

    • Just make the api docs for libkface as you would with any other KDE library. It is well-documented with doxygen. The example program is just for testing a few things here and there, not really important for users.

      • As far I understand there is not a implementation ready to use out of the box. I’m not a coder, so I think I will wait until it will be integrated in the trunk… :)
        thanks anyway for your great work!

  14. Hi,

    I already try the face recognition in digikam it’s sounds very great. I was thinking about training (I’m studying image processing too).

    I know that from the graph API of facebook you can have a lot of information for a picture, like the picture and people tagged in it with the center of the tagged face.
    With all these information, i guess it’s relatively easy to compute everything digikam needs to train.

    It could be useful isn’t it?

  15. I am sure this post has touched all the internet people, its really really fastidious piece of writing on building up new website.
    mason ripple sole Shoes – zoey ripple future life – rippln reviews

  16. This kind of game gives a real experience of building
    a farm and planting trees. That is particularly true if they
    begin to beat such games. This whole experience has made the social media giant to be mindful of downtime issues of its valued social app
    developers.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 44 other followers

%d bloggers like this: