Yet Another GSoC update : libface gets Face Recognition

So now that the GSoC community bonding period (the official one, I mean ;)) is coming to a close, I guess that a small report of my work so far is warranted.

To those who do not know: As part of my GSoC work, I’m committing to libface , an open-source face recognition library made by me and my mentor Alex Jironkin. libface shall be used for the upcoming Face Detection and Recognition support in digiKam.

I finally got around to committing a proper Eigenfaces implementation in libface. To the uninitiated, Eigenfaces is a subspace method for face recognition, that uses PCA for dimensionality reduction in the training set. You can learn more here.

Face detection is done using stock haar cascades that come bundled with OpenCV. An analysis of speed – if you have a pretty slow (1.6 GHz) computer and have set the accuracy to what shall be called 100% in libface (no, it will never be 100% in practice ;)), it’d take a max of 6 seconds per photo – which means less than a day for 10,000 photos, and about a week for 100,000 photos (seriously, not many people have 100,000 photos in their collection) – provided you start from scratch. I find the speed to be comparable to Picasa on my computer. I used a timer for that :)

The accuracy of detection is, IMHO, similar to Picasa. Now, there’s no concrete way to test that in Picasa, but I find Picasa to encounter the same problems with faces rotated in 3D. There’s always some faces that one detects but the other does not. Alex gives me similar feedback about iPhoto.

Note: Blog posts suck wihout an image, so here’s one :

A photo of KOffice developers, revealed by a random google search.

Next – Face Recognition. As my fellow GSoC candidate Kunal Ghosh for this project rightly pointed out, updating the trained database of faces with a new face requires retraining of the entire database, if you use Eigenfaces. We used a simple workaround for that – we store faces in multiple databases – each DB is a chunk of 50 faces. Updating one database (50 new faces in one go) takes only about 1 second (you’re supposed to say WOW there) on my 1.6 GHz laptop. So it can be agreed upon that speed is not much of an issue with training.

There are issues with recognizing faces with 3D rotation, Although this isn’t decided at the moment, I might apply GLR (Global Linear Regression) for that. This technique, as described in the paper you just downloaded (if you clicked on that link), can be applied to the problem at hand, to generate a virtual frontal face from a rotated face.

And tagging. I’ve just checked out a fresh copy of Adrian Bustany’s nepomuk-peopletag project, which has been abandoned since about two years or so. I’ll be modifying it – actually it involves removal of code, not much actual coding. I’ll be removing the nepomuk-specific code from it (no offense to the Nepomuk people), as digiKam shall talk to Nepomuk through it’s own interfacing.

I’ll be posting a video demonstrating Picasa’s and libface’s detection side-by-side soon!

PS: I test my code on posters of LOST. Can’t upload here, for issues of copyright. If any of the people above have an issue with the pic, just let me know, I’ll change it :)

23 thoughts on “Yet Another GSoC update : libface gets Face Recognition

  1. Will this work with faces of different size. Eg a fullsize portrait, a small group and one in a crowd like at above? This will be _very_ useful.
    And 100k photos is not much, my collection of about 9 years of work by now has approx 1mil, most of them people or groups of people (duplics and brackets not counted). I use Kphotoalbum (former Kimdaba) and try to fully tag every new photo or series each day, but correctly tagging each person is impossible to keep up.
    And this will be a tremendous help.

  2. Cool stuff.

    Does this have the capacity to improve it’s information about a person with each match that is confirmed by the user? Or is the intention that is learns once and from then on just applies that training.

    I’d be interested to hear also might the database hold a range of representations for a given person (e.g. at different ages).

    • Yes, the more confirmed faces a person has, the more the chance that he/she gets recognized more accurately.

      It is possible to hold a range of representations for a given person – quite easy actually, but I think I’ll do that later. Other work takes higher priority right now :)

  3. Since you speak of picasa… How about importing picasa’s face detection ini files? I have my photo albuns all peppered with these, with quite a long work already done identifying faces. You don’t need to import the db, as entering again the names for a specific ID isn’t that hard, the hard part is identifying to who does a face belong.

  4. If “libface, an open-source face recognition library” now gets face recognition, what was this library good for before that?

    • In terms of actual working code, libface is only about one or two months old. It had very minimal capabilities with respect to detection, and face recognition was sort of incomplete, i.e. not working. I had to scrap some of the old code and write a lot of new stuff. So, it is only NOW that libface can be said to have real face recognition :)

  5. Interesting would be integration with address book (or list of IM contacts – I guess it all is available through akonady). Instead of typing the name in, just select a contact. This would create a tag from contact name and use the face as contact’s avatar (if not set yet).

    • Yes, that is intended after the GSoC work. digiKam shall interface with nepomuk when it’s available, and nepomuk should talk to akonadi. Though I don’t know much about the internals of nepomuk/akonadi at this point.

  6. Pingback: digiKam Photo Manager To Get Face Recognition | Intertech

  7. Pingback: Tweets that mention Yet Another GSoC update : libface gets Face Recognition « Aditya Bhatt's Blog -- Topsy.com

  8. Pingback: Face recognition: presto anche per digiKam | Giovanni Raco

  9. amazing!

    just want to let you know that for me personaly your project ist one of the most interesting in this gsoc season and i’m eagerly awaiting a digikam integration :)

    keep up your good work!

  10. Hi folks,
    Doing my research, I just find one great book about Face recognition which inspire some future ideas by identifying potential research directions.
    The book consists of 28 chapters, each focusing on a certain aspect of the problem.
    This book will serve as a handbook for students, researchers and practitioners in the area of automatic (computer) face recognition.
    This is the link : http://www.intechopen.com/books/show/title/face_recognition
    Book is free to download of course.
    Hope you will enjoy it.
    Cheers!

  11. Hi Mr. Bhatt,

    Looks like you’re part of an interesting project!
    Perhaps you would be the right man to ask, where I should look to find a good script to make a good face-comparison tool? Any ideas?

    Looking forward to your reply,

    Eirik Larsen

  12. Hi Adithya,

    Is it possible to run this libface under visual studio ? Supposing that i want to port it on to some other platform what are the porting steps. Please clarify.

    Regards
    Girish

  13. By blood smears, remember that you are not searching
    for anything purple, in actual fact, its usually
    dark brown as it will get dried up. Remember that bed bugs are quite
    small so its going to take careful investigation (take your mattress
    out into the sunshine) to search out them if they are there.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 47 other followers

%d bloggers like this: