Organizer


Joseph P Robinson
Northeastern University, USA

Ming Shao
University of Massachusetts Dartmouth

Yun Fu
Northeastern University, USA

Updates

  • FIW to organize premiere Kaggle Competition [kaggle]
  • Tutorial slides available [    slides,    reduced slides]
  • Overview

    Automatic kinship recognition has relevance in an abundance of applications. For starters, aiding forensic investigations, as kinship is a powerful cue that could narrow the search space (e.g., knowledge that the Boston Bombers were brothers could have helped identify the suspects sooner). In short, there are many beneficiaries that could result from such technologies: whether the consumer (e.g., automatic photo library management), scholar (e.g., historic lineage & genealogical studies), data analyzer (e.g., social-media-based analysis), investigator (e.g., cases of missing children and human trafficking– for instance, it is unlikely that a missing child found online would be in any database, however, more than likely a family member would be), or even refugees. Besides application-based problems, and as already hinted, kinship is a powerful cue that could serve as a face attribute capable of greatly reducing the search space in more general face-recognition problems. With our FIW database, we can pose this relatively new and challenging problem at sizes much greater than ever before (e.g., kinship verification with 644,000 face pairs, opposed to just 2,000 and family classification with 1,000 families opposed to just 101. In the end, we also hope FIW serves as a rich resource to further bridge the semantic gap of facial recognition based problems to the broader human-computer interaction incentive. A fair question to ask– if so useful, why is kinship recognition capable not found or even a prototype for any real-world problem? Even with great efforts have been spent to advance automatic kinship recognition technologies dating back to 2010. We found the reasoning to be 2- fold:
    1. Existing image datasets for kinship recognition tasks are not large enough to capture and reflect the true data distributions of the families of the world.
    2. Kin-based relationships in the visual domain are less discriminant than other, more conventional problems (e.g., facial recognition or object classification), as there exists many hidden factors that affect the facial appearances among different family members
    Both points were addressed with the introduction of our FIW database, with data distributions to properly represent real world scenarios available at scales much larger than ever before. We expect the larger, more complex data will pave way to modern day data driven methods (i.e., deep learning) to be used in these problems much more effectively than before possible. In this tutorial, we will introduce the background information, progress leading us up to this point, several current state-of-the-art algorithms on the various views of the kinship recognition problem (e.g., verification, classification, tri-subject). We then will cover our FIW image collection, along with the challenges it has been used in, the winners with their deep learning approaches. This tutorial will conclude with a list of future research directions.

    Program

    Time Content Presenter
    8:30-9:00 Opening Joseph Robinson & Ming Shao
    9:00-10:30 Introduction and Background
    Various Views and Methods of Kinship Recognition
    Ming Shao
    10:30-11:00 Coffee Break
    11:00-12:30
    Our Families In the Wild (FIW) Dataset and Annual Challenges
    Future Work and Conclusions
    Joseph Robinson

    Reference

    [R-1] Joseph P. Robinson, Ming Shao, Yue Wu, Hongfu Liu, Timothy Gillis, and Yun Fu. Visual Kinship Recognition of Families in the Wild, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Special Issue: Computational Face, 2018.
    [R-2] Chao Xia, Siyu Xia, Yuan Zhou, Le Zhang L, Ming Shao. Graph Based Family Relationship Recognition from a Single Image, In: Geng X., Kang BH. (eds) PRICAI: Trends in Artificial Intelligence. Lecture Notes in Computer Science, vol 11012. Springer, Cham, 2018.
    [R-3] Yue Wu, Zhengming Ding, Hongfu Liu, Joseph P Robinson, and Yun Fu. Kinship Classification through Latent Adaptive Subspace, in IEEE Automatic Face and Gesture Recognition, 2018.
    [R-4] Shuyang Wang, Ding Zhengming, and Yun Fu. Cross-generation kinship verification with sparse discriminative metric, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018.
    [R-5] Joseph P. Robinson, Ming Shao, Handong Zhao, Yue Wu, Timothy Gillis, and Yun Fu. Recognizing Families In the Wild (RFIW): Data Challenge Workshop in Conjunction with ACM MM 2017, Proceedings of the 2017 Workshop on Recognizing Families In the Wild, 2017.
    [R-6] Shuyang Wang*, Joseph P. Robinson*, and Yun Fu. Kinship Verification on Families In The Wild with Marginalized Denoising Metric Learning, Automatic Face and Gesture Recognition (FG), 2017. (*equal contributions)
    [R-7] Junkang Zhang, Siyu Xia, Ming Shao, and Yun Fu, Family Photo Recognition via Multiple Instance Learning, ACM International Conference on Multimedia Retrieval, 2017.
    [R-8] Ming Shao, Siyu Xia, and Yun Fu. Identity and Kinship Relations in Group Pictures, Human-Centered Social Media Analytics, pages 191–206, Springer, 2014.
    [R-9] Ming Shao, Siyu Xia, and Yun Fu. Identity and Kinship Relations in Group Pictures, Human-Centered Social Media Analytics, pages 191–206, Springer, 2014.
    [R-10] Siyu Xia*, Ming Shao*, Jiebo Luo, and Yun Fu. Understanding Kin Relationships in a Photo, IEEE Transactions on Multimedia, 2012. (*equal contributions)
    [R-11] Siyu Xia*, Ming Shao*, and Yun Fu. Kinship Verification through Transfer Learning, International Joint Conference on Artificial Intelligence (IJCAI), 2011. (equal contribution)
    [R-12] Ming Shao, Siyu Xia, and Yun Fu. Genealogical Face Recognition based on UB KinFace Databases, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop on Biometrics, 65–70, 2011.
    [R-13] Siyu Xia*, Ming Shao*, and Yun Fu. Kinship Verification through Transfer Learning, International Joint Conference on Artificial Intelligence (IJCAI), 2011. (equal contribution)
    [R-14] Ming Shao, Siyu Xia, and Yun Fu. Genealogical Face Recognition based on UB KinFace Databases, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop on Biometrics, 65–70, 2011.

    Challenges

    [R-1] 3rd Recognizing Families In the Wild in conjunction with IEEE Conference on Automatic Face and Gesture Recognition, 2019. https://web.northeastern.edu/smilelab/RFIW2019/
    [R-2] 2nd Recognizing Families In the Wild in conjunction with IEEE Conference on Automatic Face and Gesture Recognition, 2018. https://web.northeastern.edu/smilelab/RFIW2018/
    [R-3] 1st Recognizing Families In the Wild in conjunction with ACM Multimedia, 2017. https://web.northeastern.edu/smilelab/RFIW2017/