Families In the Wild (FIW) Dataset

Photos of 1,000+ families with individual and relationship labels
Joseph P Robinson, Ming Shao, Yun Fu

News

Note that this project page is a work in progress. All bug reports will be added to the list, as we are actively improving the website. Thank you for your support and patience.

FIW Dataset

FIW is the largest and most comprehensive database available for kinship recognition. FIW is made up of 11,932 natural family photos of 1,000 families-- nearly 10x more than the next-to-largest, Family-101 [7] database. Also, we have 656,954 image pairs split between the 11 relationship (see Figure 1), which is much larger than the 2nd to largest Kin-Wild II with 2,000 pairs for only 4 kinship types.

For more information see Visual Kinship Recognition of Families in the Wild in PAMI (2018).

Download

If FIW is used or found useful please cite related. Please fill out registration and agree to terms for download link https://forms.gle/w1FBVX9YWheHs6VB7 

FIW README can be found on Github https://github.com/visionjo/fiw 

Terms of Use

By downloading the image data you agree to the following terms:
  1. You will use the data only for non-commercial research and educational purposes.
  2. You will NOT distribute the above images.
  3. Northeastern University makes no representations or warranties regarding the data, including but not limited to warranty of non-infringement or fitness for a particular purpose.
  4. You accept full responsibility for your use of the data and shall defend and indemnify Northeastern University, including its employees, officers and agents, against any and all claims arising from your use of the data, including but not limited to your use of any copies of copyrighted images that you may create from the data.

Please email questions, comments, bugs, ideas to Joseph Robinson.

Kinship Tasks

I. Kinship Verification

Elephant at sunset
Fig. 1 Samples of 11 pair types of FIW. Each unique pair is randomly selected from a set of diverse families to show variation in ethnicity, while four faces of each individual depict age variations.

The goal of kinship verification is to determine whether a pair of faces are blood relatives or not. This task has seen lots of attention, which mainly focus on parent-child pair-wise types-- father-daughter (F-D), father-son (F-S), mother-daughter (M-D), mother-son (M-S)-- though some have also focused on siblings pairs-- brother-brother (B-B), sister-sister (S-S). This data supports all prior pair-wise types in larger sets than previously offered to the research community. In addition, we introduce grandparent-grandchild pairs-- grandfather-granddaughter (GF-GD), grandfather-grandson (GF-GS), grandmother-granddaughter (GM-GD), grandmother-grandson (GM-GS). Figure 1 depicts each sample type.


Survey on the Analysis and Modeling of Visual Kinship: A Decade in the Making

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.

Kinship recognition is a challenging problem with many practical applications. With much progress and milestones having been reached after ten years - we are now able to survey the research and create new milestones. We review the public resources and data challenges that enabled and inspired many to hone-in on the views of automatic kinship recognition in the visual domain. The different tasks are described in technical terms and syntax consistent across the problem domain and the practical value of each discussed and measured. State-of-the-art methods for visual kinship recognition problems, whether to discriminate between or generate from, are examined. As part of such, we review systems proposed as part of a recent data challenge held in conjunction with the 2020 IEEE Conference on Automatic Face and Gesture Recognition. We establish a stronghold for the state of progress for the different problems in a consistent manner. This survey will serve as the central resource for the work of the next decade to build upon. For the tenth anniversary, the demo code is provided for the various kin-based tasks. Detecting relatives with visual recognition and classifying the relationship is an area with high potential for impact in research and practice.
PDF

Families In Wild Multimedia: A Multimodal Database for Recognizing Kinship

IEEE Transactions on Multimedia (TMM), 2021.

Kinship, a soft biometric detectable in media, is fundamental for a myriad of use-cases. Despite the difficulty of detecting kinship, annual data challenges using still-images have consistently improved performances and attracted new researchers. Now, systems reach performance levels unforeseeable a decade ago, closing in on performances acceptable to deploy in practice. Like other biometric tasks, we expect systems can receive help from other modalities. We hypothesize that adding modalities to FIW, which has only still-images, will improve performance. Thus, to narrow the gap between research and reality and enhance the power of kinship recognition systems, we extend FIW with multimedia (MM) data (i.e., video, audio, and text captions). Specifically, we introduce the first publicly available multitask MM kinship dataset. To build FIW MM, we developed machinery to automatically collect, annotate, and prepare the data, requiring minimal human input and no financial cost. The proposed MM corpus allows the problem statements to be more realistic template-based protocols. We show significant improvements in all benchmarks with the added modalities. The results highlight edge cases to inspire future research with different areas of improvement. FIW MM supplies the data needed to increase the potential of automated systems to detect kinship in MM. It also allows experts from diverse fields to collaborate in novel ways.
PDF

Publications

If you found our data and resources useful please cite our works.

2021

Joseph P. Robinson, Zaid Khan, Yu Yin, Ming Shao, and Yun Fu
The 5th Recognizing Families in the Wild Data Challenge: Predicting Kinship from Faces  
IEEE Automatic Face and Gesture Recognition, 2021
Joseph P. Robinson, Ming Shao, and Yun Fu
Visual Kinship Recognition: A Decade in the Making  
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021 CoRR arXiv:2006.16033
Joseph P. Robinson, Zaid Khan, Yu Yin, Ming Shao, and Yun Fu
Families In Wild Multimedia: A Multimodal Database for Recognizing Kinship  
IEEE Transactions on Multimedia (TMM), 2021

2020

Joseph P. Robinson, Yu Yin, Zaid Khan, Ming Shao, Siyu Xia, Michael Stopa, Samson Timoner, Matthew A. Turk, Rama Chellappa, and Yun Fu
Recognizing Families In the Wild (RFIW): The 4th Edition  
IEEE International Conference on Automatic Face & Gesture Recognition

2019

Pengyu Gao, Siyu Xia, Joseph Robinson, Junkang Zhang, Chao Xia, Ming Shao, and Yun Fu
What Will Your Child Look Like? DNA-Net: Age and Gender Aware Kin Face Synthesizer  
CoRR arXiv:1911.07014

2018

Joseph P. Robinson, Ming Shao, Yue Wu, Hongfu Liu, Timothy Gillis, Yun Fu
Visual Kinship Recognition of Families in the Wild  
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018
Yue Wu, Zhengming Ding, Hongfu Liu, Joseph P Robinson, Yun Fu
Kinship Classification through Latent Adaptive Subspace  
IEEE Automatic Face and Gesture Recognition, 2018

2017

Joseph P. Robinson, Ming Shao, Handong Zhao, Yue Wu, Timothy Gillis, Yun Fu
Recognizing Families In the Wild (RFIW): Data Challenge Workshop in Conjunction with ACM MM 2017
Proceedings of the Workshop on Recognizing Families In the Wild, 2017

Shuyang Wang, Joseph P. Robinson, Yun Fu
Kinship Verification on Families In The Wild with Marginalized Denoising Metric Learning
12th IEEE Conference on Automatic Face and Gesture Recognition, 2017

2016

Joseph P. Robinson, Ming Shao, Yue Wu, Yun Fu
Families in the Wild (FIW): Large-scale Kinship Image Database and Benchmarks
ACM on Multimedia Conference, 2016
Picture
Randomly selected families of FIW and a subset of their family photos.

Related Datasets

Below we list other kinship recognition datasets. A more detailed comparison of the datasets can be found in the paper.
  • KinFaceW: 2,000 face pairs of 4 different relationship types (i.e., parent-child)
  • TSKinFace: Tri-subject pairs (i.e., mother+father-child)
  • Family101: Image collection with 101 families (i.e., tree-like labels)
  • UB Kin dataset: Image collection of face pairs for kinship recognition.

License

Please notice that this dataset is made available for academic research purpose only. All the images are collected from the Internet, and the copyright belongs to the original owners. If any of the images belongs to you and you would like it removed, please kindly inform us, we will remove it from our dataset immediately.

Contact

For questions and result submission, please contact Joseph Robinson at robinson.jo@northeastern.edu