INESC TEC researchers create “synthetic faces” to address privacy and consent problems

INESC TEC researchers are creating faces of people who do not exist (“synthetic data”). These computer-generated faces look real but belong to no one, thus mitigating several of the privacy and consent-related issues during their collection. The pioneering work was published at the prestigious Computer Vision and Pattern Recognition (CVPR) 2024 conference and marks a significant advance in the field of facial recognition technology. 

The problem with real faces 

In an era of increasing technological integration, facial recognition emerges as a transformative tool, with applications spanning security, retail, healthcare and social media sectors. However, as its adoption expands, ethical considerations around data privacy and security become paramount. In addition, other challenges like demographic bias, adaptability to changing conditions such as aging, and lighting disparities continue to affect technology.  

Synthetic faces and the future of facial recognition  

To address these problems, INESC TEC researchers – who stood out as co-authors of a paper during the second edition of the Facial Recognition Challenge in the Era of Synthetic Data (FRCSyn), at CVPR 2024 – are generating synthetic data.  

Although promising, systems trained only with synthetic data continue to lag when compared to systems trained exclusively with real data. The second edition of the FRCSyn challenge aimed to explore the limits of facial recognition technology trained with synthetic data, allowing participants to use new methods of facial synthesis. The competition allowed the submission of models trained on datasets with approximately 500.000 synthetic images, or with an unlimited number of images.  

The team featuring INESC TEC researchers Pedro Carneiro Neto, Jaime Santos Cardoso and Ana Filipa Sequeira, together with FraunhoferIGD researchers, achieved the best score in two categories: reduction of bias using unrestricted synthetic data and use of unrestricted synthetic and real data. “The proposed solution focused mainly on mitigating demographic bias (ethnicity and gender); and we proposed a new outlook on how we consider ethnicities in balancing a dataset, discarding the previous approach of ethnic categorisation”, explained Pedro Carneiro Neto. “This allowed “the model to train with images that were truly useful in increasing the performance of each ethnicity”, concluded the researcher. 

The results achieved in the challenge will allow to analyse the improvements achieved with the use of synthetic data and the cutting-edge performance of current facial recognition technology in realistic operational scenarios, extracting very valuable contributions for advancement in this field. 

The researchers mentioned in this news piece are associated with INESC TEC and UP-FEUP.