A master’s thesis and a scientific paper, both award winners, focus on one of the main concerns when it comes to research in Artificial Intelligence (AI): machine learning models’ bias. Inês Martins and Eduarda Caldeira are studying solutions for more inclusive, transparent, fair and ethical AI systems.
“Evaluating the Impact of Pulse Oximetry Bias in Machine Learning under Counterfactual Thinking” is the title of the paper by Inês Martins, who won the “Best Student Paper” at MICCAI 2025; this work focuses on evaluating biased medical devices, like pulse oximeters, which affect the performance of machine learning (ML) models used in healthcare. How? The researcher compared an “ideal world” (unbiased SaO₂) versus a “real world” (biased SpO₂) through a healthcare dataset used to train ML models on tasks like hospital mortality prediction and organ failure. As a conclusion, pulse oximeters tend to overestimate oxygen levels in people with darker skin, due to limitations in the way they absorb light.
“Our work reinforced results from previous studies that suggested that biased pulse oximeter readings can lead to adverse outcomes and inequalities in clinical diagnoses. It also shows how these can exacerbate the existing disparities in more vulnerable populations, if used in ML models without due care,” explained Inês Martins.
The goal is to develop new approaches, like algorithms that correct or consider these biases. These approaches are easily adaptable to other devices, such as thermometers, and can be used to create fairer and more transparent AI systems. “The tool developed could be valuable in promoting fairer ML solutions in healthcare, as our counterfactual approach increases transparency and explainability of performance degradation across subgroups of patients,” she added.
Eduarda Caldeira seeks to reduce racial bias in facial recognition systems through an innovative approach to knowledge distillation. The master’s thesis “Adapting Biased Teachers for Fair Knowledge Distillation in Face Recognition” won the Award for Best Master’s Thesis, awarded annually during the RecPad event organised by the Portuguese Association for Pattern Recognition. The focus of the thesis was the creation of a fairer facial recognition model, which has a balanced performance between different ethnic groups, reducing racial bias without compromising overall accuracy.
“It is very common for these models to have an excellent level of performance in general, but when we look in detail, it is easy to see that their performance is significantly lower in ethnic minorities. And, in an ideal world, we want the model to work equally well for everyone,” said the researcher.
There are already several works developed in this area that, to a certain extent, can contribute to mitigate discrepancies, namely racial discrepancies. What makes Eduarda’s approach innovative is the fact that it uses knowledge transfer: “in this strategy, one or more ‘teacher’ models are responsible for guiding the final model, the ‘student’, during its learning, directing it in a desired direction. In this case, I used four different ‘teachers’, each trained on images of individuals of a specific ethnicity. This ensures that ‘teacher’ models learn the ethnicity concerned with confidence, and can assist the ‘student’ in their area of expertise, during their learning process.” This way, the work introduces a unique way of combining multiple specialised models to improve impartiality and reduce racial inequalities in real facial recognition applications.
Both Eduarda and Inês did their master’s theses at INESC TEC and believe that the recognitions received can pave the way in their areas of research.
News, current topics, curiosities and so much more about INESC TEC and its community!
