Machine learning systems as mentors in human learning: a user study on machine bias transmission in medical training

dc.contributor.authorVicente Holgado, Lucía
dc.contributor.authorMatute, Helena
dc.contributor.authorFregosi, Caterina
dc.contributor.authorCabitza, Federico
dc.date.accessioned2025-03-14T11:55:59Z
dc.date.available2025-03-14T11:55:59Z
dc.date.issued2025-04
dc.date.updated2025-03-14T11:55:59Z
dc.description.abstractWhile accurate AI systems can enhance human performance, exerting both an augmentation and good mentoring effect, imperfect systems may act as poor mentors, transmitting biases and systematic errors to users. However, there is still limited research on the potential for AI to transmit biases to humans, an effect that could be even more pronounced for less experienced users, such as novices or trainees, making decisions supported by AI-based systems. To investigate the bias transmission effect and the potential of AI to serve as a mentor, we involved eighty-six medical students, dividing them into an AI-assisted group and a control group. We tasked them with classifying simulated tissue samples for a fictitious disease. In the first phase of the task, the AI group received diagnostic advice from a simulated AI system that made systematic errors for a specific type of case, while being accurate for all other types. The control group did not receive any assistance. In the second phase, participants in both groups classified new tissue samples, including ambiguous cases, without any support to test the residual impact of AI bias. The results showed that the AI-assisted group exhibited a higher error rate when classifying cases where the AI provided systematically erroneous advice, both in the AI-assisted and the subsequent unassisted phase, suggesting the persistence of AI-induced bias. Our study emphasizes the need for careful implementation and continuous evaluation of AI systems in education and training to mitigate potential negative impacts on trainee learning outcomes.en
dc.description.sponsorshipLV, FC and CF acknowledge funding support provided by the Italian project PRIN PNRR 2022 InXAID - Interaction with eXplainable Artificial Intelligence in (medical) Decision-making. CUP: H53D23008090001 funded by the European Union - Next Generation EU. HM acknowledges funding support from Grant PID2021-126320NB-I00 funded by MCIN/AEI/10.13039/501100011033 and ERDF A Way of Making Europeen
dc.identifier.citationVicente, L., Matute, H., Fregosi, C., & Cabitza, F. (2025). Machine learning systems as mentors in human learning: A user study on machine bias transmission in medical training. International Journal of Human Computer Studies, 198. https://doi.org/10.1016/J.IJHCS.2025.103474
dc.identifier.doi10.1016/J.IJHCS.2025.103474
dc.identifier.eissn1095-9300
dc.identifier.issn1071-5819
dc.identifier.urihttp://hdl.handle.net/20.500.14454/2538
dc.language.isoeng
dc.publisherAcademic Press
dc.rights© 2025 The Authors
dc.subject.otherBias transmission
dc.subject.otherHuman-AI collaboration
dc.subject.otherHybrid intelligence
dc.subject.otherLearning
dc.subject.otherMachine mentoring
dc.titleMachine learning systems as mentors in human learning: a user study on machine bias transmission in medical trainingen
dc.typejournal article
dcterms.accessRightsopen access
oaire.citation.titleInternational Journal of Human Computer Studies
oaire.citation.volume198
oaire.licenseConditionhttps://creativecommons.org/licenses/by/4.0/
oaire.versionVoR
Archivos
Bloque original
Mostrando 1 - 1 de 1
Cargando...
Miniatura
Nombre:
vicente_machine_2025.pdf
Tamaño:
1.4 MB
Formato:
Adobe Portable Document Format
Colecciones