AUT PhD Research Demonstrates How Flawed Benchmarks Create Confidently Wrong Screening Agents
Phuong H. N. Dang is a PhD Candidate from the Department of Computer and Information Sciences and the AUT Institute of Biomedical Technologies (IBTec). He attended the 25th International Symposium on Knowledge and Systems Sciences (KSS 2026) hosted by the Auckland University of Technology (AUT) in Auckland, New Zealand, held from 13-15 November 2026. As autonomous, goal-directed AI agents reshape how knowledge is created, organised, and acted upon, KSS 2026 brings together researchers and practitioners to examine how knowledge science and systems thinking must evolve; and explore the methods, models, and societal implications of intelligent systems that no longer merely assist, but plan, coordinate, and decide. (https://kss2026.aut.ac.nz/)
Supported by AUT Computer and Information Sciences Research Centre (CISRC), Phuong presented the paper “What the Agent Inherits: Auditing Epistemic Hazards in Clinical Screening Benchmarks for Agentic AI” published as one volume of Springer's Communications in Computer and Information Science (CCIS) series No. 3221. As autonomous agents increasingly handle clinical screening decisions, their real-world reliability remains inseparably tied to the unseen quality of their training data. This study investigates whether hidden benchmark defects silently compromise AI decision-making by deploying an automated auditing agent to evaluate the widely used E-DAIC and DAIC-WOZ depression and PTSD corpora. The audit uncovered critical dataset flaws, including swapped participant labels and an excessively high correlation between depression and PTSD scales, making it mathematically impossible for models trained on this data to reliably differentiate the two distinct conditions. Furthermore, simulations revealed a dangerous illusion of progress: while standard performance metrics artificially improved, the model's actual accuracy on nuanced patient cases plummeted, and its incorrect predictions became significantly more confident. Ultimately, the research demonstrates that standard accuracy scores often mask fatal dataset flaws, proving that an AI screening tool's trustworthiness must be rigorously audited at the benchmark level before any clinical deployment.