Sensitivity decreased from 0.91 on a balanced refinement dataset to 0.62 on an independent dataset reflecting real-world prevalence<\/p>\n<\/h3>\n
By Elana Gotkine HealthDay Reporter<\/b>
\n<\/b><\/p>\n
THURSDAY, Jan. 15, 2026 (HealthDay News) — A fully autonomous artificial intelligence (AI) model can screen for cognitive impairment from clinical notes, according to a study published online Jan. 7 in npj Digital Medicine<\/em>.<\/p>\n Jiazi Tian, from Massachusetts General Hospital in Boston, and colleagues developed two large language model workflows for identifying cognitive concerns from clinical notes: an expert-driven workflow with iterative prompt refinement across three large language models and an autonomous agentic workflow coordinating five specialized agents for prompt optimization. The model was optimized on a balanced refinement dataset and validated on an independent dataset, which reflected real-world prevalence.<\/p>\n The researchers found that relative to the expert-driven workflow, the agentic workflow achieved comparable validation performance (F1: 0.74 versus 0.81) and superior refinement results (0.93 versus 0.87). Sensitivity decreased from 0.91 in the refinement dataset to 0.62 in the independent dataset, demonstrating the impact of a shift in prevalence on generalizability. In expert readjudication, the agentic workflow was clinically correct in seven of 16 false-negative cases (44 percent), reflecting clinically appropriate reasoning.<\/p>\n “We didn’t build a single AI model — we built a digital clinical team,” coauthor Hossein Estiri, Ph.D., also from Massachusetts General Hospital, said in a statement. “This AI system includes five specialized agents that critique each other and refine their reasoning, just like clinicians would in a case conference.”<\/p>\n