Medical LLM Error Detection
Clinical safety review — a model response approving a contraindicated Metformin dosage for a patient with severe kidney disease is flagged as a Medical Error, with expert justification detailing the correct clinical guidance.
Clinical AI Safety
Medical LLM Error Detection & Clinical Accuracy Evaluation
Expert clinical review identifying dangerous medication and dosage errors in AI-generated medical advice, with detailed correct-practice justifications.
The Challenge
Medical LLMs can produce fluent, confident-sounding advice that is nonetheless clinically dangerous — approving a contraindicated drug dosage, missing a critical interaction, or making a dosing calculation error. Catching these failures requires careful clinical reasoning, not just a fluency or coherence check, and reviewers must reason through patient-specific factors (renal function, comorbidities, lab values) exactly as a physician would before signing off on a response as safe.
Our Solution
Reviewers cross-referenced clinical guidelines to evaluate patient-scenario prompts and model responses, classifying each as Correct Medical Advice or Medical Error. Every flagged error required a detailed written justification citing the specific clinical guideline violated (e.g. eGFR-based contraindication thresholds) along with the correct recommended course of action, ensuring the dataset teaches not just what was wrong but why.
The Deliverables
Binary safety classification (correct vs. error) for every model response · Structured error categories covering contraindications, dosing errors, and drug interactions · Expert-written correct-practice justifications formatted for supervised fine-tuning and reward-model training to make medical LLMs safer before deployment.