AI diagnostic accuracy varies dramatically depending on the application domain and methodology, ranging from as low as 28-37% for general clinical reasoning tasks to as high as 98% for specialized clinical frameworks. Image-based diagnosis demonstrates notably higher performance, with deep learning models achieving 84-94% accuracy for brain tumor and pneumonia detection, while text-based clinical reasoning using large language models shows substantially lower accuracy ranges of 58-65% for emergency medical advising. A critical pattern emerges showing that specialized, fine-tuned systems designed for constrained clinical tasks (such as CLIN-LLM achieving 98% diagnostic accuracy) dramatically outperform general-purpose models, suggesting that accuracy is highly dependent on task specificity and the incorporation of safety constraints rather than model size alone. Comparatively, AI systems are approaching or matching human physician performance on certain benchmarks—with some models achieving 60-65% accuracy comparable to emergency medicine doctors—though significant gaps remain in complex diagnostic reasoning and generalization across diverse disease presentations. The wide variation in reported metrics across papers indicates that medical AI accuracy cannot be characterized as a single value but rather exists along a spectrum determined by domain (imaging versus clinical text), task complexity, dataset characteristics, and the presence of specialized training or safety mechanisms.
A Super-Learner with Large Language Models for Medical Emergency Advising
Authors: Sergey K. Aityan, Abdolreza Mosaddegh, Rolando Herrero, Haitham Tayyar, Jiang Han, Vikram Sawant, Qi Chen, Rishabh Jain, Aruna Senthamaraikannan, Stephen Wood, Manuel Mersini, Rita Lazzaro, Mario Balzaneli, Nicola Iacovazzo, Ciro Gargiulo Isacco arXiv ID:2511.08614v1 Year: 2025
Key Findings
This paper provides comprehensive accuracy data for AI-based medical diagnosis in emergency medicine. The key finding is that individual LLMs achieve 58-65% diagnostic accuracy, substantially outperforming human physicians (18-43% in literature, 41% in this study). The super-learner ensemble approach achieves 70% accuracy with basic majority voting, with potential to reach 85% with improved meta-learning. The study used 420 real emergency cases for evaluation and compared results against 12 emergency medicine physicians on a subset of 10 cases. The data clearly demonstrates that current AI systems significantly exceed human diagnostic accuracy in emergency medicine settings, with ensemble approaches offering further improvements.
Extracted Data
Data Point
Value
Individual LLM Diagnostic Accuracy Range
The range of diagnostic accuracy achieved by individual Large Language Models (Gemini, Llama, Grok, GPT, and Claude) when diagnosing acute diseases in medical emergencies. This shows how well current state-of-the-art AI models perform on their own.
58% to 65%
Gemini Diagnostic Accuracy
The diagnostic accuracy of Google's Gemini 2.5 Pro LLM for emergency medical diagnosis, tested on 420 real emergency cases.
58%
Llama Diagnostic Accuracy
The diagnostic accuracy of Meta's Llama Maverick 4 LLM for emergency medical diagnosis, tested on 420 real emergency cases.
59%
Grok Diagnostic Accuracy
The diagnostic accuracy of xAI's Grok 4 LLM for emergency medical diagnosis, tested on 420 real emergency cases.
60%
GPT Diagnostic Accuracy
The diagnostic accuracy of OpenAI's GPT-4o LLM for emergency medical diagnosis, tested on 420 real emergency cases.
65%
Claude Diagnostic Accuracy
The diagnostic accuracy of Anthropic's Claude Opus 4 LLM for emergency medical diagnosis, tested on 420 real emergency cases.
65%
Super-Learner Diagnostic Accuracy (Majority Vote)
The diagnostic accuracy achieved by the MEDAS super-learner system that combines responses from five LLMs using a basic majority vote meta-learner. This shows how combining multiple AI models improves performance.
70%
Super-Learner Best Case Accuracy
The percentage of cases where at least one of the five integrated LLMs in the super-learner produced the correct diagnosis. This represents the theoretical maximum accuracy achievable with a perfect meta-learner that could always select the correct LLM response.
85%
General Medicine Doctors Diagnostic Accuracy (Literature - Tetsuka et al.)
The reported diagnostic accuracy for general medicine doctors in emergency settings, based on published research by Tetsuka et al. (2020). This serves as a human baseline for comparison.
18%
General Medicine Doctors Diagnostic Accuracy (Literature - King & Nori)
The reported diagnostic accuracy for general medicine doctors, based on published research by King & Nori (2025). This serves as a human baseline for comparison.
20%
General Medicine Doctors Diagnostic Accuracy (Literature - Newman-Toker et al.)
The reported diagnostic accuracy for general medicine doctors without accounting for delayed diagnoses, based on published research by Newman-Toker et al. (2022). This serves as a human baseline for comparison.
43%
Emergency Medicine Doctors Diagnostic Accuracy (Study)
The average diagnostic accuracy measured in this study from 12 experienced emergency medicine physicians diagnosing 10 randomly selected acute internal emergency cases. This represents the human expert baseline.
41%
Emergency Medicine Doctors Accuracy Range (Study)
The range of diagnostic accuracy observed among 12 emergency medicine physicians in this study, showing the variability in human expert performance.
20% to 50%
Emergency Medicine Doctors Standard Deviation (Study)
The standard deviation of diagnostic accuracy among the 12 emergency medicine physicians tested in this study, indicating the variability in human expert performance.
9%
Test Dataset Size
The number of real acute internal emergency cases used to evaluate the performance of the LLMs and super-learner system.
420 cases
Human Evaluation Dataset Size
The number of emergency cases randomly selected from the main dataset to test human physicians' diagnostic accuracy for comparison purposes.
10 cases
Number of Human Physicians Evaluated
The number of experienced emergency medicine physicians who participated in the diagnostic accuracy comparison study.
12 physicians
Misdiagnosis Rate for Life-Threatening Diseases
The reported misdiagnosis rate for critical conditions such as stroke, myocardial infarction, aortic aneurysm/dissection, venous thromboembolism, spinal cord compression/injury, and spinal abscess in emergency medicine settings, according to Newman-Toker et al. (2022).
Up to 56%
Doctor-R1: Mastering Clinical Inquiry with Experiential Agentic Reinforcement Learning
Doctor-R1: Mastering Clinical Inquiry with Experiential Agentic Reinforcement Learning
Authors: Yunghwei Lai, Kaiming Liu, Ziyue Wang, Weizhi Ma, Yang Liu arXiv ID:2510.04284v1 Year: 2025
Key Findings
This paper presents Doctor-R1, an AI system specifically designed for medical diagnosis through interactive patient consultation. The research directly addresses the question of AI diagnostic accuracy by evaluating the system on multiple benchmarks. Key findings: (1) Doctor-R1 achieves 37.84% accuracy on HealthBench and 60% on MAQuE for diagnostic tasks, (2) it outperforms GPT-4.1 and other leading AI models in diagnostic accuracy, (3) the 8B parameter model exceeds much larger 70B parameter specialized medical models, demonstrating efficiency, (4) accuracy improves substantially (61% relative gain) through multi-turn strategic questioning rather than single interactions, and (5) the system maintains strong performance (83.5%) on medical knowledge tests (MedQA). The paper emphasizes that effective medical AI requires both diagnostic accuracy and consultation skills, with Doctor-R1 trained using reinforcement learning to optimize both capabilities simultaneously. Human evaluators consistently preferred Doctor-R1's clinical dialogues across all quality metrics.
Extracted Data
Data Point
Value
Doctor-R1 Diagnostic Accuracy on HealthBench
The accuracy score achieved by the Doctor-R1 AI system when making medical diagnoses on the HealthBench benchmark. This measures how often the AI correctly identified the medical condition compared to the ground truth diagnosis.
37.84% average accuracy score
Doctor-R1 Diagnostic Accuracy on MAQuE
The accuracy score achieved by the Doctor-R1 AI system on the MAQuE (Medical Agent Questioning Evaluation) benchmark, which tests the AI's ability to reach correct diagnoses through multi-turn patient questioning.
60.00% accuracy
GPT-4.1 Diagnostic Accuracy on HealthBench
The diagnostic accuracy of OpenAI's GPT-4.1 model on HealthBench, used as a comparison baseline to show how Doctor-R1 performs relative to leading proprietary AI models.
34.78% accuracy
GPT-4.1 Diagnostic Accuracy on MAQuE
The diagnostic accuracy of OpenAI's GPT-4.1 model on MAQuE benchmark, representing state-of-the-art performance from a major commercial AI system.
60.00% accuracy
Base Model (Qwen3-8B) Diagnostic Accuracy on HealthBench
The accuracy of the foundational language model (Qwen3-8B) before Doctor-R1's specialized training was applied. This shows the performance improvement gained from the training framework.
28.57% accuracy
Doctor-R1 Performance on MedQA
Doctor-R1's score on the MedQA benchmark, which consists of multiple-choice questions from medical licensing examinations. This tests core medical knowledge retention after specialized training.
83.50% on MedQA (medical licensing exam questions)
Comparison to Leading Open-Source Models
Doctor-R1's accuracy improvement compared to the best-performing open-source medical AI model of similar size (UltraMedical-8B), demonstrating its superiority in diagnostic capability.
Doctor-R1 achieved 37.84% vs UltraMedical-8B's 25.50% on HealthBench (+9.91 point improvement)
Performance vs Larger Models
Comparison showing that Doctor-R1's 8-billion parameter model outperforms much larger specialized medical models, demonstrating parameter efficiency in achieving diagnostic accuracy.
Doctor-R1 (8B parameters) scored 36.29 vs Baichuan-M2 (32B parameters) scored 33.16 on HealthBench
Improvement from Multi-turn Dialogue
The increase in diagnostic accuracy as the AI conducts longer conversations with patients, showing how strategic questioning improves diagnosis over single-turn interactions.
61.1% relative improvement from turn-1 (36.0% accuracy) to turn-5 (58.0% accuracy)
Human Evaluation Preference
The percentage of times human evaluators preferred Doctor-R1's clinical dialogues over competing models across multiple quality dimensions (coherence, adherence, clarity, empathy).
Doctor-R1 ranked first across all four qualitative evaluation categories in pairwise human comparisons
Explainable Deep Learning in Medical Imaging: Brain Tumor and Pneumonia …
This paper provides comprehensive accuracy metrics for AI-based medical diagnosis using two state-of-the-art deep learning models (DenseNet121 and ResNet50) across two important clinical tasks: brain tumor detection and pneumonia detection. DenseNet121 consistently outperformed ResNet50 in both tasks, achieving 94.3% vs 92.5% accuracy for brain tumors and 89.1% vs 84.4% for pneumonia. The paper goes beyond simple accuracy by providing multiple performance metrics (AUC, F1-score, Average Precision) and crucially includes explainability analysis using Grad-CAM. The explainability findings reveal that DenseNet121 not only performed better numerically but also focused on clinically relevant regions, while ResNet50 sometimes attended to non-pathological areas. This suggests that high accuracy alone is insufficient for clinical deployment - the AI must also be making decisions for the right reasons. The study used substantial datasets (7,023 brain MRI images and 5,863 chest X-rays) from publicly available Kaggle sources, making the findings reproducible and credible.
Extracted Data
Data Point
Value
DenseNet121 Accuracy - Brain Tumor Detection
The accuracy rate achieved by the DenseNet121 deep learning model when classifying brain tumors in MRI scans. This measures how often the AI correctly identified whether a brain tumor was present or absent.
94.3%
ResNet50 Accuracy - Brain Tumor Detection
The accuracy rate achieved by the ResNet50 deep learning model when classifying brain tumors in MRI scans. This measures how often the AI correctly identified whether a brain tumor was present or absent.
92.5%
DenseNet121 Accuracy - Pneumonia Detection
The accuracy rate achieved by the DenseNet121 deep learning model when detecting pneumonia in chest X-ray images. This measures how often the AI correctly identified whether pneumonia was present or lungs were healthy.
89.1%
ResNet50 Accuracy - Pneumonia Detection
The accuracy rate achieved by the ResNet50 deep learning model when detecting pneumonia in chest X-ray images. This measures how often the AI correctly identified whether pneumonia was present or lungs were healthy.
84.4%
DenseNet121 AUC - Brain Tumor Detection
The Area Under the Receiver Operating Curve for DenseNet121 in brain tumor detection. AUC measures the model's ability to distinguish between tumor and no-tumor cases across all classification thresholds, where 1.0 is perfect and 0.5 is random chance.
0.99
ResNet50 AUC - Brain Tumor Detection
The Area Under the Receiver Operating Curve for ResNet50 in brain tumor detection. This metric evaluates how well the model can discriminate between patients with and without brain tumors.
0.95
DenseNet121 AUC - Pneumonia Detection
The Area Under the Receiver Operating Curve for DenseNet121 in pneumonia detection from chest X-rays. This indicates the model's discriminative ability between pneumonia and normal lungs.
0.98
ResNet50 AUC - Pneumonia Detection
The Area Under the Receiver Operating Curve for ResNet50 in pneumonia detection from chest X-rays.
0.95
DenseNet121 F1-Score - Brain Tumor Detection
The F1-score for DenseNet121 in brain tumor classification. This is the harmonic mean of precision and recall, providing a balanced measure of the model's performance that accounts for both false positives and false negatives.
0.91
ResNet50 F1-Score - Brain Tumor Detection
The F1-score for ResNet50 in brain tumor classification, balancing precision and recall.
0.87
DenseNet121 F1-Score - Pneumonia Detection
The F1-score for DenseNet121 in pneumonia detection, representing the balance between correctly identifying pneumonia cases and avoiding false alarms.
0.93
ResNet50 F1-Score - Pneumonia Detection
The F1-score for ResNet50 in pneumonia detection from chest X-rays.
0.89
DenseNet121 Average Precision - Brain Tumor
The average precision score for DenseNet121 in brain tumor detection. This metric represents the area under the precision-recall curve and is particularly useful for imbalanced datasets.
0.88
ResNet50 Average Precision - Brain Tumor
The average precision score for ResNet50 in brain tumor detection.
0.82
DenseNet121 Average Precision - Pneumonia
The average precision score for DenseNet121 in pneumonia detection from chest X-rays.
0.88
ResNet50 Average Precision - Pneumonia
The average precision score for ResNet50 in pneumonia detection from chest X-rays.
0.82
Brain MRI Dataset Size
The total number of brain MRI images used to train and test the AI models for brain tumor detection.
7,023 images
Chest X-ray Dataset Size
The total number of chest X-ray images used to train and test the AI models for pneumonia detection.
5,863 images
Standardization of Psychiatric Diagnoses -- Role of Fine-tuned LLM Consortium …
Standardization of Psychiatric Diagnoses -- Role of Fine-tuned LLM Consortium and OpenAI-gpt-oss Reasoning LLM Enabled Decision Support System
Authors: Eranga Bandara, Ross Gore, Atmaram Yarlagadda, Anita H. Clayton, Preston Samuel, Christopher K. Rhea, Sachin Shetty arXiv ID:2510.25588v1 Year: 2025
Key Findings
This paper demonstrates that fine-tuned LLMs combined with a reasoning model can achieve accurate psychiatric diagnosis from conversational data. The system shows high accuracy across multiple mental disorders (Major Depressive Disorder, Bipolar Disorder, PTSD, Panic Disorder, Schizophrenia, and Generalized Anxiety Disorder), with all models producing correct DSM-5 diagnostic codes after fine-tuning. The training was efficient (under 30 minutes) and used memory-efficient techniques (QLoRA) making it accessible for deployment. The key innovation is the consortium approach where multiple fine-tuned models make predictions that are then synthesized by a reasoning LLM, improving robustness and clinical alignment. The system was developed in collaboration with the U.S. Army Medical Research Team, indicating real-world clinical validation. While specific quantitative accuracy metrics (like percentage accuracy) are not provided, the qualitative results demonstrate substantially improved diagnostic performance compared to baseline models, with consistent alignment to DSM-5 criteria across multiple psychiatric conditions.
Extracted Data
Data Point
Value
AI System Type
The type of AI system used for psychiatric diagnosis. This paper describes a consortium of fine-tuned large language models (LLMs) combined with a reasoning LLM to make diagnostic predictions.
Fine-Tuned LLM Consortium with OpenAI-gpt-oss Reasoning LLM
Medical Domain
The specific area of medicine where the AI diagnostic system is being applied. This focuses on mental health and psychiatric conditions.
Psychiatric Diagnosis (Mental Health)
Fine-Tuned Models Used
The specific large language models that were fine-tuned on psychiatric conversation data to make diagnostic predictions.
Llama-3, Mistral, Qwen2
Training Dataset Size
The number of annotated records used to fine-tune the LLMs, consisting of psychiatrist-patient conversations with diagnostic reasoning and final diagnoses.
Approximately 2,000 annotated records
Training Time
The time required to complete the fine-tuning process for the LLMs on the psychiatric dataset.
1,627 seconds (approximately 27.12 minutes)
Peak Memory Utilization During Training
The amount of GPU memory used during the fine-tuning process, indicating computational requirements.
5.853 GB (99.03% of peak allocation)
Fine-Tuned Llama-3 Diagnostic Performance - Major Depressive Disorder
The ability of the fine-tuned Llama-3 model to correctly identify Major Depressive Disorder (DSM-5 296.21) from patient-psychiatrist conversations. After fine-tuning, the model produced accurate, concise diagnoses with correct DSM-5 codes.
Accurate diagnosis with correct DSM-5 code (296.21)
Fine-Tuned Llama-3 Diagnostic Performance - Bipolar I Disorder
The ability of the fine-tuned Llama-3 model to correctly identify Bipolar I Disorder, Current Episode Manic (DSM-5 296.41). The model demonstrated improved diagnostic precision after fine-tuning.
Accurate diagnosis with correct DSM-5 code (296.41)
The ability of the fine-tuned Mistral model to correctly identify Panic Disorder (DSM-5 300.01) from conversation data. After fine-tuning, it produced concise, direct classifications matching DSM-5 diagnostic codes.
Accurate diagnosis with correct DSM-5 code (300.01)
Fine-Tuned Mistral Diagnostic Performance - PTSD
The ability of the fine-tuned Mistral model to correctly identify Post-Traumatic Stress Disorder (DSM-5 309.81). The model showed improved diagnostic specificity after fine-tuning.
Accurate diagnosis with correct DSM-5 code (309.81)
The ability of the fine-tuned Qwen2 model to correctly identify Schizophrenia (DSM-5 295.90) from patient conversations. After fine-tuning, it accurately mapped symptoms to DSM-5 diagnoses with greater clarity.
Accurate diagnosis with correct DSM-5 code (295.90)
The ability of the fine-tuned Qwen2 model to correctly identify Generalized Anxiety Disorder (DSM-5 300.02). The model showed improved clinical precision after fine-tuning.
Accurate diagnosis with correct DSM-5 code (300.02)
Training Loss Behavior
How the training error decreases during the fine-tuning process, indicating learning effectiveness. Rapid decline indicates effective learning.
Steep decline during initial steps, plateauing around step 25
Validation Loss Behavior
How the model performs on unseen data during training, indicating generalization ability. Smooth decrease suggests good generalization without overfitting.
Smooth decrease, stabilizing around step 25
Generalization Gap
The difference between training and validation performance, quantified as the area between loss curves. A smaller gap indicates better generalization.
Approximately 2.41 (relatively narrow and shrinking)
Hardware Used for Training
The computational resources used to fine-tune the models, important for understanding accessibility and cost.
Google Colab with NVIDIA A100 GPUs and Tesla TPUs
Optimization Technique
The method used to make the models efficient enough to run on consumer-grade hardware while maintaining performance.
Quantized Low-Rank Adaptation (QLoRA) with 4-bit quantization
Deployment Framework
The software framework used to deploy and run the fine-tuned models for inference.
Ollama
Reasoning LLM Role
How the OpenAI-gpt-oss reasoning model contributes to final diagnosis. It synthesizes predictions from multiple fine-tuned models to determine the most consistent and clinically appropriate diagnosis.
Evaluates, cross-validates, and synthesizes outputs from LLM consortium to produce final DSM-5-aligned diagnosis
System Architecture Layers
The number of functional layers in the proposed platform, each serving a specific role in the diagnostic workflow.
4 layers: Data Lake, LLM Agent, LLM, and OpenAI-gpt-oss Reasoning
Consensus Mechanism
The approach used to combine predictions from multiple fine-tuned models. This improves diagnostic robustness by leveraging diversity in model reasoning.
Multi-model consensus with reasoning LLM synthesis
Clinical Collaboration
The real-world medical organization involved in developing and validating the prototype system.
U.S. Army Medical Research Team in Norfolk, Virginia, USA
Innovation Claim
What makes this research novel compared to prior work in AI-assisted psychiatric diagnosis.
First application of a fine-tuned LLM consortium integrated with a reasoning LLM for clinical mental health diagnosis
CLIN-LLM: A Safety-Constrained Hybrid Framework for Clinical Diagnosis and Treatment …
CLIN-LLM demonstrates high diagnostic accuracy (98%) for medical diagnosis using AI, significantly outperforming baseline models including ClinicalBERT (88.8% F1), BioClinicalBERT (93.1% F1), and GPT-5 (87.5% F1). The system's key innovations include uncertainty quantification (flagging 18% of low-confidence cases), retrieval-augmented generation (78% top-5 precision), and integrated safety mechanisms that reduced unsafe antibiotic recommendations by 67% with zero hallucinated treatments. Clinicians rated the system highly (4.2/5), confirming clinical validity. The system maintains this high accuracy across multiple datasets and achieves real-time performance (2.5 seconds per case). These results demonstrate that AI can achieve very high diagnostic accuracy when properly designed with medical domain knowledge, uncertainty awareness, and safety constraints.
Extracted Data
Data Point
Value
CLIN-LLM Diagnostic Accuracy
The overall accuracy of the CLIN-LLM system in correctly diagnosing diseases from patient symptoms. This measures the percentage of correct disease classifications out of all predictions made.
98%
CLIN-LLM F1 Score
The F1 score balances precision (how many predicted diagnoses were correct) and recall (how many actual diseases were identified). A score of 98% indicates excellent performance in both correctly identifying diseases and minimizing false diagnoses.
98%
CLIN-LLM Precision
The precision metric measures what percentage of the system's positive disease predictions were actually correct. A 98% precision means very few false positive diagnoses.
98%
CLIN-LLM Recall
Recall measures what percentage of actual diseases present in patients were correctly identified by the system. A 98% recall means the system successfully detected nearly all cases of disease.
98%
ClinicalBERT Accuracy
The diagnostic accuracy of the baseline ClinicalBERT model, used for comparison. This shows how well a standard biomedical language model performs on the same task without the safety enhancements.
88.5%
ClinicalBERT F1 Score
The F1 score for the baseline ClinicalBERT model, showing its balanced performance in disease classification compared to CLIN-LLM.
88.8%
BioClinicalBERT Accuracy
The diagnostic accuracy of BioClinicalBERT, another biomedical transformer baseline used for comparison.
91.1%
BioClinicalBERT F1 Score
The F1 score for BioClinicalBERT, indicating its balanced diagnostic performance.
93.1%
GPT-5 Zero-shot Accuracy
The diagnostic accuracy of GPT-5 when used without specific medical training (zero-shot learning). This tests how well a general-purpose large language model can diagnose diseases.
92.3%
GPT-5 Zero-shot F1 Score
The F1 score for GPT-5 in zero-shot mode, showing its balanced diagnostic performance without medical fine-tuning.
87.5%
Performance Improvement Over ClinicalBERT
The percentage improvement in F1 score that CLIN-LLM achieved compared to the ClinicalBERT baseline. This quantifies how much better the proposed system performs.
7.1% improvement (statistically significant, p < 0.001)
Low-Certainty Case Flagging Rate
The percentage of diagnostic cases where the AI system's confidence was low enough to automatically flag the case for human expert review. This safety mechanism ensures uncertain diagnoses get human oversight.
18%
Top-5 Treatment Retrieval Precision
The accuracy of the system in retrieving relevant treatment dialogues from the medical database. A 78% precision means that about 4 out of 5 retrieved treatment examples were clinically relevant.
78%
Clinician-Rated Treatment Validity
Medical professionals rated the quality and appropriateness of the AI-generated treatment recommendations on a scale of 1 to 5. A score of 4.2 indicates that clinicians found the recommendations to be highly valid and clinically appropriate.
4.2 out of 5
Reduction in Unsafe Antibiotic Recommendations
The percentage decrease in inappropriate or unsafe antibiotic prescriptions compared to GPT-5. This measures how much the safety mechanisms (stewardship rules and drug interaction checks) improved prescription safety.
67% reduction compared to GPT-5
Hallucinated Treatment Rate
The number of completely fabricated or non-existent treatments generated by the system. Zero hallucinations means the system never invented fake medications or procedures.
0 (zero hallucinated treatments)
Training Dataset Size
The number of clinical cases used to train the diagnostic model. This relatively small dataset demonstrates the system's efficiency in learning from limited data.
1,200 clinical cases from Symptom2Disease dataset
Treatment Dialogue Corpus Size
The size of the medical dialogue database used for retrieving relevant treatment information. This large corpus provides extensive clinical knowledge for generating recommendations.
260,000 doctor-patient dialogues from MedDialog
Number of Disease Classes
The number of different diseases the system was trained to diagnose. This shows the breadth of diagnostic capability.
24 disease classes
Perfect AUC Score Diseases
The number of diseases where the system achieved a perfect Area Under the Curve (AUC) score of 1.00 in ROC analysis. AUC measures the model's ability to distinguish between positive and negative cases, with 1.00 being perfect discrimination.
16 out of 23 diseases achieved AUC = 1.00
System Response Time
The total time required for the system to process a patient case and generate a diagnosis and treatment recommendation. Fast response time is critical for real-world clinical deployment.
Under 2.5 seconds (classification: 0.7s, retrieval: 1.1s, generation: 0.6s) on NVIDIA A100 GPU
Performance on Symptom-Disease Prediction Dataset
The system's accuracy when tested on the SDPD dataset, demonstrating generalization to different data sources beyond the training set.
94.1% accuracy, 94.3% F1 score
Performance on Disease Diagnosis Dataset
The system's accuracy on a semi-structured disease diagnosis dataset, showing robustness with multimodal clinical data.
91.3% accuracy, 91.5% F1 score
Performance on MedDialog Diagnosis Subset
The system's accuracy when tested on free-text conversational medical dialogues, demonstrating ability to extract structured diagnostic information from natural language.
92.1% accuracy, 92.2% F1 score
Ready to create your own data extraction?
This is an example of what Future Scan can do. Head to your dashboard to create your own custom data extraction for systematic reviews and meta-analyses.