Evaluation Methodology & Limitations
How AI-detection accuracy should be measured — and why a single headline number, without the dataset and testing details behind it, cannot be trusted.
Last verified: July 13, 2026
What the Numbers Would Mean
These are the standard metrics used to evaluate any classifier. A responsible benchmark reports all of them together, at a stated decision threshold, on a described dataset.
Accuracy
The percentage of all texts (both AI and human) correctly classified. It is misleading on its own when the mix of AI vs. human texts (the base rate) differs from the real world.
Recall (True Positive Rate)
Of the AI-generated texts, the percentage correctly identified as AI. Trading recall up usually raises false positives.
False Positive Rate (FPR)
Of the human-written texts, the percentage wrongly flagged as AI. This is the number that matters most when a flag can harm a person.
Precision
Of the texts flagged as AI, the percentage that are actually AI. Precision falls sharply as real-world AI content becomes rarer.
What a Trustworthy Benchmark Must Publish
We do not publish accuracy percentages or head-to-head rankings here, because a number without the material below cannot be independently checked. A reproducible study needs, at minimum:
The dataset
Sample counts by language, domain, and length; human-text sources and licenses; deduplication method; and a clean train/validation/test split.
Generation details
The exact AI model IDs and versions, generation dates, prompts, and decoding settings used to create the AI samples.
Thresholds & calibration
How the decision threshold was chosen, whether scores were calibrated, and confusion matrices with 95% confidence intervals.
Reproducibility
Sample-level results and runnable code or a notebook, so anyone can reproduce the figures. For competitor numbers: their version, plan, and test date.
What This Tool Can and Cannot Tell You
The result is an AI-likeness signal, not a calibrated probability of authorship, and it cannot identify who or what wrote a text. Accuracy varies with language, text type, length, the source model, generation settings, and any human editing. Short, paraphrased, translated, and heavily edited texts are especially hard, and false positives are more likely for non-native English writing and formulaic genres.
Use any result as one fallible signal — never as proof, and never as the sole basis for an academic, employment, disciplinary, legal, healthcare, or compliance decision.