Methodology

How IsabiCheckAI reaches a result. Version 1.0.

Principle: evidence first, transparent reasoning, traceable sources, honest uncertainty.

1. How claims are selected

Every submission is first analysed to find its distinct factual assertions. Opinions, predictions, jokes and rhetorical questions are not fact-checkable and are labelled Not Fact-Checkable. A pasted article is split into up to five of its most important checkable claims, each assessed separately. If a statement depends on missing context (for example, “Who did he kill?” with no person or event named) we return Insufficient Context and do not guess what was meant.

2. How sources are retrieved

For each claim we run web searches through a search provider, then try to read the public pages found. Pages that forbid automated access (robots.txt), cannot be reached, or are unsafe to fetch are not read; where only the search-result snippet is available, the report says so. A report can only cite a source that was actually retrieved during the check; we never create or recall links from memory.

3. How sources are evaluated

The language model is given only the retrieved text and asked to compare it with the claim. It must quote evidence verbatim. We then verify, in code, that every cited quotation really appears in the retrieved text and that every link is a valid http(s) address; citations that fail are discarded. A valid link alone never proves that a source supports a claim. Many pages repeating one statement are not counted as independent confirmation.

4. How verdicts are assigned

  • Supported: the evidence substantially supports the claim.
  • False: reliable evidence directly contradicts the central assertion.
  • Misleading: partly accurate but creates a materially wrong impression.
  • Partly True: some elements are supported, others are not.
  • Unverified: the evidence is not enough to decide.
  • Insufficient Context: too vague or incomplete to evaluate.
  • Not Fact-Checkable: an opinion, prediction or other non-factual statement.

A verdict that says a claim is true, false, misleading or partly true is kept only if verified citations back it; otherwise it becomes Unverified. Not finding evidence is never treated as proof that something is false. The verdict is separate from the processing status (queued, processing, completed, failed, needs review).

5. Evidence strength and sentiment

The “evidence strength” label (low / medium / high) is computed by fixed rules from the verified citations: how many independent publishers support the verdict, whether one is an institutional or established fact-checking source, how many were read in full rather than as snippets, and whether conflicting evidence exists. It is not the model's own certainty and is not a probability. Domain type is a rough heuristic, not a judgement of any outlet's quality. Sentiment describes the tone of the claim only and never influences the verdict.

6. How uncertainty is handled

If the search service or the AI service is unavailable, the check fails honestly and no verdict is produced; we do not fall back to the model's memory. Claims on sensitive topics (elections, health, finance, serious allegations about people) with a definitive verdict are held for an editor before they can be shown.

7. Limitations

  • Automated analysis can misread sources, miss relevant material, or reflect gaps in what search engines return.
  • Recent events may not yet be well covered; time-sensitive claims are flagged.
  • Language coverage and local-source coverage are uneven.
  • Reports are not legal determinations.

8. AI-assisted analysis versus human verification

Reports are produced automatically. A report is labelled Reviewed by an editor only after a real review is recorded in our system; otherwise it is labelled AI-assisted. When an editor changes a verdict, both the change and its reason are logged and the report says it was revised.

9. Reporting an error

Use “Report an error” on any report or visit Corrections & feedback. Published corrections appear on the report itself.