4.8%
of the 420 answers were wrong: 20 in all.
When AI reads a sustainability report,
how much comes back wrong?
20 of 420 answers came back wrong (4.8%), and with 21 more only partly right, one answer in ten was not fully right. We put ten factual questions about each of 14 published sustainability reports to ChatGPT, Gemini and a text-layer reader, and scored every answer against a key written from the pages.
Ask for this check on one of your reports ↗Main numbers
1 in 10
answers was not fully right, counting the 21 partial ones.
13 of 20
wrong answers were a true value from the same report, just not the one asked for.
The question
When AI tools read a sustainability report, how many of their factual answers come back wrong?
Method
- 14 public sustainability reports from 2025 and 2026, from 9 countries and 10 sectors, in English, Spanish and Portuguese. Six were made in InDesign and eight in Word.
- Ten questions per report, always of the same types: five exact figures (two only in a table, one only in a chart or image), two chart readings, a target with its baseline, scope, and method or assurance.
- An answer key written from the pages before any run.
- Three readers: ChatGPT (temporary chat, PDF attached, Medium reasoning); Gemini (temporary chat, PDF attached; Pro on 11 reports, Flash on 3); and a text-layer reader (Claude reading the extracted text, no page images), which stands in for what a search index, a retrieval pipeline or a screen reader on an untagged PDF receives.
- One run per report and reader, each answer scored correct, partial, wrong or not found: 420 answers.
Results
Correct, partial, wrong and not found, by reader
140 answers per reader. ChatGPT got 93% right and 2 wrong. Gemini gave the most wrong answers, 15 of 140 (11%), most of them looking right. The text-layer reader rarely got a figure wrong (3), but could not find 22, most of them on chart questions and figures shown only as an image.
| Reader | Correct | Partial | Wrong | Not found |
|---|---|---|---|---|
| ChatGPT | 130 (93%) | 7 (5%) | 2 (1%) | 1 (1%) |
| Gemini | 113 (81%) | 10 (7%) | 15 (11%) | 2 (1%) |
| Text-layer reader | 111 (79%) | 4 (3%) | 3 (2%) | 22 (16%) |
Correct answers by question type
Chart readings had the lowest correct rate for all three readers: ChatGPT 23 of 28, Gemini 19 of 28, the text-layer reader 15 of 28.
| Question type | ChatGPT | Gemini | Text layer |
|---|---|---|---|
| Exact figure | 28/28 (100%) | 22/28 (79%) | 24/28 (86%) |
| Exact figure, only in a table | 28/28 (100%) | 26/28 (93%) | 24/28 (86%) |
| Exact figure, only in a chart or image | 11/14 (79%) | 10/14 (71%) | 9/14 (64%) |
| Chart reading | 23/28 (82%) | 19/28 (68%) | 15/28 (54%) |
| Target and baseline | 13/14 (93%) | 12/14 (86%) | 13/14 (93%) |
| Scope | 13/14 (93%) | 13/14 (93%) | 13/14 (93%) |
| Method or assurance | 14/14 (100%) | 11/14 (79%) | 13/14 (93%) |
The 20 wrong answers, by cause
Most errors were not invented. They were the right report’s wrong number. Classified from the scoring notes; the cause is not a column in the CSV.
| Cause | Wrong answers | What it means |
|---|---|---|
| Neighbouring value | 13 | A true value from the same report, from the next column, year, scope or chart |
| Not in the document | 4 | A number or an auditor name that does not appear anywhere in the report (all four from Gemini) |
| Other misreads | 3 | Wrong answers that were neither a neighbouring value nor missing from the document |
The common errorsWhat goes wrong,
What goes wrong,
and what fixes it in the file.
The neighbouring value
In 13 of the 20 errors the tool took a true value from the report, but from the next column, year, scope or chart: the 2023 figure instead of 2024, Scope 1 without biogenic emissions, another score from the same rating, the chart next door.
Fix in the filePut the year and scope label next to each number, in table headers and on chart labels, so no value can be read without its context.
Lost in an image
22 of the text-layer reader’s 140 answers were not found, most of them on chart questions and on figures shown only as an image: tables pasted as pictures, highlight cards, unlabelled charts. The chat tools, which look at the page image, found most of them.
Fix in the fileBuild real tables, not images of tables, so the figures exist as text that search, retrieval and screen readers can reach.
The misread chart
Chart readings were where every reader stumbled, with the lowest correct rate in all three columns.
Fix in the fileGive every chart direct data labels and a short text description of what it shows.
Today the chat tools make up for a lot by reading the page as an image. The risk is the tool that answers the neighbouring number with confidence, and every reader that depends on structure.
Limits
- One run per report and reader; there was no second run.
- Claude.ai was not tested. The text-layer column is Claude reading the extracted text, not the claude.ai product.
- Gemini answered with Flash on 3 of the 14 reports, because Pro was overloaded.
- A small convenience sample. Of 20 reports listed, 3 had no data chart for the chart questions and 3 were over the 10 MB upload limit even after compression. In 9 of the 14 PDFs only the photos were recompressed to fit the upload; text and tags stayed identical, checked page by page.
- InDesign versus Word did not separate the chat tools. The text-layer reader did better on the InDesign files (90% against 71%), but the Word files in this sample have more tables and charts pasted as images and are almost all in Spanish or Portuguese. With 14 reports that cannot separate tool from language: a hint, not a finding.
- This is a test of extraction and retrieval, not an accessibility audit.
Open data
All 420 scores, one row per question (140) and one column per reader, scored C (correct), P (partial), I (wrong) or N (not found). Reports are anonymised as R01 to R14, with sector, country, language, authoring tool, page count and tagging only.
Download the CSV ↓See it in one report
The same four-page report, laid out twice and exported two ways, and what AI assistants, a screen reader and an accessibility checker found in each file.
A Report, Read Two Ways →