Prove the AI's answer, or take its word for it

1. Why it matters: confident answers nobody can check

An AI model states every figure with the same confidence. It cannot know whether a figure is the right one from the right table, and the analyst, auditor or bank relying on it cannot know either, unless the answer can be checked against the source independently of the AI. A citation is that check only if it leads to one exact place in the source. In financial work an unverified figure that looks cited is the most expensive kind of error: it gets used. Independent studies of AI search tools find citations routinely fail this test¹. This test measures what that costs, in risk, time and money, for the same AI on the same filings.

2. What was compared

2.1 The documents: SEC Form 10-Q, published as Inline XBRL HTML

A Form 10-Q is the quarterly report a US public company files with the SEC². On sec.gov it is one HTML page in Inline XBRL (iXBRL): the complete filing (cover page, financial statements, notes, management's discussion, risk factors) readable in a browser, with machine-readable tags on many of its figures³. It is not "the financials only": the 30 pages hold 967,751 words and 1,295 tables with numbers. Tags cover 756/1,295 (58%) of those tables and 18,928/32,313 (59%) of their number cells (statements and notes; most narrative-section tables are untagged); 107 of the 120 values behind this test's questions are tagged. The tags live only in the HTML code: the AI tools that read web pages and the text converters used for RAG discard them, so no AI answer can cite them.

2.2 Three formats: the same content, different structure

One value (Apple's Greater China operating income, three months ended 28 March 2026) as each format holds it:

<ix:nonFraction unitRef="usd" contextRef="c-135" decimals="-6" name="us-gaap:OperatingIncomeLoss" format="ixt:num-dot-decimal" scale="6" id="f-744">9,189</ix:nonFraction>

Operating income/(loss)| $| 19,373 | | | $| 13,052 | | | $| 9,189 | | | $| 3,839 | | | $| 4,127 | | | $| (13,695)| | | $| 35,885 |

U000001.D20005.PG1.S36.T20.CD: Three Months Ended March 28, 2026 — Greater China
U000001.D20005.PG1.S36.T20.CA.R6: Operating income/(loss)
U000001.D20005.PG1.S36.T20.CD.R6: $9,189

The content is equivalent (the prototype converter still misses a small share of table numbers, which is being closed). What separates the formats is structure: only the agentic file gives every value an address.

2.3 What the citations actually look like: the same three questions, verbatim from the test

The test instructed every set-up to "cite every claim with the document and the exact quote": the most help a raw-HTML or RAG answer can give. In everyday use a web chat AI's own citation is a link to the page, and a document chat's is the file name.

SD001: In Apple's Q2 2026 Form 10-Q (period ended March 28, 2026), what was Greater China segment operating income for the three months ended March 28, 2026, per the segment information table (in millions)? Correct value: 9,189.

According to Apple's Q2 2026 Form 10-Q segment information table, the Greater China segment operating income for the three months ended March 28, 2026, was $9,189 million. 

**Citation Format: DIRECT QUOTE / PARAPHRASE**
Greater China operating income was $9,189 million [@U000001.D20005.PG1.S36.T20.CD.R6] for the three months ended March 28, 2026 [@U000001.D20005.PG1.S36.T20.CD].

Based on Apple's Q2 2026 Form 10-Q (for the period ended March 28, 2026), the Greater China segment operating income for the three months ended March 28, 2026, was **$9,189 million**. 

This can be found in **Note 10 – Segment Information**, under the table for "Three Months Ended March 28, 2026":

> **Exact Quote:** 
> "Three Months Ended March 28, 2026 [...]
> Operating income/(loss) $ 19,373 $ 13,052 $ 9,189 $ 3,839 $ 4,127 $ ( 13,695 ) $ 35,885"

*(Note: The third numerical column corresponds to the "Greater China" segment).*

Based on Apple's Q2 2026 Form 10-Q, the Greater China segment operating income for the three months ended March 28, 2026, was **$9,189 million**. 

This is supported by the Segment Information table:
> "Three Months Ended March 28, 2026 [...] Operating income/(loss) | $| 19,373 | | | $| 13,052 | | | **$| 9,189** |" (Source 11: AAPL 10-Q 2026-03-28.htm, page 11).

SD016: In Apple's 10-Q for the period ended June 27, 2026, what was the percent change in Europe net sales in the geographic table from the nine months ended June 28, 2025 to the nine months ended June 27, 2026? Correct value: 16.11.

**PARAPHRASE**
According to Apple's 10-Q for the period ended June 27, 2026, the percent change in Europe net sales from the nine months ended June 28, 2025 to the nine months ended June 27, 2026 was 16% [@U000001.D20006.PG1.S43.T23.CG.R2].

Based on Apple's 10-Q for the period ended June 27, 2026, the percent change in Europe net sales in the geographic table from the nine months ended June 28, 2025 to the nine months ended June 27, 2026 was 16%.

**Citation:**
* **Document:** [https://www.sec.gov/Archives/edgar/data/320193/000032019326000020/aapl-20260627.htm](https://www.sec.gov/Archives/edgar/data/320193/000032019326000020/aapl-20260627.htm)
* **Exact Quote:** "16%" (from the Net Sales by Geographic Segment table displaying the nine-month period data: 95,596 for 2026 and 82,329 for 2025, corresponding to a "16%" change).

Based on Apple's 10-Q for the period ended June 27, 2026, the percent change in Europe net sales from the nine months ended June 28, 2025 to the nine months ended June 27, 2026 was 16%.

This is supported by the "Segment Operating Performance" table:
> "Europe| 29,395 | | | 24,014 | | | 22%| | | 95,596 | | | 82,329 | | | 16%|" (Source 2: AAPL 10-Q 2026-06-27.htm, Page 14)

SX001: Using the SEC 10-Q filings: (1) In Apple's Q2 2026 Form 10-Q (period ended March 28, 2026), what was Greater China segment operating income for the three months ended March 28, 2026, per the segment information table (in millions)? (2) In Adobe's 10-Q for the period ended May 29, 2026, what was net cash provided by operating activities for the six months ended May 29, 2026 (in millions) in the cash flow summary? What is (1) minus (2), in millions? Correct value: 4066.00.

**1. Apple's Greater China Segment Operating Income**
In Apple's Q2 2026 Form 10-Q, the Greater China segment operating income for the three months ended March 28, 2026 was **$9,189 million** [@U000001.D20005.PG1.S36.T20.CD.R6].

**2. Adobe's Net Cash Provided by Operating Activities**
In Adobe's 10-Q for the period ended May 29, 2026, the net cash provided by operating activities for the six months ended May 29, 2026 was **$5,123 million** [@U000001.D20011.PG1.S14.T10.CB.R15].

**3. Difference**
Subtracting Adobe's operating cash flow from Apple's Greater China segment operating income ($9,189 million - $5,123 million) yields **$4,066 million**.

*(Citation format: Synthesis)*

Based on the SEC 10-Q filings provided, here are the answers to your questions:

**(1) Apple's Greater China segment operating income**
According to Apple's Q2 2026 Form 10-Q (period ended March 28, 2026), in "Note 10 – Segment Information", the table for the "Three Months Ended March 28, 2026" reports the following for the Greater China segment:
* **Quote:** "Operating income/(loss) [...] $ 9,189" (in millions).

**(2) Adobe's net cash provided by operating activities**
According to Adobe's Form 10-Q for the period ended May 29, 2026, in the cash flows summary section: 
* **Quote:** "A summary of our cash flows is as follows: Six Months Ended (in millions) May 29, 2026... Net cash provided by operating activities $ 5,123". 
*(Note: This aligns with the subsequent textual explanation that states "Net cash provided by operating activities of $5.12 billion for the six months ended May 29, 2026...")*

**(3) Calculation: (1) minus (2)**
* $9,189 million (Apple Greater China operating income) 
* - $5,123 million (Adobe net cash provided by operating activities)
* **Result:** **$4,066 mill…

Based on the provided documents:

(1) In Apple's Q2 2026 Form 10-Q, the Greater China segment operating income for the three months ended March 28, 2026, was **$9,189 million**. 
* **Source:** [Source 12: AAPL 10-Q 2026-03-28.htm] 
* **Exact Quote:** Under the "Three Months Ended March 28, 2026" segment table (page 11): "| GreaterChina | ... Operating income/(loss) | ... | $| 9,189 |"

(2) Adobe's 10-Q for the period ended May 29, 2026, and its net cash provided by operating activities are **Not in the documents**.

Because the Adobe data is not in the documents, the calculation for (1) minus (2) cannot be performed.

3. How each set-up finds and cites the answer

Agentic files (ADAP): read only what is needed, cite the exact cell, check it automatically. The shipped Tasset 1.2.0 in-file AI ran with all 30 files open. For each question the model works in steps through the file's own tools: (1) outline: the list of a file's sections (67 outline reads in 100 answers); (2) search: find the lines and cells that match (175 searches); (3) section: read only the section that holds the answer (129 section reads); (4) resolve: confirm an exact address (40 identity / resolve calls). Only what the tools return is sent to the model: a median of 13,239 characters per answer out of 6,447,618 characters in the 30 filings. The model then cites each figure by its address. Before the answer is shown, the file checks every cited address (it exists exactly once; the model actually read it): about 1 microsecond per answer, and records each result in its Verification Log. Clicking a citation scrolls the file to that cell and highlights it in its table. For a question across two filings the model opens both files and cites a cell in each.⁴

Raw HTML: the AI given the 30 sec.gov links. How people use web pages with AI: the prompt holds the links and the question; the model decides what to open with Gemini's URL-reading tool and may run Google searches. The tool attempted 121 page fetches; 79 succeeded and 42 failed. The question's own filing was read for 56 of 100 answers; for the rest the model answered from searches or its own knowledge (38 of those were still correct: with nothing from the filing behind them). For a cross-filing question it must open both filings itself.⁵

RAG over the HTML. The standard set-up behind document-chat products: each page converted to text, cut into 800-token chunks (4,168 chunks for the 30 filings), and for each question the 20 chunks that look most similar to the question are given to the model. The model sees only those 20 pieces; for a cross-filing question both filings must happen to be among them.

4. The tests: what, how and why

All grading is done by programs from the recorded test data; no person and no AI judges the results. Each set-up ran once; none was told which filing a question is about.

5. Definitions

6. Results

6.1 Risk: can the user prove the answer?

With raw HTML and RAG most answers were right, but for 55 and 79 of them the user cannot tell right from wrong without redoing the work. With agentic files every answer ended in a certain state.

6.2 Time: verifying every answer by hand

The reviewer's steps for each value an answer depends on, the same in every format. Open the cited place: agentic file: click the address; the file has already checked it and scrolls to the highlighted cell (0.2 s in the same file; a 2.5 s page load when another file opens). Quote: open the filing, Find the quote, check every hit. Link or file name only: open the filing, search the figure, check every hit (reading the text around it), or skim the filing when the figure is not printed as written. Confirm the value with its row and period. Recompute a calculation. Times: Keystroke-Level Model (Card, Moran & Newell 1983), web page-response standards (Google: 0.2 s interaction, 2.5 s load), reading 238 words/min (Brysbaert 2019), skimming 700 words/min (Rayner et al. 2016).⁶

Time is half the difference; the other half is the result. An agentic-file check ends in certainty: the value is at the highlighted cell or it is not. A link-only check means searching a 60-100-page filing for a figure that may appear in several places or not as written at all.

6.3 Cost and return

The agentic file sends the model more input than raw HTML or RAG (it reads the sections it needs rather than one page or 20 chunks), so its AI cost is a little higher. It is still a fraction of giving the AI the documents whole: $0.458 per answer for the filings that fit in one request (966,975 input tokens per answer), against $0.083. The cost that decides the return is the analyst's: at the BLS mean wage for financial analysts ($56.15 per hour⁷) verification dominates the total, and only the agentic file keeps it small. Gemini 3.1 Pro list prices, each answer priced from its recorded token counts.⁸ All Gemini test runs, including earlier rounds, cost $265.69 in tokens.

6.4 Do the citations point to one exact place?

A quote is not an address: even when found, it may occur in several places and does not identify the cell. RAG quotes often copy the converted text the model saw rather than the page.

6.5 Across filings

With agentic files the AI found both values in every case and cited each at its exact cell. Raw HTML and RAG often could not reach the second filing at all.

6.6 The built-in check

The shipped 1.2.0 citation check, run directly on addresses (no AI).

6.7 Every wrong or unfound answer

The one wrong agentic-file answer (SD090) cited U000001.D20030.PG1.S51.T28.CA.R3 = "AWS", U000001.D20030.PG1.S51.T28.CC = "June 30, 2026", U000001.D20030.PG1.S51.T28.CC.R3 = "263,750". The highlighted table is introduced by "Property and equipment, net by segment is as follows (in millions):"; the question asks for total assets, in the table before it ("Total segment assets reconciled to consolidated amounts are as follows (in millions):"). The reviewer sees the mismatch at the cited cell: the wrong figure could not pass unnoticed.

Some "could not find" replies add a figure after saying "Not in the documents"; they are counted as not found because the AI itself said it did not find the value.

6.8 Can an AI check the AI?

With exact addresses the re-check was right every time. Without them it erred both ways: it confirmed a wrong raw-HTML answer and rejected correct ones. A second AI is not independent verification; an address a program resolves is.

6.9 Risk control built into the format: automatic verification, then a one-click human pass

What: which formats can check an AI's citations by themselves, and how fast can a person double-check what remains? How: Layer 1 is read from the test records (the file's check of every citation) and from running the shipped check directly; Layer 2 is the verification time of §6.2. Why: regulated firms need a control on every AI figure and a record of it (§7).

Layer 1 is a program: it runs on every answer, gives the same result every time and leaves a record, so no AI citation reaches the user unchecked. It proves that every cited place exists and was read; it does not judge meaning: a real cell from the wrong table passes it (§6.6). That is what Layer 2 is for: because every citation is an exact address, the reviewer's double-check is a click per figure, with the line or cell highlighted in context: enough to see, as in SD090, that a figure came from the wrong table. Raw HTML and RAG have neither layer: nothing is checked automatically, nothing is logged, and the human pass starts with finding the figure in a 60-100-page filing. This is one test of one model on 100 questions; it points to AI workflows in which software checks every figure first and people review by exception, with a record an auditor can follow.

7. What this means for regulated finance

7.1 Why firms have not let AI run unchecked

• The firm stays responsible for every AI output. ESMA: "firms' decisions remain the responsibility of management bodies, irrespective of whether those decisions are taken by people or AI-based tools"⁹. FINRA reminds members that existing supervision rules apply to generative AI and lists "concerns about accuracy, privacy, bias"¹⁰.
• Wrong answers look right. The Financial Stability Board describes "a new type of model inaccuracy, the hallucination, where a model provides a seemingly confident but inaccurate response to user inputs"¹¹; it warns that "Hallucinations can be difficult to detect and evaluate"¹¹, and that issues such as "explainability remain a barrier to full-scale adoption"¹¹. The US Treasury records the concern that "a model convincingly produces an incorrect output"¹², and that "it is still challenging for AI models to pinpoint the source of the errors"¹².
• Regulation asks for logs and human oversight. The EU AI Act classes AI used to "evaluate the creditworthiness of natural persons or establish their credit score"¹³ as high-risk. Article 12: "High-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system"¹⁴. Under Article 14 the people overseeing them must remain aware of the "possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system"¹⁵ "(automation bias)"¹⁵.
• So almost nothing runs unattended. In the Bank of England and FCA survey, "Only 2% of use cases have fully autonomous decision-making"¹⁶, and "46% of respondent firms reported having only"¹⁶ 'partial understanding' of the AI they use.
• US supervision has no settled framework for agentic AI yet. The April 2026 model-risk guidance that replaced SR 11-7 states: "Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance"¹⁷.

7.2 What the human check costs

The total cost of financial-crime compliance in the US and Canada alone has "reached U.S.$61 billion"¹⁸; IDC forecasts that "the financial services industry will account for more than 20% of all AI spending"¹⁹. Keeping a person on every answer protects the firm but, as Deloitte notes for agentic AI in banking, "placing a human at every step of an agent's decision chain could defeat the purpose and negate efficiency gains"²⁰; what removes the need is that every decision leaves a "clear audit trail"²⁰. When unchecked AI output reaches a client the costs are real: a US court sanctioned lawyers for filing "fake quotes and citations created by the artificial intelligence tool ChatGPT"²¹; Deloitte "refunded over 97,000 Australian dollars"²² to the Australian government over a report with "a fabricated quote from a federal court judgment and references to nonexistent academic research papers"²².

7.3 What this test indicates

The agentic file supplies the control these frameworks look for, at machine speed: every figure the AI cites is checked by the file's own deterministic code: about 1 microsecond per answer, with no statistical model involved, and logged with the exact address of its source. Under the April 2026 US guidance, software of that kind sits outside the definition of a "model": "deterministic rule-based processes and software where there are no statistical, economic, or financial theories underpinning their design or use"¹⁷. In this test the file verified all of its citations automatically and logged each check; a reviewer can then confirm any figure with one click at the highlighted cell, in context (§6.9). Human oversight moves from searching documents to reviewing exact, pre-checked sources: the layered control (program first, people on the final pass) that the frameworks above ask for. Raw HTML and RAG cannot offer it: they have no address to check, no automatic check and no log.

7.4 The rules are arriving, and they ask for exactly these controls

United States. There is no federal AI act. Federal policy favours one national framework over state rules: Executive Order 14365 directs that the administration "shall jointly prepare a legislative recommendation establishing a uniform Federal policy framework for AI that preempts State AI laws"²³. Meanwhile states legislate: Colorado's automated-decision law (in force from 1 January 2027) states: "Both developers and deployers are required to retain records necessary to demonstrate compliance with the act for at least 3 years"²⁴. The federal standards body already names the exact problem this test measures: NIST's Generative AI Profile warns that "GAI outputs may also include confabulated logic or citations that purport to justify or explain the system's answer, which may further mislead humans into inappropriately trusting the system's output"²⁵, and recommends: "Review and verify sources and citations in GAI system outputs during predeployment risk measurement and ongoing monitoring activities"²⁵. FINRA's 2026 oversight report points firms to "storing prompt and output logs for accountability and troubleshooting"²⁶ and warns: "Complicated, multi-step agent reasoning tasks can make outcomes difficult to trace or explain, complicating auditability"²⁶.

European Union. The AI Act requires high-risk AI systems to log automatically: Article 12: "High-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system"¹⁴, "In order to ensure a level of traceability of the functioning of a high-risk AI system"²⁷. Article 19: "the logs shall be kept for a period appropriate to the intended purpose of the high-risk AI system, of at least six months"²⁷. Article 26(6): "Deployers that are financial institutions subject to requirements regarding their internal governance, arrangements or processes under Union financial services law shall maintain the logs as part of the documentation kept pursuant to the relevant Union financial service law"²⁷. The high-risk obligations now apply from "2 December 2027 as regards AI systems classified as high-risk pursuant to Article 6(2) and Annex III"²⁸. EIOPA asks insurers to keep records "to enable their reproducibility and traceability"²⁹.

International. The OECD AI Principles: "AI actors should ensure traceability, including in relation to datasets, processes and decisions made during the AI system lifecycle"³⁰. IOSCO notes that a "model's outputs are typically probabilistic, rather than deterministic, in which case they will not always produce the same result for a given input"³¹. Singapore's final guidelines (October 2026): "An FI should document the AI development process to enable reproducibility and auditability"³²; "An FI should also take into account automation bias and decision fatigue as the use of AI increases in speed and scale"³². Canada's model-risk guideline expects data to be "traceable (that is, having documented lineage and provenance)"³³.

7.5 Why programmatic checks are trusted over manual ones

Auditing standards already treat automated controls as the stronger kind. The international auditing standard ISA 315: "Automated controls may be more reliable than manual controls because they cannot be as easily bypassed, ignored, or overridden, and they are also less prone to simple errors and mistakes"³⁴. The US PCAOB: "Entirely automated application controls are generally not subject to breakdowns due to human failure"³⁵. Manual checking, by contrast, has a measured error rate: "when humans do simple mechanical tasks, such as typing, they make undetected errors in about 0.5% of all actions"³⁶, and in field audits "88% of the 113 spreadsheets audited"³⁶ contained errors. The agentic file puts the program first (every citation, every time, logged) and leaves people the part only people can do: judging the highlighted source in context.

This is a test result and a description of alignment, not legal advice or a compliance opinion; whether a given workflow meets a given rule is for each firm and its supervisors to decide.

8. Scope of this test

• One AI model, one document type (SEC 10-Q), one run per set-up, 100 + 20 questions; the cross-filing pairings are synthetic and test retrieval across files.
• Correct values were computed by program from the filings' cells and confirmed by a second model.
• Verification time is a model with published standards, applied identically to every format; it favours raw HTML and RAG.
• Raw HTML ran through the Gemini API's URL tool; many sec.gov fetches failed in this run; a consumer chat app may read pages differently.
• The agentic files were made by the authors' prototype converter; the in-file AI ran the shipped 1.2.0 code with the reading cap lifted. The authors build the agentic-file format; every figure and program is published for re-running (§9).

9. How the figures and quotes were produced and audited

Every figure was re-derived from the raw test records by audit programs that reuse none of the original scoring code, computed a second time in plain SQL, and must agree: 329 of 329 claims agree (claims marked PYTHON ONLY are measured outside the database, e.g. the check timing, or need a rounding rule SQL cannot express). Planted wrong data must make both checks fail before any real run; the shipped file check was run, not assumed; all 30 source pages match their recorded fingerprints; a full rebuild reproduces this page byte for byte.

• Audit process log · claims ledger · every claim as SQL · definitions · sourced constants · regulatory and industry sources (quotes verified word for word) · rebuild everything

The audit package (ledger, SQL, definitions, sources and the rebuild script) and the raw test records are not yet published on this site.