AI Observation & Attestation — how the AI helps without replacing your judgment
For: all
Tier: starter+
Time: ~6 min
Why you'd do this
ComplianceLint puts AI on both sides of the table: your IDE AI scans your repo and forms a per-obligation judgment, and YOU open the dashboard and answer Yes or No yourself in the wizard. Both layers exist on purpose — the AI accelerates the reading-and-thinking step, but only your wizard Save counts as the legal attestation. AI observations are decision support, never substitution. This chapter explains how to read AI observations, what the red dots mean, and what to do when the AI disagrees with you.
Before you start
- Starter+ tier — Starter (€29/month or €25/month yearly) and above have wizard access. Free tier sees the upgrade prompt instead of AI observations
- At least one scan completed (the IDE AI writes AI observations as part of its scan-and-classify flow)
- MCP host configured (Claude Code, Cursor, Continue, or any compatible AI IDE — the AI observation flow relies on the IDE's built-in fetch + read tools to look at evidence)
Step 1
The mental model has 3 layers:
Layer 1 — AI scans + forms a per-obligation judgment. Your IDE AI reads your code + docs, decides whether each obligation looks fulfilled, and stores its judgment as an AI observation tied to that finding.
Layer 2 — You review. Open the wizard in the dashboard. Each obligation section now shows the AI's view next to the legal text — what it thinks (Yes / No / Needs review) and a 2-4 sentence explanation of why.
Layer 3 — You attest. Click Yes or No yourself, then Save. ONLY this writes a legal attestation; layers 1 + 2 are advisory.
Step 2
When you open a Human Gates questionnaire for an article, each obligation section now shows a small panel between the legal-reference banner and the Yes/No question:
🤖 AI Observation — AI thinks: Yes
docs/risk-management.md Section 2 lists 7 identified risks with mitigations and is dated 2026-04, satisfying the continuous risk management requirement of Art 9(1).
The block is read-only — you can't edit what the AI thinks from the dashboard. That judgment comes from the IDE AI's scan + classification. To change it, you correct the AI from your IDE (see step 4 below).
Step 3
When the AI's view and your wizard answer differ on the same obligation, a small red dot appears at three places — your eye picks up the signal at increasing scopes:
| Layer | Where | When it shows | |-------|-------|---------------| | 1 — Inline badge | In the AI Observation block in the wizard | The AI's view differs from YOUR answer for THIS obligation | | 2 — Article-row dot | Top-right of the article row in the Human Gates list | ANY obligation in this article has a disagreement | | 3 — Sidebar nav dot | Tiny red dot on the 'Human Gates' tab in the left nav | ANY repo, ANY article, ANY obligation has a disagreement |
The signal flows one way: fine-grained at the obligation level, summarised up to the article + portfolio levels. You can spot 'I have something to look at' at the nav level, drill into the right repo + article, and find the specific obligation in the wizard — all by following red dots.
What does NOT trigger a red dot:
- AI says 'Needs review' — that's an invitation, not a contradiction
- You haven't answered the obligation yet (null human answer) — nothing to disagree with
This keeps red dots meaningful: the AI saw something you didn't, or you saw something the AI missed — worth a second look.
Step 4
The AI's observations live and die by the evidence it saw at scan time. If you have evidence the AI missed (or the AI misread a file), correct it from the IDE in plain English. Example prompts that work:
'AI, ART09-OBL-1 is yes because docs/risk-management.md Section 2 lists 7 risks with mitigations. The Q2-2026 review log is at docs/risk/2026-q2-review.md.'
'AI, you marked ART51-OBL-1 as yes but our system is not a GPAI provider — please re-classify as needs_review.'
'AI, please re-look at the Art. 50 transparency notice. The disclosure is at https://acme.dev/about/ai-disclosure.'
In each case, your IDE AI runs a 3-step pattern:
- Read prior — fetch the existing AI Observation for this obligation (if one exists from an earlier scan) so it knows what it previously said and why.
- Read evidence — read what you cited (file path or URL — see next step for the URL flow).
- Form a fresh judgment — write a new AI Observation with the updated verdict + a 2-4 sentence explanation. Once written, the new observation replaces the prior in the dashboard.
There's no server-side check that the AI explained why it changed its mind — the dashboard trusts whatever the IDE AI writes. The safety signal you actually look at is the red dot: if the AI's new verdict ends up disagreeing with YOUR wizard answer, the inline / article-row / sidebar dots light up so you see the gap. That's the cross-check that matters; the AI's own verdict over time is just an audit-trail row.
Step 5
When you cite a URL (or a local file path the AI hasn't already read), your IDE AI's built-in fetch / read tool kicks in. The exact behaviour depends on your IDE, but the standard flow is:
- The IDE AI tries to fetch the URL or open the file
- The IDE prompts you for permission — exactly like when AI asks to read any other file or fetch any other URL during your conversations
- You approve → the AI reads the actual content + forms its judgment based on what's there
- The AI writes an updated AI Observation (with source marked as 'web fetch' in the metadata)
If you decline the fetch (or the URL is unreachable, behind auth, etc.), the AI is instructed to write a 'needs_review' observation with the reason: 'Evidence URL not fetched (user declined or unreachable: 404). Cannot verify whether content satisfies obligation.' — so the dashboard reflects the gap honestly rather than the AI hallucinating a 'yes' verdict on something it didn't read.
This works the same for file:// paths, local-network paths on a LAN, or remote URLs — the IDE AI's read/fetch tool handles all of them with its own consent dialog.
Context size: the IDE AI manages its own chunking + summarisation. If a fetched page is huge, the AI will summarise rather than dump 5 MB into its context. ComplianceLint doesn't need to manage that.
Step 6
This is the legal-substance bit:
| Surface | Is it a legal attestation? | What it's for | |---------|---------------------------|----------------| | AI Observation | No — never. | Decision support; surface AI's read of your project to help you decide. | | Wizard Save (Yes/No answer) | Yes. Recorded with your identity, timestamp, and the exact answers JSON. | The auditable record an EU AI Act regulator can read. | | Evidence collection (text, URL, file upload) | No — collection is enrichment, not attestation (post-2026-05-21 Option B). | Audit-trail material attached to your wizard Save. |
Concretely: if the AI says 'yes' but you click No in the wizard, your No wins. The dashboard renders Non-Compliant, the PDF reports Non-Compliant, and the regulator sees your No. The AI's 'yes' lives only in the ai_observation row as a record of the disagreement — useful for your own audit (why did I disagree with the AI?), not for legal attestation.
This is intentional. The whole point of having human attestation is that legal accountability requires a human signature, not an AI verdict. The AI helps you read faster + spot things you missed; you decide.
Step 7
A typical session looks like this:
- In your IDE, ask your AI to scan the project ('scan this repo for EU AI Act compliance'). The AI uses its own scan tool + writes a finding + an AI Observation per obligation.
- In the SaaS dashboard, open Human Gates → pick an article. Each obligation section now shows the AI's view next to the legal text.
- Read the legal text. Read the AI's reasoning. Decide for yourself — click Yes or No in the wizard.
- If your answer differs from the AI's, the red dot stays as a reminder you saw the disagreement and chose your answer deliberately.
- Click Save. That's the legal attestation moment.
- Want to dig into a disagreement? Go back to your IDE and ask the AI to re-look at the evidence ('AI, why did you say no on ART09-OBL-1? The risk register is at docs/risk-management.md.'). The AI re-classifies + the dashboard updates.
The red dot persists until either (a) the AI changes its mind via your IDE correction, or (b) you change your wizard answer to match the AI. Both paths are valid.
Step 8
Scope notes:
- AI Observations are read-only via the dashboard. There's no 'edit AI Observation' button on purpose — that would defeat the AI-First model.
- AI Observations are NOT shown in the per-article PDF (Declaration of Conformity, Technical Documentation, etc.). PDFs report your wizard Save + evidence only. AI observations live in the SaaS UI for in-flight review; they're not part of the legal artefact.
- Cross-repo AI Observation summary view — the sidebar dot just says 'you have something'; a future 'AI disagreement dashboard' could enumerate which obligations across which repos have the open disagreements. Not built yet; raise on GitHub if you want it.
Related
Last updated: 2026-05-22