Before you trust an AI answer, check each claim against its source

Use a technical question with a clear boundary

This is the October 5 lesson in our daily learning series. It is an evergreen verification exercise, not a claim about a newly released AI model. The task is to assess one answer about website accessibility. You can do it with a deliberately written sample answer, so the exercise does not require a paid AI subscription.

Consider this fictional answer: an automated accessibility scan can find some issues, and if it passes, the entire website is accessible. The first part is narrower than the second. Treat the sentence as two separate claims before you decide whether to accept it. This small separation prevents one correct detail from making a much broader assertion look proven.

Make a claim-and-evidence log

Create five columns: claim, source, relevant section, decision and correction. Use decisions such as supported, contradicted and not established. These labels describe the relationship to the evidence you actually checked. They are more informative than giving the whole answer an unexplained confidence percentage or counting how many links appear at the end.

NIST's Generative Artificial Intelligence Profile is a companion to its voluntary AI Risk Management Framework and addresses trustworthiness in the use and evaluation of AI systems. For this beginner exercise, borrow the habit of explicitly evaluating an output. The log below is our practice method, not a claim that NIST has certified a particular answer or tool.

Source: NIST: Generative Artificial Intelligence Profile

Open the source and compare the scope

Read W3C's accessibility evaluation overview and the Playwright accessibility-testing guide. W3C states that tools alone cannot determine conformance, and Playwright explains the limits of automated checks. The sample claim that an automated scan can detect some issues is supported; the claim that passing proves whole-site accessibility is not.

Write down the specific documentation section that led to your decision. Check whether the answer discusses one page, a complete site or a particular interface state. A scan that examined only the initial page cannot establish what happens after opening a dialog. This is a scope problem even if the linked source is authentic and the tool ran successfully.

Source: W3C WAI: Evaluating Web AccessibilityPlaywright documentation: Accessibility testing

Correct the answer without overcorrecting it

A useful revision is: automated accessibility checks can identify some common problems, but a passing result does not establish that the whole site is accessible; combine them with manual evaluation and user testing. The revision keeps the supported benefit and removes the unsupported guarantee. Attach the sources to the relevant statement so readers can inspect the reasoning.

Now add an uncertainty note from your own exercise: the practice review checked documentation claims and did not audit a deployed website. This prevents a reader from confusing verification of the explanation with verification of an actual product. Use the same discipline when reviewing generated instructions about a framework, database, API or deployment service.

Source: Playwright documentation: Accessibility testing

Repeat with a date or version claim

For a second round, write a fictional statement that a future release is already available. Find the project's own release announcement and separate its publication date, planned release date and actual availability. A roadmap may support a plan without supporting a claim that the product has shipped. Record that distinction rather than inventing a date to complete the answer.

If the source is unavailable, outdated or unclear, label the claim not established and explain what evidence would resolve it. Do not ask another AI to vote on the first answer and treat agreement as proof. You can use a tool to locate candidate documents, but the final decision should still point to inspectable evidence.

Keep the exercise useful for your next project

Save the original fictional answer, your claim log, the source URLs and check date, and the corrected paragraph. Include one supported claim and one rejected or narrowed claim. A reviewer should be able to understand why your conclusion changed. Avoid including private customer information or confidential code when building a shareable portfolio example.

This method improves traceability; it does not guarantee that every source is complete or that you noticed every error. Important decisions may require deeper domain review and direct testing. As a next step in the related Full Stack + AI program, apply the same method to one generated implementation suggestion and pair the documentation review with a small reproducible test.

Sources and further reading

• NIST: Generative Artificial Intelligence Profile · checked 2026-10-05
• W3C WAI: Evaluating Web Accessibility · checked 2026-10-05
• Playwright documentation: Accessibility testing · checked 2026-10-05

Spot an error? Email info@spothub.in with the article link and correction.