Checking AI answers: how to catch its mistakes

Key takeaways

• →AI can produce false information and present it as fact. The DfE calls this hallucination.
• →A clear, confident answer is not evidence that it is correct.
• →References that look real may not exist, so find and read every source yourself.
• →Asking another AI, or asking the same one to check itself, does not count as checking.
• →Compare the answer with your course material, your marking criteria and your own working.

1. Why does AI get things wrong?

The Department for Education says generative AIAI that writes text or makes images, such as ChatGPT. can produce “nonsensical, inaccurate or false information presented as fact”, which it calls hallucinationWhen AI states something false or invented as if it were true.. It adds that outputs can also be out of date or unreliable, and that the tools may be trained on data that is not specific to our curriculum.^[1]

Glasgow’s guidance for students says AI tools do not “know” the meaning of what they produce, and that they are prone to biases, inaccuracies and mistakes. In other words, the answer is written to sound right, not checked to be right.^[2]

2. Why is a confident answer not enough?

Abertay University’s study skills guide puts it plainly: a clear and confident answer is not evidence that it is correct. It also says polished writing, technical detail, links and references do not prove that an answer is reliable.^[3]

The same guide warns that a repeated answer, agreement from another AI tool, or an AI saying it has checked its own work does not verify anything, because tools may share the same errors. Asking an AI to list its assumptions is not independent checkingChecking with a source that does not depend on the AI. either.^[3]

3. Can you trust the references it gives?

Not without looking. Glasgow says AI tends to produce text that “looks like a citationMade-up references that have the right shape but no real source.” and cannot verify that a source exists, so it can invent references that look real. Its advice is to always find and read your sources.^[2]

Abertay adds a second test: confirm that each cited source exists, is suitable and supports the exact claim it is attached to. A real source can still be used to back up something it does not say.^[3]

4. What should you check?

Abertay suggests separating the parts of an answer: facts you can check, suggestions, and conclusions. You do not have to judge the whole answer as good or bad. Check each important claim against an original or authoritative source, your course material, a calculation or a qualified person.^[3]

It also says to compare the answer with your original question and the marking criteriaThe list of what earns marks in a piece of work., to check that the evidence supports the exact claim being made, and to look for missing evidence or other explanations. For maths and science, it says to reproduce calculations and to run any code yourself, and for languages to check translated terms against the original.^[3]

For GCSE and A-level, our own suggestion is to hold a school answer up against the exam board’s specificationThe exam board's list of what you must know. and mark schemeThe exam board's guide to what earns each mark., which show what a mark is actually given for. That tip is ours, not from the three sources above.

5. A quick checklist

The table sums up the checks from the sources in this guide.^[1]

6. Who is responsible for the final answer?

You are. Glasgow says you are responsible for checking AI output carefully, and that using AI at any point has to be acknowledged. The DfE makes the same point for professionals: whoever produces a document remains responsible for its quality and content, whatever tools were used, and AI cannot replace the judgement and deep subject knowledge of a human expert.^[2]

Our guide to whether using AI is cheating covers the rules, and using AI alongside tutoring shows how a tutor can help you spot weak answers.^[2]

7. How Brainhub can help

A tutor is a human check. In a lesson we can go through an AI answer with you, find what is wrong and explain why, so you build the knowledge to catch it next time. We coach and explain, and we never complete assessed work for a student. See our online tutoring page for how lessons work, and the contact form to ask.

1. 1. What does the DfE call false information that AI presents as fact?

   Show answer

2. 2. Does agreement from a second AI tool prove an answer is right?

   Show answer

3. 3. What should you do with an AI reference?

   Show answer

4. 4. Who is responsible for the final answer?

   Show answer

Your next steps

1. 01

   Pick one answer an AI gave you this week.

2. 02

   Check each fact against your notes or an official page.

3. 03

   Find and read any source it named.

4. 04

   Ask your teacher or tutor about anything you cannot check.