1. Objective
By the end of this lesson, you will be able to:
- Write a prompt that asks the chatbot to name its source, flag uncertainty, and list claims worth checking.
- Verify one important claim by opening a reliable source yourself and comparing.
- Decide when a reply needs checking and when it doesn't.
2. Hook
Transcript
You've learned to build a strong prompt and steer the reply. Now the habit that protects you: a sure tone is not evidence. Today you'll practice checking a claim yourself, calmly and without fear.
A chatbot tells you, in a calm, polished paragraph, exactly how much of something you should have each day. It names a government agency. It even gives a link.
Is it right? It might be. The lesson is that you can't tell from how it sounds. You can only tell by looking.
3. Warm-up (from lessons 1–7)
- (Lesson 5) Rewrite this weak follow-up: "Make it better."
- (Lesson 2) What should you do with a detail in a reply that you never provided?
- (Lesson 1) What's the difference between fluency and truth?
- (Lesson 7) What does the check line in your template ask the chatbot to do, and what doesn't it guarantee?
Answers
- Any follow-up that names the problem and what to keep, for example: "Keep the tone; cut it to three sentences."
- Treat it as invented; remove or correct it.
- Fluency is how smooth the writing sounds. Truth is whether it's correct. A reply can have the first without the second.
- To ask you questions if something is missing and to say when it's unsure. It doesn't guarantee it catches every error, which is why this lesson matters.
4. The idea
Transcript
Chatbots can produce wrong information that sounds confident. That's often called a hallucination: a wrong date or number, a quote no one said, or a source that doesn't hold up. The reply can also be completely right. So the skill isn't catching the chatbot in a lie. It's checking the claim. Check when being wrong would matter: numbers, rules, quotes, studies, and health, legal, or money claims, or anything you'll repeat or act on. Your prompt can make checking easier. Ask it to say if it's not sure, to name the organization and page you should check, and to list the claims you should verify. But the real check happens outside the chat, on a source the chatbot didn't write.
Fluency is not the same as truth
OpenAI, which makes ChatGPT, puts it plainly: ChatGPT can produce incorrect or misleading output, and "it might sound confident — even when it's wrong." They call this a hallucination: a reply that isn't factually accurate. Examples they list include wrong dates or facts, and fabricated quotes, studies, citations, or references to sources that don't exist. Their advice: "verify important information from reliable sources." (See Sources.)
xAI, which makes Grok, says similar: Grok may sometimes include inaccurate information or hallucinations, "and you should carefully consider and verify Grok's responses before using them." (See Sources.)
This isn't a bug in one product. In a 2025 paper and article, OpenAI researchers argued that models often guess when unsure, partly because common tests reward a confident answer more than an honest "I don't know." (See Sources.) The practical upshot: a sure tone is not evidence.
A hallucination can look like:
- a number, date, name, or rule that's simply wrong;
- a quote no one said, or a study that can't be found;
- a source that looks official (a title, an author, a link) but doesn't hold up when you look.
The reply can also be completely right. So this lesson isn't about catching the chatbot in a lie. The skill is the same either way: check the claim.
When checking matters
You don't have to verify every joke, packing list, or subject line. Do check when:
- you'd be embarrassed, harmed, or unfair to someone if it were wrong;
- the reply includes a number, a rule, a quote, a health, legal, or money claim, or "a study shows…";
- you're about to repeat it to someone else, or act on it.
Health, law, and money deserve extra care. A chatbot can help you prepare questions or understand a term. It's not your doctor, lawyer, or financial adviser.
Your new prompting skill: ask for a check (P6)
You can make checking easier with your prompt. Add one or more of these:
- "Say if you're not sure." This invites the model to flag uncertainty instead of covering it.
- "Name the organization and the document or page I should check." This gives you somewhere to look.
- "List the claims in your answer that I should verify." This turns a smooth paragraph into a checklist.
- "Don't invent a URL. If you can't give a real one, say so."
Important: these requests make the reply easier to check. They don't do the checking. Asking a chatbot "are you sure?" isn't a check, because the same program is grading its own work. The check happens outside the chat, on a source the chatbot didn't write.
A reliable source for a public rule or figure is usually the organization that publishes it (a government agency, a standards body, or a company's official page for its own policy), not a social post, and not the chatbot.
Tool notes: search inside chatbots (November 2026)
As of November 2026, many free chatbots can search the web and show sources. That helps, especially for current events. It doesn't finish the job. OpenAI notes that even with search, a model may not be able to read a given page (for example, because of technical problems or paywalls), and that confidence still isn't reliability. Its advice includes treating the reply as a first draft, not a final source, and visiting links directly. (See Sources.)
So if your chatbot cites a page, open it, find the part that's supposed to support the claim, and read it. If you can't find the claim, or the page doesn't exist, don't use the claim.
5. Worked example
Before:
How much sodium should I have a day?
After:
What is the recommended daily sodium limit for adults in the United States, according to the FDA or the Dietary Guidelines for Americans? Give the number, say who published it, and name the document or page I should check. If you are not sure, say so.
What changed:
| Change | Row |
|---|---|
| "For adults in the United States, according to the FDA or the Dietary Guidelines" makes the question specific and names who should be the authority | 2. Context |
| "Give the number, say who published it, and name the document" says what you want back | 1. Task / 4. Shape |
| "Name the document or page I should check. If you are not sure, say so." | 6. Check built in (new) |
Notice what the "after" prompt doesn't do: it doesn't ask for personal diet advice. It asks for a public, checkable figure.
6. Guided practice
This exercise uses a public, checkable number: the U.S. recommended daily limit for sodium (salt is one source of sodium). We're using it to practice fact-checking, not to give dietary advice. If you have questions about your own diet, ask a qualified professional.
Start a new chat.
Prompt 1 (copy and paste):
What is the recommended daily sodium limit for adults in the United States, according to the FDA or the Dietary Guidelines for Americans? Give the number, say who published it, and name the document or page I should check. If you are not sure, say so.
What to look for: a number, an organization, and a document or page. Notice the tone too. Does it sound sure?
Now do the check, outside the chat. Open one of these official pages yourself. Type or paste the address into your browser; don't rely only on a link the chatbot gave you.
- U.S. Food and Drug Administration, "Sodium in Your Diet": fda.gov/food/nutrition-education-resources-materials/sodium-your-diet
- Dietary Guidelines for Americans (current edition as of this writing: 2025–2030): dietaryguidelines.gov
As of November 2026, the FDA page and the Dietary Guidelines both use a limit of less than 2,300 milligrams of sodium per day for people 14 and older, unless a doctor has given someone different advice. If those pages have changed since this lesson was written, believe the official page in front of you.
Compare:
- Did the chatbot's number match?
- Did it name the right organization or document?
- If it gave a link, did the link go where it claimed, and did the page say what the chatbot said it said?
If everything matched, excellent; you still did the right thing by checking. If something was off, also excellent; you caught it before relying on it. Either result is a successful exercise.
Prompt 2 (copy and paste, same chat):
Name two official U.S. government pages that state the adult daily sodium limit. For each, give the exact page title and the full URL. Do not invent a URL. If you cannot provide a real URL, say so.
Try to open both URLs.
- If a link works and the page supports the claim, you have a source you can keep.
- If a link leads nowhere, to the wrong page, or to a page that doesn't say what was claimed, you've seen why "it came with a citation" isn't enough.
Some chatbots will give working official URLs. Some will be slightly off, or confident about a page they didn't really retrieve. You can't know which you have until you look.
7. Independent practice
Spot the weak prompt
You want to know when U.S. federal income tax returns are due. Which prompt makes checking hardest?
- A.
When are taxes due? - B.
What is the usual deadline to file a U.S. federal individual income tax return? Name the IRS page I should check, and tell me if the date can change in some years. - C.
When is the federal tax deadline this year? Include a link.
Answer
A. It doesn't say which country, which tax, or where the answer should come from, and it asks for nothing checkable (rows 2 and 6 = 0). C asks for a link (row 6 = 1) but doesn't invite uncertainty or name the authority. B names the authority and invites a caveat (row 6 = 2). With any of them, you'd still check the IRS site yourself.
Fix-it drill
Rewrite this prompt so the reply is easy to check. You don't need to look up the answer; this drill is about writing the prompt.
Is it true you should replace a smoke alarm every few years?
One strong version (not the only one)
How often do U.S. fire-safety authorities recommend replacing a home smoke alarm? Name the organization and the page or document I should check, and tell me if recommendations differ by type of alarm or if you are not sure.
Why it's stronger: it names the kind of authority (row 2), asks for a specific answer and a source (rows 1 and 4), and invites uncertainty (row 6). If you want to go further, open the source it names and check. Your local fire department or the U.S. Fire Administration are reasonable places to start.
8. Rubric self-check
You met row 6 briefly in lessons 3 and 7 (the check line). Here it gets its full treatment. Score your fix-it prompt:
| Row | Question | 0 | 1 | 2 |
|---|---|---|---|---|
| 1. Task | What do I want done and back? | Neither | One | Both |
| 2. Context | Situation and facts? | None | Some | Enough for a stranger |
| 3. Reader and tone | Who's it for, how should it sound? | Neither | One | Both (or n/a for a fact question) |
| 4. Shape and limits | Length, format, include, avoid? | None | Vague | Specific |
| 6. Check built in | If it matters, did I ask for sources or uncertainty, and do I plan to check outside the chat? | Stakes are real and no check | Asked for a link, no plan to check | Asked for source and uncertainty, and I'll verify myself |
(Row 5, Safe to send, has applied since lesson 3; lesson 9 covers it in depth.)
Where to next: If row 6 is low, add "Name the source I should check, and say if you're unsure," then actually open that source.
9. Retrieval check
- What is a hallucination, as this lesson uses the word?
- Why is a confident tone not evidence that an answer is correct?
- The chatbot gives a statistic and a link. What do you do before repeating the statistic?
- True or false: if your chatbot can search the web, you no longer need to check important facts.
- True or false: asking the chatbot "Are you sure?" counts as checking.
- (From lesson 5) Name the three follow-up rows.
Answers
- A reply that isn't factually accurate (a wrong fact or date, or a made-up quote, study, or source), even if it sounds plausible.
- Models can guess when they're unsure and still write in a sure-sounding way.
- Open the link yourself and see whether the page actually states it. If you can't confirm it from a reliable source, don't repeat it.
- False. Search helps, but the chatbot still writes the answer. Links can be wrong, unreadable, or misquoted.
- False. The same program is grading its own work. Real checking happens on a source outside the chat.
- F1 Points at the problem; F2 Says what to keep; F3 One main change.
10. Recap
Transcript
Recap. A smooth, confident answer can still be wrong. Check when being wrong would matter. Prompt for checkability: name the authority, ask for the source, and invite I'm not sure. Then check outside the chat. The chatbot isn't the source. Confirming a right answer and catching a wrong one are both wins. This week, when a chatbot gives you a number you'd repeat, open the original source first.
- A smooth, confident answer can still be wrong. That's often called a hallucination.
- Check when being wrong would matter: numbers, rules, quotes, studies, and health, legal, or money claims.
- Prompt for checkability: name the authority, ask for the source, invite "I'm not sure."
- Then check outside the chat. The chatbot isn't the source.
- Confirming a right answer and catching a wrong one are both wins.
You can now: write a prompt that makes a reply easy to check, and do the check. Use it this week: the next time a chatbot gives you a number you'd repeat, open the original source before you repeat it.
Next: lesson 9, what not to paste into a chatbot, and how to get help anyway.
11. Talk it through with your coach
- Quiz me: your coach reads three short chatbot-style claims aloud and asks, for each, "Would you check this? Where?" It gives feedback on where to check, not on the facts themselves.
- Text-only alternative: the "When checking matters" list and retrieval check above.
Talk to your coach
River is an AI voice coach, not a person. River knows this lesson and can quiz you, listen to a prompt you're working on, and give hints. River won't do the exercise for you. You can talk out loud or type.
Ideas to start:
Sources
- OpenAI Help Center, "Does ChatGPT tell the truth?" (definition of hallucination; confidence isn't reliability; verify important information, including when search is used): help.openai.com
- OpenAI, "Why language models hallucinate": openai.com
- Kalai, Nachum, Vempala, and Zhang, "Why Language Models Hallucinate" (2025): cdn.openai.com
- xAI, Consumer FAQs (inaccurate information; verify responses): x.ai/legal/faq
- U.S. Food and Drug Administration, "Sodium in Your Diet," checked November 2026: fda.gov
- Dietary Guidelines for Americans, 2025–2030: dietaryguidelines.gov