S0 · #6, Skill Lesson, 15 min, Published

AI Assistants for Learners: Catch the Mistakes

Track Your Progress

Watched it and tried the exercises? Mark it done: it counts toward your credentials.

About This Lesson

AI assistants answer almost any question in seconds, in confident, fluent language. Most of the time that is a gift for a learner. Sometimes the answer is wrong, and it sounds exactly as sure. This lesson teaches you to use an assistant as a tutor and to check what it tells you, the skill real job ads now ask for by name.

It starts with a real court case: in Mata v. Avianca, two lawyers filed a brief with six court decisions that an assistant had invented, and a New York judge fined them 5,000 dollars. Then the why: an assistant writes one likely word at a time, so the same question can get two different answers, and nothing inside it checks the facts. A guess scores better than "I don't know", and one invented detail can snowball into many.

Hands-on, you open two free accounts, ChatGPT and Claude, ask the same question twice and compare the answers side by side. You learn five steps to check any answer, see them used on a practice answer about ibuprofen 200 mg checked against its official label, and learn to use an assistant as a tutor without fooling yourself. The mini project, Tutor and Check, puts it all together.

What You'll Learn

  • What an AI assistant is, and why the same question gets different answers
  • Why assistants hallucinate, and the snowball effect
  • ChatGPT, Claude, Gemini and Copilot, and two free accounts
  • The five steps to check any answer
  • How to use an assistant as a tutor
  • How to answer "Why do LLMs hallucinate?" in an interview
Free Workbook

Practise This Lesson on Paper

  • The whole lesson, step by step
  • Exercises and homework, with answers
  • Interview questions, with model answers
  • A reading list to go deeper

PDF, 31 pages, Version 1.2

Get the Free WorkbookFree account, no payment.

Sources and References

Everything this lesson states was checked against these sources before it was published.

1 Job Posting4 Documents and Papers3 Books2 Further Reading

Job Postings in the Lesson

  • Experience using common AI tools, including generative technologies such as ChatGPT or Microsoft Copilot, to support problem solving and enhance productivity.

    Associate Scientist, Clinical Immunology, Pfizer, Pearl River, NY. Pfizer careers (requisition 4966580), 2026.

Official Documents and Papers

  • Mata v. Avianca, Inc., opinion and order on sanctions

    US District Court, Southern District of New York, 2023. Sanctions of 22 June 2023: a filing cited six court decisions that did not exist; the lawyers and their firm were fined 5,000 dollars.

  • Why Language Models Hallucinate

    Kalai, Nachum, Vempala and Zhang, OpenAI and Georgia Tech (arXiv:2509.04664), 2025. Training and testing reward a confident guess over "I don't know", so models learn to guess.

  • AI Search Has a Citation Problem

    Tow Center for Digital Journalism, Columbia University (Columbia Journalism Review), 2025. Eight AI search tools, 1,600 queries: incorrect answers to more than 60% of them (6 March 2025).

  • Ibuprofen 200 mg Drug Facts labels

    DailyMed, US National Library of Medicine, 2026. The official label used to check the practice answer. The ibuprofen example is practice in checking, not medical advice.

The Books Behind This Lesson

This lesson was researched from these books. Read them to go deeper:

Further Reading

Corrections

None so far. If you spot an error, tell us in the comments under the video or at info@ai2hired.org.

Job postings belong to their companies; ai2hired is not affiliated with them. How sources are chosen and checked.

Transcript

Read the Full Transcript

The Job Line: ChatGPT or Copilot

You'll find this line in job descriptions for several roles, on LinkedIn and on job boards. Experience using common AI tools, including generative technologies such as ChatGPT or Microsoft Copilot, to support problem solving and enhance productivity. By the end of this video, you'll have two of these assistants working as your tutors, and you'll know how to catch them when they're wrong.

AI Assistants for Learners: Catch the Mistakes

This is AI assistants for learners, and how to catch their mistakes. Let's get into it.

The Big Picture

Remember the car from our lesson on AI jobs, where every AI application was a car built around an engine? Here it is again, and this time we only need two of its parts. You've already met the whole car, with its dashboard, its fuel and its brakes, and the road that keeps it running. The simplest AI application of all is a chat assistant, and it's little more than a driver talking to an engine. You're the driver who types a question, and the engine is the model that writes the answer.

Today, we'll make an assistant teach us something, and catch it when it's wrong, because both halves of that job belong to the same skill.

Mata v. Avianca: Six Invented Cases

Let's start with a true story, because it shows exactly what can go wrong with these tools. In twenty twenty-three, two lawyers in New York gave a judge a document that quoted court cases in support of their client. Six of those cases had never existed, because an AI assistant had invented them, with names, dates and quotes that looked completely real. One lawyer had even asked the assistant whether the cases were real, and it answered that they were.

In June of that year, the court fined the lawyers and their firm five thousand dollars, and the story travelled around the world.

How an LLM Writes

You already know how that can happen, from our lesson on what AI really is, so before I remind you, how does an LLM write its answer? It guesses the next word, then the one after it, and it keeps going until a whole answer is written. That's why it gives you the likely answer and not a checked one, and a confident answer that's false is what we called a hallucination.

Same Question, Different Answers

Here's the part that's new, and it explains a lot of what you'll see on your own screen. The model doesn't always pick the most likely word, because it picks by chance among the likely ones, like dice that favor some sides over others. So ask the same question once and you might get blue, and ask it again and you might get green. This is called inconsistency, which simply means the same question getting different answers.

It's a bit like asking a friend the same question on two different days, except that a friend remembers what they said the first time. The model isn't trying to agree with what it said before, so when two of its answers disagree on a fact, at least one of them is wrong.

Why an Assistant Invents

So why does an assistant invent an answer, when it could simply say that it doesn't know? The first reason you already have, since it produces likely words, and nothing inside it checks them against the facts. The second comes from how these models are trained and tested, where a guess often scores better than admitting it doesn't know. It's like a student in an exam where a blank answer earns zero, so guessing is always worth a try.

The Snowball

There's also a third reason, and it's exactly what happened to those two lawyers and their six invented cases, so let's call it the snowball. Once a first sentence is wrong, the model keeps building on it, and each new detail makes the invention look more real.

Four Free Assistants

Assistants stay useful even with these weaknesses, and four of them will follow you through this course and all the way into your job. They are ChatGPT from OpenAI, Claude from Anthropic, Gemini from Google and Copilot from Microsoft, and each one is a product with an LLM working inside. They all have a free plan, and they all share the weakness you've just seen.

Verify Before You Trust

So the rule for this lesson, and for the rest of your career with these tools, is to verify before you trust. An assistant's ideas and explanations are yours to use freely, while any fact you plan to rely on gets checked first. Engineers shrink the problem by handing the model the right documents to answer from, a method called RAG, which we'll build ourselves in Season nine. Until then the checking is your job, so let's set up two assistants and put that rule to work.

Hands-On: Free Accounts

We'll work with two assistants, ChatGPT and Claude, because comparing two answers is one of the simplest checks you have. Both are free to start and neither asks for a payment card, so open ChatGPT dot com in your browser and choose to sign up. Type your email address, confirm it from your inbox, and you land on a page with one box waiting for your question. Then do exactly the same at Claude dot AI, and in about five minutes you have two tutors ready to go.

The free plans have limits that change, so if one of them asks you to wait, switch to the other and keep going.

Ask the Same Question Twice

Let's test what we just learned, by typing one message into ChatGPT and watching what comes back. The message you type is called a prompt, the text you send to an assistant, and ours asks for three medicines you can buy without a prescription. There's the first answer, so now I'll open a new chat and send exactly the same words a second time. Before the second answer appears, do you expect it to match the first one word for word?

Both Answers, Side by Side

The same three medicines came back, but the line about ibuprofen was worded differently the second time. Neither answer is wrong here, and that's the point, because an assistant hands you one possible answer out of many.

The Five Steps

Checking an answer comes down to five steps, and they work on any answer you plan to rely on. First you spot the checkable claims, which are the names, numbers and dates an answer gives you, and then you ask twice in a new chat and compare the two answers. Next you open every link, because a source is a claim too, and you confirm in an official source, like the label, the law or the company's own page. At the end you decide which claims are right and which are wrong or unproven, and you note what you checked.

A Practice Answer, Checked

A real assistant rarely gets things wrong exactly when we want it to, so we'll practise like a fire drill. Let's run those five steps on an answer we wrote ourselves, with mistakes hidden in it on purpose. The question was what ibuprofen two hundred milligrams is used for, and which warnings come with it. Our sample says it relieves minor aches and reduces fever, that it's safe with aspirin, that stomach bleeding isn't a concern, and it names an official-looking source.

The real label is free to read on DailyMed, the label database of the US National Library of Medicine. It agrees on aches and fever, but on aspirin it tells you to ask a doctor or pharmacist first. It also carries a clear bleeding warning, so two of our four claims are wrong. And the source doesn't exist at all, even though it came with a year and a page number.

Learning With a Tutor

Checking is half the skill, and the other half is learning with an assistant without fooling yourself. Remember retrieval practice and the learning log, from our lesson on how to learn? Both come back here, inside a loop of five moves that you can use on any topic. You try first on your own, and only then do you ask for a simple explanation, written for a complete beginner. You ask follow-up questions on every word that's still unclear, and you ask for an example from your world, like a pharmacy or your own job.

Then you turn the tables and ask it to quiz you, which is retrieval practice with a very patient partner. And you finish the way we just practiced, because you verify the facts you'll keep, and you write them in your learning log.

Common Mistakes

Three mistakes catch almost everyone, and the first is believing an answer because it sounds sure. An assistant writes a wrong answer in exactly the same confident voice as a right one, so you judge the claim and never the tone. The second mistake is trusting an answer because it came with a link. In twenty twenty-five, researchers at Columbia University tested eight AI search tools on questions about news articles, and more than sixty percent of their answers were wrong.

Many of the links led to pages that didn't exist, so a link only counts once you've opened it yourself. The third mistake is pasting private information into a free assistant, like passwords, health details or your company's files. At work, that information only goes into tools your company has approved, where the company decides what the assistant can see and what it keeps. Those are called enterprise assistants, and we'll open them up in the lesson on enterprise AI assistants, in Season sixteen.

Mini Project: Tutor and Check

Time to do the whole thing for real, with a real assistant and a real label, in a mini project called Tutor and Check. It takes about thirty minutes, and it's practice in checking facts, so it's never medical advice. Pick a common medicine that you aren't taking right now, and follow the four steps on the screen. Step one is to ask the tutor, so it's the same question as our sample, this time to a real assistant. We ask ChatGPT what ibuprofen two hundred milligrams is used for, with each warning on its own line and the source it relied on.

It gives us eight uses and a long list of warnings, and it names two sources, the FDA and DailyMed. Step two is to find the official label, and for a medicine sold in the United States that means DailyMed. We search for ibuprofen two hundred milligram tablets, open one label, and read its uses and its warnings with our own eyes. In Europe you'd look on the site of the European Medicines Agency, or of your own country's medicines agency.

Step three is to mark every claim, one line at a time, as right, wrong or not on the label, and here are six of them. All six match the label we opened, so nothing we checked was invented, and now we know that because we checked, not because the answer sounded sure. Step four happens in the same learning log we brought back a few minutes ago, with one line in your own words. Write what you learned and what you caught, because that line is what you'll remember next month.

You've just used an AI assistant as a tutor and checked its answer against an official source, which is exactly what that job line asks for. For your turn, run the same check in Claude and see which assistant made fewer mistakes.

Interview Angle

In interviews for almost any role that touches AI, you can expect a question like this one. Why do LLMs hallucinate, and how do you reduce it? A strong answer has three parts, and after this lesson you own all of them. Start with what it is, a confident answer that's false, and then explain why it happens, because the model produces likely words that nothing checks. Add that guessing was rewarded when the model was trained and tested, which is why it rarely admits that it doesn't know.

Then finish with what you do about it, which is asking for sources and opening them, asking twice, and confirming every important fact in an official source.

Recap

Here's the short version of everything we covered in this lesson. An AI assistant is a product with an LLM inside, and because it picks likely words by chance, the same question can bring back different answers. Some of those answers are hallucinations that sound confident and are false, so before you rely on a fact, you run the five steps and confirm it in an official source. Used as a tutor, an assistant explains, gives examples and quizzes you, as long as you try first and verify before you trust.

The exercises, further reading and interview questions for this lesson are in the free workbook on ai2hired. org, linked in the description. So the next time an assistant sounds completely sure of itself, you'll know how to check its answer at the source.

Your New CV Line

Here's your new CV line. Use ChatGPT and Claude to learn and research, and verify every fact at its source. It's yours to add, starting today.

Never Miss a Lesson

Subscribe on YouTube, or get an email the day each new lesson goes live.