Can teachers use ChatGPT to check for AI? Not reliably. OpenAI says ChatGPT’s answers to “did AI write this?” are random, and pasting a named student’s essay into a consumer chatbot can break student privacy rules. Use process evidence and a conversation instead.
Last updated: 20 November 2026
Key Takeaways
- The chatbot does not know. OpenAI says ChatGPT has no knowledge of what is AI-generated and its answers to “did you write this?” are “random and have no basis in fact” (OpenAI Help Center).
- In a study from Arizona State University, ChatGPT correctly labeled only 2 of 2,000 ChatGPT-written news articles as AI, and GPT-4 misclassified over 95% of human-written text as AI.
- OpenAI retired its own purpose-built AI classifier in July 2023 for low accuracy. It caught 26% of AI text and wrongly flagged 9% of human text.
- NYU Steinhardt says sharing student identifiers with ChatGPT or Gemini violates FERPA, and that unlicensed tools “will never be FERPA-compliant”.
- Gemini’s built-in “Is this AI-generated?” check reads Google’s SynthID watermark in images, video and audio. It is not a detector for a typed essay.
- Better evidence: version history, drafts and a short conversation about the work. A purpose-built detector can add one data point, never proof.
- The one safe chatbot use: run your own assignment prompt through it and compare the pattern. No student text leaves your hands.
Can teachers use ChatGPT to check if an essay is AI?
They can type the question, but the answer means nothing. ChatGPT is a text generator, not a classifier. It has no record of what other people asked it to write, and it does not see watermarks in plain text. When you ask “was this written by AI?”, it produces a plausible-sounding reply, which is what it does for every prompt.
OpenAI says this directly. Its guidance for educators states that ChatGPT “has no knowledge” of what content could be AI-generated, and that it “will sometimes make up responses” to questions like “did you write this [essay]?” or “could this have been written by AI?”. In OpenAI’s words, these responses “are random and have no basis in fact” (OpenAI Help Center).
Teachers still try it, because it is free and already open in another tab. This month a professor on r/Professors described uploading a batch of suspect assignments to Gemini to confirm their suspicions. The top replies pushed back on two fronts: use a purpose-built detector rather than a chatbot, and check whether uploading student work to an outside AI breaks privacy rules. Those are the two problems this article covers.

Why does ChatGPT say it wrote essays it never wrote?
Because a yes is an easy, plausible completion to the question. A language model predicts likely next words. A question that already suggests AI authorship tends to pull the reply toward agreement, and the model has no ground truth to check against.
The best-known case shows the cost. In May 2023 a Texas A&M University-Commerce instructor told a class he had pasted their last three essays into ChatGPT, twice each, and given a zero to any paper the bot “claimed”. He wrote that ChatGPT “will tell me if the program generated the content” (Business Insider). Diplomas were put on hold for seniors who had already graduated, according to Rolling Stone.
The university later said that no students failed the class or were barred from graduating, and that it was developing policies on AI use. MIT’s AI Risk Repository now lists the episode as an incident of a professor misusing ChatGPT to detect AI text. The lesson has not changed since: the chatbot will “claim” work it never saw.
What does the research say about ChatGPT as an AI detector?
It says ChatGPT is lopsided and unreliable in both directions. Amrita Bhattacharjee and Huan Liu at Arizona State University tested ChatGPT and GPT-4 as zero-shot detectors on public datasets of human and machine-written news text, published in SIGKDD Explorations (arXiv 2308.01284).
Their results, from the camera-ready paper:
| Model asked “AI or human?” | On AI-written text | On human-written text | What that means in a classroom |
|---|---|---|---|
| ChatGPT (GPT-3.5) | Identified AI text in under 50% of samples for most generators; only 2 of 2,000 ChatGPT-written articles | Good at calling human text human | Misses most AI work, so a “human” verdict tells you little |
| GPT-4 | Caught about 97 to 100% of AI text | Misclassified over 95% of human text as AI | Calls almost everything AI, so a “yes, AI” verdict tells you nothing |
Read together, the two rows are the problem. One model says “human” to nearly everything, the other says “AI” to nearly everything. Neither gives you a signal you could put in front of a student. The study used 2023 models and news text rather than student essays, so treat the exact numbers as an illustration, but OpenAI’s 2026 guidance makes the same point in plain words.
OpenAI also tried the problem with a tool built for it. Its AI text classifier caught 26% of AI-written text and wrongly labeled human text as AI 9% of the time, and OpenAI withdrew it on 20 July 2023 “due to its low rate of accuracy” (OpenAI). If a dedicated classifier failed, a general chatbot answering a leading question will not do better.
People are not much better by eye either. In a study of 89 pre-service teachers, participants identified only 45.1% of AI-generated texts and 53.7% of student-written texts correctly (Computers and Education: Artificial Intelligence, 2024). That is why the evidence you collect matters more than the verdict you get.
Is it legal to paste a student’s essay into ChatGPT or Gemini?
In the US it can break FERPA when the essay identifies the student and the tool is not one your institution has licensed. FERPA protects education records, and student work with a name, ID or personal story attached is usually treated as one. A consumer chatbot account is a third party with its own data terms.
NYU Steinhardt’s faculty guidance is direct: sharing student identifiers with services like ChatGPT or Google Gemini “violates” FERPA, and sharing data that identifies students “with tools that NYU has not licensed will never be FERPA-compliant” (NYU Steinhardt). Johns Hopkins’ generative AI guidelines make the related point that student work can include identifiable information when students write about themselves.
Teachers on Reddit raise the same flag without being asked. Under the r/Professors Gemini post, one reply warned that unless the college has a contract with Google that isolates Gemini on campus, entering student work “may very well be a FERPA violation”. Under a viral r/mildlyinfuriating post about a grader pasting every essay into ChatGPT (2,843 comments), a commenter who said they were a professor called feeding student work to a non-local LLM without consent a privacy problem and “possible misconduct”.
Three practical rules follow. Check whether your school licenses an AI tool with an education data agreement, and use only that one. Strip names, ID numbers and file metadata before any upload. And outside the US, ask your school’s privacy officer which law applies; FERPA is American, but most systems have a version of the same idea. This is not legal advice, and your institution’s policy is the one that counts.
Does Gemini’s “Is this AI-generated?” check work on essays?
No. Google added a verification feature to the Gemini app where you can ask “Is this AI-generated?”, but it looks for SynthID, Google’s invisible watermark, in images, video and audio made with Google AI (Google). It does not certify that a typed essay was written by a person.
Even for text, SynthID could only ever find text produced by Google’s own models, and only if the watermark survived editing. An essay drafted in ChatGPT or Claude, or copied into a new document and retyped, carries nothing for it to find. We covered the limits in detail in whether Google’s SynthID Detector can catch ChatGPT essays. When Gemini answers a pasted essay with “this looks AI-generated”, it is the same guessing chatbot as above.
Which ways of checking for AI actually hold up?
Process evidence holds up best, a conversation comes next, and a detector score is supporting evidence at most. The chatbot verdict is the only method on this list with no reliable signal at all.

| Method | Reliable signal? | Student privacy risk | Source of the rating |
|---|---|---|---|
| Ask ChatGPT or Gemini “did AI write this?” | No. Answers are made up | High if names are included | OpenAI guidance; ASU study; NYU |
| Purpose-built AI detector (Turnitin, Pangram, GPTZero) | Partial. One piece of evidence | Depends on the school’s contract | Vendor docs; what academic rules say about detectors as proof |
| Version history, drafts, process trackers | Strong process evidence | Low | How far version history goes as proof |
| Talk through the work with the student | Strong | None | OpenAI guidance; teacher reports |
| Run your own prompt through ChatGPT and compare | Useful pattern check, not proof | None. No student text shared | Teacher reports on r/Professors |
If your school pays for a detector, read its report for what it is. Our guides to the Turnitin AI checker for teachers and what AI detector reports actually show explain the score, and why a percentage is a probability rather than a finding. If your school has none, the comparison in AI detectors for professors without Turnitin covers the options; professors in these threads most often named Pangram, which we review in our Pangram review.
Detectors also have known blind spots. Formulaic or second-language writing gets flagged more often, which we document in the published data on neurodivergent and ESL writers. And two detectors can disagree wildly on the same text, as our ZeroGPT vs GPTZero comparison shows.
Before you paste: a 30-second check
Pick what you are about to do with a suspect essay.
Is there a safe way to use ChatGPT when you suspect AI?
Yes: put your own assignment prompt into it, not the student’s essay. You learn what generic AI output for your exact question looks like, and no student data leaves your hands.
Several teachers in the threads use this. On r/Professors one wrote that they put their instructions into an AI, read the output, then read the student reports, and the overlap “will be obvious”. Under the Gemini post, another described giving the essay prompt to an AI and getting something “remarkably similar” to a student’s essay, which, together with the student being unable to talk about the work, settled the case.
Notice what made that case hold. The similarity was a reason to ask questions, and the student’s answers were the evidence. That matches the “totality of evidence” approach most academic integrity offices use, and it is the same reason a detector score alone rarely survives an appeal.
A few limits keep this honest. Chatbots give different answers each run, so generate three or four versions before comparing. Shared phrasing can also come from a common source or a class handout. And a student who drafted with AI and rewrote heavily will not match any single output.
What should you do instead when you suspect AI?
Start with the work, not a tool. The r/Teachers thread “I strongly suspect a student used AI but can’t prove it” (79 comments) landed on the same few moves that hold up in a misconduct process:
- Compare with earlier work. A sudden jump in vocabulary or structure is a reason to look closer, not a finding.
- Ask for the process. Version history, notes, outlines and sources. Process trackers can help where your school uses them; see what writing process trackers actually prove. Note that detectors themselves cannot read Google Docs history or ChatGPT logs.
- Have the conversation. The top advice in that thread was to quote a passage and ask the student to explain it in simpler words. A writer can usually say more than the page does.
- Check citations. Invented sources and quotes are concrete evidence, unlike a style hunch.
- Follow your institution’s process. Most published policies treat detector output as a lead, not proof. Our survey of AI detection policies at the top 50 universities shows how they phrase it.
Prevention is cheaper than proof. Many teachers in these threads moved key writing into class time. Data on how often students are actually caught is in our roundup of students caught using AI, and the K-12 side is in AI detection in K-12 schools.
Choosing a detector for your school? Start with what professors use without Turnitin and our Pangram vs GPTZero comparison.
If you are the student on the other side of this, read whether an AI flag goes on your academic record.
Methodology and limits
For this article I read OpenAI’s educator guidance and its 2023 classifier announcement, the Bhattacharjee and Liu paper from Arizona State University, the 2024 pre-service teacher study in Computers and Education: Artificial Intelligence, NYU Steinhardt’s and Johns Hopkins’ faculty AI guidance, Google’s SynthID verification announcement, and Business Insider and Rolling Stone coverage of the Texas A&M-Commerce case. I also read the top 10 Google US results for this question and 6 Reddit threads from r/Professors, r/Teachers, r/AustralianTeachers, r/AskTeachers and r/mildlyinfuriating, posted between 18 October and 16 November 2026. I ran no test of ChatGPT, Gemini or any detector, so every accuracy figure here is the source’s, not mine. The research figures come from 2023 models; newer models have not been shown to fix the underlying problem, and OpenAI’s current guidance still says the answers are random.
Frequently Asked Questions
Can ChatGPT tell if something was written by AI?
No. OpenAI says ChatGPT has no knowledge of what content is AI-generated and that its answers to questions like “did you write this?” are random and have no basis in fact. In an Arizona State University study, ChatGPT labeled only 2 of 2,000 ChatGPT-written articles as AI.
Can Gemini detect AI-written essays?
Not reliably. Gemini’s built-in verification checks for Google’s SynthID watermark in images, video and audio made with Google AI. It does not detect text written by ChatGPT, Claude or other tools, and asking Gemini whether an essay sounds like AI is a guess.
Is it a FERPA violation to put student work into ChatGPT?
It can be. NYU Steinhardt says sharing student identifiers with ChatGPT or Gemini violates FERPA and that unlicensed tools will never be FERPA-compliant. Use only tools your institution licenses, and remove names and IDs before any upload. This is not legal advice.
Why did ChatGPT say it wrote my student’s essay?
Because “yes” is a plausible reply to a leading question, and ChatGPT has no record to check. In 2023 a Texas A&M-Commerce instructor zeroed papers that ChatGPT “claimed”; the university later said no student failed or was barred from graduating over it.
What is the most reliable way to tell if a student used AI?
Process evidence and a conversation. Version history, drafts, notes and the student’s ability to explain the work hold up better than any single score. A purpose-built detector can add one data point, but most institutional policies treat it as a lead, not proof.
Should teachers use an AI detector instead of ChatGPT?
A purpose-built detector is a better tool than a chatbot, but still not proof on its own. OpenAI withdrew its own classifier in 2023 after it caught 26% of AI text and wrongly flagged 9% of human text. Use any detector score alongside process evidence.
Can I use ChatGPT safely at all when I suspect AI?
Yes, by running your own assignment prompt through it a few times and comparing the outputs with the suspect essay. No student text is shared, and the comparison gives you specific questions to ask the student. It is a reason to talk, not proof.
