AI detectors can detect AI-generated code only weakly: essay detectors like GPTZero score close to a coin flip on code in published studies, Turnitin says its AI model does not reliably read code, and code-specific checkers do best on longer files.
Key Takeaways
- Turnitin says its AI model does not reliably detect code, and it only scores prose sentences in long-form writing (Turnitin FAQ).
- On 5,069 Python problems, five essay-style detectors performed poorly on code; GPTZero sat at about 0.50 accuracy, the same as guessing (Pan et al., ICSE-SEET 2024).
- A 2026 study of 280,000+ real student samples found detectors show "systematic failures" on engineering code (Computers & Education).
- Code-specific tools report much better numbers, but those are vendor tests: Pangram reports 96.2% accuracy on code over 40 lines and 89.4% across all snippets (Pangram).
- In one vendor-published run on 600 pre-ChatGPT Java files, GPTZero flagged 9.2% and Copyleaks 11.8% of human code as AI (Codequiry).
- What usually decides a CS integrity case is not a score but whether you can explain your own functions in a meeting.
Can AI detectors detect AI-generated code?
Sometimes, and far less reliably than they detect essays. General AI text detectors were built on natural language, and code breaks most of the signals they rely on. Short functions, fixed syntax and textbook solutions look the same whether a student or a model wrote them.
That gap is why the question keeps coming up. On r/learnprogramming, a student wrote that they were afraid to keep commenting their code in case it looked like AI, and the thread drew over 120 replies. A first-year student at Toronto Metropolitan University asked the same thing in plainer words: flagged for AI for code?
The short version: there are three kinds of checks, and they behave very differently. Essay detectors are weak on code. Code-specific detectors are stronger, but their numbers come mainly from the vendors themselves. The instructor conversation is where most cases are actually decided.
Does Turnitin check code for AI?
Not in any way you should rely on, and Turnitin says so itself. Its own FAQ states that the model "does not reliably detect AI-generated text in the form of non-prose, or code" and that it only analyses prose sentences in long-form writing (Turnitin AI writing detection FAQs).
The same page sets hard limits. A submission needs at least 300 words of prose before it gets an AI score, files must be .docx, .pdf, .txt or .rtf, and lists and bullet points are excluded. A Python file with a short README does not clear that bar, so the AI percentage either never appears or reflects only the written parts.
This matters for mixed assignments. If your lab report has a written analysis plus code listings, the AI indicator is about the analysis. The code may still be checked for similarity against other students and online sources, which is a separate system and a separate question. Our breakdown of which detector is closest to Turnitin covers how its prose model compares with others.
How accurate are AI detectors on code, really?
Independent studies and vendor claims disagree by a wide margin, and that disagreement is the most useful thing to know. Peer-reviewed work on general-purpose detectors puts them near chance on code, while companies selling code detectors report accuracy above 90%.
What the independent studies found
Pan and colleagues built a dataset of 5,069 coding problems with human-written Python solutions from Kaggle, LeetCode and Quescol, generated AI versions with 13 prompt variants, and ran five detectors. Their conclusion was blunt: existing detectors "perform poorly in distinguishing between human-written code and AI-generated code" (arXiv 2401.03676). GPTZero's accuracy hovered around 0.50 across variants in their results table.
A paper at AAAI 2024 on detecting AI-generated code assignments reported GPTZero as a baseline with an average AUC of 0.56, against 0.87 for a detector designed for code (AAAI proceedings). An AUC of 0.5 is random, so 0.56 is only slightly better than a guess.
The largest study is from 2026. Sun, Liao and Ma assembled over 280,000 authentic samples, including an engineering code set, and evaluated 13 detectors. They found acceptable results on long theses but "systematic failures in engineering code and short-form coursework", and the paper page reports GPTZero at an AUC of 45.79% with a 73.97% false negative rate on code (Computers & Education, vol. 249). An earlier evaluation by Wang and colleagues reached the same direction: detectors do worse on code than on natural language (arXiv 2304.05193).
What the code detector vendors report
Pangram published its own study of AI code detection. It reports 96.2% accuracy and a 0.3% false positive rate on code over 40 lines, falling to 89.4% accuracy with a 23.3% false negative rate across all snippets (Pangram). Copyleaks offers a dedicated endpoint for AI-generated source code (Copyleaks release notes).
Codequiry published a comparison credited to a CS instructor who ran 1,200 Java submissions through three tools. On 600 files written before ChatGPT launched, Codequiry flagged 3.5%, GPTZero's code beta 9.2%, and Copyleaks 11.8% as likely AI (Codequiry blog). Treat that as a vendor result: it was published by one of the three tools it compares.
| Check | Built for code? | Best published number | Who measured it |
|---|---|---|---|
| Turnitin AI indicator | No, prose only | Does not reliably detect code (own statement) | Turnitin |
| GPTZero (text model) | No | About 0.50 accuracy; AUC 0.56 | Independent researchers |
| GPTZero code beta | Yes | 81% of known AI files caught; 9.2% false positives | Codequiry (vendor) |
| Copyleaks AI code | Yes | 83% caught; 11.8% false positives | Codequiry (vendor) |
| Codequiry | Yes | 94% caught; 3.5% false positives | Codequiry (own test) |
| Pangram | Yes | 96.2% on 40+ lines; 89.4% overall | Pangram (own test) |
The pattern is consistent with what happens on essays. Independent numbers on text detectors are usually worse than the marketing, as our look at GPTZero false positives on ESL writing shows, and the code results follow the same shape.
Why are short assignments the hardest to judge?
Because there is very little for any detector to measure. Pangram's own figures show accuracy dropping from 96.2% on code over 40 lines to 89.4% across all lengths, with the miss rate rising from 8.5% to 23.3% (Pangram). That is the vendor's best case, and it still gets markedly worse on short code.
Intro programming courses live in that short range. A loop that reverses a string or sums a list has only a handful of sensible answers, so a careful student and a chatbot often write nearly identical code. Codequiry's write-up traced most of its own false positives to heavily commented starter code, very short methods with no branching, and methods returning a single arithmetic result.
The 2026 Computers & Education study saw the same thing from the other side: detectors did better on long theses than on short coursework (Sun et al.). The shorter the task, the less a score means.
What makes human code look AI-written?
Mostly style that happens to overlap with what models produce. In the vendor-published Java comparison, GPTZero's false positives skewed toward files using Java streams and lambda expressions, and Copyleaks' toward files with long comments and method-level Javadoc (Codequiry). Both styles were common long before ChatGPT.
Students report a second trigger that no detector measures: using techniques the course has not taught yet. In the Toronto Metropolitan thread, a commenter warned that reaching for something like multithreading in a first-year assignment invites a flag. That is a human judgment, and it can be wrong, but it is real.
Comments are the most debated signal. The top replies on the r/learnprogramming thread urged the student to keep writing them, and one pointed out that models comment code because they learned from human code that is commented. Removing good comments to look less like AI makes the code worse and proves nothing. Comments can also carry hidden instructions in assignment files, which we covered in hidden AI traps in assignments.
How do instructors actually check programming work?
Usually with three layers: similarity checking, an optional AI code tool, and a conversation. Similarity tools such as Stanford's MOSS compare submissions against each other to find shared structure. They were designed to catch copying between students, not AI, but two students pasting the same chatbot answer can end up matching each other.
When something looks off, many instructors simply ask the student to explain the code. Turnitin's own guidance for educators frames a detection result as the start of a conversation and suggests questions to ask the student. A professor replying in a Purdue thread about an AI code integrity procedure described doing exactly that, with follow-up questions to see whether the student understood the work or had memorised it.
This is the same shift we have seen with essays, where many classrooms moved from detector scores to process evidence. Our guide to what writing-process trackers prove explains why process logs alone are not proof either.
What should you do if your code gets flagged as AI?
Treat it as a request to show your work, not a verdict. Two students in the Purdue thread described meetings where they were asked what a few functions did and why they made certain choices, and the matter ended once they explained. Those are reader reports, not a policy, but they match how integrity processes are usually framed.
Before you submit
Commit early and often if you use Git. A history of small commits with real messages, failed attempts and fixes is the code equivalent of essay drafts. The same logic sits behind using track changes as evidence and keeping version history as supporting proof.
Keep your scratch files, test inputs and notes on why you chose an approach. If your course allows AI help for some tasks, write down what you used it for. Check the syllabus first: rules for AI in programming courses vary widely, and an approved use is not misconduct.
In the meeting
Walk through the code line by line without notes, explain alternatives you considered, and offer to modify a function live. Ask which parts looked suspicious and what tool, if any, produced the concern. If a general essay detector was used on code, point to the Turnitin statement and the published studies above, calmly. If the case goes further, our explainer on whether an AI flag goes on your academic record covers what happens next.
Which AI code detectors exist in 2026?
The tools that claim to handle code are mostly code-specific products, not the essay checkers students know. Pangram, Codequiry and Copyleaks each publish code detection features, and GPTZero has offered a code beta. Our Pangram review and Pangram vs GPTZero comparison cover their text models in depth.
Free essay checkers are a poor pre-check for code. A tool like ZeroGPT was built for prose, and our ZeroGPT review explains its limits even there. Running a Python file through one and getting a low score tells you very little about what a code-specific tool or an instructor will conclude. The same caution applies to Copyleaks and Originality.ai scores on mixed text and code.
Frequently asked questions
Can Turnitin detect ChatGPT code?
Turnitin says its AI model does not reliably detect code and only scores prose sentences in long-form writing, with a 300-word prose minimum. Code can still go through similarity checking, which is a separate system.
Does GPTZero work on code?
Its text model performs near chance on code in independent studies, with accuracy around 0.50 on 5,069 Python samples and an AUC of 0.56 in an AAAI 2024 paper. GPTZero has offered a separate code beta, which a vendor comparison credited with catching 81% of known AI files.
Can professors tell if code was written by AI?
Often they look for mismatches: techniques the course has not covered, a style that changes between assignments, or a student who cannot explain their own functions. A conversation about the code usually carries more weight than any score.
Will code comments get me flagged as AI?
Comments alone are weak evidence. One vendor test found Copyleaks false positives skewed toward files with long comments and Javadoc, but good comments are normal practice and removing them proves nothing about authorship.
How do I prove I wrote my code?
Keep a Git history with small, regular commits, save scratch files and test cases, and be ready to explain and modify your code live. That combination is far stronger than any detector result.
Are AI code detectors more accurate on longer code?
Yes, by their own numbers. Pangram reports 96.2% accuracy on code over 40 lines but 89.4% across all lengths, with its miss rate rising to 23.3% on the full set.
Does MOSS detect AI-generated code?
MOSS measures similarity between submissions, not AI authorship. It can still surface AI use indirectly when several students submit near-identical code from the same chatbot answer.
Sources
- Turnitin: AI writing detection capabilities FAQs
- Turnitin: What questions do I ask my student if AI is detected?
- Pan et al. (2024), Assessing AI Detectors in Identifying AI-Generated Code: Implications for Education, ICSE-SEET
- Detecting AI-Generated Code Assignments Using Perplexity of Large Language Models, AAAI 2024
- Sun, Liao and Ma (2026), Trusting AI to detect AI?, Computers & Education 249
- Wang et al. (2023), Evaluating AIGC Detectors on Code Content
- Pangram: How well does Pangram work on AI code? (vendor test)
- Codequiry: AI code detector comparison (vendor-published)
- Copyleaks API release notes
- Stanford MOSS: A System for Detecting Software Similarity
