With the recent surge in AI-made content, people across the world are having existential crises trying to figure out what’s real and what’s fake. I mean... have you seen deepfakes recently?
In the writing realm, ChatGPT has exploded since it's release, months hitting a million users within the first week of its launch and currently boasting over 100 million.
After people wrapped their head around what this actually is, it left tons of people trying to figure out how to reverse-engineer artificially-produced writing.
What is AI writing? And how do you detect it? Reverse engineering words on a screen require a little bit of backward thinking.
You could use AI detection tools to produce a bit of quantifiable evidence to check if ones writing was made with ChatGPT, but AI detection leaves no actual watermark.
Trying to detect AI writing requires a bit of critical thinking, detection tools, and ultimately trust. Here's a few tools and tips you can follow to help figure out what you're looking at.
What is ChatGPT?
ChatGPT is a casual-speaking, large language model created by OpenAI – the team behind DALL-E 2. You might've heard about it on TikTok, Twitter, or a few months ago when the NYC public school system banned the entire thing.
The goal of ChatGPT is to combine AI with casual conversation. You simply ask it any question you want and you'll get a customized response. You can ask it things like:
- Can you write me an essay about the industrial revolution, in the style of a southern boy in the 1900s?
- What is the meaning of life and how can we find purpose in our existence?
- Write me a workout plan for the next 6 months that focus heavily on legs. I weigh 180 lbs and am 5'11.
- Can you write me marketing copy for a company that sells glow-in-the-dark crayons?
It's kind of crazy. Here's an example:
How Does Artificial Intelligence Work?
AI tools are like computer brains trained using heaps of data, helping them spot patterns in text after reading countless articles.
Imagine AI as a computer buddy named Robby who wants to learn chess. You teach him the basics, then he plays against others. The more he plays, the better he gets.
What makes Robby smart? Algorithms – they're like instructions for decision-making. A special type called "machine learning" lets Robby improve over time. He uses "training data," which are examples of good chess moves, to become better.
The more examples Robby sees, the smarter his moves become.
How Do You Detect ChatGPT?
Teachers, students, business owners, freelance writers, and pretty much anyone else in between has been trying to figure out if a piece of writing they've come across (or wrote) has been created by ChatGPT.
It's not easy to figure out, and it's also not 100% accurate. You're looking at words on a computer screen, and words leave no trail.
The issue is ChatGPT [currently] produces no watermark on the content it produces. If you ask it a question, you could simply copy and paste it & do as you wish. It's not like plagiarism which has actual sources you can the backtrack writing to. I'm not sure if OpenAI will even be able to watermark ChatGPT, we'll have to wait..
Some of the best AI detection tools for teachers don't even fully rely on scanning for AI. They check plagiarism, AI detection, and replay the writing to see if a lot of copy/pasting was involved. Even though tools are getting better by the day, it's all still based on prediction.
But these tools do showcase a fairly solid way to help predict AI-writing, especially writing produced ChatGPT, and that's by looking really close into patterns.
Once you have an essay written (like a 1200-word piece about the industrial revolution) how do you actually check where it came from?
How AI Detection Works
Tools that predict AI analyze how likely each word would be predicted next, based on the previous context of words to the left.
We wrote a more in-depth explanation if you want to take a deep dive.
AI-generated articles tend to have a certain level of polish and consistency that is often associated with perfection. They are designed to adhere to grammatical rules and maintain a structured flow. However, this perfection often comes at the cost of creativity and the human touch. Humans make mistakes, AI writing is engineered to be objectively "perfect."
AI operates fundamentally on the basis of pattern recognition. It doesn’t “think” or “feel” in the way humans do but instead processes and reproduces patterns from vast datasets.
For a practical example, consider the process of predictive text. If you were to type "The worst part of my day is when I wake up for _", AI would attempt to complete the sentence based on the patterns it recognizes from its training data.
Given the context, it might predict the word "work" to follow, drawing from countless instances where such sequences appear in its data. If "work" is the highest likelihood of a word that should occur at that position, it's going to write it.
Spread this concept across a longer piece of text, like a paragraph or an article, and you begin to see distinct patterns.
Highly predictable sequences of text are indicative of AI generation, simply because algorithms thrive on consistency. Meanwhile, less predictable, more creative sequences suggest a human touch, characterized by our inherent creativity and the nuances of human experience.
Remember, this is all about patterns. AI isn't inherently smart – it just recognizes and reproduces existing patterns based on an unfathomable amount of data. And it does this really, really well.
The Best ChatGPT AI Writing Detection Tools
All of the following tools will use some derivation of this technology combined with other algorithms and language processing models reverse-engineer the writing.
So much really goes into checking for AI writing, but these online tools can help you get a decent understanding of if ChatGPT really did write what you're looking at.
Please keep in mind there is no mathematically definite way to confirm AI-generated writing and that all of these tools work on predictions, but they are decent indicators.
Here are my top picks after analyzing detectors for months on end:
Tool 1: Undetectable.AI
I really like Undetectable because they made a tool that analyzes every other detector you'll run into online. They created their own detectors by sampling the models of a bunch of existing ones. It's a great way to combine some of the most popular tools into one.
They run your writing through some of the most popular AI detection tools like Content At Scale and GPTZero (which we'll get into later) by analyzing how those tools would classify your writing if you tested them individually. They base this on thousands of writing samples.
I've noticed both Undetectable and Content at Scale are fairly conservative with their predictions & try not to false flag anyone. But always take these results with a grain of salt.
I also like the fact that Undetectable tries to be hesitant about writing. I don't think you should classify AI writing unless you're fairly confident about it.
You open up massive problems like what's happening with schools that implemented TurnItIn's very own detection tools. Students are getting written up for using AI when they haven't. It's failing students, impacting their chances of getting into college, and creating a bitter relationship between students and teachers. All based on something you can't truly "prove." AI detection should be speculative, but that's a story for another day.
To use Undetectable, paste a sample of your writing into the tool, agree with the terms of service, then click "check for AI"
In a few seconds you'll see the result of your writing get thrown through a bunch of other AI detection models and predict the likelihood that you'd get picked up by them. It's that easy:
Tool 2: CopyLeaks
CopyLeaks is probably the most straightforward & fairly strict AI detection tool. It detects ChatGPT-generated writing regardless of which model somebody is using. It's quick, free, and I'd consider it to be reliable.
They also are pretty good at figuring text that's in the middle (writing that is a mix of ChatGPT & human writing).
Paste your writing, press check, give it a few seconds, and you'll have a result. You can also hover over paragraphs and individual lines of writing to get a individual percentage if you want to break it down into smaller bits.
I wouldn't really recommend doing this though, because it's a lot harder (almost impossible honestly) to determine ChatGPT from a single sentence.
Let it test the whole sample and judge things based on that. Here's what it looks like:
Tool 3: Originality AI
While CopyLeaks is meant for testing quick content (and it does it pretty well), Originality does have the tendency to over-predict at times, which I don't like.
I've submitted my writing a few times and it's told me it was AI-written. So be cautious and try a bunch of samples from the same person to get a really reliable result.
Originality is meant for industry-level & bulk testing. It works the same as all the others tools: Copy & paste suspected ChatGPT content and let it do the rest of the work for you. If you want to read an in-depth review of how Originality works, we broke it down in this review.
Originality will display a percentage showing if a text is more or less likely to have been written with AI. Remember to do more testing than just submitting a single writing sample & basing an entire decision on that.
Also, with all types of AI detectors, the more writing Originality has to work with and analyze, the more reliable of a prediction you'll have. It's safer to say 10 AI-written verdicts on the same writer are a lot more reliable of a prediction than a single declaration.
I'd say my favorite feature in Originality is the writing visualizer.
It works by re-typing the article directly in front of you.
For it to work, you need to have access to a Google doc with editing permissions enabled. Simply install the chrome extension & visualize the writing.
You'll see the article get crafted back right in front of you. SUPER useful if you want to try to catch people copy and pasting ChatGPT content directly into a paper. There's no other tool that comes as close to Originality in that regard – & I use it all the time.
Tool 4: Content At Scale
The next tool you could use to check for ChatGPT is using Content at Scale's AI Detector.
I've been using this tool from the very day it was released and love it because it tends to not "over-predict" writing. It's also completely free.
I'd rather let some AI get through if it meant not flagging someone on accident. If you want a more strict tool, Originality is probably the better choice.
Here's an example of a snippet I generated using ChatGPT & pasted directly into their tool:
Paste it in the tool, wait about 5-10 seconds, and you'll get a line-by-line breakdown on top of a percentage about if it was written by ChatGPT.
Once submitted, you'll see this content was very likely to be written by AI. The content was easily predictable, followed a lot of patterns, and wrote probable content.
The more content you test from the same author (and length) will increase the likelihood of these prediction results being reliable. I'd still recommend taking these with a grain of salt.
Content at Scale will also highlight each line it believes to be written by ChatGPT (the orange text on the right). In this case, it was the entire piece! (which is true in this example)
Here's an example of what my writing from the last paragraph outputted when using C@S:
I'm glad I'm a human... ha!
If you need something more strict though, maybe something a little harder on both ChatGPT writing & plagiarism, look into Originality.
Tool 5: Gold Penguin's AI Detector
Yes... we actually made on ourselves! After testing these tools throughout the last few months it was only a matter of time. I decided to list our AI writing detection tool last because it generally under-predicts AI writing.
We created it to minimize the risk of falsely flagging someone as using ChatGPT to write something. Some of the other detectors have been caught flagging the constitution of the United States as being AI-written! That's insane!
It's free & extremely simple to use. Just paste the text into the input box and check for AI:
It Started with GLTR
You used to be able to run any text through GLTR. It was the start of AI detection before that really become a thing. It was trained on GPT-2 years ago. It worked as a free tool and to see a "heatmap" type of diagram showing AI predictability.
Here's AI content compared to a professional academic article. The darker words mean the more predicable they are:
While it doesn't work anymore (presumably because it wasn't reliable as AI writing got better) it helps visually explain what AI detection really is.
Can Teachers and Professors Detect ChatGPT?
They can't prove it, but can get a fairly decent inclination, especially based on repeated samples of writing. The most reliable way would be testing multiple samples of a students writing through different AI detectors & writing visualizers, but there's no foolproof method.
You're looking at words on a screen when you really think about it. The only way teachers, professors, and other educators can really get around the ChatGPT craze is to have students complete writings on paper, in class.
Does ChatGPT Have a Watermark?
Not yet! Rumors are out that OpenAI is planning on are adding some kind soon though. A watermark is added by placing a secret code, called a token, that is unique to the GPT-4 (ChatGPT) model somewhere in the generated text.
A token is like a building block of the text, it can be a word, a punctuation mark, or even a part of a word. By adding these tokens to the text, the output will still be coherent and grammatically correct, but it will be unique to the GPT-4 model, and can be used to identify if something was written by it.
This is similar to how a watermark is added to a digital image, it is not visible to the naked eye, but it can be used to identify the source of the image. This is done by analyzing the text for the presence of the unique tokens, which are known only to OpenAI.
If these tokens are found in the text, it can be determined that the text was likely generated by GPT-4. This process is similar to how a digital image can be analyzed for the presence of a watermark to determine its source.
Whether there will be a public-facing access to this tool is currently unknown. It might become specifically available to educators or academic administrators who would need to test content to determine its true integrity.
What's Next?
It's hard to tell, I don't really know. Of course there are still many things ChatGPT cannot accomplish, but it seems overnight we've upgraded societies technical skills. The education or professional industry won't be ruined because of this – but it'll certainly cause a disruption.
Remember to take any result with a grain of salt, as no tool can 100% determine if AI was used to generate if ChatGPT wrote something. Use your best judgment and always use more than one route when determining the likelihood of something being written by ChatGPT. Also, this article wasn't pro or anti-AI, but rather an educational piece describing a few ways to detect it. The next few years will certainly be exciting!
