Gold Penguin Logo with Text

Is GPTZero a Game Changer for AI Recognition? A Review

GPTZero can help detect whether text is written by ChatGPT or not. Developed over an explosion of fears in academia, can this tool really help predict AI? And how well does it work?
Updated August 24, 2023
An expressive oil painting of a book with a pencil on it, sitting on a table, depicted as an explosion of nebula explosion
An expressive oil painting of a book with a pencil on it, sitting on a table, depicted as an explosion of nebula explosion

A few months ago, a tool called GPTZero blew up seemingly overnight on Twitter. A Princeton student by the name of Edward Tian created a tool to help detect whether a human or AI wrote something.

It works similar to other online testing tools like Originality, TurnItIn, and PassedAI, but specifically seems to break text down into a more detailed analysis and scrutinize it more before classifying it as AI-written or not.

I took a deep dive into the tool to see how it works and what you could use it for.

If you're not familiar with how AI detection works, it works by analyzing text for patterns to determine how predictable writing is. The easier the text is to predict by an AI, the higher chance it overlaps with actual AI-produced content.

These tools all work by reverse engineering text prompts to determine if the AI can recreate what was entered (and with what accuracy).

What is GPTZero?

GPTZero was launched on January 3, 2023, by Edward Tian, who created the tool as his thesis project. The freemium AI-detection tool had a whopping 1.2 million users after 5 months and 2.5 million users as of posting. But it’s not because Tian was a 22-year-old computer science senior at Princeton. It’s because of the mission & purpose of how the tool detects AI content. 

The app’s name and the “Humans Deserve the Truth” tagline speak for themselves. So yes, GPTZero aims to fight the misuse of ChatGPT by detecting content written by AI tools such as ChatGPT, GPT-3, GPT-4, and LlaMA, a large language model (LLM) by Meta AI. They've also raised $3.5 million in capital funding.

Who is GPTZero for?

GPTZero was mainly designed for educators. It was created specifically to evaluate the work of students. Their model also promises to avoid false positives, which means that it is most likely to release accurate scores. Other professionals such as publishers, editors, and those who hire writers can also use this tool (including students).

How Does GPTZero Work?

Technically, GPTZero has only one feature – to detect AI content. But as I said earlier, it’s the way it releases the details of the result. You use GPTZero by pasting text into the paragraph box and submitting it for detection. It analyzes text based on 2 characteristics: "perplexity" and "burstiness"

Perplexity – How random your text is based on predictability. The model runs text through GPT-2 (345 million parameters). The range of perplexity is not quite known, but values closer to 0 are very likely to be artificially generated, while those closer to 100 has a higher chance of being human-written.

Burstiness – The occurrence of non-common items appearing in random clusters over time (aka creative variability). Perplexity is uniformly distributed and consistently low for machine-generated content. As humans naturally include more variability in their writing, you'll notice it has a lower chance of being predicted by patterns.

Testing GPTZero with ChatGPT

According to the company, GPTZero was trained with an equal balance of human and AI-written articles. Moreover, their tool is designed to classify 99% of the human-written articles correctly, and 85% of the AI-generated articles correctly. 

To put GPTZero to the test, I put content from GPT 4, GPT-3, and my own sentences without AI assistance on the tool. You can see the results and scores in the summary table below.

GPT-4GPT-4GPT-3GPT-3GPT-4Human
Number of Characters1,7962,9212,2184,2964,4371,071
Perplexity score 138.113 123.64351.55647.53364.417172.077
Burstiness score 212.272168.02728.55831.11849.166178.575
ResultMost likely human writtenMost likely human writtenMost likely human writtenMay include parts written by AIMost likely human writtenMost likely human written

I'll go ahead and test GPTZero with an example I asked ChatGPT. This seems fairly predictable but also a tad bit creative. Let's see how it does.

In short, GPTZero does a decent job at detecting short-form AI content, but does a lot better at detect long-form AI writing. GPTZero concluded that the short texts from GPT-4, GPT-3, and human were likely to be written by a human. When I increased the number of characters from GPT-3 to 4,296, GPTZero detected it as partly written by AI.

ChatGPT response describing the meaning of life is about personal fulfillment

GPTZero resulted in a text perplexity of 12. A very low number – indicating a higher probability of being generated by AI (this is correct). Regarding burstiness, we got a score of 45. After scrolling down the page you can click "Get GPTZero Result" and you'll get a final score and a predictor. This is what the above paragraph received:

GPTZero result for AI generated content (it detected AI)

Ok so it looks like it did a good job. I'm going to test an academic thesis paragraph now & then I'll test something I wrote in a past blog. I'm assuming the thesis will be confidently human-generated, and my writing be somewhere in the middle. Here's the thesis:

GPTZero Thesis result returning 50. More likely text to be produced by a human

With a huge sentence perplexity (especially among the average sentence perplexity), GPTZero predicted a very low chance of this being AI-generated, showing high signs of it being human-produced:

GPTZero showing high sentence perplexity and burtiness, very likely human-generated content

Now for my personal text from a recent blog on detecting ChatGPT (ironically a funny article to use for this example). Just like predicted, we met in the middle between AI-generated content & a professional academic thesis abstract – but thankfully I still came out as human-generated 😎

Human generated text result with GPTZero result.

Support and Community

GPTZero has a Facebook community called GPTZero Educators, which now has more than 4.3K members. But as expected, most of them are in the education industry. The topics are usually about how to help students avoid cheating via ChatGPT. If you need technical support, you can send them a message via their contact page. GPTZero also accepts requests for new features.

Pricing

GPTZero has a free version (GPTZero Classic), which you can use without signing up. It has a limit of 5,000 characters (about 700-1,200 words) per input/document. Aside from the character limit, the only difference between the free and paid plans is that the latter has a “better” detector threshold designed for educators. It seems like they're putting most of their effort into these premium models.

GPTZero ClassicGPTZero EducatorGPTZero Pro
PriceFree$9.99/month$19.99/month
Character limit per document5,00050,00050,000
Upload files limit per batch3 UnlimitedUnlimited
Number of words per monthUnlimited1 million2 million
AI detection modelFreeFinetunedPremium with high limits
API Access*NoneNoneNone

Pros and Cons

PROS

CONS

  • Free version (no sign-up required)

  • Very easy to use

  • Highlights AI content

  • Batch files upload

  • Chrome Extension

  • Canvas Integration

  • Dark/Light mode

  • MS Word Extension

  • Cannot save results

  • Paid API Key (still in beta)

  • Supports English only

  • No money-back guarantee

Is GPTZero Groundbreaking for AI Detection?

I think it's a great tool to give insight, but it's clearly not polished enough for everyday use. It's extremely impressive that a 22 year old created an amazing company and vows to only detect things as AI that really are. The issue in AI detection isn't finding AI content, it's more about not flagging people as using AI when they weren't.

Compared to other tools on the market, I think GPTZero has great potential to keep growing. Especially within the education industry. It's definitely worth trying and has a very promising and authentic team beside the product. If you've used the tool as an educator, I’d love to hear your experience in the comments below!

Want To Learn Even More?
If you enjoyed this article, subscribe to our free monthly newsletter
where we share tips & tricks on how to use tech & AI to grow and optimize your business, career, and life.
Written by Justin Gluska
Justin is the founder of Gold Penguin, a business technology blog that helps people start, grow, and scale their business using AI. The world is changing and he believes it's best to make use of the new technology that is starting to change the world. If it can help you make more money or save you time, he'll write about it!
Subscribe
Notify of
guest

3 Comments
Most Voted
Newest Oldest
Inline Feedbacks
View all comments
Join Our Newsletter!
If you enjoyed this article, subscribe to our newsletter where we share tips & tricks on how to make use of some incredible AI tools that you can use to grow and optimize a business
magnifiercross