Originality.ai Review – The Harsh GPT-4 AI Detection Tool (& How Well It Works)

If you hire writers to produce articles, you probably want to ensure they're not producing boilerplate, AI-spammy content. At least I do, and it's not easy to tell. Same thing with schools — as an educator, you expect students to be on their best behavior and not use ChatGPT for homework.

Personally, I don't really pay much attention to AI detection tools unless it says something is 100% AI (which doesn't happen too much). These detectors are great to check if someone just copied ChatGPT and pasted it right into a document. Other than that, make sure to take everything with a grain of salt as they're honestly not very accurate.

I've been using Originality since it was released and used to be super impressed with it.

I think it's very harsh, tries to be fair, but ultimately really just gives you a hint of where what you're looking at came from.

What Is Originality & What Does It Include?

Working as both a comprehensive plagiarism & AI content detection tool, Originality allows you to easily check the uniqueness of text by comparing it against public-facing web content all while running it through an AI checker.Originality was pretty much the first public AI detector (behind GLTR, which was super old). It can help detect GPT-4 writing & also has a writing visualizer tool (which is the only thing I religiously use from them after OpenAI's recent announcement + my own research on how AI detection actually works)

AI detectors were getting better for quite some time until AI humanizers like Undetectable AI also got better. It's since been a cat-and-mouse game, but for the most part, AI detection tools are winning.

Regardless of the type of writing you test with Originality, you'll see a percentage determining the odds that text was written with AI (other detectors like Sapling and Winston call this “AI likelihood score” or something similar). The tool will also provide the link to pages it finds plagiarism violations on.

Who Should Use Originality?

The best use case for Originality, in my honest opinion, are publishers looking to check their writers. Anyone looking to hire talent that you don't fully trust yet could benefit from having their writing get put through Originality.

Students

Students can also use Originality to check their own writing if they used ChatGPT to outline or draft content. It's interesting because I actually wouldn't recommend teachers to use Originality. There's just too much risk in using AI detection in an educational setting.

That said, even students who didn’t use AI could benefit from this tool since false positives exist. It’s best to test your writing on every detector you could get your hands on to prevent issues down the line.

Journalists/Editors

If you frequently edit content that you don't necessarily write, Originality could lift some stressful weight off your back. Being able to quickly run a quality assurance check across the content you work with could potentially save you hours validating content. This works well for plagiarism checking on top of AI writing.

Educators are actually advised against using Originality as the risk of falsely flagging a student is generally waaay worse than for other use cases. It also doesn't help that OpenAI said detectors don't work. So take all of this with a grain of salt. Although Originality did have a response to that if you want to check it out.

How Accurate Is It?

Originality claims they have a 99% accuracy rate on GPT-4o, ChatGPT (GPT-4 powered) and Claude 3.

That said, I always try to stay vigilant in mentioning how AI detection is not something explicitly measurable. This entire process works on predictions, so you really should only use these with a grain of salt.

If you are trying to determine if a writer used AI, this tool would be best used to check content over a period of time (10 articles from the same author). This would give you a better indication than a 1-off analysis.

Originality seems to result in false positives more than false negatives (it will sometimes detect human content as being AI-written, but AI-written content usually comes back as AI). After testing academic content to complex ChatGPT responses, I'd say ...

You can pretty easily detect ChatGPT content!

I tested high school to graduate level content and got 90%+ original reviews. Academic content is a lot more complex and nuanced, so it makes a lot of sense seeing results like these:

We also included Originality AI in our latest intensive testing of AI writing detection tools which included heavy hitters like Copyleaks, Sapling, Winston, and more. Full disclosure: the results were, let’s just say… not ideal. It came 5th out of 8 detectors in true positive tests (68.83%) and 4th in false positive tests (73.8%).

So, is it accurate? Somewhat, but in 2024, there are tools like Winston and Copyleaks that are more consistent.

Highlightable Features

Originality used to be at the forefront of AI detection, when things were still new in the AI world. But now, there are too many detectors that are just as good, if not better. Still, it has a lot to offer when it comes to features. Some of these are:

Check For AI Content & Plagiarism Line By Line

The biggest feature is AI detection. According to case studies, Originality can detect the AI on all text generated by GPT-4, GPT-3, GPT-3.5, and ChatGPT an average of 99.41%. GPT-3 had the highest maximum average score of 99.95%, followed by GPT-3.5 at 99.65%, and ChatGPT coming in last at 98.65%.

But again, take this with a grain of salt, since these “studies” can somewhat be influenced, especially when the company behind the product is the same one who did the study.

But going back to features. Not only will Originality return the percent likely writing has been created with AI, but you can also view exactly which lines come back the most suspicious.

You can see the AI predicted text is highlighted in red, while the human estimate is shown in green.

This also applies to plagiarized text. Anything suspicious will be highlighted. As you can see, it found the exact text from this review! Pretty impressive. It seems to be crawling web content quite frequently (or even live for that matter).

Content Readability Checker

Originality also has a readability checker which aims to give you more information about your content score, stats, reading level, etc. This tool is completely free and is great for testing the level of writing you or your contractors have been producing.

You'll see text highlighted in different colors based on their syllable count. It looks like I should revisit the ones in red and update a few things... You can also see a grade level based on the Flesch-Kincaid Grade Level scale. My sample was written at a 9th grade level, so if I wanted to target the average American grade level I would reduce certain words and syntax structures to fit a 7-10th grade level.

Fast Chrome Extension

The team recently released a Chrome Extension that lets you 1-click check content for both plagiarism and AI across any website. The extension has features including:

1. One-Click AI Detection inside Google Docs
2. Watch the Writer Write (it's actually a little creepy)
3. See Who Contributed to an Article
4. See a Table Showing Writing Sessions and Characters Added
5. See a Graph Showing Characters Added vs Time

The best feature about Google Doc integration is the multiple writer support. Now you can easily see if a writer:

• Pasted in content from ChatGPT or another source and only made minor edits
• Had other writers work on the articles specifically assigned to them

The extension is specifically useful for those who work with writing as part of their daily workflow.

No More False Positives (For Writers)

If you are a writer and are concerned about false positives or the need to defend that your work was actually created by you, this can help. This update provides you with the ability to show that you created the document yourself.

No More Uncertainty with Clients (For Content Marketing Agencies)

You can arm yourself with the ability to ensure you are only getting articles that have been human created. There's no more accusations of writing coming from a robot (not that it would be likely without proof anyways).

Confidence In What You Are Publishing (For Web Publishers)

If you want to be sure what you're putting out into the world doesn't seem like AI – here's your confidence score. You don't need to stress about having a site getting picked up as low-quality, AI content.

Scan Entire Websites

This is super, super cool. This feature was released early in 2024 and allows you to run full content audits scanning an entire site for AI. I've tested this out on our blog and it shows the word count & originality percentage the page ranks by. Hopefully in the future plagiarism support will be added, as this would be a wonderful tool for detecting the authenticity of a website’s content origin. 

This does use a lot of credits, it used about 600 and scanned 142 pages. I found the $6 equivalent to scan the site to be a pretty reasonable deal. I don't see myself needing to scan an entire site more than once a year. Once they add plagiarism checks, it'll be great.

Pros & Cons

Originality Pricing

The tool costs $0.01 per 100 words to scan (equal to 1 credit). You also pay as you go, there's no obnoxious or predatory recurring pricing model. When you need to buy more credits, you just need to buy more credits! When checking your content you can decide whether to check for plagiarism, AI, or both (using 2 credits per 100 words). 

Current credit purchase options are as follows:

• $60 as a one-time payment: 6,000 credits with AI detection, Plagiarism, and Readability scans.
• $19.95 per month: 1,000 credits with the complete package including full site scan, file upload and access to future features. 
• $179 per month: 15,000 credits, inclusive of everything plus priority support, API access and 365 day scan history.

Where Can They Improve?

It’s already 2024 and we’re way into  the late stage of development. It works and that's ultimately the most important thing that matters — but there are a few tweaks that we can file on the “nice to have” pile..

Firstly, the UI/UX is a bit wonky — an issue that exists even now in 2024. The recent content scans could be organized in a better format & the AI detection score screen could look a bit prettier. When I first started using it, there was a slight learning curve where I needed to understand how to check something as fast as I could.

But that’s just a nitpick, what I really want is a strong improvement of their accuracy. We live in an age where students can’t afford to be flagged as AI if they’re innocent. Sure, it works, but only to a certain extent. As long as this issue isn’t minimized, I hesitate to recommend Originality AI to schools.

Meet The Founder

We sat down with the founder of Originality, Jon Gillham, to talk about how & why he created the tool. Hear about his future plans, goals, and what he has in store:

Should You Use Originality?

Everything said, I still would recommend trying it! I use it almost daily. I don't use the plagiarism checker as much as I run AI tests, but I've seen great success with both. The only thing to keep in mind is to never put your full reliance on a single tool for testing content (or anything for that matter). Also, if something does get flagged as AI, take another look into the writing to see if you can pick up on any patterns yourself. 

Originality is a great tool to have in your content editing arsenal, and it's pretty affordable! Whether you’re a writer, academic, or web developer, Originality is worth a try.

Here's what it had to say about this article: