Gold Penguin Logo with Text

The AI Detection Dilemma: When Students Become Collateral Damage

When AI tools TurnItIn and GPTZero falsely accuse diligent students of cheating, what are they supposed to do? And why are professors taking the guilty-until-proven-innocent approach for a non-deterministic technology that's under a year old.
Updated September 3, 2023
a class of robots taking an exam in a school classroom as 4k digital art
a class of robots taking an exam in a school classroom as 4k digital art

Picture this: It’s the night before the final submission of your thesis that you've poured your heart into. You hit the submit button, expecting to feel a sense of accomplishment. But instead, you receive a notification that your paper has been flagged for AI-plagiarism by an AI tool, and you are now under investigation.

Your heart sinks as you realize that your academic career, which you have tirelessly built, is hanging by a thread because of a tool that's supposed to catch cheaters.

This isn’t a hypothetical scenario for some students; it’s their real-life horror story.

A Nightmare Unfolds

Let us start by looking at the cases of Louise Stivers and William Quarterman, both students at the University of California Davis. They were falsely accused of employing AI chatbots to write their papers, based on the analysis of TurnItIn and GPTZero - AI detection tools used by their institution.

In Louise’s case, Turnitin flagged her paper for plagiarism. This unexpected incident not only caused immense stress but also adversely affected her academic performance and took a toll on her mental health.

Louise, who was a political science student in her last semester, had to take the task of defending herself, all while trying to keep up with her studies and applications to law school.

William Quarterman, on the other hand, was falsely accused of plagiarism by his professor based on the analysis of the AI-detection tool GPTZero, and failed him as a result.

For both students, the initial accusations seemed like an uphill battle. However, their paths crossed and Quarterman, along with his father, was able to provide Louise with much-needed advice and support.

The irony here is that the tools designed to maintain academic integrity have caused innocent students an overwhelming amount of stress and distraction from their actual academic goals.

The Flawed Crusaders of Academic Integrity

AI detection tools like Turnitin and GPTZero are increasingly being used by educators to monitor and check for plagiarism and content generation using AI chatbots. However, as in the cases of Stivers and Quarterman, these tools have shown significant flaws.

OpenAI’s ChatGPT, for instance, has been acknowledged by its own makers to be unreliable in discerning human-written content from AI-generated text.

In another alarming incident at Texas A&M University, Dr. Jared Mumm, an instructor, allegedly used AI detection inaccurately and informed a large portion of his class that they would receive zeros on assignments.

He believed that the assignments were written by an AI chatbot, ChatGPT. Dr. Mumm’s hasty actions without adequate evidence or understanding of the tool’s limitations placed several students’ academic futures in jeopardy.

These incidents reveal the gaps in the deployment and utilization of AI tools in academic settings. Turnitin's AI detection tool, which was still in beta testing during the Stivers incident, claimed a 98% accuracy rate but also acknowledged the presence of false positives. While TurnItIn just released new guidelines on their software, concerns still remain regarding the technology behind what they've made.

The Human Factor: A Missing Link?

An important aspect to consider is the reliance on AI tools as the sole arbiters of academic integrity. The human factor - the critical assessment by educators - is often missing.

Some educators across the world have been relying solely on the AI program's feedback without applying any personal judgment or allowing room for students to present their defense. This is an extremely unfair stance to take on such a new technological revelation.

It is imperative for educators to strike a balance between technology and human discernment. While AI can be an excellent tool for initial screening, it is the responsibility of the educators to ensure fairness and accuracy by critically analyzing any AI-generated results.

Towards a Balanced Approach

What then can be done to prevent further cases of false accusations?

  1. Educating the Educators: Educators should be educated on the limitations of AI detection tools and be encouraged to use them only as preliminary tools and not as definitive proof.
  2. Incorporating Human Judgment: A balanced approach that incorporates human judgment is essential. Educators should critically analyze the results from AI tools and give students the opportunity to present their case.
  3. Transparent Communication: Institutions should communicate transparently about the tools being used, their limitations, and the steps involved in the case of any accusations.
  4. Policy Revisions: Academic institutions should revisit their policies regarding academic integrity, with an emphasis on fairness and giving students an adequate chance to defend themselves.
  5. Feedback Loops: There should be feedback loops to improve AI tools. Users of these tools should be encouraged to report inaccuracies to the developers.

The Road Ahead

As AI technology continues to evolve, it is essential to acknowledge its limitations and approach its deployment in academic settings with caution and sensitivity. The cases of Stivers, Quarterman, and the students of Texas A&M University underscore the need for a more balanced and human-centric approach to upholding academic integrity in the age of AI.

It is time for academic institutions to recognize that in the quest for integrity, the undue reliance on imperfect tools should not jeopardize the futures of the very students they are meant to educate and empower.

Want To Learn Even More?
If you enjoyed this article, subscribe to our free monthly newsletter
where we share tips & tricks on how to use tech & AI to grow and optimize your business, career, and life.
Written by Justin Gluska
Justin is the founder of Gold Penguin, a business technology blog that helps people start, grow, and scale their business using AI. The world is changing and he believes it's best to make use of the new technology that is starting to change the world. If it can help you make more money or save you time, he'll write about it!
Notify of

Inline Feedbacks
View all comments
Join Our Newsletter!
If you enjoyed this article, subscribe to our newsletter where we share tips & tricks on how to make use of some incredible AI tools that you can use to grow and optimize a business