Trained AI vs Teachers
Ravneet Khangura Ravreet Kaur Bhullar
Grade 8
Presentation
Hypothesis
We hypothesize that most teachers will be able to distinguish between student-written and AI-generated essays. Since teachers have extensive experience assessing student writing, we believe they will recognize AI-generated content by analyzing vocabulary, punctuation, tone, and emotional depth. Additionally, we predict that ChatGPT will outperform DeepSeek in mimicking human writing due to its conversational nature and broader application scope.
Research
Background Research 1-
Human-written essays often have three key qualities that artificial intelligence generated one’s lack. First, humans bring personal experiences, emotional depth, and unique perspectives. This makes their writing more authentic, relatable, and engaging. Second, they excel at complex, layered thought, exploring multiple perspectives, addressing subtle contradictions, and delivering deeper meanings that AI may overlook or oversimplify. This may include emotional tones and personal connections and a degree of emphasis on certain parts of the body of text. Lastly, human writers offer creativity, and originality using language in innovative ways and presenting fresh ideas. While AI tends to rely on established patterns and can lack groundbreaking originality. These qualities highlight the cognitive and emotional depth that human writers bring to their work, which AI struggles to replicate fully.
Background Research Number 2-
AI-generated essays have several unique advantages that human-written ones typically don’t. First, AI excels in speed and efficiency, producing essays in a fraction of the time it would take a human writer. In other words, it solely does its job with that of being a machine. It's also capable of quickly gathering information and generating text without needing breaks or extended research periods. Second, AI offers unmatched consistency and objectivity, maintaining a steady tone, style, and structure throughout an essay, unaffected by mood, fatigue, or personal biases. Lastly, AI has access to vast amounts of data and information, instantly incorporating up-to-date facts, statistics, and examples from a broad range of sources, enabling it to pull from a wealth of knowledge much faster than a human could research manually. These capabilities highlight AI’s strengths in speed, consistency, and knowledge access, where human writers may face limitations.
What Is Chat Gpt-
ChatGPT is an advanced language model developed by OpenAI. It's designed to understand and generate human-like text based on the input it receives. It can perform a wide range of tasks, from answering questions and engaging in conversation to writing essays, generating creative content, assisting with coding, and more.
The core technology behind ChatGPT is the GPT (Generative Pre-trained Transformer) architecture, specifically the GPT-4 version which is the version we were interacting with. It's trained using vast amounts of text data available on the internet and what it’s programmed with Thus allowing it to understand context, syntax, and meaning across multiple domains.
What Is DeepSeek-
DeepSeek is an advanced search technology that leverages deep learning and natural language processing (NLP) to enhance search results by understanding the intent behind user queries. Unlike traditional search engines, which focus mainly on keyword matching, DeepSeek uses deep neural networks, such as transformer models (e.g., BERT, GPT), to interpret the meaning and context of queries, providing more accurate and relevant results. This allows it to deliver nuanced answers based on semantic understanding rather than just surface-level keyword matches. Additionally, DeepSeek continuously improves its performance by learning from user interactions and feedback, refining its algorithms to adapt and become more effective over time. This approach can be applied to various search domains, including web search, image recognition, and video search.
What Is Trained AI-
Trained AI refers to an artificial intelligence system that has been taught or "trained" to perform specific tasks or make decisions based on data. This process typically involves feeding large amounts of data into a machine learning model, allowing it to recognize patterns, make predictions, or take actions.Like for essays we would give it data for our essay as in the tone,which vocab level we are at what grade we are in and depending on that it will make a essay close to the way you write.
Variables
- Manipulated Variable: Source of the essays (ChatGPT, DeepSeek, or student-written).
- Controlled Variables: Font style, essay topic, and text format.
- Responding Variable: Teachers’ ability to differentiate between AI-generated and human-written essays.
Procedure
- Write a student-composed essay and generate two AI-written essays using ChatGPT and DeepSeek.
- Print all three essays.
- Label the essays as Essay A, Essay B, and Essay C (without revealing their sources).
- Present the essays to each teacher for evaluation.
- Ask each teacher to determine which essay was student-written and which were AI-generated.
- Record their reasoning and observations.
- Reveal the true authorship of each essay.
- Repeat the process with all five teachers.
- Document the accuracy of the teachers’ guesses and analyze the results.
Observations
We observed that our hypothesis was WRONG because we said that the teachers will be able to guess which one is written by a student and which one is generated by a Trained AI. We had 5 trials and in 4 trials our teacher guessed the essays wrong and mixed them, but teacher 2 had guessed them right. We had thought that the teachers would have guessed them right since they know what vocabulary level and what tone there teachers use. But as our research question states TRAINED AI VS Human Written it was trained to write like us and make up its own moments on the tones which made them think that perhaps that a student wrote the essay. We also thought ChatGPt would be a better AI generator from our experience but most teachers found out Essay C was Ai generated 3 out of 5 teachers thought Essay B was human written but in reality it was written by DEEPSEEK which shows DEEPSEEK is a better human mimicker and teachers can't tell the difference between deepseek and student written essays.
Analysis
Our hypothesis was incorrect—most teachers struggled to distinguish between AI-generated and human-written essays. In four out of five trials, teachers misidentified the essays. However, one teacher (Trial 2) correctly identified all three essays, citing punctuation inconsistencies as key indicators of AI-generated content. Surprisingly, most teachers believed DeepSeek's essay was human-written, suggesting it is better at mimicking human writing than ChatGPT.
Key Findings:
- 4 out of 5 teachers misclassified the essays.
- DeepSeek was more deceptive—many teachers believed its essay was written by a student.
- ChatGPT was more easily identified due to its structured flow and formulaic expressions.
(Results are from all 5 trials)
Conclusion
Our results were teachers couldn't tell the difference between 2 Trained AI Essay and Human Written essay. We thought that teachers would tell because of the tones and the humanized way human talk.We were wrong.As our trials above only 1 out 5 teacher guessed the essay right because teacher 2 said the AI had used weird word choices and has very specific punctuation. So, in conclusion our study demonstrates that trained AI can convincingly replicate human writing, making it difficult for teachers to distinguish between AI and student essay.Contrary to our hypothesis, DeepSeek was more effective at mimicking human writing than ChatGPT. This suggests that AI tools are evolving to become more human-like, making their detection increasingly challenging.
Application
The result of this experiment would be useful so next time you or somebody needs to know whether teachers can tell between trained AI generated essay or human written essays.Or if you need to know information as into how teachers can tell between AI and Human Written and how TRAINED AI works and to see which AI is better. Many people ask this question so we came up with an answer. Your science project can be the answer to somebody new question.
Sources Of Error
It was hard for us because one of us where in India so we could not communicate properly. We also could not communicate when we had a problem or question because of the time difference. So next time we would probably try to think of the topic and get the experiment out of the way before doing anything else. We also didn't realize how hard it would be to research about chatGPT and DEEPSEEK and how we had to play around with them and train them.We also had only 3 trials at first but last minute decided to do 5 to make the project more accurate so next time we would have more time to do the 5 trials and add them into our slides vs doing it last minute and stressing about it.
Citations
https://zapier.com/blog/how-does-chatgpt-work/
https://www.sciencedirect.com/science/article/pii/S2666920X24000109
Acknowledgement
We would like to acknowledge our teachers for helping us. They took the time out of their day to read our essays. We are thankful for our homeroom teacher to guide us and help fix our errors. We would also like to thank our friends for helping us with things we didn't understand. We would like to thank each other for being so cooperative and working together. Lastly, we would like to to thank you judges for listening to our presentation.