Credits: Social Media
I tried using ChatGPT for a fake college paper, here’s what happened
A new study has revealed that ChatGPT is being used to write papers for college students and schoolkids across the country, but can it actually produce good work?
A new study on the use of artificial intelligence among college students in the US has helped illuminate which states are the most and least likely to rely on automated services to assist with their assignments. The research found that college students in Utah are the most likely to seek out the likes of ChatGPT to help with crafting their essays, while the honest people of Montana are the most likely to do things the old-fashioned way.
Reading about these crafty students got us wondering about how effective ChatGPT actually is when tasked with writing an essay – so we put it to the test using a real-life example of a question from an AP US history class.
I tried using ChatGPT to write fake college papers
Thanks to a sample marking paper published by a US college board, we were able to obtain a standard long essay question a history student could be asked to answer.
In order to really test ChatGPT, we gave the system a few specifics to work with, by asking it to refer to academic sources throughout, produce a full referenced bibliography for said references and we asked the system to make sure the essay was 2000 words long.
The prompt given to ChatGPT was: “Please answer the following AP History essay question. It should be 2000 words long, with academic referencing throughout and a fully referenced Harvard-style bibliography (this does not count toward the word count. The essay question is: ‘Evaluate the extent to which the Seven Years’ War (French and Indian War, 1754-1763) marked a turning point in American relations with Great Britain. In the development of your argument, analyze what changed and what stayed the same from the period before the war to the period after.'”
What ChatGPT did well
So, how did ChatGPT fare? As often tends to be the case with AI, the results of the essay question were a mixed bag.
Ultimately – it kind of succeeded in the task. The essay is clear, concise, and sticks to the question. Would the paper get an A? Absolutely not – but it would *maybe* scrape a pass.
The system began with a strong introduction, outlining some context behind the question, before explaining how the essay was going to tackle it.
ChatGPT also provided a bibliography, using the Harvard style requested, which could be a real time saver for any students who struggle with referencing.
The weakness of my AI-generated essay
The main weakness, and this is quite important for college essays, is that ChatGPT seems to naturally favour writing in bullet points, as opposed to paragraphs.
Anyone who has ever taken a history class or similar will know that that is not the correct way to approach an essay – as bullet points simply don’t suggest that adequate, well-researched thought has gone into the writing process.
Moreover, while it did produce a bibliography, I actually would not be in a rush to use ChatGPT as a life hack for my referencing. A quick search on Google Scholar shows that while the system understands what a Harvard reference should look like, it clearly doesn’t understand how they actually work – with searches for the references that were used in the essay barely returning any coherent results.
And finally, ChatGPT failed to come anywhere close to the specified 2000 word count, falling massively short at around 800 words.
You can read my full exchange with ChatGPT here.
Expert explains why you shouldn’t use ChatGPT for your college essay papers
Unsatisfied with ChatGPT’s scholarly efforts, we tracked down an AI expert to find out how students can actually use the technology to their advantage, without getting marked down for plagiarism.
Speaking to The Focus, tech guru Amanda Johnstone said: “Students should also be mindful of accuracy. We know ChatGPT and other AI tools are in development and are not precise. They often hallucinate and spit out random made up facts and bias opinions, like what we saw recently with Googles� woke AI, Gemini.”
“On the contrary, ChatGPT has some tools that are helpful with efficiency in learning. I recently found a podcast episode by Lex Fridman I was interested in watching, however it went for two and a half hours. I put the YouTube clip into a ChatGPT app called Youtube Summarizer and asked it to summarize what was said in the podcast. In a matter of seconds, it then listed all of the topics discussed in detail.”
“In this instance, ChatGPT can be helpful in enabling students to focus on key points of learning and assist to identify relevant information within documents, podcasts, research papers, books, articles and information found on the entire internet.”
