Bermuda Post

Thursday, Jun 01, 2023

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."
Researchers found students to have fared better at accounting exams than ChatGPT, OpenAI's chatbot product.

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."

The researchers from Brigham Young University (BYU), US, and 186 other universities wanted to know how OpenAI's technology would fare on accounting exams. They have published their findings in the journal Issues in Accounting Education.

In the researchers' accounting exam, students scored an overall average of 76.7 per cent, compared to ChatGPT's score of 47.4 per cent.

While in 11.3 per cent of the questions, ChatGPT was found to score higher than the student average, doing particularly well on accounting information systems (AIS) and auditing, the AI bot was found to perform worse on tax, financial, and managerial assessments. Researchers think this could possibly be because ChatGPT struggled with the mathematical processes required for the latter type.

The AI bot, which uses machine learning to generate natural language text, was further found to do better on true/false questions (68.7 per cent correct) and multiple-choice questions (59.5 per cent), but struggled with short-answer questions (between 28.7 and 39.1 per cent).

In general, the researchers said that higher-order questions were harder for ChatGPT to answer. In fact, sometimes ChatGPT was found to provide authoritative written descriptions for incorrect answers, or answer the same question different ways.

They also found that ChatGPT often provided explanations for its answers, even if they were incorrect. Other times, it went on to select the wrong multiple-choice answer, despite providing accurate descriptions.

Researchers importantly noted that ChatGPT sometimes made up facts. For example, when providing a reference, it generated a real-looking reference that was completely fabricated. The work and sometimes the authors did not even exist.

The bot was seen to also make nonsensical mathematical errors such as adding two numbers in a subtraction problem, or dividing numbers incorrectly.

Wanting to add to the intense ongoing debate about how how models like ChatGPT should factor into education, lead study author David Wood, a BYU professor of accounting, decided to recruit as many professors as possible to see how the AI fared against actual university accounting students.

His co-author recruiting pitch on social media exploded: 327 co-authors from 186 educational institutions in 14 countries participated in the research, contributing 25,181 classroom accounting exam questions.

They also recruited undergraduate BYU students to feed another 2,268 textbook test bank questions to ChatGPT. The questions covered AIS, auditing, financial accounting, managerial accounting and tax, and varied in difficulty and type (true/false, multiple choice, short answer).
AI Disclaimer: An advanced artificial intelligence (AI) system generated the content of this page on its own. This innovative technology conducts extensive research from a variety of reliable sources, performs rigorous fact-checking and verification, cleans up and balances biased or manipulated content, and presents a minimal factual summary that is just enough yet essential for you to function as an informed and educated citizen. Please keep in mind, however, that this system is an evolving technology, and as a result, the article may contain accidental inaccuracies or errors. We urge you to help us improve our site by reporting any inaccuracies you find using the "Contact Us" link at the bottom of this page. Your helpful feedback helps us improve our system and deliver more precise content. When you find an article of interest here, please look for the full and extensive coverage of this topic in traditional news sources, as they are written by professional journalists that we try to support, not replace. We appreciate your understanding and assistance.
Newsletter

Related Articles

Bermuda Post
Close
0:00
0:00
America's Got Talent…
Nvidia Joins Tech Giants as First Chipmaker to Reach $1 Trillion Valuation
Drone Attack on Moscow's Wealthiest Neighborhoods Suspected to be Launched by Ukraine
Elon Musk Meets Chinese Finance Minister in Beijing
AI ‘extinction’ should be same priority as nuclear war – experts
Prominent Hacker Forum RaidForums Suffers Substantial Data Breach
Nvidia CEO Huang says firms, individuals without AI expertise will be left behind
WPP Revolutionizes Advertising with NVIDIA's AI Powerhouse
Two US Employees Fired For Chasing Robbers Out Of Store As They Broke ''Company Policy''
If you donated to BLM, you got played
Pfizer, the EU, and disappearing ink - Smoke, Mirrors, and the Billion-Dose Pfizer Vaccine Deal: EU's 'Open Secret
Actor Tom Hanks told Harvard University graduates to be superheroes in their defense of truth and American ideals, and to resist those who twist the truth for their own gain
The Sussexes' Royal Rebound: Could Harry and Meghan Markle Return to the UK?
A provocative study suggests: Left-Wing Extremism and its Unsettling Connection to Psychopathy and Narcissism
France Arrests 10 on Suspicion of Failing to Respond in Time to Migrant Drowning
Neuralink Receives FDA Approval for First-in-Human Clinical Study
Is Saudi Arabia the holiest place in the world? Ancient Hebrew Inscriptions from "The Mount Sinai Stand" Discovered in Saudi Arabia
Ukrainian Intelligence Official Admits to Assassination Attempts on Putin
WATCH THIS: democracy in Russia is so bad!
Ed Davey: It is possible for a woman to have a penis
Bernard Arnault Loses $11.2 Billion in One Day as Investors Fear Slowdown in US Growth Will Reduce Demand for Luxury Products
Russian’s Wagner Group leader: “I am not a chef, I am a butcher. Russia is in danger of a revolution like in 1917.”
TikTok Sues Montana Over Law Banning the App
Ron DeSantis Jumps Into 2024 Presidential Race, Setting Up Showdown With Trump
Steve Jobs introducing Apple's iPhone, exactly 16 years ago.
China overtakes Japan as world's top car exporter
Last Walmart in North Portland Closing Down
Banking Behemoth vs Ex-Boss: The PMorgan-Epstein Entanglement
Florida's DeSantis seeks to disqualify judge in Disney case
Talks between US House Republicans and President Biden's Democratic administration on raising the federal government's $31.4tn debt ceiling have paused
Biden Administration Eyeing High-Profile Visits to China: The Biden Administration is heating things up by looking into setting up a series of top-level visits to Beijing by top officials in the coming months
New evidence in special counsel probe may undercut Trump’s claim documents he took were automatically declassified
A French court of appeals confirmed former President Nicolas Sarkozy's three-year jail term for corruption and influence peddling
Debt Ceiling Crises Have Unleashed Political Chaos
Weibao Wang, a former software engineer at Apple, was charged with stealing trade secrets related to autonomous systems, including self-driving cars
Mobile phone giant Vodafone to cut 11,000 jobs globally over three years as new boss says its performance not good enough
Elon Musk compares George Soros to Magneto, the supervillain from the Marvel Comics series.
Warren Buffett Sells TSMC Shares Over Concerns About Taiwan's Stability
New Study Finds That Secondary Bacterial Pneumonia Is a Major Cause of Death in COVID-19 Patients Who Require Ventilator Assistance
The Prime Minister of St. Vincent and the Grenadines calls the British monarchy "an absurdity" he wants to remove in his lifetime
Remarkable video from our Fox drone team
The official tapes of Trump's deposition in the E. Jean Carroll battery (rape) and defamation case have been released.
Here is the full video that the media are purposely not sharing
King Charles III being crowned.
New York procedures cover up cold blooded first degree murder of black man by white man
Newly released video of Donald Trump’s rape trial deposition:
Video: Ukraine MP Punches Russian Representative At Global Meet
El Paso mayor has declared a state of emergency
Russia accused Ukraine of attacking the Kremlin with drones in an attempt to kill Putin
Ahead of King Charles’ coronation, British guards rehearse in the middle of the night in London
×