Breathe Easy - AI Not Ready to Teach

ChatGPT won’t be replacing human teachers anytime soon with a study finding it can’t help students critically understand academic assessment solutions. 

Charles Darwin University (CDU) researchers explored whether ChatGPT could help students comprehend the answers in two financial accounting course unit assessments.

CDU Professor of Accounting Indra Abeysekera fed ChatGPT and later version ChatGPT-4 a series of multiple-choice questions from assessments from an introductory and an advanced financial accounting course unit respectively.

Mostly, ChatGPT’s perfromance was decidedly mediocre.

“It can meet some criteria that meet originality, such as synthesising information differently and providing a new interpretation using known information. However, there are originality tasks that ChatGPT is incapable of meeting, at least for now. These include testing someone else’s idea in a different context or developing a research tool. Hence, academic focus on learning and assessment should shift toward original knowledge,” said Prof Abeysekera.

“ChatGPT does not provide scaffolding for novice learners to take over their learning and develop individual competencies to be less or not reliant on it. It can be destructive to an incompetent learner and can serve as a platform to simply find the solution.

“Furthermore, the findings showed that ChatGPT is not a foolproof solution provider, especially when questions have discipline-specific underlying assumptions and increased technical and task complexity.”

ChatGPT answered correctly eight out of 10 questions from the introductory test, and scored 5 out of 10 in the advanced test. ChatGPT-4 correctly answered nine out of 10 questions and scored seven out of 10 in the advanced test.

While the technology had the ability to correctly answer questions, it couldn’t support beginner learners to develop the knowledge and skills to find solutions.

“The solutions provided by ChatGPT showed that it is a solution provider rather than a teacher or instructor,” Professor Abeysekera said.

“ChatGPT can be constructive to a competent learner who has reached the competency level to further develop critical understanding.”

While ChatGPT had many capabilities it could not surpass the critical thinking benefits of academia.

“ChatGPT can process unstructured data, convert it into information, and share it as knowledge,” Professor Abeysekera said. 

ChatGPT and academia on accounting assessments was published in the Journal of Open Innovation: Technology, Market, and Complexity.