|
Abstract
|
Large Language Models (LLMs) are becoming more popular among students. Although using LLMs can improve academic performance [2], there is a risk that students will stop learning basic skills when their homework is solved by LLMs. Previous studies [6, 8] indicate that allowing usage of LLMs for homework negatively affects grades. However, to the best of our knowledge, no previous research has systematically tracked students who attempt to hide their use of AI systems. In this paper, we present a method to detect students who cheated by using LLMs or copying answers from old sample solution. We examined the homework submissions and exam grades of a first-semester programming course with more than 800 students. We investigated the impact of different forms of LLM use (e.g. requesting full solutions or asking questions) and different kinds of cheating, including use of LLMs, on final exam grades. Students who used artificial intelligence systems to generate homework code performed significantly worse in the exam. The negative effect on the exam grades was weaker when LLMs were used to answer questions or debug own code, or when traditional forms of cheating, such as copying from others, were used. We conclude that simply prohibiting the use of LLMs in programming courses is not appropriate, but sensible use of these systems should be taught.
|
