Large language models present significant challenges for programming-based homework integrity
the verdict
SUPPORTED
the evidence backs this
refutedsupported
the weight of evidence
3 sources for · 0 against
Peer-reviewed literature and educational studies report that large language models and conversational coding assistants pose significant challenges to academic integrity by generating full-code solutions and fostering over-reliance among students.
Since 2021, the rapid integration of large language models (LLMs), such as OpenAI’s Codex and ChatGPT, into programming has reshaped how software is written, learned, and maintained. Tools such as GitHub Copilot, Amazon CodeWhisperer, Tabnine, and Sourcegraph Cody have evolved from experimental aids to core elements of modern workflows, while academic prototypes continue to explore new interfaces and teaching applications. This meta-analysis synthesizes empirical research, user evaluations, and product-level comparisons to provide a comprehensive view of the opportunities and challenges posed by LLM-based programming assistants. The analysis considers novice programmers, professional developers, researchers, and educators, highlighting recurring human-computer interaction (HCI) themes of trust calibration, cognitive load management, interface modalities, and the balance between automation and user control.The methodology followed a systematic review of studies published between 2021 and early 2025 in ACM, IEEE, arXiv, and other recognized repositories. Industry reports and tool documentation were included to capture emerging developments. A qualitative thematic synthesis integrated findings across varied research contexts, including user studies, classroom evaluations, and professional development workflows, revealing consistent patterns in tool use, learning outcomes, and professional practice, while also identifying gaps in current understanding.Novice programmers benefit from immediate feedback, reduced syntax errors, and increased confidence. Yet these advantages can foster over-reliance if tools are used as answer generators. Structured support, such as hint-based prompting and code validation, helps students engage more deeply with core concepts. Professional developers report productivity gains in routine tasks and code navigation but remain cautious about correctness, security, and workflow disruptions. Vulnerability checks, auto-generated tests, and explana
Large language models in programming: a meta-analysis of tools, users, and human-computer interaction themes | AHFE Open Access AHFE Open Access Journal / New York, USA Publications AHFE Access Instructions Aims and scope Volumes & Issues Editorial Board AHFE Conference Series Follow us on Social Media AHFE International Accelerating Open Access Science in Human Factors Engineering and Human-Centered Computing Large language models in programming: a meta-analysis of tools, users, and human-computer interaction themes Open Access Article Conference Proceedings Authors: Daniel Olivares , Charles Bennington , Abigail Skillestad Abstract Since 2021, the rapid integration of large language models (LLMs), such as OpenAI’s Codex and ChatGPT, into programming has reshaped how software is written, learned, and maintained.
Tools such as GitHub Copilot, Amazon CodeWhisperer, Tabnine, and Sourcegraph Cody have evolved from experimental aids to core elements of modern workflows, while academic prototypes continue to explore new interfaces and teaching applications. This meta-analysis synthesizes empirical research, user evaluations, and product-level comparisons to provide a comprehensive view of the opportunities and challenges posed by LLM-based programming assistants.
Professional developers report productivity gains in routine tasks and code navigation but remain cautious about correctness, security, and workflow disruptions. Vulnerability checks, auto-generated tests, and explanation features are especially valued. Researchers and educators employ LLM-based programming tools to streamline analysis, generate assessments, and create interactive teaching
For learners, risks arise when practice is bypassed, limiting skill growth. For professionals, challenges involve accuracy, security, and workflow integration. Effective use treats LLMs as collaborators that support reflection and experimentation rather than replacements for human reasoning. Students benefit when tools provide hints and guidance instead of complete solutions, encouraging deeper understanding.In conclusion, LLM-based programming tools present strong potential for advancing productivity, education, and research. Benefits include faster coding, improved learning, and streamlined teaching. Persistent challenges remain related to correctness, cognitive load, and trust.
Keywords: large language models, programming tools, developer productivity, human-computer interaction, software education, user experience, meta-analysis, AI DOI: 10.54941/ahfe1006934 Cite this paper Downloads 872 Visits 1104 Download PDF More from this volume ← Secure Authentication Design For AI Agents Human-Centered AI for Automotive Systems: Towards Explainable, Intercultural, and Standardized Integration → Warnings and Multilingual Audiences EAT Da Vinci 3.0_Translating Cinematic Narrative into Media Art Installation From Manual to Automated: Enhancing Inclusivity in Foreign Language Education with Technology The effect of multi-sensory physical experiences in daily emotional self-tracking service for emotion self-awareness Parametric generation based graphic design and spatial expression research Gender Stereotypes in Video Gaming: Impacts of Anxiety Levels, Verbal Communication, and Performance Exploring Usability And User-experience Metrics With A Novel AR App In The MASTERLY Project Drawing Dialogues Between Generative AI and Children with Autism: A Qualitative Study on the Externalization of “Understanding” Human-Centered Design of Integrated Food Service Management Systems: Reducing Cognitive Load in Resource-Constrained Kitchen Operations The Design Futures Art-driven (DFA) Method: Structuring Art-Tech Collaboration for Sustainable Future of Food System Increasing importance of Instinct Bridging the Privacy Gap: Stakeholder Solutions to Support Transparent Data Management Practices in Digital Health Research View all articles in Human Factors in Design, Engineering, and Computing →
For many beginner programmers, encountering errors in code can be frustrating and disheartening—leading some to questions their belonging in computer science (CS). In these moments, timely debugging help is essential to sustain motivation and foster learning. While students have traditionally turned to peers or teaching assistants for guidance, many now seek debugging support from conversational Large Language Models (LLMs). These chatbots offer promise in providing immediate help, but their ability to generate full-code solutions raises concerns about learning and over-reliance. As these tool
founded in 1993. Coinciding with the AI boom of the 2020s, the use of large language models in the global north has been promoted and funded by venture capital
Artificial intelligence in education (often abbreviated as AIEd) is a subfield of educational technology that studies how to use artificial intelligence to create learning environments.
Considerations in the field include data-driven decision-making, AI ethics, data privacy and AI literacy. Concerns include the potential for cheating, over-reliance, equity of access, reduced critical thinking, and
AI-directed, learner-as-recipient: AIEd systems present a pre-set curriculum based on statistical patterns that do not adjust to learner's feedback.
AI-supported, learner-as-collaborator: Systems that incorporate responsiveness to learner's feedback through, for example, natural language processing, wherein AI can support knowledge construction.
AI-empowered, learner-as-leader: This model seeks to position AI as a supplement to human intelligence wherein learners take…
AI-directed, learner-as-recipient: AIEd systems present a pre-set curriculum based on statistical patterns that do not adjust to learner's feedback.
AI-supported, learner-as-collaborator: Systems that incorporate responsiveness to learner's feedback through, for example, natural language processing, wherein AI can support knowledge construction.
AI-empowered, learner-as-leader: This model seeks to position AI as a supplement to human intelligence wherein learners take agency and AI provides consistent and actionable feedback.
Some scholars place AI in education within a socio-technical framework. This positions AI alongside other emerging educational technologies, such as computing, the internet, and social media.
The framework of Tsao, Heinrichs and Camit (2025) draws on new materialism and posthumanism, specifically Donna Haraway's concept of sympoiesis (making-with). This perspective views learning as an entanglement of human and non-human actors (students, teachers, and AI algorithms), where knowledge is co-composed in contact zones between human context and algorithmic prediction.
AI agents have been trained on biased datasets and thus continue to perpetuate societal biases. Since LLMs were created to produce human-like text, algorithmic bias can be introduced and reproduced. AI's data processing and monitoring reinforce neoliberal approaches to education rather than addressing inequalities.
Everything we examined (3)
This check searched the claim as stated. It did not run a separate search for evidence against it.