Skip to main content
Articles | Acta Pedagogia Asiana | Volume 4 - Issue 1 - 2025 | 55-65 | https://doi.org/10.53623/apga.v4i1.553
© 2025 by the author(s), and is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) License Creative Commons Attribution 4.0 International (CC BY 4.0) License

Comparative Analysis of the Effectiveness of Informatics Course Learning Utilizing Chatgpt

Author(s): Arneitta Dwicahya Utami 1 , Mia Kamayani 1 , Estu Siduningrum 1 , Nur Chalik Azhar 2
Author(s) information:
1 Department of Informatics Engineering, Faculty of Industrial and Informatics Technology, Universitas Muhammadiyah Prof. Dr.HAMKA, Indonesia
2 Faculty of Industrial and Informatics Technology, Universitas Muhammadiyah Prof. Dr.HAMKA, Indonesia

Corresponding author

Acta Pedagogia Asiana

Volume 4 - Issue 1 - 2025

 About the journal
Submit your article

This study examined the effectiveness of conventional teaching methods and ChatGPT in an introductory Algorithms and Programming course at the university level. ChatGPT, an AI-based NLP technology, assisted students in understanding course material through automated responses. However, its effectiveness relative to conventional methods required further evaluation, particularly concerning motivation, interaction, self-regulation, instructional structure, and the instructor's role. Using a sample of 10 students for pretest-posttest analysis, 38 respondents for the User Experience Questionnaire (UEQ), and accuracy analysis via prompt engineering, the results revealed that conventional methods better enhanced motivation and interaction. ChatGPT demonstrated strengths in attractiveness (1.982) and efficiency (2.053) but scored lower in accuracy (1.395) and novelty (1.053). Prompt engineering significantly improved response accuracy when tailored to learning modules, highlighting the importance of precise inputs. The findings suggested that while ChatGPT excelled as a supplementary tool, it was less effective as a standalone teaching method. This study contributed to the growing field of educational technology by providing insights into the integration of AI tools in learning environments.

Read Full-Text
Previous article
Next article

Herbold, S.; Janisz, A.H.; Heuer, U.; Kikteva, Z.; Trautsch, A. (2023). OPEN A large-scale comparison of human-written versus ChatGPT-generated essays. Scientific Reports, 1, 1‒11. https://doi.org/10.1038/s41598-023-45644-9.

Zhou, M.; Duan, N.; Liu, S.; Shum, H.Y. (2020). Progress in Neural NLP: Modeling, Learning, and Reasoning. Engineering, 6(3), 275‒290. https://doi.org/10.1016/j.eng.2019.12.014.

Xue, Y.; Chen, H.; Bai, G.R.; Tairas, R. (2024). Does ChatGPT help with introductory programming? An experiment of students using ChatGPT in CS1. ICSE 2024 - Software Engineering Education and Training, 1, 1‒11. https://doi.org/10.1145/3639474.3640076.

Joshi, I.; et al. (2022). ChatGPT in the classroom: An analysis of its strengths and weaknesses for solving undergraduate computer science questions. Proceedings of the ACM, 1, 625‒631. https://doi.org/10.1145/3626252.3630803.

Jackaria, P.M.; Hajan, B.H.; Mastul, A.R.H. (2024). A comparative analysis of the rating of college students’ essays by ChatGPT versus human raters. International Journal of Learning, Teaching, and Educational Research, 23(2), 478‒492. https://doi.org/10.26803/ijlter.23.2.23.

Septiyani, S.; Sutabri, T. (2024). Analisis perbandingan antara cara konvensional dan Chat GPT dalam keberhasilan mahasiswa mengerjakan tugas menggunakan metode pengamatan partisipan. Indonesian Journal of Multidisciplinary, 2, 391‒397.

Hamzah, F.; Mujib, A.; Firmansyah. (2022). Efektivitas pembelajaran blended learning menggunakan Schoology pada pelajaran matematika. DELTA: Jurnal Ilmiah Pendidikan Matematika, 10(1), 95‒104.

Ekawardhana, N.E. (2020). Efektivitas pembelajaran dengan menggunakan media video conference. Seminar Nasional Ilmu Terapan, 1, 1‒7. [Online]. Available: https://ojs.widyakartika.ac.id/index.php/sniter/article/view/218.

Eom, S.B.; Ashill, N. (2016). The determinants of students’ perceived learning outcomes and satisfaction in university online education: An update. Decision Sciences Journal of Innovative Education, 14, 185‒215. https://doi.org/10.1109/CONMEDIA46929.2019.8981813.

Saleh, A.M.; Abuaddous, H.Y.; Alansari, I.S.; Enaizan, O. (2022). The evaluation of user experience of learning management systems using UEQ. International Journal of Emerging Technologies in Learning, 17(7), 145‒162. https://doi.org/10.3991/ijet.v17i07.29525.

Kochanek, M.; et al. (2024). Improving training dataset balance with ChatGPT prompt engineering. Electronics, 13(12), 1‒20. https://doi.org/10.3390/electronics13122255.

Masus, S.B.; Fadhilaturrahmi, F. (2020). Peningkatan keterampilan proses sains IPA dengan menggunakan metode eksperimen di sekolah dasar. Jurnal Pendidikan dan Konseling, 2(2), 161‒167. https://doi.org/10.31004/jpdk.v2i1.1129.

Chen, C.A.; Hsieh, C.W.; Chen, D.Y. (2021). Can training enhance public employees’ public service motivation? A pretest–posttest design. Review of Public Personnel Administration, 41(1), 194‒215. https://doi.org/10.1177/0734371X19872244.

Pratama, A.; Faroqi, A.; Mandyartha, E.P. (2022). Evaluation of user experience in integrated learning information systems using User Experience Questionnaire (UEQ). Journal of Information Systems and Informatics, 4(4), 1019‒1029. https://doi.org/10.51519/journalisi.v4i4.394.

A prompt pattern catalog to enhance prompt engineering with ChatGPT. (accessed on 1 November 2025 Year) Available online: http://arxiv.org/abs/2302.11382.

Nurakhim, B.; Madiistriyatno, H.; Wijayanto, A. (2023). The influence of leadership on employee performance in Class IIA Central Institution, Karawang. Return: Study of Management, Economics, and Business, 3(4), 400‒408. https://doi.org/10.57096/return.v3i04.93.

Read Full-Text
About this article

SUBMITTED: 30 November 2024
ACCEPTED: 20 January 2025
PUBLISHED: 26 January 2025
SUBMITTED to ACCEPTED: 51 days
DOI: https://doi.org/10.53623/apga.v4i1.553

Cite this article
Utami, A. D., Kamayani, M. ., Siduningrum, E. ., & Azhar, N. C. . (2025). Comparative Analysis of the Effectiveness of Informatics Course Learning Utilizing Chatgpt. Acta Pedagogia Asiana, 4(1), 55–65. https://doi.org/10.53623/apga.v4i1.553
Keywords
earning effectiveness, ChatGPT, UEQ, prompt engineering
Accessed
46
Citations
0
Share this article