Automated Assessment of Knowledge and Skills at the National Russian Therapy Olympiad with International Participation in 2023

The development of modern systems and technologies creates new requirements for the education system every day, provides new opportunities for the development of educational methods, consumer coverage and data science. The experience gained as a result of the transfer of the Therapy Olympiad in 2021 to a remote format proved its success and prompted the development of the Olympiad in this direction. On March 21–22, 2023, the III National Russian Therapy Olympiad with international participation in online format was organized. As a result, 2,120 students, residents and schoolchildren from Russia, Uzbekistan, Kazakhstan, Belarus, Kyrgyzstan, Tajikistan, Azerbaijan, and India took part in the Olympiad. This article analyzes the Olympiad in order to study the possibility of conducting an automated assessment of knowledge and skills formed by students of 4–6 courses of specialty and residents of the therapeutic profile and other medical specialties. The following evaluation tools were used: knowledge testing using a distance learning system and assessment of practical skills using a Dimedus computer virtual on-screen patient simulator. The analysis of the results showed that the base of the formed test mini-cases and clinical cases is valid, reliable and can be used for automated assessment of knowledge and skills of students in specialty and residency programs of therapeutic profile. Statistical data and feedback data showed the involvement in the process and the interest of students in the chosen assessment format.

1. Профессиональные стандарты // Официальный интернет-ресурс Министерства труда и социальной защиты Российской Федерации. URL: https://www.profstandart.rosmintrud.ru/ (дата обращения: 09.05.2023).

2. Федеральные государственные образовательные стандарты высшего образования. Официальный интернет-ресурс федеральной службы по надзору в сфере образования и науки. URL: https://www.obrnadzor.gov.ru/gosudarstvennye-uslugi-ifunkczii/7701537808-gosfunction/acts_list2021/mandatory_requirements_2021/fgos_specialitet/ (дата обращения: 09.05.2023).

3. Roy M. J., et al. Simulation and Virtual Reality in Medical Education and Therapy: a Protocol // Cyberpsychol Behav. 2006, Apr. No. 9(2). S. 245–247. DOI: 10.1089/cpb.2006.9.245Watari T. et al. The Utility of Virtual Patient Simulations for Clinical Reasoning Education // Int J Environ Res Public Health. 2020. Jul 24. No. 17(15). P. 5325. DOI: 10.3390/ijerph17155325Thampy H., Collins S., Baishnab E., et al. Virtual Clinical Assessment in Medical Education: an Investigation of Online Conference Technology // J Comput High Educ. 2022, 21 Apr. DOI: 10.1007/s12528-022-09313-6

4. Zachari Swiecki, Hassan Khosravi, Guanliang Chen, Roberto Martinez-Maldonado, Jason M. Lodge, Sandra Milligan, Neil Selwyn, Dragan Gašević. Assessment in the Age of Artificial Intelligence (9 May 2022). DOI: 10.1016/j.caeai.2022.100075

5. Бугубаева М. М., Джумаева Л. М., Калматов Р. К., Горшков М. Д. Применение многопрофильной университетской виртуальной клиники Димедус в качестве оценочного средства на итоговой государственной аттестации выпускников // Виртуальные технологии в медицине. 2022. № 4. С. 285–289. DOI: 10.46594/2687-0037.2022.4.1570

6. Михайлов О. В., Денисова Я. В. Дистанционное обучение в российских университетах: «шаг вперед, два шага назад?» // Высшее образование в России. 2020. № 10. С. 65–76.

7. Мерецков О. В. Проектирование тестовых систем и тренажеров для электронного обучения: метод. пособ. Рига: LAMBERT Academic Publishing, 2020.

8. Лопатин З. В. Дистанционная оценка профессиональных компетенций с применением симуляционных технологий в условиях COVID-19 // Виртуальные технологии в медицине. 2021. № 3. С. 104–110. DOI: 10.46594/2687-0037_2021_3_1311