ChatGPT fails medical exams: Not ready to replace doctors yet

Artificial intelligence is undoubtedly a key element of our future. Already, many tasks are undertaken by robots, which on the one hand is beneficial as it reduces the burden on workers, but on the other hand, raises concerns about future jobs. Nevertheless, ChatGPT is still not capable of making medical diagnoses as it has not passed most medical exams.

People are increasingly wondering what artificial intelligence's limits are. We know it can mimic voice, create images, and write scientific papers. Hence, the question arises: Could it also diagnose human ailments? Theoretically, yes, as it would suffice to input symptoms, and the technology could match them to diseases in a database. In practice, ChatGPT still cannot pass the medical exam to make this possible.

Researchers from Collegium Medicum UMK conducted an experiment, using artificial intelligence to solve medical exams. ChatGPT performed the worst in internal medicine, scoring between 48% and 53%, which is well below the passing requirements. For comparison, the average student scores on the same exam ranged from 65% to 72%.

Researchers observed that ChatGPT handles simple questions better. Complex issues make the AI's responses less precise. Besides internal diseases, ChatGPT also took other exams, performing best in allergology (71%) and worst in cardiology (44%).

At the moment, artificial intelligence's capabilities are too limited to compete with qualified doctors, but this is expected to change over time. ChatGPT is constantly being developed, making it one of the fastest-growing tools.

Interestingly, ChatGPT is considered a more empathetic source of knowledge on medical forums. Patients claim its responses are more accurate than those from human specialists on these platforms.