AI has surpassed doctors in diagnosing diseases and even communicating with patients

AI trained to conduct medical interviews with patients matched or even exceeded the ability of conventional doctors to communicate with patients and compile a list of possible diagnoses based on medical history, reports Nature.

Photo: IStock

A chatbot based on Google's Large Language Model (LLM) was found to be more accurate than board-certified primary care physicians in diagnosing respiratory and cardiovascular diseases. Compared to human doctors, he was able to obtain a similar amount of information during medical interviews and scored higher on empathy.

“To our knowledge, this is the first time that a conversational AI system has been optimally designed for diagnostic interviewing and medical history collection,” says Alan Karthikesalingam, a clinical research fellow at Google Health in London and co-author of the study. It was published on January 11 in the arXiv preprint repository, but has not yet been peer-reviewed.

On the subject: Robot companion, transparent TV and home smokehouse: the main new products from CES 2024

The chatbot, called Articulate Medical Intelligence Explorer (AMIE), remains purely experimental for now. It was not tested on people with real health problems - only on actors trained to portray people with diseases.

“We want the results to be interpreted with caution,” says Karthikesalingam.

Although the chatbot has not yet been used in clinical practice, the authors argue that it could eventually play a role in democratizing healthcare. The tool can be useful, but it shouldn't replace interaction with doctors, says Adam Rodman, a physician at Harvard Medical School in Boston, Massachusetts. “Medicine is about more than just collecting information: it’s all about human relationships,” he says.

Delicate task

There have been only a few attempts to explore whether systems can mimic a doctor's ability to take a person's medical history and use it to make a diagnosis. Medical students spend a lot of time learning to do just that, Rodman says.

“This is one of the most important and difficult skills to teach physicians,” he said.

One of the developers' concerns was the lack of real-life medical conversations that could be used as training data, said Vivek Natarajan, a research scientist at Google Health in Mountain View, California, and co-author of the study. To solve this problem, researchers have developed a way to train a chatbot on its own “conversations.”

The researchers conducted an initial round of fine-tuning of the basic LLM using existing real-world data sets such as electronic medical records and medical conversation transcripts. To further train the model, the researchers asked the LLM to play the role of a person with a specific disease and an empathetic clinician seeking to understand the person's history and develop potential diagnoses.

The team also asked the model to play another role: a critic who evaluates the doctor's interaction with the person being treated and provides feedback on how to improve that interaction. This critique is used to further LLM teaching and improve dialogue.

To test the system, the researchers recruited 20 people who were trained to impersonate patients and asked them to conduct text-based online consultations with both AMIE and 20 board-certified physicians. They were not told whether they were communicating with a human or a bot.

The actors simulated 149 clinical scenarios and were then asked to rate their experiences. The panel also assessed the work of AMIE and doctors.

AMIE beats doctors

The artificial intelligence system matched or exceeded the diagnostic accuracy of doctors in all six medical specialties examined. The bot outperformed doctors on 24 of 26 conversation quality measures, including politeness, explanation of condition and treatment, honesty, and expression of care and commitment.

“This in no way means that the language model is better at taking medical history than doctors,” says Karthikesalingam. He notes that the primary care physicians in the study were likely not used to interacting with patients via text chat, and this may have impacted their performance.

AMIE has a special advantage: it can quickly compose long and beautifully structured answers, says Karthikesalingam. This allows him to be constantly attentive without getting tired.

Unbiased chatbot

An important next step in the research, he said, is to conduct more detailed studies to assess potential biases and ensure the system is fair to different populations. The Google team is also starting to explore the ethical requirements for testing the system on people who have real-life health problems.

Daniel Ting, an artificial intelligence specialist at Duke-NUS Medical School in Singapore, agrees that testing a system for bias is necessary to ensure that the algorithm does not discriminate against racial groups that are poorly represented in training data sets.

You may be interested in: top New York news, stories of our immigrants, and helpful tips about life in the Big Apple - read all this on ForumDaily New York.

According to Ting, the privacy of chatbot users is also an important aspect to consider.

“We are still not sure where the data is stored and how it is analyzed in many commercial large language model platforms,” he says.

Read also on ForumDaily:

'No war and no help to the Armed Forces of Ukraine': Pepsi banned mention of the Russian invasion of Ukraine

How to effectively manage money: tips from an American billionaire

New wars and economic crisis: troubles to expect in 2024

Do you want more important and interesting news about life in the USA and immigration to America? — support us donate! Also subscribe to our page Facebook. Select the “Priority in display” option and read us first. Also, don't forget to subscribe to our РєР ° РЅР ° Р »РІ Telegram  and Instagram- there is a lot of interesting things there. And join thousands of readers ForumDaily New York — there you will find a lot of interesting and positive information about life in the metropolis.