Google's AI-based medical chatbot has successfully tested medicine in the United States, but its results are lower than those achieved by humans, according to a study published Wednesday in the journal Nature.
The tech giants have been locked in a frantic competition in artificial intelligence, which has been booming since the launch of ChatGPT, designed by Microsoft-backed rival OpenAI last year.
Health care is one area where technology has already made significant progress, as some algorithms have been shown to ensure better reading of radiological images than those performed by humans.
Last December, Google announced in an article the artificial intelligence tool "Med-PaLM" dedicated to medical questions. But this tool has not been made available to the public, unlike Chat GPT.
Google confirmed that Med-Palme was the first major program based on the language model, an artificial intelligence technology trained by large amounts of human-generated text, to pass the U.S. medical licensing exam.
Passing this test qualifies the student to practice medicine in the United States, and requires a score of approximately 60%.
In February, a study revealed that Chat GPT had a satisfactory score close to that required to pass the test.
In a new study reviewed by other researchers and published Wednesday in the journal Nature, Google researchers reported that Med-Palm scored 67.6 percent in answering multiple choice questions on the medical licensing exam.
The study described the results as "encouraging, but still lower than those of humans."
In an effort to reduce so-called "hallucinations" — a word that clearly indicates a wrong answer provided by an artificial intelligence model — Google has developed a new standard for evaluation, it announced.
Google researcher and lead author of the new study, Karan Singhal, told AFP that his team had tested a newer version of the model.
A study published in May but not reviewed by other researchers found that Med-Palme 2 scored 86.5 percent on a medicine test, beating the previous version of the program by about 20 percent.
The Wall Street Journal also reported that Med-Palm 2 has been undergoing testing at the prestigious US research hospital Mayo Clinic since April.