A new artificial intelligence tool developed at Stanford Medicine combines data from medical images with text to predict cancer prognoses and treatment responses. Stanford Medicine researchers developed an artificial intelligence model that accurately predicted prognoses for patients with cancer.
FG Trade via Getty ImagesThe melding of visual information (microscopic and X-ray images, CT and MRI scans, for example) with text (exam notes, communications between physicians of varying specialties) is a key component of cancer care. But while artificial intelligence helps doctors review images and home in on disease-associated anomalies like abnormally shaped cells, it’s been difficult to develop computerized models that can incorporate multiple types of data.
Now researchers at Stanford Medicine have developed an AI model able to incorporate visual and language-based information. After training on 50 million medical images of standard pathology slides and more than 1 billion pathology-related texts, the model outperformed standard methods in its ability to predict the prognoses of thousands of people with diverse types of cancer, to identify which people with lung or gastroesophageal cancers are likely to benefit from immunotherapy, and to pinpoint people with melanoma who are most likely to experience a recurrence of their cancer.
The researchers named the model MUSK, for m ultimodal transformer with u nified ma sk modeling. MUSK represents a marked deviation from the way artificial intelligence is currently used in clinical care settings, and the researchers believe it stands to transform how artificial intelligence can guide patient care.
“MUSK can accurately predict the prognoses of people with many different kinds and stages of cancer,” said Ruijiang Li , MD, an associate professor of radiation oncology. “We designed MUSK because, in clinical practice, physicians never rely on just one type of data to make clinical decisions. We wanted to leverage multiple types of data to gain more insight and get more precise predictions about patient outcomes.”
Li, who is a member of the Stanford Cancer Institute, is the senior author of the study , which was published Jan. 8 in Nature . Postdoctoral scholars Jinxi Xiang , PhD, and Xiyue Wang , PhD, are the lead authors […]
Unique Stanford Medicine-designed AI predicts cancer prognoses, responses to treatment
