Researchers from Facebook and NYU Langone Health have created AI models that scan X-rays to predict how a patient’s condition with COVID-19 will develop.
The team says its system can predict whether a patient may need more intensive care resources up to four days in advance. They believe that hospitals could use it to anticipate demand for resources and avoid sending patients at risk home too soon.
Its approach differs from most previous attempts to predict the deterioration of COVID-19 by applying machine learning techniques to X-rays.
They usually use supervised training and single schedule images. This method has shown promise, but its potential is limited by the time-consuming process of labeling data manually.
[Read: How Netflix shapes mainstream culture, explained by data]
These limits have led researchers to use self-supervised learning.
They first pre-trained their system on two sets of public X-ray data, using a self-supervised learning technique called Momentum Contrast (MoCO). This allowed them to use a large amount of non-COVID X-ray data to train their neural network to extract information from the images.
Predicting deterioration of COVID-19
They used the pre-trained model to build classifiers that predict whether a patient’s condition with COVID-19 is likely to get worse. Then they adjusted the model with an extended version of the NYU COVID-19 data set.
This smaller data set of about 27,000 X-ray images of 5,000 patients received labels indicating whether the patient’s condition deteriorated within 24, 48, 72 or 96 hours after scanning.
The team built a classifier that predicts patient deterioration based on a single X-ray. Another makes his predictions using an X-ray sequence, aggregating the image’s features using a Transformer model. A third model estimates the amount of supplemental oxygen that patients may need by looking at an X-ray.
They say that using an X-ray sequence is particularly valuable, as they are more accurate for long-term predictions. This approach is also responsible for the evolution of infections over time.
His study showed that the models were effective in prediction of ICU needs, mortality forecasts and general forecasts of long-term adverse events (up to 96 hours):
The performance of our multi-image model surpassed that of all single-image models. In comparison with radiologists, our multiple image prediction model was comparable in its ability to predict patient deterioration and stronger in its ability to predict mortality.
The team opened the code for the pre-trained models so that other researchers and hospitals can adjust them with their own COVID patient data – using a single GPU.
You can read the study article on the Axiv.org prepress server.
Published on January 15, 2021 – 17:00 UTC