Google proposes hybrid approach to AI transfer learning for medical imaging

Medical imaging is among the most popular application of AI and machine learning, and with good reason. Computer vision algorithms are naturally adept at spotting anomalies experts sometimes miss, in the process reducing wait times and lightening clinical workloads. Perhaps that’s why although the percentage of health care organizations that have adopted AI remains relatively low (22%) globally, the majority of practitioners (77%) believe the technology is important to the medical imaging field as a whole.

Unsurprisingly, data scientists have devoted outsize time and attention to developing AI imaging models for use in health care systems, a few of which Google scientists detail in a paper accepted to this week’s NeurIPS conference in Vancouver.  In “Transfusion: Understanding Transfer Learning for Medical Imaging,” coauthors hailing from Google Research (the R&D-focused arm of Google’s business) investigate the role transfer learning plays in developing image classification algorithms.

In transfer learning, a machine learning algorithm is trained in two stages. First, there’s retraining, where the algorithm is generally trained on a benchmark data set representing a diversity of categories. Next comes fine-tuning, where it is further trained on the specific target task of interest. The pretraining step helps the model to learn general features that can be reused on the target task, boosting its accuracy.

According to the team, transfer learning isn’t quite the end-all, be-all of AI training techniques. In a performance evaluation that compared a range of model architectures trained to diagnose diabetic retinopathy and five different diseases from chest x-rays, a portion of which were pretrained on an open source image data set (ImageNet), they report that transfer learning didn’t “significantly” affect performance on medical imaging tasks. Moreover, a family of simple, lightweight models performed at a level comparable to the standard architectures.

In a second test, the team studied the degree to which transfer learning affected the kinds of features and representations learned by the AI models. They analyzed and compared the hidden representations (i.e., representations of data learned in the model’s latent portions) in the different models trained to solve medical imaging tasks, computing similarity scores for some of the representations between models trained from scratch and those pretrained on ImageNet. The team concludes that for large models, representations learned from scratch tended to be much more similar to each other than those learned from transfer learning, while there was greater overlap between representation similarity scores in the case of smaller models.

To rectify these and other issues, the team proposes a hybrid approach to transfer learning, where instead of reusing the full model architecture, only a portion of is resused and the rest is redesigned to better suit the target task. They say that it confers most of the benefits of transfer learning while further enabling flexible model design. “Transfer learning is a central technique for many domain,” wrote Google Research scientists Maithra Raghu and Chiyuan Zhang in a blog post. “Many interesting open questions remain, [and we] look forward to tackling these questions in future work.”

The work comes shortly after Google detailed an AI capable of classifying chest X-rays with human-level accuracy. In another recent study, teams from the tech giant claimed to have developed a machine learning model that detects 26 skin conditions as accurately as dermatologists and a lung cancer detection AI that outperformed six human radiologists.