Publication trend
Radiomics aims to capture the informative content hidden in medical images, overcoming the limitations of the human eyes and human cognitive patterns: the rationale is that medical images (anatomical, functional, or metabolic) can carry information about the physiological response to cancer and therapy stress [1]. The wealth of relevant information provided by medical images is key in decision-making and follow-up of treatment response. The ability to extract more quantitative data from medical images will reinforce the position of medical imaging in clinical decision-making and patient management.
Medical and information technology imaging specialists have produced a number of algorithms, statistical analyses, and models, fuelling an exponential growth in the number of publications since 2012 (Fig. 3, Additional file 1: Table S1). These have gradually become more complex and study more diverse pathologies, modalities, and applications.
While radiomics make their way into clinical workflows, large or even dynamically growing prospective patient cohorts are required to reach the level of robustness and precision needed to be adopted by clinicians (Fig. 4). To better understand these challenges, it is important to grasp the workflow of radiomics, as well as the variety of algorithms and analytical models currently under development.
Handcrafted radiomics techniques in radiomics research
The current state-of-the-art approach in radiomics relies on a relatively conventional image analysis workflow, which is referred to as “handcrafted” radiomics based on four successive processing tasks [5]: (1) image acquisition/reconstruction, (2) image segmentation, (3) feature extraction and quantification, and (4) feature selection/statistical analysis. In handcrafted radiomics, image features are known and are selected depending on their correlation with clinical outcome.
Image acquisition (step 1) and image segmentation (step 2) have been a lesser focus of radiomics publications. However, the risk of bias and potential lack of reproducibility of algorithms across different machines, protocols, and sites are well recognised [6,7,8,9] and standardisation issues are further discussed in the next section.
Feature extraction (step 3) in handcrafted radiomics has established several techniques based, for example, on image morphology, histograms, textures, wavelet, or fractal techniques, with their application in original works reviewed in detail by Avanzo et al. [10]. In the same review, the authors showed that statistical analysis/feature selection (step 4) has not reached a disease-specific or technical consensus on feature selection strategies, predictive models and performance estimation techniques. Linear regression models were shown to be more frequently used, which can be explained by more simple models providing graphical results (nomograms) [11], and a similar trend is noted in a 2019 sample of publications analysed (Additional file 1: Table S2). Avanzo et al. [10] further identified limitations of radiomics related to the size of the cohorts, and issues of standardisation and benchmarking, which are discussed in the next paragraphs.
The emerging role of deep learning
Deep learning radiomics (DLR) can be applied to any aspect of the workflow described above. Many studies use DLR to either automatically identify or extract features (step 3). In other instances, a classical handcrafted radiomics approach extracts features so that a DLR algorithm can select them (step 4). Single artificial neural networks (ANN) can perform both tasks [5].
DLR is based on a subtype of machine learning technique based on ANN with a high number of interconnected layers. Given a training set, such networks can autonomously build image filters and extract image features for classification without the need to pre-determine (handcraft) them. In the field of medical imaging, several applications have already been successfully designed, for example, in detecting lung nodules [12] or in building computed tomography images from magnetic resonance images for the purpose of positron-emission tomography (PET) attenuation correction [13].
These multi-layer ANN are often convolutional (cANN) or recurrent (rANN) and, after training, produce image filters and features in a much higher number than other techniques. The major challenge of deep learning algorithms in radiomics is however the need for a higher number of observations (patients) than in many handcrafted radiomics studies.
However, once trained, a deep learning algorithm can be extremely fast and accurate [5]. Recently, Xu et al. [14] have developed a model based on a cANN and a rANN for overall survival prognosis in non-small cell lung cancer using a seed point tumour localisation instead of a classical segmented contour. The authors analysed how the tumour evolved during the time, exploiting several successive computed tomography scans acquired at different time points (pre-treatment, 1, 3, and 6 months after radiation therapy). Algorithms that can include multiple imaging time points are a key step to bring radiomics to the bedside and into personalised oncology data streams.