Skip to main content

Table 3 PRIMAGE platform testing methodologies and performance metrics

From: PRIMAGE project: predictive in silico multiscale analytics to support childhood cancer personalised evaluation empowered by imaging biomarkers

 

Main testing methodologies

Main performance metric

Cloud infrastructure

Definition of unitary and integration tests based on the application requirements, monitoring along time, design-time vulnerability analysis.

Performance (deployment and reconfiguration overheads, performance penalties, scalability), reliability (correct results with respect references), reproducibility (predictability of performance and automation), robustness (reliability along time and under different stress conditions), security (identification of vulnerabilities and isolation), privacy (privacy risk estimation).

High-performance computing infrastructure

Continuous monitoring of infrastructure. Alerts for administrators in case of malfunctions or failures.

VM start-up time. Resource consumption. Number of concurrently running computational tasks. Availability. Measured through monitoring statistics, experiments and benchmarks.

Data repositories

Testing on MR, 131I-MBIG imaging, CT, PET/CT data, from retrospective studies of neuroblastoma and diffuse intrinsic pontine glioma patients

Correlation between clusters of imaging biomarkers. Correlation between radiomic signatures and genomic profiles, and/or circulating tumour biomarkers from liquid biopsy (circulating tumour cells, tumour nucleic acids, etc.)

Imaging biomarkers

Testing on images (MR, 131I-MBIG imaging, CT, PET/CT) from retrospective data of neuroblastoma and diffuse intrinsic pontine patients

Precision, accuracy and clinical relationship measured in terms of quantified limit of detection and limit of quantification, reproducibility, sensitivity/specificity, coefficient of variation, correlation to diagnosis/prognosis of a specific disease

Multiscale modelling framework

Qualitative and quantitative comparison of numerical predictions with retrospective data

Quantitative correlation of the shape and size of tumour between image-based data and computer-based results. Qualitative correlation of vascular level and extracellular matrix properties in the tumour surroundings.

  1. CT Computed tomography, MR Magnetic resonance, MBIG Metaiodobenzylguanidine, PET Positron emission tomography