Table 4 Metrics assessed in PRIMAGE platform
Metrics to be assessed | Methodology |
|---|---|
Security/privacy | Provision of authentication and authorisation and analysis of vulnerabilities from public databases. Assessment of the platform’s robustness to preserve data integrity according to GDPR, evaluating the privacy risk (e.g., as the capability of a model to infer information previously anonymised), and managing the fine-grain consent as GDPR requires. |
Correctness/reliability | Assess the correctness of the predictive results for the established clinical end points using testing datasets from clinical data repositories for over 2000 neuroblastoma patients and over 500 diffuse intrinsic pontine glioma patients with complete diagnosis and follow-up data, including treatment and outcomes. |
Sensitiveness to incomplete data | Assess dependence of correctness of the predictive results with the completeness of the diagnosis datasets, to establish how new biomarkers modify minimum datasets required for correct diagnosis/prognosis. |
Reproducibility | Statistical assessments: dispersion in the results obtained for a subgroup of patients with a common clinical diagnosis, belonging to different hospitals where the diagnosis studies were undertaken |
Interoperability | Assessment of failures in the integration with hospital picture archive and communication system and electronic health record systems, as well as to on-premise and public cloud services |
Malfunction | Occurrence of any fatigue of integrity or potential to induce to use errors |
Relevance | Interviews to assess users’ own judgement of helpfulness of the platform to guide them beyond obvious decisions for a given data available set |
Added value | Statistical assessment of occurrence of correct predictions for clinical end points that current diagnosis/prognosis standard protocols could not predict correctly |
Usability/user friendliness | Observation of use patterns, users’ eye tracking, interviews to users |