Skip to main content

Multiparametric prostate MRI quality assessment using a semi-automated PI-QUAL software program

Abstract

The technical requirements for the acquisition of multiparametric magnetic resonance imaging (mpMRI) of the prostate have been clearly outlined in the Prostate Imaging Reporting and Data System (PI-RADS) guidelines, but there is still huge variability in image quality among centres across the world. It has been difficult to quantify what constitutes a good-quality image, and a first attempt to address this matter has been the publication of the Prostate Imaging Quality (PI-QUAL) score and its dedicated scoring sheet. This score includes the assessment of technical parameters that can be obtained from the DICOM files along with a visual evaluation of certain features on prostate MRI (e.g., anatomical structures). We retrospectively analysed the image quality of 10 scans from different vendors and magnets using a semiautomated dedicated PI-QUAL software program and compared the time needed for assessing image quality using two methods (semiautomated assessment versus manual filling of the scoring sheet). This semiautomated software is able to assess the technical parameters automatically, but the visual assessment is still performed by the radiologist. There was a significant reduction in the reporting time of prostate mpMRI quality according to PI-QUAL using the dedicated software program compared to manual filling (5′54″ versus 7′59″; p = 0.005). A semiautomated PI-QUAL software program allows the radiologist to assess the technical details related to the image quality of prostate mpMRI in a quick and reliable manner, allowing clinicians to have more confidence that the quality of mpMRI of the prostate is sufficient to determine patient care.

Key points

  • A semiautomated tool for prostate multiparametric MRI (mpMRI) quality can be built using the Prostate Imaging Quality (PI-QUAL) score.

  • Our PI-QUAL software program allows a quick assessment of the technical parameters of prostate mpMRI.

  • The application of this tool will help in the future refinement of the PI-QUAL score.

Background

Image quality plays a fundamental role in multiparametric magnetic resonance imaging (mpMRI) of the prostate; it is essential that the images are of adequate diagnostic quality in order to rule in and rule out clinically significant prostate cancer [1, 2]. In addition to the Prostate Imaging Reporting and Data System (PI-RADS) guidelines v.2.1 [3], which outline the minimum technical requirements (Table 1) and standards for the conduct and reporting of mpMRI of the prostate, there have been two important publications—one from the UK and one from the European Society of Urogenital Radiology (ESUR)/EAU Section of Urologic Imaging (ESUI)—that have stressed the importance of specific quality criteria for the acquisition of prostate mpMRI [4, 5]. In this regard, the recently proposed Prostate Imaging Quality (PI-QUAL) score [6] represents the first attempt to address this issue.

Table 1 Minimal technical requirements for multiparametric prostate MRI according to the PI-RADS v. 2.1 guidelines

This scoring system aims to assess the image quality against a set of predefined criteria (as per PI-RADS guidelines) together with objective criteria obtained from mpMRI of the prostate using a dedicated scoring sheet (Fig. 1). It is based on a 1 to 5 scale where 1 means that none of the three mpMRI sequences (i.e., T2-weighted imaging, diffusion-weighted imaging and dynamic contrast-enhanced sequences) has sufficient diagnostic quality, while a score of 5 means that all of the sequences are of optimal diagnostic quality, and therefore, it is possible to rule in and rule out clinically significant prostate cancer.

Fig. 1
figure1

Scoring sheet for assessing the quality of multiparametric magnetic resonance imaging using the PI-QUAL score. ADC, Apparent diffusion coefficient; DCE, Dynamic contrast-enhanced; DWI, Diffusion-weighted imaging; T2-WI, T2-weighted imaging. Reprinted with permission from [6]

As adequate image quality is the prerequisite for a diagnostic scan, it is essential that tools that can help the clinician evaluate the quality of a mpMRI study before reporting are developed. When evaluating the image quality, the manual extraction of technical parameters (e.g., in-plane resolution, field of view or temporal resolution) from the metadata of a mpMRI study requires a significant amount of time, which adds to the time needed for the visual assessment of the anatomical structures and for manually filling in the PI-QUAL scoring sheet.

We developed the dedicated PI-QUAL software program that allows the radiologist to evaluate the quality of mpMRI according to the PI-QUAL scoring sheet using a semiautomated step-by-step workflow. A semiautomated workflow means that some of the steps (e.g., the quantitative extraction of technical parameters from the metadata) are performed by the software while other steps (e.g., the qualitative assessment of specific anatomical structures) are manually inserted by the radiologist.

In this technical note, we discuss this PI-QUAL software program, which we have specifically developed with the intent of assisting the radiologist in daily clinical practice.

Methods

The PI-QUAL scoring sheet (Fig. 1) was created to standardise the assessment of prostate mpMRI quality and streamline the collection of data both for clinical and research purposes.

All patients included in this report gave written informed consent to have their images used for research and teaching purposes.

PI-QUAL workflow

After a general window in which the radiologist is asked to enter their name and the name of the scan site (if the program is used for research purposes), the PI-QUAL software automatically extrapolates the technical parameters of T2-weighted imaging, diffusion-weighted imaging and dynamic contrast-enhanced sequences from the raw Digital Imaging and Communications in Medicine (DICOM) images as outlined in the PI-QUAL scoring sheet (Fig. 1) and checks their compliance against the PI-RADS technical requirements. It should be noted that the PI-QUAL software has been built following PI-RADS v. 2.1 guidelines [1].

The radiologist then manually evaluates the scans for the presence (or absence) of the items listed in the ‘visual assessment’ box of the scoring sheet. Finally, the radiologist uses both results to state whether the images for each sequence are of diagnostic quality.

In the final step, the operator inserts the PI-QUAL score and has also the possibility to include additional comments and relevant snapshots from the different sequences (Figs. 2, 3 and 4).

Fig. 2
figure2

Structured report obtained using the dedicated PI-QUAL software program

Fig. 3
figure3

Structured report obtained using the dedicated PI-QUAL software program

Fig. 4
figure4

Structured report obtained using the dedicated PI-QUAL software program

Technical aspects

When the DICOM data is imported, the software parses the header and stores the elements of this information in its metadata repository for the user’s reference when browsing the patient list. The header contents of DICOM datasets are also stored in the memory for retrieval in other software functions. The PI-QUAL workflow is configured to directly read the values of specific tags embedded in the DICOM data from memory storage, then the workflow returns the values of these tags as variables for display to the user or for further manipulation in the workflow.

For example, the slice thickness of the axial T2-weighted image is pulled from the DICOM tag, then the workflow compares this value to the PI-QUAL minimum standard value (as per PI-RADS v. 2.1 guidelines) for slice thickness. The workflow displays a table to the user that contains the image-specific value, the value of the PI-QUAL minimum standard and a boolean pass (yes)/fail (no) result when comparing the image-specific value against the minimum standard value. The workflow gives the user the ability to adjust the pass (yes)/fail (no) results for any DICOM-based technical parameters and perform other visual-based assessments for the image that cannot be directly derived from DICOM information (e.g., clear delineation of certain anatomical structures).

Therefore, the overall quality scores are based upon the radiologists’ interpretations of both the automatic technical parameter results and the visual assessment according to the PI-RADS v. 2.1 guidelines.

Image analysis

In this study, two specialist consultant radiologists (C.A. and F.G. reporting more than 3,000 and 2,000 prostate MRI scans per year, respectively) analysed in consensus the image quality of 10 multiparametric scans (all without endorectal coil) from different MRI systems and vendors. First, they filled in the PI-QUAL scoring sheet manually and then, after an interval of 8 weeks between the two readings to avoid any recall bias, they re-assessed the image quality using a dedicated PI-QUAL software program (MIM® Symphony Dx v. 7.1.2 - Cleveland, OH, USA).

The time needed to assess the image quality for each scan was recorded for both methods.

Statistical analysis

Data are presented as medians and interquartile ranges (IQR) and were compared using a two-tailed Wilcoxon test. All statistical analyses were performed by using SPSS (version 27.0; SPSS, Chicago, IL, USA); p values were considered to indicate a significant difference when < 0.05.

Results

Four out of 10 (40%) patients were scanned on a 1.5-T scanner and 6/10 (60%) on a 3-T scanner. In detail, 7/10 (70%) scans were conducted on a Siemens platform (three Skyra, two Verio, one Avanto, and one Prisma), 1/10 (10%) on a General Electric platform (Signa) and 2/10 (20%) on a Philips platform (one Ingenia and one Intera).

There was a significant reduction in the reporting time of prostate MR quality using the dedicated PI-QUAL software program compared to the manual filling of the scoring sheet (5′54″ [IQR 5′40″–6′40″] versus 7′59″ [IQR 7′26″–8′29″], respectively; p = 0.005).

Discussion

Our study suggests that the use of a semiautomated PI-QUAL software program can be of help to accelerate the assessment of image quality of mpMRI of the prostate.

There has been a lot of interest in the application of artificial intelligence (AI) in radiology over the last few years [7], and currently, there are at least 100 commercially available software products [8]. As far as mpMRI of the prostate is concerned, different AI tools have been developed for a more efficient image interpretation [9,10,11,12,13] but not for the assessment of image quality, which is actually the prerequisite for a correct and reliable image interpretation.

It should be made clear that at present, our PI-QUAL tool is not based upon an AI algorithm, but it is simply a semiautomated software program that combines human and machine steps. However, our PI-QUAL software program could pave the way to the creation of a fully automated AI application by using PI-QUAL evaluations for a large subset of data to train a neural network. This could help assess the image quality during the acquisition of the different prostate mpMRI sequences, for example, recommending a new scan acquisition if the quality is suboptimal.

Since the PI-QUAL score is an aggregate over each modality, two AI approaches would be possible for each sequence. First, given an overall visual assessment, it has been shown to be possible to train an AI to mimic a human grader with no additional information than the visual assessment score [14]. This assessment could replace the entire “visual assessment” ranking for a sequence, or it could be used only on the “absence of artefact” and “adequate ADC map” scores. The second method would apply to the subscores concerning the delineation of various anatomic sub-structures. If a dataset with segmentations could be obtained, a network could be trained both to segment the structure in question and also to express confidence in the segmentation. The usual method would be to train a family of similar networks (e.g., trained on different subsets of the image data or trained with a probabilistic dropout technique) and then compare the segmentations produced. It has been shown that variance in the produced segmentations is a good correlate for the quality of the segmentation [15]. Therefore, experiments would be devised to show that if the network is unable to confidently segment a structure, it is because the structure is not clearly delineated in the image and the poor segmentation is not due to limitations of the segmentation model. If so, this technique could be used to grade the clear delineation of various structures in the T2-weighted and dynamic contrast-enhanced sequences.

In addition, our PI-QUAL tool could contribute to the creation of a cloud-based platform for multiple centres with multiple readers, in order to facilitate the assessment of the inter-reader variability of the PI-QUAL score and also to promote the use of this scoring system (and its future iterations) for clinical and teaching purposes.

There are some limitations to our study. First and foremost, the small sample size (n = 10). Second, two radiologists analysed the images in consensus, so we cannot provide the inter-reader variability of the PI-QUAL score in this cohort. However, this has been recently investigated in another cohort, and the results have shown a strong reproducibility in the assessment of PI-QUAL between two expert radiologists [16].

In conclusion, our initial results demonstrate how a semiautomated program can be used to analyse the image quality of prostate mpMRI in a quicker and reliable manner. An assessment of image quality should be performed prior to reporting mpMRI of the prostate so that the clinician and patient can be confident in the result. As stated in the original publication [2], the PI-QUAL score will be refined in the future and an international group is currently working on the next version to see if this scoring system should be still based on a 5-point scale or simplified into a 3-point scale. The application of our tool will help in the future refinement of the PI-QUAL score, and we plan periodic updates of the software and, possibly, the creation of a fully automated AI application.

Availability of data and materials

Not applicable.

Abbreviations

AI:

Artificial intelligence

DICOM:

Imaging and Communications in Medicine

IQR:

Interquartile ranges

mpMRI:

Multiparametric magnetic resonance imaging

PI-QUAL:

Prostate Imaging Quality

PI-RADS:

Prostate Imaging Reporting and Data System

References

  1. 1.

    Giannarini G, Valotto C, Girometti R, Dal Moro F, Briganti A, Padhani AR (2021) Measuring the quality of diagnostic prostate magnetic resonance imaging: a urologist’s perspective. Eur Urol 79:440–441. https://doi.org/10.1016/j.eururo.2020.09.015

    Article  PubMed  Google Scholar 

  2. 2.

    Giganti F, Kasivisvanathan V, Kirkham et al (2021) Prostate MRI quality: a critical review of the last 5 years and the role of the PI-QUAL score. Br J Radiol 20210415:20210415. https://doi.org/10.1259/bjr.20210415

  3. 3.

    Turkbey B, Rosenkrantz AB, Haider MA, et al (2019) Prostate Imaging Reporting and Data System version 2.1: 2019 update of Prostate Imaging Reporting and Data System Version 2. Eur Urol 0232:1–12. https://doi.org/10.1016/j.eururo.2019.02.033

  4. 4.

    Brizmohun Appayya M, Adshead J, Ahmed H et al (2018) National implementation of multi-parametric MRI for prostate cancer detection - recommendations from a UK consensus meeting. BJU Int 122:13–25. https://doi.org/10.1111/bju.14361

    Article  PubMed  PubMed Central  Google Scholar 

  5. 5.

    De Rooij M, Israël B, Tummers M et al (2020) ESUR/ESUI consensus statements on multi-parametric MRI for the detection of clinically significant prostate cancer: quality requirements for image acquisition, interpretation and radiologists’ training. Eur Radiol 30:5404–5416. https://doi.org/10.1007/s00330-020-06929-z

    Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Giganti F, Allen C, Emberton M, Moore CM, Kasivisvanathan V (2020) Prostate Imaging Quality (PI-QUAL): a new quality control scoring system for multiparametric magnetic resonance imaging of the prostate from the PRECISION trial. Eur Urol Oncol 3:615–619. https://doi.org/10.1016/j.euo.2020.06.007

    Article  PubMed  Google Scholar 

  7. 7.

    Wichmann JL, Willemink MJ, De Cecco CN (2021) Artificial intelligence and machine learning in radiology: current state and considerations for routine clinical implementation. Invest Radiol 55:619–627. https://doi.org/10.1097/RLI.0000000000000673

  8. 8.

    Van Leeuwen KG, Schalekamp S, Rutten MJCM, van Ginneken B, de Rooij M (2021) Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur Radiol 31:3797–3804. https://doi.org/10.1097/RLI.0000000000000673

    Article  PubMed  PubMed Central  Google Scholar 

  9. 9.

    Twilt JJ, van Leeuwen KG, Huisman HJ, Fütterer JJ, de Rooij M (2021) Artificial intelligence based algorithms for prostate cancer classification and detection on magnetic resonance imaging: a narrative review. Diagnostics (Basel) 11 :959. doi: https://doi.org/10.3390/diagnostics11060959, 6.

  10. 10.

    Stocker D, Finkenstaedt T, Kuehn b et al (2018) Performance of an automated versus a manual whole-body magnetic resonance imaging workflow. Invest Radiol 53:463–471. https://doi.org/10.1097/RLI.0000000000000471

  11. 11.

    Esser M, Zinsser D, Kündel M, et al (2020) Performance of an automated workflow for magnetic resonance imaging of the prostate: comparison with a manual workflow. Invest Radiol 55:277–284. https://doi.org/10.1097/RLI.0000000000000635

  12. 12.

    Bardis MD, Houshyar R, Chang P, et al (2020) Application of artificial intelligence to prostate multiparametric MRI (mpMRI): current and emerging trends. Cancers (Basel) 12(5):1204. doi: https://doi.org/10.3390/cancers12051204

  13. 13.

    Sanford T, Harmon SA, Turkbey EB, et al (2020) Deep-learning-based artificial intelligence for PI-RADS classification to assist multiparametric prostate MRI Interpretation: a development study. J Magn Reson Imaging 52:1499–1507. https://doi.org/10.1002/jmri.27204

  14. 14.

    Yang L, Zhang Y, Chen J, Zhang S, Chen DZ (2017) Suggestive annotation: a deep active learning framework for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017; Springer 399-407. https://doi.org/10.1007/978-3-319-66179-7_46

    Book  Google Scholar 

  15. 15.

    Piccini D, Demesmaeker R, Heerfordt J et al (2020) Deep learning to automate reference-free image quality assessment of whole-heart MR images. Radiol Artif Intell 2:e190123. https://doi.org/10.1148/ryai.2020190123

    Article  Google Scholar 

  16. 16.

    Giganti F, Dinneen E, Kasivisvanathan V et al (2021) Inter-reader agreement of the PI-QUAL score for prostate MRI quality in the NeuroSAFE PROOF trial. Eur Radiol. https://doi.org/10.1007/s00330-021-08169-1

Download references

Acknowledgements

The authors would like to express special thanks to Mr. Paul Jacobs, research team lead, at MIM Software.

Funding

FG is a recipient of the 2020 Young Investigator Award funded by the Prostate Cancer Foundation/CRIS Cancer Foundation. VK is funded by the UK National Institute for Health Research (NIHR). ME is a United Kingdom National Institute of Health Research (NIHR) senior investigator and receives research support from the UCLH/UCL NIHR Biomedical Research Centre. CMM is supported by the UKNIHR, Movember, PCUK and the EAU Research Foundation.

Author information

Affiliations

Authors

Contributions

FG and CA conceived and carried out the study. All authors contributed to the study and approved the final version.

Corresponding author

Correspondence to Francesco Giganti.

Ethics declarations

Ethics approval and consent to participate

All patients included in this report gave written informed consent to have their images used for research and teaching purposes. No IRB approval was needed for this study.

Consent for publication

Not applicable.

Competing interests

SL is an employee for MIM® Software Inc., Cleveland, OH, USA. JWP is a stockholder and employee for MIM® Software Inc., Cleveland, OH, USA. Ms. Lindner and Mr. Piper did not have any financial or business interests in this study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Giganti, F., Lindner, S., Piper, J.W. et al. Multiparametric prostate MRI quality assessment using a semi-automated PI-QUAL software program. Eur Radiol Exp 5, 48 (2021). https://doi.org/10.1186/s41747-021-00245-x

Download citation

Keywords

  • Prostatic neoplasms
  • Multiparametric magnetic resonance imaging
  • Quality improvements
  • Software