No institutional review board was needed for this prospective phantom study.
AR in situ US
The AR in situ US system is composed of a conventional US system (SuperSonic Aixplorer Ultimate, SuperSonic Imagine, Aix-en-Provence, France), custom developed software, industry-grade head-mounted AR displays (Microsoft Hololens, Redmond, Washington, USA) and physical extensions of the US transducer handles (Fig. 1).
The positioning of the US image is based on calculating a relative offset of the image to the tracking marker on the US probe with respect to the head-mounted device coordinate system. To perform the offset calculation, a calibration station was used with an exact cut-out for the probe head and whose geometrical dimensions are precisely known. The station also has a tracking marker, which is recognised by the head-mounted device. To properly register, the US image is placed relative to the station marker according to the known offset to the cut-out. From this positioning, the offset to the probe’s tracker marker is measured within the head-mounted device’s coordinate system.
By observing the tracking pattern attached to the US transducer, the system calculates the correct positioning of the US image in space. The same tracking pattern is used to first calibrate the system together with the interpupillary distance of the operator’s eyes. Using these coordinate system transformations and the information attained in the calibration step, the US image is transferred form the conventional US monitor to the AR headset. Thus, this transferred image is directly superimposed to the imaged anatomical structures. While scanning with the AR in situ US, the displayed superimposed image is update in real time as the operator moves the transducer, displaying the anatomical structures at their correct anatomical location in a 1:1 scale.
Additionally, to the superimposed imaged displayed in a 1:1 scale, the AR system also displays an enlarged version of the same image in an overhead position in the virtual three-dimensional space. This allows the operator to consult both the superimposed 1:1 image at the anatomical location, as well as the same, but larger, image when looking up.
The accurate image size is calibrated based on the scanning depth and width given by the external US system. Changes in any of these two parameters make recalibration of our system necessary. Usually, this information is readily available from the US devices. The US image is displayed on a two-dimensional plane with the correct scan depth and width. The plane is then rendered onto the head-mounted device display and is scaled, positioned, and rotated in such a way that it is perceived at the correct three-dimensional location. For example, for in situ visualisation, the plane is shown beneath the US probe where the image is generated.
Phantom puncture
Three untrained operators (orthopaedic surgeons) without any experience in US and two experienced radiologists (with 9 and 11 years of training) performed 200 US-guided punctures with a linear transducer using a conventional US (SL 18-5, SuperSonic Aixplorer, SuperSonic Imagine, France) and with the AR in situ US (Microsoft Hololens and SuperSonic Aixplorer, SuperSonic Imagine, Aix-en-Provence, France).
The punctures were performed using a 20-gauge yellow needle (7-cm long) in a leg phantom (leg model with soft tissue biopsy insert, blue phantom, CAE Healthcare, Synmedic AG, Zurich, Switzerland) including 20 soft tissue lesions of varying lesions ranging from 4 to 11 mm in diameter. The leg phantom size was 81 × 20 × 20 cm; the weight 11.5 kg. Ten of those lesions were marked and numbered, ensuring that each operator punctured the same lesions in the same order (Figs. 2 and 3a). Further, each operator had to puncture the same vessel (6 mm in size) in a vessel phantom (Blue phantom, CAE Healthcare, Synmedic AG, Zurich, Switzerland) ten times (Fig. 3b). These punctures were initially performed using AR in situ US, and > 4 weeks later, each operator repeated the punctures using the conventional US, in order to avoid a possible training effect after the first puncture session. The location of the needle tip was documented after each puncture by an experienced radiologist in US in the leg phantom and verified by fluid aspiration in the vessel phantom. Time to puncture and number of needle passes as well as location (correct or incorrect) of needle tips were documented for each puncture using the Redcap software (Vanderbilt University Medical Center, Version 8.11.5, Nashville, USA) [18].
Data presentation
Descriptive statistic was performed using the software PRISM (Version 8, Graphpad software, La Jolla (CA), USA). Medians and ranges were used to report the non-parametric data. By purpose, being this a proof-of mechanism phantom study for which sample size was not preliminarily estimated, and based on consultation with the statistician, we did not perform statistical testing for significance.