.png){kind=icon}
.png){kind=icon}
Return
From XR-derived performance metrics to an objective performance score in point-of-care ultrasound
.png){kind=icon}
OA
AI
T
D
K
Y
M
M
F
B
A
DOI：10.1186/s12909-026-10034-y.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Point-of-care ultrasound (POCUS) quality depends on operator proficiency. As POCUS training expands, scalable approaches to performance evaluation are needed. Current performance evaluation relies largely on examiner-intensive Objective Structured Clinical Examinations (OSCE). Extended Reality (XR) technologies offer opportunities to capture objective performance metrics and provide the digitization needed to support automated and standardised performance evaluations. The objective of this study was to explore the use of XR-derived performance metrics and examine how an automated performance score using these metrics relates to OSCE performance. Medical students (novices), physicians with limited ultrasound experience (intermediates), and experienced physicians and sonographers (experts) performed a cardiac ultrasound task on a phantom, while using a phone as a probe and wearing a HoloLens that immerses them in an XR environment. They were tasked to obtain four cardiac POCUS views. OSCE was scored by two independent experts. Probe trajectories were derived from the phone and were used to extract kinematic metrics, such as time, path length, straightness index, and dimensionless squared jerk. These were complemented with image quality surrogates (position and angle deviation), and gaze behaviour (gaze shifts to the display). A metric-based performance score was constructed using multivariable regression and compared with OSCE scores. Ninety participants were included (34 novices, 31 intermediates, 25 experts). The OSCE scoring demonstrated excellent inter-rater reliability (ICC 0.97) and clear discrimination across experience levels. Kinematic and gaze metrics aligned with experience level and OSCE performance, whereas image quality surrogates showed weaker associations. Factor analysis of the XR-derived performance metrics identified two underlying domains of technical performance; probe handling and image quality, with gaze behaviour emerging as a distinct construct. The automated performance score showed a moderate to good correlation with OSCE performance (ρ = 0.69). Replacing XR-derived image quality surrogates with expert-rated image quality substantially improved associations (ρ = 0.91). XR-derived, automated performance scoring represents a promising, scalable approach for formative feedback and early-stage screening, particularly in novice learners, while reducing examiner burden. These findings lay the foundation for further development toward fully automated ultrasound performance tracking. Further validation is required before use in high-stakes or fully automated assessment.
Keywords:
Point-of-care ultrasound
Extended Reality
Simulation-based training
Automated performance evaluation
Kinematic metrics
Gaze behaviour
Objective structured clinical examination
Journal
IF:
3.2
Papers:
2.9K
Citations:
2.0W
Organization
Cited Papers