Introduction:

Endoscopic ultrasound (EUS) is an advanced endoscopic modality requiring a high level of technical skill that allows the operator to view extraluminal structures (e.g. the pancreas). There is a growing unmet need for EUS services and interventions to accelerate trainees toward competency are required [1]. The Simbionix EUS Mentor simulator offers this potential but has not been validated. We devised and validated a curriculum for novice endosonographers utilising the EUS Mentor.

Methods:

The intervention’s validity was assessed utilising Messick’s unified validity framework [2], covering the content, response process, internal structure, relationship to other variables and consequences of the curriculum.

Eight novices and 5 experts were recruited. Each participant had standardised instructions. All novices completed a web based interactive EUS module to orientate them. Performance was rated using a validated DOPS tool (TEESAT) [3] by one expert assessor comprising a 1 - 4 Likert for each anatomical landmark and a 1 - 10 “global” assessment. Differences in expert and novice performance were assessed by Mann Whitney-U. A pass-fail score was derived using the contrasting groups method.

Results:

Content: Thirteen anatomical landmarks for identification by trainees performing simulated EUS were derived from the national training curriculum in EUS and thus corresponded with real world practise [1].

Response Process: The simulation task was standardised using written instructions and strict inclusion/exclusion criteria minimised potential bias.

Internal Structure: Internal consistency of the TEESAT assessment was assessed by comparing participants’ global rating with their cumulative TEESAT. This demonstrated a statistically significant correlation for novice echoendosonographers (R2 = -0.815, p<0.05).

Relationship to other variables: TEESAT scores for all of the 13 anatomical landmarks were significantly different between the novice and expert groups (p <0.05)

Consequences: The mean cumulative TEESAT scores for experts was 13.4 (SD = 1.52) and 40.5 (SD = 6.48) for novices. The contrasting groups method derived a pass/fail score of 19; i.e. a trainee could receive a “2” (“achieves with minimal verbal cues”) for 6/13 landmarks and a “1” (“achieves without instruction”) for the remaining 7 to achieve a “pass”.

Discussion:

The Messick framework is the gold standard for validating medical educational interventions. To date, no studies of any endoscopic simulator have utilised this framework. Here, we have demonstrated strong validity evidence for the utilisation for the Simbionix EUS mentor as a tool for novice trainees. This lays the groundwork for further studies to assess whether the EUS learning curve could be shortened.

Ethics statement:

Authors confirm that all relevant ethical standards for research conduct and dissemination have been met. The submitting author confirms that relevant ethical approval was granted, if applicable.

References

1. El Menabawey T, McCrudden R, Shetty D, et al. UK and Ireland Joint Advisory Group (JAG) consensus statements for training and certification in diagnostic endoscopic ultrasound (EUS). Gut 2024;73:118–130.

2. Cook DA, Hatala R. Validation of educational assessments: a primer for simulation and beyond. Advances in Simulation. 2016;1.

3. Wani S, Keswani R, Hall M, et al. A prospective multicenter study evaluating learning curves and competence in endoscopic ultrasound and endoscopic retrograde cholangiopancreatography among advanced endoscopy trainees: the rapid assessment of trainee endoscopy skills study. Clinical Gastroenterology and Hepatology. 2017;15:1758–1767.e11.