The Language of Surgery: Build the Alphabet With Us (Part 2)

In Part 1, we discussed how and why surgical knowledge is being lost, and that the language of surgery is being written without surgeons. As stewards of the field, students of your mentors, and representatives of your patients, it should be the surgical community that takes the lead and builds the systems that will define how surgery is measured, taught, and improved in the future.

With that in mind… surgical video has become increasingly important over the past decade. However, on its own, it’s not enough to make a huge change in the field. Not only that, but the hours it takes to review, compare, and learn from difficult footage – these are not hours that surgeons have to spare.

The key idea is legibility.

We want someone who did not watch the entire operation to still understand what actually happened. To do this, SDSC began turning surgical video into data by identifying instruments, tracking instrument movements, identifying surgical phases, and converting unstructured video into structured information. We are beginning to turn a surgical video from a recording into a series of signals that we can analyze, compare, and learn from.

The beginning of making surgery readable.

How it all began

Thanks to NASBS, this work started ten years ago. During Dr. Donoho’s time at the University of Southern California (USC) with Dr. Gabriel Zada, he trained hundreds of surgeons to manage internal carotid artery injury in pituitary surgery. This is an adverse scenario estimated to occur in 0.1-1% of transsphenoidal pituitary cases, and it frequently causes trouble because it requires the rapid employment of a set of skills and maneuvers that are very idiosyncratic and rarely practiced.¹

On this occasion, Dr. Donoho approached the training program in an entirely unconventional way. All the raw videos and data needed to make those surgeries legible were saved and published, and with the sheer volume of standardized videos from the simulator, he was able to assess the video frames and manually catalogue them to create the first publicly available pituitary surgery video dataset.² Then, methods were created to make the videos readable by machines, subsequently helping surgeons to see and understand data that is too vast for human eyes and minds.

With the help of colleagues at Stanford University, SDSC was able to produce an automated performance metrics (APM) report that matches what human beings can provide during coaching and training.³ APMs could be used to assess surgeons for proficiency and provide specific, actionable feedback; it showed us what was possible in the field, and now we are turning to the real operating room.

Video → Data → Understanding

The transformation from video to insight happens in stages: first, we capture surgical video; second, we extract the data from that video; finally, we extract meaning.

This pipeline is how we are beginning to make surgery legible at scale, and we are starting to see this take shape in real world applications of SDSC’s work:

1. Our work in Endoscopic Third Ventriculostomy with Choroid Plexus Cauterization (ETV-CPC) is using surgical video to better understand technique and variation.

• The ETV-CPC machine learning model – available to all users on Surgical Video Platform (SVP) – analyzes ETV/CPC surgeries to help surgeons understand both what they did and how well they did it. Find out more here.

2. The Surgical Metrics Scoring System (SMSS), now available on SVP.

• SMSS is an automated system that analyzes tool-tracking data from Endoscopic Endonasal Approach (EEA) procedures to generate a surgical technique score from 0 to 100.
• The score is derived from a set of twelve objective metrics, including tool movement efficiency, workspace coverage, instrument switching patterns, and idle time.
• These are measured across different phases of the operation - nasal, sphenoid, sellar, and closure - without requiring manual annotation or expert review.
• The score is not significant by itself, but it represents a shift from subjective assessment to measurable, reproducible signals of surgical performance.

We and others (see image below) believe that intraoperative metrics are both correlated with intraoperative events, like cerebrospinal fluid (CSF) leaks, and postoperative outcomes, like tumor resections rates, even after controlling for tumor size and invasiveness.

It’s interesting to see that there is a new dimension no one has seen before. We can receive powerful signals of both process and outcomes, suggesting that we can map the relationship between surgeon performance, surgical processes, and outcomes in a way that could translate to any given surgical procedure. 

This is a fundamentally new way of studying surgery.

Build the alphabet with us

It is important to note that we are at the very beginning of this journey. We don’t know if these metrics will be the final ones, or if they will work in every case. But there is one thing we are certain about: the signals are clear. There are messages in the surgical videos we ignore every day that we can decode and study at scale.

Part 1 was about why surgery needs a language; Part 2 is about how that language is beginning to take shape.

There is a long way to go, but the direction is clear: video to data, data to understanding, understanding to improvement. The language must be built piece by piece, for surgeons, by surgeons. We invite you to help build the alphabet with us. For more information on how to get involved, please contact info@surgicalvideo.io. 

‍

^{1. Mishra R, Konar SK, Shukla DP. Internal carotid artery injury during the endoscopic transsphenoidal surgery of pituitary adenoma: Case illustration, introspection, and systematic review. Acta Neurochirurgica Supplement. 2024 Nov 22;89–97. doi:10.1007/978-3-031-61601-3_15} 

^{2. Donoho DA, Pangal DJ, Kugener G, Rutkowski M, Micko A, Shahrestani S, et al. Improved surgeon performance following cadaveric simulation of internal carotid artery injury during endoscopic endonasal surgery: Training outcomes of a nationwide prospective educational intervention. Journal of Neurosurgery. 2021 Nov;135(5):1347–55. doi:10.3171/2020.9.jns202672}

^{3. Pangal DJ, Kugener G, Cardinal T, Lechtholz-Zey E, Collet C, Lasky S, et al. Use of surgical video–based automated performance metrics to predict blood loss and success of simulated Vascular Injury Control in Neurosurgery: A pilot study. Journal of Neurosurgery. 2022 Sept 1;137(3):840–9. doi:10.3171/2021.10.jns211064}