PN-005CompleteAll projects

SAAR, surgical navigation by projected augmented reality

Surgical navigation that projects the reconstruction directly onto the surgical site, instead of putting the anatomy on a screen across the room.

perceptionsurgicalaugmented reality

Line drawing of the projector rig, a cylindrical projector head carried on a motorised frame of linear rails and belt drives with a stepper motor at each end
  1. Marked at upper left of the drawing.
  2. Marked at centre left of the drawing.
  3. Marked at centre of the drawing.
  4. Marked at lower centre of the drawing.
  5. Marked at centre right of the drawing.
  6. Marked at lower right of the drawing.
  7. Marked at centre right of the drawing.

SAAR, surgical navigation by projected augmented reality

Part
PN-005
Scale
NTS
Drawn
TM
Started
TODO: date
Type
drawing
The projector head and its three-armed linkage seen from the side, the carbon-fibre arms running from the head down to their carriages on the aluminium rail
From the side
The projector head seen from directly below, its lens ring facing the camera, with the three carbon-fibre linkage arms angling down to their carriages on the aluminium rail
From under the frame

Description

The problem

A surgeon operating on something they cannot see directly works from a screen: a fluoroscopy feed, or a camera view on a monitor across the room. Every glance at it is a glance away from the patient, and the mapping between what is on the screen and what is under the hands has to be held in the surgeon’s head.

SAAR removes the screen. It projects the patient’s own anatomy, reconstructed from their scans, onto the surface of the patient, so the thing being looked at and the thing being operated on are in the same place.

How it works

A registration pipeline turns preoperative CT into a 3D reconstruction and lines that model up with the body on the table. A calibrated projector, carried on a motorised frame, casts it onto the surgical site.

Bodies move during a procedure, so a fixed projection would drift out of register within minutes. A splotch-based tracking algorithm watches the site and drives the frame to keep the overlay sitting where it should.

Two networks add to what is projected. A U-Net convolutional network segments the preoperative scans and marks internal abnormalities, and those regions are highlighted in the projection rather than left for the surgeon to find. A second network, a CNN-LSTM, predicts how organs in the chest will move. An LSTM is built to work on sequences, which is what breathing and a heartbeat are, so the overlay can follow the anatomy in real time instead of showing where it was when the scan was taken.

Where it stands

The models were trained and validated on large public CT and MRI datasets, and the accuracy of the projection alignment was measured separately. The case being made is as much economic as clinical: a projector and a frame against the cost of existing image-guidance systems.

This description is drawn from the project abstract. The fuller write-up, with per-model accuracy, alignment error and the registration maths, is not yet published here.