Haptics for Surgical Simulation CPSC 599.86 / 601.86 Sonny Chan - - PowerPoint PPT Presentation

▶
haptics for surgical simulation
SMART_READER_LITE
LIVE PREVIEW

Haptics for Surgical Simulation CPSC 599.86 / 601.86 Sonny Chan - - PowerPoint PPT Presentation

Sep 02, 2022 227 likes •554 views

Haptics for Surgical Simulation CPSC 599.86 / 601.86 Sonny Chan University of Calgary Motivation I taught you all this great, futuristic stuff this semester - but where would I ever use it? Few applications can support the high cost

slide-1
SLIDE 1

Haptics for Surgical Simulation

CPSC 599.86 / 601.86 Sonny Chan University of Calgary

slide-2
SLIDE 2

Motivation

  • I taught you all this great,

futuristic stuff this semester…

  • but where would I ever use it?
  • Few applications can support

the high cost of haptic devices

  • Luckily for us (me), the medical

community is well funded

  • surgery is a high-risk endeavour with

very, very costly mistakes!

slide-3
SLIDE 3

A C C E P T E D

NeuroTouch (NRC Canada) and LAP Mentor (Simbionix)

slide-4
SLIDE 4
slide-5
SLIDE 5

neuroArm project, University of Calgary

slide-6
SLIDE 6

Opportunities for Surgical Simulation

What one is the simulator?

slide-7
SLIDE 7

My Research Mission

Virtual Surgical Environment Operating Room

slide-8
SLIDE 8

Volumetric Isosurfaces

slide-9
SLIDE 9

Volume Rendering

  • Implicit representations are now

uncommon, but...

  • 3D medical imaging (CT, MRI,

etc.) has resulted in an abundance of volume data

  • Can be rendered with (almost)

the same algorithm for implicit surfaces!

slide-10
SLIDE 10

Sampled Volume Data

  • Image values sampled on rectilinear grid
  • CT scans measure radiodensity in Hounsfield Units
slide-11
SLIDE 11

Interpolation

What is the value here?

(0,0) (1,1) 0.3 0.6 0.7 0.9 F(x, y) = (1 x + bxc)(1 y + byc) I(bxc, byc) + (x bxc)(1 y + byc) I(dxe, byc) + (1 x + bxc)(y byc) I(bxc, dye) + (x bxc)(y byc) I(dxe, dye) for x, y, z 2 R

slide-12
SLIDE 12

Isocontours & Isosurfaces

Choose a threshold value, T, to determine the surface function

S(x, y, z) = T − F(x, y, z)

T = -600 HU T = 300 HU

slide-13
SLIDE 13

Isosurfaces in 3D

slide-14
SLIDE 14

Rendering Algorithm

  • Our implicit surface rendering

algorithm has two specific requirements:

  • Inside-outside function for S(p)
  • The gradient of S(p)

S(x, y, z) = T − F(x, y, z) rS(x, y, z) = ???

slide-15
SLIDE 15

Gradient Estimation

  • Estimate gradient using central

difference:

  • What’s the best choice for the δ

value?

rS(x, y) = ∂S

∂x ∂S ∂y

! ⇡ @

S(x+δ,y)−S(x−δ,y) 2δ S(x,y+δ)−S(x,y−δ) 2δ

1 A

∂S/∂x ∂S/∂y

slide-16
SLIDE 16

Isosurface Rendering

  • The full rendering algorithm:
  • Detect initial contact when S(p) < 0
  • Find surface point with initial seed
  • Update surface point as the device

moves by using tangent plane

  • Contact breaks when device is

moved outside the constraint plane

  • Repeat from start…
  • Any issues?
slide-17
SLIDE 17

Demo?

slide-18
SLIDE 18

Deformable Tissue Simulation

slide-19
SLIDE 19

Continuum Mechanics

  • Governs the stress/strain

behaviour of materials

  • Includes Young’s modulus

(elasticity) and Poisson ratio (incompressibility)

  • Think of it as an “advanced”

version of mass-spring systems

[from continuummechanics.org]

slide-20
SLIDE 20

Finite Element Simulation

  • Partitions an object into small, discrete elements
  • Computes continuum mechanics on solid element mesh

x x ) ( x x − ) (

1

x x R −

− e

x R

1 − e e

R ) (

1

x x R K R −

− e e e

) (

1

x x R K −

− e e

plastic

f

total

f

elastic

f

plastic

  • f
  • riginal state

plastic rest state deformed state

[from M. Müller & M. Gross, Proc. Graphics Interface, 2004]

slide-21
SLIDE 21

Data Representation

isosurface CT tetrahedral mesh

slide-22
SLIDE 22

Data Processing

slide-23
SLIDE 23

Deformation Simulation

  • Co-rotational linear elastic FEM
  • Computes nodal displacements

from external forces

  • Implementation from VEGA FEM

library (Jernej Barbič, USC)

slide-24
SLIDE 24

Haptic Rendering

  • Extension of original isosurface

rendering algorithm:

  • sampling deformed volume
  • force transfer
  • moving the proxy
slide-25
SLIDE 25

Sampling the Deformed Volume

  • How do we find the image intensity value

at a given location in deformed space, x?

p0

deformed tetrahedron

rest vertices barycentric coordinates  xm 1

  • =

 r0 r1 r2 r3 1 1 1 1  p0 p1 p2 p3 1 1 1 1 −1  x 1

  • r2

r1 r0 xm

x

p1 p2

slide-26
SLIDE 26

Force Transfer

  • Virtual coupling force is applied to mesh nodes with barycentric weighting

xp x

Fc F1 F2 F0

slide-27
SLIDE 27

Moving the Proxy

  • To avoid interpenetration, we must keep the proxy outside the isosurface, even

when the deformable mesh elements themselves move!

  • If a tetrahedron’s vertices move,
  • we displace the proxy’s with the tetrahedron:

x0

p

1

  • =

 q0 q1 q2 q3 1 1 1 1  p0 p1 p2 p3 1 1 1 1 1  xp 1

  • pi → qi
slide-28
SLIDE 28

Architecture

Haptic Rendering Deformation Simulation Visual Rendering

forces displacements displacements

1000 Hz ~10-100 Hz 60 Hz

slide-29
SLIDE 29

Results

slide-30
SLIDE 30

Results

slide-31
SLIDE 31

Summary

  • Surgical simulation is a popular

and promising application of high-fidelity computer haptics

  • can tolerate cost of hardware
  • Sampled volumetric images

(medical image data) can be described as implicit surfaces

  • adapt the implicit surface rendering

algorithm to compute forces