Shane Transue, Phuc Nguyen, Tam Vu, and Min-Hyung Choi IEEE - - PowerPoint PPT Presentation

▶
shane transue phuc nguyen tam vu and min hyung choi
SMART_READER_LITE
LIVE PREVIEW

Shane Transue, Phuc Nguyen, Tam Vu, and Min-Hyung Choi IEEE - - PowerPoint PPT Presentation

Dec 29, 2022 682 likes •937 views

Shane Transue, Phuc Nguyen, Tam Vu, and Min-Hyung Choi IEEE Conference on Connected Health: Applications, Systems and Engineering Technologies Introduction: Real-time Tidal Volume Estimation Tidal-volume estimation Goal: Monitor a patients

slide-1
SLIDE 1

Shane Transue, Phuc Nguyen, Tam Vu, and Min-Hyung Choi IEEE Conference on Connected Health: Applications, Systems and Engineering Technologies

slide-2
SLIDE 2

Introduction: Real-time Tidal Volume Estimation

 Tidal-volume estimation

Goal: Monitor a patient’s tidal volume remotely

Evaluate medical conditions:

Chronic Pulmonary Disease (COPD), Cystic Fibrosis

 Tidal-volume Estimation Methodologies

Accelerometers, pressure, etc. [A. Fekr et al., 2015]

Invasive deployment, expensive

 Proposed Camera-based Volume Estimation

Input: Depth-image, Estimated skeletal posture

Training: Spirometer + Camera monitoring

Output: Real-time tidal volume waveform

Camera-based Vision Challenges

Occlusion, clothing, monitoring distance, depth-image error

Assumptions: Correct posture, form-fitting clothing, limited movement Proposed: Two phase real-time tidal volume estimation using depth imaging 2/24

slide-3
SLIDE 3

Methodologies and Related Work

 Camera-based Respiratory Monitoring

Respiration rate [vs] tidal volume estimation

Recent developments for respiration rate monitoring

 Remote infrared using Kinect [A. Loblaw et al., 2013]  Real-time Vision-based Monitoring [K. Tan et al.,2010]

Recent developments within tidal volume estimation

 Chest surface monitoring [M.-C. Yu et al., 2012]  Emerging: Fine-grain tidal volume estimation (continuous)

 Surface Reconstruction

Non-restraint pulmonary test [Y. Mizobe et al., 2006]

Non-contact measurement (structured light) [H. Aoki et al., 2012]

[H. Aoki et al., 2012] [Y. Mizobe et al., 2006] [Implemented Reconstruction] Chest Surface Reconstructions 3/24

slide-4
SLIDE 4

Chest Volume Extraction Methodology

 2 Phase Monitoring System

(1)

Initial Training

Spirometer + Camera-based monitoring

Correlation: Deformation to Volume

Models surface-to-volume correlation

Defines per-patient breathing profile (2)

Real-time Monitoring

Camera-based Monitoring (no spiro)

Allows patient to breath naturally

Patient Guidelines:

Patient movement should be minimized

Correct posture should be maintained Real-time Monitoring: After training, patient can breath freely (1.0 – 2.0[m]) Input: Depth-cloud and Skeletal Structure 4/24

slide-5
SLIDE 5

Omni-directional Deformation Model

  • Deformation Model: Chest volume displacement surface modeling
  • Orthogonal Models (current methodologies)

2D distance lattice as chest surface (depth surface)

Inaccurate representation of lung displacement

Lungs act as balloons not as a planar surface

Boundary curvature information is distorted

  • Omni-directional Model

Represents natural chest displacements (lung expansion)

Unique to each patients breathing characteristics

Retains boundary curvature

Challenge: Patient movement

Cross-sectional orthogonal model view

  • f prior methods (top), introduced
  • mni-directional model (bottom)

5/24

slide-6
SLIDE 6

Chest Volume Extraction Methodology (1)

Proposed Chest Reconstruction: Monitoring device to Chest Volume Computation Overview Depth + Skeletal 6/24

slide-7
SLIDE 7

Chest Volume Extraction Methodology (2)

Proposed Chest Reconstruction: Monitoring device to Chest Volume Computation Overview Depth + Skeletal Clipping Regions 7/24

slide-8
SLIDE 8

Chest Volume Extraction Methodology (3)

Proposed Chest Reconstruction: Monitoring device to Chest Volume Computation Overview Depth + Skeletal Clipping Regions Chest Depth Cloud 8/24

slide-9
SLIDE 9

Chest Volume Extraction Methodology (4)

Proposed Chest Reconstruction: Monitoring device to Chest Volume Computation Overview Depth + Skeletal Clipping Regions Chest Depth Cloud Chest Surface 9/24

slide-10
SLIDE 10

Chest Volume Extraction Methodology (F)

Proposed Chest Reconstruction: All operations are performed per-frame Depth + Skeletal Clipping Regions Chest Depth Cloud Chest Surface Real-time Chest Mesh Volume 10/24

slide-11
SLIDE 11

Chest Surface Acquisition

 Focus region: Patient’s Chest using Skeletal Tracking

Depth-image Segmentation

Bounded region (cylinder) to clip chest region

Based on skeletal and depth data

 Stable Depth Chest Sampling:

Bit-history of stable points (only saturated points included in reconstruction)

Depth + skeletal data (left), cylinder bounding region (center), depth bit-history (right) 11/24

slide-12
SLIDE 12

Surface Normal Estimation

 Volumetric Requirements

(1) Surface boundary definition (characteristic function) (2) Surface orientation (surface normals)

 Stencil-based Surface Normal Estimation

Defined as a spatial filter (neighboring concentric squares)

Efficient computation (for real-time monitoring)

Chest-cloud with Normals 12/24

slide-13
SLIDE 13

Chest-region Surface Filling

 Clip Region Hole Filling

Clip regions require synthetic data to enclose the chest volume

Planar uniform surfaces introduced to fill holes in the surface

Chest points projected onto back-plane to fill back

 Planar hole filling algorithm

Skeletal joints (shoulders, neck, waist), with depth edges

2D Convex-hull of projected edge points

Generate synthetic grid to close hole

Generated Clip-regions: Shoulders, back, neck, and waist (a) Chest depth-cloud neck edge points, (b) planar hole fill algorithm applied to (a) 13/24

slide-14
SLIDE 14

Chest Surface Reconstruction

 Chest-deformation to Tidal Volume Estimation

Observation: Chest deformation over time indirectly correlates with tidal volume

Objective: Infer tidal volume from enclosed iso-surface and spirometer-based training

 Chest-based Iso-surface Reconstruction and Volume

Iso-surface reconstruction from oriented points (MC Variant) [M. Kazhdan, 2005]

Signed tetrahedral volume [C. Zhang and T. Chen, 2001]

14/24

slide-15
SLIDE 15

Tidal Volume Estimation

 Chest Volume to tidal Volume Correlation

Iso-volume Chest Region to Tidal Volume Estimation Mapping

Per-patient Spirometer-based Training

 Chest deformations used to infer changes in tidal volume  Quantifies relationship between chest deformations and tidal volume (per-patient)  Patient Requirement: 30 second initial training period (with spirometer)

15/24

slide-16
SLIDE 16

Tidal Volume Estimation

 Chest surface encloses arbitrary volume, does not represent tidal volume

Influenced by body shape, clothing, posture, etc.

Per-patient training establishes predictive model to estimate future tidal volume

 Volume Correlation: Bayesian Back-propagation Neural Network Training

Spirometer directly measures tidal volume

Direct chest volume measured as change dV of the patient’s chest

Data processed with simple smoothing filters (windowed zero mean, band-pass)

Inhale/Exhale Deformation 16/24

slide-17
SLIDE 17

Real-time Tidal Volume Estimation (Video)

17/24

slide-18
SLIDE 18

Initial Trial: Tidal Volume Estimation Results

(Top) Raw Data: Chest mesh volume and spirometer tidal volume (Center) Result: Correlated Result (estimated tidal volume) (Bottom) Error: Between spirometer and estimated volumes 18/24

slide-19
SLIDE 19

Initial Trial: Tidal Volume Estimation Results

 Real-time Tidal Volume Monitoring Results

Illustration of four tidal volume waveforms (unique to each patient) Initial trial with limited patient count based on the proposed training and real-time monitoring system 19/24

slide-20
SLIDE 20

Tidal Volume Estimation Error Sources

 Patient Related Sources

Patient movement (omni-directional model)

Clothing (occluding chest surface)

Patient distance results in depth-image density changes:

Closer Distances: Higher depth-image resolution, higher frame time, lower error

Further Distances: Lower depth-image resolution, lower frame time, higher error

20/24

slide-21
SLIDE 21

Tidal Volume Estimation Error Sources

 Device and Methodology Related Sources

Depth-image distance measurement errors

Patient chest region clipping (cylindrical volume)

Distance-based processing time (decreases sampling)

Closer Distances: Higher depth-image resolution, high depth accuracy, longer frame time

Further Distances: Lower depth-image resolution, low depth accuracy, shorter frame time

21/24

slide-22
SLIDE 22

Conclusion

 Novel Omni-directional deformation model

Mimics omni-directional lung deformations

Incorporates patients unique deformations

Monitors surface deformation patterns

Provides a complete 3D iso-surface

 Tidal Volume Estimation

Training: Introduces patient-specific breathing characteristics and monitoring

Enabled non-contact tidal-volume estimation in real-time (with visualization)

92.2% - 94.19% Accuracy compared to spirometer ground-truth values

22/24

slide-23
SLIDE 23

Future Work

 Continued Challenges

Video-based monitoring (occlusion, clothing, measurement errors)

Body-shape, clothing interference

Non-linear deformation to tidal volume correlation

Air is compressible (error within mesh to volume correlation)

Per-patient waveform characteristic experimentation

 Patient Requirements

Limit impact of movement (signal fluctuations)

Relax posture requirements (especially arm segmentation)

Simplification of training procedure

Curvature Analysis Chest Segmentation 23/24

slide-24
SLIDE 24

References

[1] A. R. Fekr et al., “Design and evaluation of an intelligent remote tidal volume variable monitoring system in e-health applications,” in IEEE Journal of Biomedical and Health Informatics, 2015. [2] M.-C. Yu et al., “Noncontact respiratory measurement of volume change using depth camera,” IEEE EMBS, 2012. [3] Y. Mizobe et al., “Proposal on nonrestraint pulmonary function test using active 3d measurement for body surface,” in World Congress on Medical Physics and Biomedical Engineering, vol2, pp.849-852, 2006. [4] A. Loblaw et al., “Remote respiratory sensing with an infrared camera using the Kinect infrared projector,” in the International Conference on Image Processing (IPCV), 2013. [5] K. Tan et al., “Real-time vision based respiration monitoring system,” in the International Symposium on Communication Systems Networks and Digital Signal Processing (CSNDSP), pp. 770-774, 2010. [6] P. Nguyen et al., “Poster: Continuous and fine-grained respiration volume monitoring using continuous wave radar,” in ACM Mobicom, pp. 266-268, 2015. [7] H. Aoki et al., “Non-contact respiration measurement using structured light 3-d sensor”, in SICE Annual Conference, pp. 614- 618, 2012. [8] M. Kazhdan, “Reconstruction of solid models from oriented point sets,” in Proceedings of the Third Eurographics Symposium

  • n Geometry Processing. Eurographcis Association, 2005.

[9] W. E. Lorensen and H. E. Cline, “Marching Cubes: A high resolution 3d surface construction algorithm,” in SIGGRAPH. ACM, 1987. [10] D. MacKay, “A practical Bayesian framework for back-propagation networks,” vol. 4, no. 3, pp. 448-472, 1992. [11] C. Zhang and T. Chen, “Efficient feature extraction for 2d/3d objects in mesh representation,” in Image Processing,

  • International. Conference on, vol. 3, pp. 935–938 vol.3, 2001.

24/24