Time-Varying Behavior of Motion Vectors in Vection-Induced Images - - PowerPoint PPT Presentation

▶
time varying behavior of motion vectors in vection
SMART_READER_LITE
LIVE PREVIEW

Time-Varying Behavior of Motion Vectors in Vection-Induced Images - - PowerPoint PPT Presentation

May 14, 2023 45 likes •200 views

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to Autonomic Regulation Tohru Kiryu 1 , Hiroshi Yamada 1 , Masahiro Jimbo 1 , and Takehiko Bando 2 1 Graduate School of Science and Technology, 2 Graduate School of

slide-1
SLIDE 1

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to Autonomic Regulation

Tohru Kiryu1, Hiroshi Yamada1, Masahiro Jimbo1, and Takehiko Bando2

1Graduate School of Science and Technology,

2Graduate School of Medical and Dental Sciences, Niigata University, Niigata, Japan

Abstract—Virtual reality (VR) is a promising technology in biomedical engineering, but at the same time enlarges another problem called cybersickness. Aiming at suppression of cybersicknes, we are investigating the influences of vection-induced images on the autonomic regulation quantitatively. We used the motion vectors to quantify image scenes and measured electrocardiogram, blood pressure, and respiration for evaluating the autonomic regulation. Using the estimated motion vectors, we further synthesized random-dot pattern images to survey which component of the global motion vectors seriously affected the autonomic regulation. The results showed that the zoom component with a specific frequency band (0.1 – 3.0 Hz) would induce sickness. Keywords—cybersickness, autonomic regulation, motion vector, vection-induced image, random-dot pattern

slide-2
SLIDE 2

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

Background

  • Development of digital imaging technology is producing many image

formats, resolutions, frame rates, in addition to conventional factors.

  • Current digital imaging technology is also creating extraordinary special

effects that we have never seen or experienced.

  • Contrary to the benefits, digital imaging technology is widely spreading

unexpected visual stimulus.

  • Not only entertainment, but also practical problems are emerging

especially in the virtual reality (VR) or the virtual environment (VE).

  • Regarding visually induced illusions of self-motion, it has been reported

that the mismatch between visual system and vestibular system causes sickness (sensory conflict theory) . Cybersickness

slide-3
SLIDE 3

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

Approaches

  • 1. Database of Biosignals under Vection-Induced Images
  • 2. Quantifying the Image Components by Motion Vectors
  • 3. Featuring Motion Vectors around Sickness Intervals that

were Determined by Biosignals.

  • 4. Estimation of System Function by Multivariate ARX Model

2 4

  • 0.2

0.2 0.5 1 1.5 2

RRLF RRHF

120 time [sec] 50 100 0.1-3.0 Hz band power of zoom component motion of zoom component HF and LF power from R-R interval time-series

Band power of zoom component zoom component HF & LF power of R-R interval

slide-4
SLIDE 4

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

Overview

motion vector motion vector real image real image

biosignal biosignal

random dot pattern by CG random dot pattern by CG

extraction extraction synthesis synthesis biosignal biosignal

watching watching

slide-5
SLIDE 5

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

Experiments under Real & RDP Images

ECG: chest Respiration: tube sensors around the chest and the abdomen Blood Pressure: tonometry method ten healthy young subjects (eight males and two female from 21 to 24 yrs. old)

Subjects Subjects

real images real images

at Niigata University December, 2002

Measured Measured Biosignals Biosignals Parachute Bobsleigh boat boat Go cart Hang glider Mountain-bike Mountain-bike Car race Car race Bungee jump diving diving Bike race Bike race

Vehicle experiencing video

real images rest rest

3 min 3 min about 18 min

rest rest

3 min 3 min about 18 min

s pt s zpt s zt s zpt s zp s zpt

s: still, z: zoom, p: pan, t: tilt task # 3 5 7

s s

... ...

Experimental Protocol Experimental Protocol

real RDP

slide-6
SLIDE 6

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

Quantification of Image by Motion Vectors

Global Motion Vector Global Motion Vector Local Motion Vector Local Motion Vector

post frame current frame

y

x

camera tilt pan

Block matching method

x

y motion of camera motion of camera

zoom

local motion in a screen local motion in a screen

tilt

distant view

Bottom up approach

slide-7
SLIDE 7

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

(1,1) (1,2) (1,3) (1,4) (1,5) (2,1) (2,2) (2,3) (2,4) (2,5) (3,1) (3,2) (3,3) (3,4) (3,5) (4,1) (4,2) (4,3) (4,4) (4,5) (5,1) (5,2) (5,3) (5,4) (5,5) time [sec]

Correlation between Pan and Right/Left

mountain-bike mountain-bike

| correlation coefficient | 0.7

slide-8
SLIDE 8

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

(1,1) (1,2) (1,3) (1,4) (1,5) (2,1) (2,2) (2,3) (2,4) (2,5) (3,1) (3,2) (3,3) (3,4) (3,5) (4,1) (4,2) (4,3) (4,4) (4,5) (5,1) (5,2) (5,3) (5,4) (5,5) time [sec]

Correlation between Pan and Right/Left

bobsleigh

| correlation coefficient | 0.7

slide-9
SLIDE 9

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

time [sec] [magnification] Frequency [Hz] [pixel] Frequency [Hz] [pixel] Frequency [Hz]

mountain-bike mountain-bike

GMV and Time-Frequency Representation

zoom pan tilt

slide-10
SLIDE 10

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

Sickness Interval and Trigger Point

Estimation of Trigger Points

% 120 _ RR % 120 _ RR % 80 _ RR

LF ) LF HF (

  • %

120 _ BP % 120 _ BP % 80 _ RES

LF ) LF HF (

  • Sickness Interval Z

Zs

s

  • HF

HFRES

RES

HF HFRR

RR (Respiration, HRV)

(Respiration, HRV) LF LFBP

BP

LF LFRR

RR (Blood Pressure

(Blood Pressure HRV) HRV) Normalized data by values during 3-min rest

  • local minimum of LF

LFBP

BP

  • local minimum of LF

LFRR

RR

Signal Intensity

t t

  • Signal Intensity

1.8 1.4 1.0 0.6 0.2 1.8 1.4 1.0 0.6 0.2 20 40 60 80 100 120 20 40 60 80 100 120

Time [sec] time [sec]

HF HFRES

RES

HF HFRR

RR

LF LFBP

BP

LF LFRR

RR

0.8 0.8 1.2 1.2

Z Zs

s

Trigger Pointt t

slide-11
SLIDE 11

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

Distribution of Trigger Points

Trigger Points tg as a function of time

  • real image (mountain bike)
  • random dot pattern image

t1:1-10 [sec], t2:11-20 [sec], t3:21-30 [sec], t4:31-40 [sec], t5:41-50 [sec]

t5:41-50 [sec], t6:51-60 [sec] t6:51-60 [sec],

t7:61-70 [sec], t8:71-80 [sec], t9:81-90 [sec], t10:91-100 [sec]

t10:91-100 [sec], t11:101-110 [sec] t11:101-110 [sec], t12:111-120 [sec]

t tg

g= 41, 49, 93, 99 [sec]

N=19 Repetition

4 3 2 1 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12

Segment tg

20 40 60 80 100 120 Time [sec]

t tg

g= 50, 59, 103, 110 [sec]

20 40 60 80 100 120

tg N=52 Repetition

t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12

Segment

8 6 4 2 Time [sec]

slide-12
SLIDE 12

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

Time-Frequency Structure of GMVs at Trigger Points

t tg

g

frequency [ Hz ] zoom tilt pan time [ sec ] 15 10 5 15 10 5 41 41 43 45 39 47 37 35 15 10 5

t tg

g

frequency [ Hz ] zoom tilt pan time [ sec ] 15 10 5 15 10 5 93 93 95 97 91 99 89 87 15 10 5

t tg

g

frequency [ Hz ] zoom tilt pan time [ sec ] 15 10 5 15 10 5 49 49 51 53 47 55 45 43 15 10 5

t tg

g

frequency [ Hz ] time [ sec ] 15 10 5 15 10 5 99 99 101 103 97 105 95 93 15 10 5 zoom tilt pan

slide-13
SLIDE 13

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

Extraction of Trigger GMV by Similarity

Survey for Similar Time-Frequency Structure

  • frequency band: 0.01- 15 [Hz]
  • n. of bands, n = 31
  • mP( t ): mean of Power

for each section (k1, k2, , kj, )

  • interval: 3 [sec] 90 [point]
  • shift: 1 [sec] 30 [point]
  • =

+ =

n 1 i 2 i 2 i

)}) t , f ( MV (Im{ )}) t , f ( MV (Re{ ) t ( P MV( t ): GMV ( zoom, pan, tilt )

Wavelet Transform

f : frequency, t : time

V0 = ( mPzoom, mPpan, mPtilt )

(t) = cos

2 (v0 v)

v0 v

  • V( t ) = {mPzoom( t ), mPpan( t ), mPtilt ( t ) }

Similarity Similarity

time Signal Intensity

( j-1) j ( j+1)

  • t = 1, 2
  • kj-2

kj

P( t )

kj-1 Power of MV

slide-14
SLIDE 14

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

Similarity

1 0.8 0.6 0.4 0.2 10 20 30 40 50 60 70 80 90 100110

time [sec] V0 : mP 89-92[sec] V0 : mP 46-49 [sec]

Distribution of Trigger Points and Similarity of GMV

N=19 Repetition

4 3 2 1 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12

N=52 Repetition

t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 8 6 4 2

Similarity

1 0.8 0.6 0.4 0.2 10 20 30 40 50 60 70 80 90 100110

time [sec] V0 : mP 100-103[sec] V0 : mP 56-59 [sec]

Looks similar !

  • real image (mountain bike)
  • random dot pattern image

Comparison between biosignal-related index and GMV-related index

slide-15
SLIDE 15

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A utonomic RegulationIEEE EMBS04 at San FranciscoSeptember 4, 2004

  • We studied influences of vection-induced images in the relationships

between autonomic nervous activity related indices and motion vectors of images.

  • Autonomic nervous activity was evaluated from R-R interval, blood pressure,

and respiration. The motion vectors including global and local motion vectors were estimated by the data compression technique.

  • According to the time-varying behavior of motion vectors, the temporal

high-frequency (over 3 Hz) with steady low-frequency of GMVs possibly caused cybersickness.

  • Similarity function of GMV showed a similar behavior of the number of

trigger points as a function of time. Cybersickness could be predicted by the similarity function of GMV

  • However, we have not yet concluded whether the unpleasant feeling was

caused by the content of the vection-induced image or the structure of the image scene (the frame rate, the vibration of objects, etc).

Conclusion