Soun So und d te techn hnolog logies: ies: Topo pograp aphy - - PowerPoint PPT Presentation

SLIDE 1

So Soun und d te techn hnolog logies: ies: Topo pograp aphy hy for r qu quie iet t ar areas as an and qu d quie iet t sid ides

Jin Yong Jeon

Hanyang University, SEOUL, KOREA

4 April 2019

URBA BAN N SOU OUND ND SYMPO POSI SIUM UM

April ril 3-5, , 2019 19 in Ghen ent, t, Belgi lgium Ghent ent Uni niver versi sity ty

SLIDE 2

Soundscape undscape de desc scriptor riptors

• Pleasantness-Eventfulness model

Typical approach

Pleasant Unpleasant Eventful Uneventful Exciting Boring Calm Chaotic

Soundscape Design

불쾌한 쾌적한 활기찬 정온한 비활동적인 단조로운 혼란스러운 활동적인

Ö. Axelsson, M.E. Nilsson, and B. Berglund, “A principal components model of soundscape perception.”The Journal of the Acoustical Society of America,128(5),2836–2846,2010.

Nature appreciation and Tranquility Vibrant city life

SLIDE 3

From

• m no

noise ise contr ntrol

• l to so

soundsc undscape ape

• Soundscape concepts
• ISO 12913-1 “acoustic environment as perceived or experienced and/or understood by a person or people, in context”
• Correlation between acoustic indicators and soundscape descriptors

(Yang & Kang, 2005; Yu & Kang, 2009; Kang & Zhang, 2010; Hong & Jeon, 2013; Meng, Sun, & Kang, 2017)

• Relationship between soundscape and context (Jeon, Lee, Hong & Cabrera, 2011; Galbrun & Calarco, 2014)
• Soundscape perception model
• Correlation between soundscape and landscape

(Southworth, 1969; Pheasant, Horoshenkov & Watts, 2008; Joynt and Kang, 2010; Liu, Kang, Behm, Luo & Tao, 2014)

• Urban environment (soundscape & landscape) interpretation model

(Liu et al., 2013; 2014; Yu, Behm, Bill & Kang, 2017)

• Audio-visual interaction
• Relationship between landscape spatial patterns (e.g., urban morphology) and soundscapes for entire cities

(Ge, Lu, Morotomi & Hokao, 2009; ; Mazaris, Kallimanis, Chatzigianidis, Papadimitiou & Pantis, 2009; Liu, Kang, Behm & Coppack, 2013; Hong & Jeon, 2017)

• Audio-visual interaction on soundscape perception (traditional 2D photographs, collage, or photoshop)

(Stamps, 1993; Lange, 2001; Daniel, 2001; de Val, Atauri, & de Lucio, 2006; Hong & Jeon, 2013)

Previous studies

SLIDE 4

Rel Related ated audio io-visual visual studies udies (2 (2013 13-20 2018) 18)

SLIDE 5

• Preference
• Natural elements increase the aesthetic preference
• Laboratory experiment
• Photomontage method

Audi udio-visual visual in inter teraction action (H (Hong

• ng & Je

Jeon

• n,

, 2013) 13)

Progressive evolvement

J.Y. Hong and J. Y. Jeon, “Designing sound and visual components for enhancement of urban soundscapes” The Journal of the Acoustical Society of America,134(3),2026–2036,2013.

SLIDE 6

• Preference
• Vegetated (Ve) > concrete (Co) > wood (Ti) >

translucent acrylic (Tr) > aluminum (Al)

• Different types of noise barrier

Audi udio-visual visual in inter teraction action (H (Hong

• ng & Je

Jeon

• n,

, 2014) 4)

Progressive evolvement

Hong, J. Y. and Jeon, J. Y. “The effects of audio–visual factors on perceptions of environmental noise barrier performance,” Landsc. Urban Plan., 125, 28–37, 2014.

SLIDE 7

Inf nfluence luence of ur f urban ban mo morphology phology (H (Hong

• ng & Je

Jeon

• n,

, 2017) 7)

• Urban morphological indices

More expanded recognition model

Indicators Definition Formula Range Bldg (A) Sum of building area

෍

𝐣 𝐨

𝐂𝐣𝐞𝐡 𝐛𝐬𝐟𝐛

0.00-144543.70 Bldg (P) Sum of building perimeter

෍

𝐣 𝐨

𝐂𝐣𝐞𝐡 𝐪𝐟𝐬𝐣𝐧𝐮𝐟𝐬

0.00-4819.35 Bldsf (A) Sum of building surface area

෍

𝐣 𝐨

𝐂𝐣𝐞𝐡 𝐭𝐯𝐬𝐠𝐛𝐝𝐟 𝐛𝐬𝐟𝐛

0.00-10387198 BPAF The ratio of the plan area of buildings to the total surface area

𝐂𝐣𝐞𝐡(𝐁) 𝐇𝐬𝐣𝐞(𝐁)

0.00-0.64 CAR The summed area of roughness elements and exposed ground divided by the total surface area of the study region

𝐂𝐣𝐞𝐭𝐠(𝐁) 𝐇𝐬𝐣𝐞(𝐁)

0.22-4.82 Gr (A) Sum of green area

෍

𝐣 𝐨

𝐇𝐬𝐟𝐟𝐨 𝐛𝐬𝐟𝐛

0.00-1575.57 Gr (P) Sum of green perimeter

෍

𝐣 𝐨

𝐐𝐟𝐬𝐣𝐧𝐟𝐮𝐟𝐬 𝐩𝐠 𝐡𝐬𝐟𝐟𝐨 𝐛𝐬𝐟𝐛

0.00-557.03 Op (A) Sum of open public area including urban squares, green and water areas

෍

𝐣 𝐨

𝐏𝐪𝐟𝐨 𝐪𝐯𝐜𝐦𝐣𝐝 𝐛𝐬𝐟𝐛

0.00-15754.57 Op (P) Sum of open perimeter including urban squares, green and water feature areas

෍

𝐣 𝐨

𝐐𝐟𝐬𝐣𝐧𝐟𝐮𝐟𝐬 𝐩𝐠 𝐩𝐪𝐟𝐨 𝐪𝐯𝐜𝐦𝐣𝐝 𝐛𝐬𝐟𝐛

0.00-686.20 OSR The ratio of the open area divided by the total surface area of the study region

𝐏𝐪(𝐁) 𝐇𝐬𝐣𝐞(𝐁)

0.00-0.70 Grd (A) Sum of exposed ground area

෍

𝐣 𝐨

𝐅𝐲𝐪𝐩𝐭𝐟𝐞 𝐡𝐬𝐩𝐯𝐨𝐞 𝐛𝐬𝐟𝐛

630.29-174169.47 Rd (A) Sum of road area

෍

𝐣 𝐨

𝐒𝐩𝐛𝐞 𝐛𝐬𝐟𝐛

32.07-14765.52 EGR The ratio of the exposed ground area divided by the total surface area of the study region

𝐇𝐬𝐞(𝐁) 𝐇𝐬𝐣𝐞(𝐁)

0.03-0.77 RAF The ratio of the road area to the study region

𝐒𝐞(𝐁) 𝐇𝐬𝐣𝐞(𝐁)

0.00-0.66 Wt (A) Sum of water feature area

෍

𝐣 𝐨

𝐗𝐛𝐮𝐟𝐬 𝐠𝐟𝐛𝐮𝐯𝐬𝐟 𝐛𝐬𝐟𝐛

0.00-2106.92 Wt (P) Sum of water feature perimeter

෍

𝐣 𝐨

𝐐𝐟𝐬𝐣𝐧𝐟𝐮𝐟𝐬 𝐩𝐠 𝐱𝐛𝐮𝐟𝐬 𝐠𝐟𝐛𝐮𝐯𝐬𝐟 𝐛𝐬𝐟𝐛

0.00-342.11

• J. Y. Hong and J. Y. Jeon, “Relationship between spatiotemporal variability of soundscape and urban morphology in a multifunctional urban area: A case study in Seoul, Korea” Build Environ 2017;126:382–95.

SLIDE 8

• Main space functions
• Morphological factors

Inf nfluence luence of ur f urban ban mo morphology phology (H (Hong

• ng & Je

Jeon

• n,

, 2017) 7)

More expanded recognition model

Component 1 (41.29%) 2 (19.44%) 3 (16.81%) 4 (10.54%) Open space Gr (A) 0.89

• 0.11
• 0.04
• 0.09

Gr (P) 0.84

• 0.16
• 0.10
• 0.09

Op (A) 0.90

• 0.29

0.10 0.04 Op (P) 0.77

• 0.40

0.08 0.31 OSR 0.89

• 0.32

0.11 0.17 Building Bldg (A)

• 0.18

0.91

• 0.10
• 0.19

Bldsf (A)

• 0.30

0.87 0.06

• 0.09

Bldg (P)

• 0.09

0.75 0.06

• 0.14

BRAF

• 0.18

0.94

• 0.11
• 0.13

CAR

• 0.36

0.81

• 0.06
• 0.01

Water feature Wt (A) 0.04

• 0.20

0.10 0.96 Wt (P) 0.02

• 0.21

0.09 0.96 Road Rd (A)

• 0.23
• 0.18

0.90

• 0.16

Grd (A)

• 0.39
• 0.17
• 0.79
• 0.30

RAF

• 0.22
• 0.27

0.92 0.12 EGR

• 0.42
• 0.25
• 0.85
• 0.14
• J. Y. Hong and J. Y. Jeon, “Relationship between spatiotemporal variability of soundscape and urban morphology in a multifunctional urban area: A case study in Seoul, Korea” Build Environ 2017;126:382–95.

SLIDE 9

Inf nfluence luence of ur f urban ban mo morphology phology (H (Hong

• ng & Je

Jeon

• n,

, 2017) 7)

• Perceived affective model
• Pleasantness model was developed using LAeq, open space and water feature components
• Pleasantness model show higher R2 than eventfulness model

More expanded recognition model

• J. Y. Hong and J. Y. Jeon, “Relationship between spatiotemporal variability of soundscape and urban morphology in a multifunctional urban area: A case study in Seoul, Korea” Build Environ 2017;126:382–95.

Pleasantness Eventfulness Total P1 P2 P3 Total P1 P2 P3 R2 0.49 0.54 0.53 0.50 0.13 0.08 0.22 0.21 Acoustic Laeq

• 0.67**
• 0.45**
• 0.65**
• 0.81**

0.22** 0.02 0.32** 0.17 LCeq-Aeq

• 0.05

0.13

• 0.10
• 0.11
• 0.17**
• 0.30*
• 0.24*
• 0.04

L10-90 0.12** 0.16 0.06 0.13

• 0.11
• 0.09
• 0.15
• 0.09

Sharpness

• 0.03

0.27

• 0.07
• 0.13
• 0.06
• 0.09
• 0.05
• 0.04

Morphological Open space 0.12** 0.12 0.14*

• 0.01

0.05

• 0.01

0.17*

• 0.01

Building 0.00 0.04

• 0.06

0.09 0.11*

• 0.06

0.11 0.25** Road 0.00

• 0.02
• 0.02

0.07 0.10 0.05 0.08 0.23* Water feature 0.26** 0.24** 0.27** 0.25** 0.10 0.09 0.10 0.14

P1 (09:00–11:00), P2 (13:00–15:00) and P3 (18:00–20:00)

SLIDE 10

Soundscape undscape ma mapp pping ing (H (Hong

• ng & Je

Jeon,

• n, 2017)

7)

• Soundscape perception
• Temporal variation: pleasantness > eventfulness

Visualization

• J. Y. Hong and J. Y. Jeon, “Relationship between spatiotemporal variability of soundscape and urban morphology in a multifunctional urban area: A case study in Seoul, Korea” Build Environ 2017;126:382–95.

SLIDE 11

De Development velopment of VR f VR tool

• l

SLIDE 12

• Spatial audio in VR (Hong et al., 2017)
• Visual image
• 360 camera (Insta 360, Samsung gear, …)
• 8k ultra-high definition, 30 fps resolution
• Head mounted display (Vive, oculus, …)
• Spatial audio recording
• Stereo and surround recording
• Microphone array
• Ambisonics (well suitable)
• Binaural recording (most common used)
• Spatial audio reproduction
• Perceptual reconstruction
• Binaural, transaural
• Physical reconstruction
• Stereo, multichannel, ambisonics, wave field synthesis
• Calibration with dummy torso (B&K)

Imm mmer ersive sive so soundscape undscape ev evaluation aluation tools

• ls

Characteristics of the Acoustic Environ. Recommended Techniques Spatial Fideli Movements Virtual Sound Source Localization Reproduction Techniques Recording Techniques Listener Pos. Head Low X X 0D Mono loudspeaker; stereo headphone Mono X X 1D Stereo/surround loudspeaker; stereo headphone Stereo/surround X X 2D Surround sound Loudspeakers with height Array Ambisonics (2D) Ambisonics Med X X 3D- Ambisonics; Binaural Ambisonics; Binaural; X X 3D+ Personalized binaural (PB) Personalized binaural; Ambisonics X  3D+ Binaural/PB with head tracking Ambisonics High   3D+ WFS; Binaural/PB with positional & head tracking Mono (anechoic); Ambisonics   3D+ WFS; Binaural/PB with positional & head tracking Mono (anechoic); Ambisonics

Hong et al. Spatial Audio for soundscape Design: Recording and Reproduction. Applied Sciences 2017;7:627–49.

SLIDE 13

Val alidation idation of f VR te R techniques chniques (H (Hong

• ng et al

al., , 2019) 9)

• Compare soundscape in situ and VR environment
• In situ, FOA-static binaural, FOA-tracked binaural, FOA-2D octagonal array
• Overall soundscape quality
• No significant differences in 4 different environments
• Sufficient spatial aural fidelity
• FOA-tracked binaural play back, FOA-octagonal speaker array

Subjective attributes Acoustic reproduction methods FOA-static binaural FOA-tracked binaural FOA-2D octagonal array Overall soundscape quality Dominance of sound sources ○ ○ ○ Affective quality of soundscape ○ ○ ○ Source-related spatial attributes Distance ▲ ▲ ▲ Directivity ▲ ○ ○ Width ○ ○ ○ Distinctiveness ▲ ○ ○

Hong et al. Quality assessment of acoustic environment reproduction methods for cinematic virtual reality in soundscape applications, Building and Environment 149. 1-4 (2019)

SLIDE 14

VR R technology chnology in in in indo door

• r env

nvir ironment

• nment

SLIDE 15

Ef Effect ect of vis f visual ual in information

• rmation (Je

Jeon

• n et al

al., , 2019) 9)

• Head mounted display (HMD)
• Sound sources: water supply and drainage noise
• The difference of acceptance limit and annoyance : 6% and 8%
• J. Y. Jeon, H. I. Jo, S. M. Kim, H. S. Yang, “Subjective and objective evaluation of water-supply and drainage noises in apartment buildings by using a head-mounted display” Applied Acoustics 48:289-99 (2019).

SLIDE 16

Audi udio-visual visual in inter teraction action (Je Jeon

• n & Jo

Jo, , 2019) 9)

• HMD + Head related transfer function (HRTF)
• Sound sources: road traffic noise
• Experiment environment: None, HRTF, HMD, HRTF+HMD
• Effect of HRTF and HMD on subjective evaluation: 77% and 23 %
• Source-related spatial attributes: HRTF dominant effect
• Environment-related spatial attributes: HMD dominant effect
• J. Y. Jeon, H. I. Jo, “Three-dimensional virtual reality-based subjective evaluation of road traffic noise heard in urban high-rise residential buildings”, Building and Environment, 148, 468-477 (2019).

Source-related spatial attributes Environment-related spatial attributes

SLIDE 17

Audi udio-visual visual in inter teraction action (J (Jo

• & Je

Jeon

• n,

, 2019) 9)

• HMD + Head related transfer function (HRTF)
• Sound sources: heavy-weight impact noise
• Experiment environment: None, HRTF, HMD, HRTF+HMD
• Annoyance: HRTF dominant, HMD (higher than 53 dBA)
• Allowance limit: HRTF odd ratio (2.90) , HMD odd ratio (1.30)
• Lowered the criterion for satisfaction by 6-7 dB

Class %A LA,Fmax [dBA] NONE HRTF HMD HRTF+HMD I 0 − 20% <48.0 <41.5 <48.0 <42.5 II 20 − 40% <52.0 <45.5 <52.0 <46.0 III 40 − 60% <56.0 <49.0 <55.0 <49.0 IV 60 − 80% <59.5 <52.5 <58.0 <52.0 V 80 − 100% ≥59.5 ≥52.5 ≥58.0 ≥52.0 Participant Television Living room Balcony Kitchen

A A’ B

• H. I. Jo, J. Y. Jeon, “Downstairs resident classification characteristics for upstairs walking vibration noise in an apartment building under virtual reality environment”, Building and Environment, 150, 21-32 (2019).

SLIDE 18

VR R technology chnology in in out utdoor door env nvir ironment

• nment

SLIDE 19

Ma Main in is issu sue

• Main objective
• To investigate the correlations of the overall satisfaction of urban environment with soundscape and landscape,

respectively, and examine the influence of audio-visual interactions

Introduction

STEP 1 STEP 2

Capturing & Analysis

• f Urban Soundscape

Psychoacoustic Evaluation

STEP 3

Deployment of Soundscape Recognition model

• 3D Audio-Visual Recording
• Analyzing psychoacoustic indicators
• Psychoacoustic evaluation using VR
• Determination landscape and soundscape indicator
• Development of soundscape mapping
• Applying in the smart city

SLIDE 20

Hum uman an be beha havior vior

• Urban park soundscape
• Activity: chatting, loitering, talking on the phone, stroll, …
• Group: alone, group

Recent soundscape research

SLIDE 21

Audi udio-visual visual in inter teraction action in in VR R env nvir ironments

• nments
• Virtual reality techniques
• Recording method
• 360° Camera (Insta 360) + Soundfield microphone (SPS 200)
• Reproduction method
• Head mounted display (HMD) + 3D auralization (FOA + head tracking)

Methods

3 different experiment set-ups

1) Only audio 2) Only visual 3) Combined audio and visual

H1 H2 H3 H4 H5 H6 H7 H8 H9

H1 H2 H3 H5 H4 H6 H7 H8 H9 Locations LAeq LA10 LA50 LA90 LA10-A90 LCeq-Aeq H1 79.4 82.6 78.3 70.1 12.5 16.5 H2 71.1 73.2 70.5 60.9 12.3 9.5 H3 68.4 71.1 67.0 61.0 10.1 13.4 H4 69.4 71.8 68.5 62.9 8.9 16.5 H5 65.7 68.8 63.3 57.3 11.6 6.6 H6 72.8 75.9 66.2 60.2 15.8 9.2 H7 65.1 67.1 64.9 61.5 5.6 7.7 H8 57.2 69.4 59.9 58.0 11.4 9.3 H9 60.1 63.0 56.8 55.2 7.8 17.9

SLIDE 22

Audi udio-visual visual in inter teraction action

• Correlation between soundscape elements and landscape elements
• Soundscape and landscape components: PCA analysis
• Pleasantness show positive correlation with overall quality, regularity, naturalness
• Eventfulness show negative correlation with regularity

Discussion

Visual elements Landscape components Vehicle Building Road Open Green People Sky Overall quality Regularity Spatial impression Naturalness Sound Sources Traffic 0.72** 0.29** 0.63**

• 0.04
• 0.07

0.09 0.01

• 0.28**
• 0.22**

0.09

• 0.32**

Human 0.07 0.16** 0.06 0.04 0.05 0.43** 0.22** 0.05 0.01 0.10

• 0.09

Bird

• 0.22**
• 0.04
• 0.19**

0.18** 0.25** 0.04

• 0.05

0.30** 0.17**

• 0.01

0.23** Wind 0.05 0.13* 0.06 0.01 0.15* 0.18** 0.22** 0.11 0.12 0.12 0.08 Music

• 0.12
• 0.04
• 0.04
• 0.06

0.17**

• 0.01
• 0.14*

0.01 0.03

• 0.03

0.12** Soundscape components Pleasantness

• 0.56**
• 0.21**
• 0.53**

0.02 0.08

• 0.01
• 0.01

0.50** 0.30**

• 0.06

0.26** Eventfulness 0.05 0.08

• 0.03
• 0.12
• 0.11

0.05 0.24** 0.17**

• 0.24**

0.11

• 0.11

SLIDE 23

Soundscape undscape sa satisfaction isfaction mo model del

• Regression model using sound and visual element
• Bird sound and the visual element of vehicles are major factors
• Regression model using soundscape and landscape components
• Pleasantness, overall quality, regularity, and naturalness are major factors

Conclusion

Environment R2 Pleasantness Eventfulness Overall quality Regularity Spatial impression Naturalness Visual effect Only Audio 0.45 0.66** 0.12**

• Audio + Visual

0.32 0.56**

• 0.05**
• Audio effect

Only Visual 0.49

• 0.62**

0.28** 0.16**

• Audio + Visual

0.49

• 0.55**

0.29** 0.05 0.32** Interaction Audio + Visual 0.51 0.21**

• 0.05

0.45** 0.22** 0.06 0.25** Environment R2 Traffic Human Bird Wind Music Vehicle Building Road Open Green People Sky Visual effect Only Audio 0.14

• 0.27**

0.12* 0.12*

• 0.14*

0.02

• Audio + Visual

0.25

• 0.29**

0.03 0.31**

• 0.01

0.07

• Audio effect

Only Visual 0.15

• 0.28**

0.06

• 0.05

0.17* 0.12

• 0.08

0.05 Audio + Visual 0.21

• 0.36**
• 0.02
• 0.10

0.07 0.11 0.02

• 0.06

Interaction Audio + Visual 0.31

• 0.02

.005 0.29** 0.02 0.04

• 0.30**
• 0.03
• 0.07

0.03 0.04

• 0.01
• 0.04

SLIDE 24

Co Conclusion nclusion

• VR technology in noise evaluation
• Visual information: Head mounted display (HMD)
• Audio information
• Head related transfer function (HRTF)
• Steam audio technology: combine accurate occlusion, reflection, reverb and HRTF effects for natural sounding immersion
• Audio-visual interaction in urban environment perception
• Soundscape elements
• Dominant sound sources, perceived affective quality, and so on
• Landscape elements
• Urban morphology, dominant visual elements, human behavior, and so on
• Further plan
• Utilizing urban big-data with deep learning technology
• Real-time 3D soundscape mapping technology

3D soundscape mapping

SLIDE 25

Thank you for your attention!

jyjeon@hanyang.ac.kr www.researchgate.net/profile/Jin_Yong_Jeon