People Watching: Human Actions as a Cue for Single-View Geometry - - PowerPoint PPT Presentation

▶
people watching human actions as a cue for single view
SMART_READER_LITE
LIVE PREVIEW

People Watching: Human Actions as a Cue for Single-View Geometry - - PowerPoint PPT Presentation

Jan 22, 2024 42 likes •387 views

People Watching: Human Actions as a Cue for Single-View Geometry David Fouhey, Vincent Delaitre, Abhinav Gupta, Alexei Efros, Ivan Laptev, Josef Sivic Presented by Ashwini Venkatesh Slide Credit: Fouhey et al Slide Credit: Fouhey et al Where

slide-1
SLIDE 1

Slide Credit: Fouhey et al Slide Credit: Fouhey et al

People Watching: Human Actions as a Cue for Single-View Geometry

David Fouhey, Vincent Delaitre, Abhinav Gupta, Alexei Efros, Ivan Laptev, Josef Sivic Presented by Ashwini Venkatesh

slide-2
SLIDE 2

Slide Credit: Fouhey et al

Where are the people?

slide-3
SLIDE 3

Slide Credit: Fouhey et al

People – Cues not Clutter

slide-4
SLIDE 4

Slide Credit: Fouhey et al

Goal – Inverse Problem

Sensors ! Clutter

slide-5
SLIDE 5

Slide Credit: Fouhey et al

Approach

Pose Detections

Timelapse

slide-6
SLIDE 6

Slide Credit: Fouhey et al

Approach

Pose Detections

Estimate Functional Regions from Poses

Timelapse

slide-7
SLIDE 7

Slide Credit: Fouhey et al

Approach

Functional Regions Pose Detections Timelapse

3D Room Hypotheses From Appearance

slide-8
SLIDE 8

Slide Credit: Fouhey et al

Approach

Functional Regions Pose Detections Timelapse

Score 3D Room Hypotheses With Appearances + Affordances #1 #49

slide-9
SLIDE 9

Slide Credit: Fouhey et al

Approach

Functional Regions Pose Detections Timelapse

Estimate Free-Space

Pose Detections

Estimate Free-Space

Pose Detections

Estimate Free-Space

slide-10
SLIDE 10

Slide Credit: Fouhey et al

Approach

Functional Regions Pose Detections Timelapse Pose Detections Pose Detections

slide-11
SLIDE 11

Slide Credit: Fouhey et al

Detecting Human Actions

Standing Sitting Reaching Standing Sitting

Train Separate Detectors for Each Pose

Deformable Parts Model Articulated Pose Estimator

slide-12
SLIDE 12

Slide Credit: Fouhey et al

Approach

Functional Regions Pose Detections Timelapse Pose Detections Pose Detections

slide-13
SLIDE 13

Slide Credit: Fouhey et al

From Poses to Functional Regions

Sittable Regions at Pelvic Joint

slide-14
SLIDE 14

Slide Credit: Fouhey et al

From Poses to Functional Regions

Walkable Regions at Feet

slide-15
SLIDE 15

Slide Credit: Fouhey et al

From Poses to Functional Regions

Reachable Regions at Hands

slide-16
SLIDE 16

Slide Credit: Fouhey et al

Approach

Pose Detections Timelapse

#1 #49

Functional Regions Pose Detections

#1 #49

Pose Detections

#1 #49

slide-17
SLIDE 17

Slide Credit: Fouhey et al

3D Room Hypotheses

Vanishing-point aligned hypotheses

slide-18
SLIDE 18

Slide Credit: Fouhey et al

Constraints

Containment

Volume occupied by human should be inside a room

Support

Object surfaces which can make the pose physically stable

Free Space

Volume occupied by human cannot intersect any object in the room

slide-19
SLIDE 19

Slide Credit: Fouhey et al

Putting it together – picking a room

φ ( )+ψ ( , , )+ρ( )

Compatibility of room layout with surface geometry

slide-20
SLIDE 20

Slide Credit: Fouhey et al

Putting it together – picking a room

φ ( )+ψ ( , , )+ρ( )

Compatibility of human poses and room layout

slide-21
SLIDE 21

Slide Credit: Fouhey et al

Putting it together – picking a room

φ ( )+ψ ( , , )+ρ( )

Relative room size regularizer

slide-22
SLIDE 22

Slide Credit: Fouhey et al Slide Credit: Fouhey et al

Reranking Results

#1 #49

Appearance Alone Appearance + People

...

#82 #1

Score = -1.7754 Score = -1.8859 Score = -1.8865

...

Score = -2.0319

slide-23
SLIDE 23

Slide Credit: Fouhey et al

Approach

Functional Regions Pose Detections Timelapse

Estimate Free-Space

Pose Detections

Estimate Free-Space

Pose Detections

Estimate Free-Space

slide-24
SLIDE 24

Slide Credit: Fouhey et al

Estimating free space

LEGEND Floor Wall 1 Wall 2 Wall 3 Ceiling Clutter

Hedau et al. ’09

slide-25
SLIDE 25

Slide Credit: Fouhey et al

Estimating Free Space

slide-26
SLIDE 26

Slide Credit: Fouhey et al

Estimating Free Space

slide-27
SLIDE 27

Slide Credit: Fouhey et al

Results

slide-28
SLIDE 28

Slide Credit: Fouhey et al Slide Credit: Fouhey et al

Qualitative Example

slide-29
SLIDE 29

Slide Credit: Fouhey et al Slide Credit: Fouhey et al

Qualitative Results

Appearance Alone

slide-30
SLIDE 30

Slide Credit: Fouhey et al Slide Credit: Fouhey et al

Qualitative Results

Appearance + People

slide-31
SLIDE 31

Slide Credit: Fouhey et al

Single Images with People

Appearance Alone

slide-32
SLIDE 32

Slide Credit: Fouhey et al

Single Images with People

Appearance + People

slide-33
SLIDE 33

Slide Credit: Fouhey et al Slide Credit: Fouhey et al

Quantitative Results

Location Appearance Only People Only Appearance + People Lee et al. '09 Hedau et al. '09 Timelapses 64.1%

70.4% 74.9% 70.8% 82.5%

Single Image

66.4% 71.3% 77.0% 79.6%

slide-34
SLIDE 34

Slide Credit: Fouhey et al

Discussion Points

  • 1. Do some human action exceptions such as can sitting on

table, standing on a sofa cause trouble for the algorithm?

  • 2. Role of background subtraction during testing
  • 3. Semantic relationships between humans and objects
  • 4. Performance on odd-shaped rooms and outdoor scenes
  • 5. Evaluation metric used - only evaluates the 3D room scene

and the free space estimate