Using humans to analyze robot hand capabilities John Morrow HF - - PowerPoint PPT Presentation

▶
using humans to analyze robot hand capabilities
SMART_READER_LITE
LIVE PREVIEW

Using humans to analyze robot hand capabilities John Morrow HF - - PowerPoint PPT Presentation

Dec 18, 2023 255 likes •750 views

Using humans to analyze robot hand capabilities John Morrow HF Seminar 5/18/2018 Quick History of Robot Hands Morrow - Evaluating Hands 2 [Graphic borrowed from RI Seminar given by Matei Ciocarlie -

slide-1
SLIDE 1

Using humans to analyze robot hand capabilities

John Morrow HF Seminar 5/18/2018

slide-2
SLIDE 2

Morrow - Evaluating Hands 2

Quick History of Robot Hands

[Graphic borrowed from RI Seminar given by Matei Ciocarlie - https://www.youtube.com/watch?v=wiTQ6qOR8o4]

slide-3
SLIDE 3

Morrow - Evaluating Hands 3

Quick History of Robot Hands

slide-4
SLIDE 4

Morrow - Evaluating Hands 4

Fully-Actuated Hands

[Jacobsen et al, 1986; Bae et al, 2011]

slide-5
SLIDE 5

Morrow - Evaluating Hands 5

Quick History of Robot Hands

slide-6
SLIDE 6

Morrow - Evaluating Hands 6

Quick History of Robot Hands

slide-7
SLIDE 7

Morrow - Evaluating Hands 7

Under-Actuated Hands

https://www.youtube.com/watch?v=BMLJBVPb7qM

slide-8
SLIDE 8

Morrow - Evaluating Hands 8

Under-Actuated Hands

[Odhner et al, 2012]

slide-9
SLIDE 9

Morrow - Evaluating Hands 9

Where do we go from here?

slide-10
SLIDE 10

Morrow - Evaluating Hands 10

Our Questions:

What is the most effective addition we can make to our robot hands? How can we evaluate these hands?

slide-11
SLIDE 11

Morrow - Evaluating Hands 11

What additions are others making?

[Ma and Dollar, 2016; Odhner et al, 2012; Aukes et al, 2014]

slide-12
SLIDE 12

Morrow - Evaluating Hands 12

Versatility

slide-13
SLIDE 13

Morrow - Evaluating Hands 13

Physical Human Interactive Guidance

[Balasubramanian et al, 2012]

slide-14
SLIDE 14

Morrow - Evaluating Hands 14

The Power of Human Grasping

77% vs. 97%

Humans Robots

[Balasubramanian et al, 2012]

slide-15
SLIDE 15

Morrow - Evaluating Hands 15

The Power of Human Grasping

93% vs. 97%

Humans Robots

[Balasubramanian et al, 2012]

slide-16
SLIDE 16

Morrow - Evaluating Hands 16

Physical Human Interactive Guidance

[Balasubramanian et al, 2012]

slide-17
SLIDE 17

Morrow - Evaluating Hands 17

Our Study

  • 18 subjects, 2 hands per subject
  • 10 minute warm-up
  • Two tasks:

– Drawing with a pen – Spraying a spray bottle

  • Comparing hands that were...

– Under-Actuated – Fully-Actuated – Fully-Actuated and Compliant

slide-18
SLIDE 18

Morrow - Evaluating Hands 18

Barrett Hand (BH)

  • Under-actuated
  • Controlled via sliders
  • Limited joint

movement

– Coupled per finger

[Townsend et al, 2000]

slide-19
SLIDE 19

Morrow - Evaluating Hands 19

Posable Barrett Hand (PH)

  • Fully-actuated
  • Controlled by hand
  • No limitations on joint

pose

slide-20
SLIDE 20

Morrow - Evaluating Hands 20

OpenHand Model O (OH)

  • Fully-actuated
  • Controlled by hand
  • Compliant joints can

twist

slide-21
SLIDE 21

Morrow - Evaluating Hands 21

Pen Task

slide-22
SLIDE 22

Morrow - Evaluating Hands 23

Spray Task

slide-23
SLIDE 23

Morrow - Evaluating Hands 24

Results

Metric Task BH PH OH

  • Avg. Task

Completion (sec)

Bowl

191 256 85

Spray

398 344 163 Avg Manipulation Time (%)

Bowl

30% 21% 22%

Spray

23% 22% 21% Avg Attempted Grasps

Bowl

3 6 2

Spray

4 5 2

slide-24
SLIDE 24

Morrow - Evaluating Hands 25

Results

slide-25
SLIDE 25

Morrow - Evaluating Hands 26

Results

slide-26
SLIDE 26

Morrow - Evaluating Hands 27

Hand Comparisons

Metric Task BH PH OH

  • Avg. Task

Completion (sec)

Bowl

191 256 85

Spray

398 344 163 Avg Manipulation Time (%)

Bowl

30% 21% 22%

Spray

23% 22% 21% Avg Attempted Grasps

Bowl

3 6 2

Spray

4 5 2

slide-27
SLIDE 27

Morrow - Evaluating Hands 28

Survey Data

slide-28
SLIDE 28

Morrow - Evaluating Hands 29

What is the PH missing?

slide-29
SLIDE 29

Morrow - Evaluating Hands 30

Study Limitations

  • No data for OH
  • Finger ‘loops’ difficult to use
  • ‘Powered’ by humans

– Not the same interface

slide-30
SLIDE 30

Morrow - Evaluating Hands 31

Our Questions:

What is the most effective addition we can make to our robot hands? How can we evaluate these hands?

slide-31
SLIDE 31

Morrow - Evaluating Hands 32

Conclusions

  • Refined control of distal link

– Bending it backwards – Twisting it

  • Evaluating hands with humans as the guide
  • Observations:

– Soft finger pads are important to us –

slide-32
SLIDE 32

Morrow - Evaluating Hands 33

References

  • Bae, Ji-Hun, et al. "Development of a low cost anthropomorphic robot hand with high capability." IEEE/RSJ International

Conference on Intelligent Robots and Systems (IROS), 2012 .

  • Jacobsen, Steve, et al. "Design of the Utah/MIT dextrous hand." IEEE International Conference on Robotics and Automation.
  • Proceedings. Vol. 3. IEEE, 1986.
  • Lael U. Odhner, Raymond R. Ma, and Aaron M. Dollar, "Precision grasping and manipulation of small objects from flat surfaces

using underactuated fingers," 2012 IEEE International Conference on Robotics and Automation (ICRA), pp.2830-2835, 14-18 May 2012.

  • Raymond R. Ma and Aaron M. Dollar, "In-Hand Manipulation Primitives for a Minimal, Underactuated Gripper with Active

Surfaces," ASME International Design Engineering Technical Conferences (IDETC), 2016.

  • Aukes, Daniel M., et al. "Design and testing of a selectively compliant underactuated hand." The International Journal of

Robotics Research 33.5 (2014): 721-735.

  • Balasubramanian, Ravi, et al. "Physical human interactive guidance: Identifying grasping principles from human-planned

grasps." IEEE Transactions on Robotics 28.4 (2012): 899-910.

  • Townsend, W. T. "MCB—industrial robot feature article—Barrett hand grasper." Industrial Robot: An International Journal 27.3

(2000): 181-188.

slide-33
SLIDE 33

Expertise modeling and training: Manual 3D Image Segmentation Process Cindy Grimm

Ruth West (UNT) Chris Sanchez (OSU) Anahita Sanandaji (PhD) Max Parola, Meghan Kajihara, Deniece Yates, Brandon Lane

slide-34
SLIDE 34

Problem area and goals

  • There are many tasks that require a mix of spatial ability, domain

knowledge, verbal skills, and mathematical skills

  • Mechanics of solid materials
  • 2D/3D image segmentation
  • Biological processes
  • How do we transfer (spatial) knowledge from experts to novices?
  • Capture and representation of expert’s mental models
  • Decomposition into hierarchical and orthogonal skills
  • Training materials
  • Tools
  • [Capture] Eye-tracking, EMG, video, spatial relationships
  • [Training] AI, Virtual/Augmented reality, computer interfaces

1

slide-35
SLIDE 35

3D Image Segmentation

§ A fundamental process in:

  • Scientific and medical applications

§ Medical Imaging and Segmentation

  • Locating tumors
  • Measuring tissue volumes
  • Computer guided surgery

§ Performed (or evaluated) on 2D slices of the 3D data

  • Stack of CT Scans

2

Stack of CT scan of a liver

slide-36
SLIDE 36

3D Image Segmentation Approach

§ Drawing contours on selected cross-sections by human experts

7

slide-37
SLIDE 37

3D Image Segmentation Approach

§ Drawing contours on selected cross-sections by human experts

8

Selected cross-section of a developing chicken heart

Contour s

slide-38
SLIDE 38

Time-intensive Process

§ Performing segmentation manually on a slice-by-slice basis

9

Manual segmentation by human experts Reconstruction

slide-39
SLIDE 39

Expertise: What does it consist of?

§ Domain knowledge – expected shapes/patterns/shape

relationships

§ Spatial skills

  • Relationship of 2D slices to 3D shape
  • Relationship of image properties to contours
  • Software interface: Amira, etc, are big, complicated

pieces of software

10

slide-40
SLIDE 40

Expertise: How do we capture it?

§ Field studies: In-depth and per-expert analysis

  • Eye-tracking (spatial data)
  • Video and audio recoding
  • Task analysis
  • Retrospective think aloud

11

slide-41
SLIDE 41

Analysis Methodology: Combining spatial/visual data with task structure

  • Primary source of analysis : Observation data (gaze and actions)
  • Match to: Participant’s mental task model

12

slide-42
SLIDE 42

Analysis: Conceptual task -> gaze and actions

13

P5 Expert

D r a w Navigation E d i t

P3 Novice

D r a w Navigation E d i t

Action Pairs: Novice (P3) vs Expert (P5) Origin to Subsequent action pairs for expert vs novice for the task Annotate Cell Same site 1 novice 1 expert Same task

slide-43
SLIDE 43

Repeated Task Results

20

slide-44
SLIDE 44

Task structures

P4 P5 P6 P7 TASKS ST-L1 ST-L2 ST-L3 TASKS ST-L1 ST-L2 TASKS ST-L1 ST-L2 TASKS ST-L1

21

slide-45
SLIDE 45

Insights and Hypothesis

§ Comparing novices to experts:

  • Have better knowledge of 3D structures
  • Tend to use 3D views more frequently

§ Mental model hypothesis:

  • Given a 3D structure and slicing plane experts can:
  • 1. Predict the 2D contour
  • 2. Predict how 2D contour changes
  • 3. Identify invalid 2D contours
  • 4. Image characteristics that correspond to boundaries

22

slide-46
SLIDE 46

Next step: Domain-independent expertise transfer Spatial skills

23 Category Base:0 Range of Difficulty

Viewpoint View Point -Fixed

Base 0: All 4 attributes are true (3/3) 1. Simple object* AND 2. Viewpoint wrt. the plane is orth. AND 3. Viewpoint wrt. the object is orth. to the major/minor axis Level 1: Only one of the base attributes is false (2/3) Level 2: Only one of the base attributes is true (1/3) Level 3: None of the base attributes is true (0/3)

View Point -Rotating

Base 0: All 3 attributes are true (2/2) 1. Simple object AND 2. Simple Rotation (Having viewpoint rotation around major/ minor axis of the object OR rotation around
  • ne of the axes of the plane)
Level 1: Only one of the base attributes is true (1/2) Level 2: None of the base attributes is true (0/2)

View Point -Freeform

Base 0: Get to the Base viewpoint fixed AKA “Simplest Viewpoint” with 0 mental rotation (See View Point-Fixed Base:0 for the attributes) Level 1: Get to the Simplest Viewpoint with only 1 mental rotation Level 2: Get to the Simplest Viewpoint with 2 mental rotations Level 3: Get to the Simplest Viewpoint with 3 mental rotations Level 4: There does not exist a Simplest Viewpoint (e.g., Object is complex): In this case complexity increases if we have no bases rotations

Mental Rotation/Translation (Adjust the plane to complete a task)

Base 0: Complete the assigned task* with 0 translation AND 0 rotation of the plane wrt to the object Level of complexity increases with increasing the number of translation and rotations

2D Object Representation from Cut- away

Base 0: All Attributes are true (2/2) 1. From Primitive 3D object 2. From an orthogonal plane wrt to object (parallel to the cross-section) Level 1: Only one of the base attributes is true (1/2) Example: oval from a cylinder with non-orthogonal plane Level 2: None of the base attributes is true (0/2) Example: Branching structure with non-orthogonal plane

3D Object Representation

Base 0: Primitive Objects OR Symmetric simple organic (e.g. symmetric potato) shapes Level 1: Attached/Nested Primitive objects Level 2: Symmetric nested organic objects Level 3: Asymmetric simple organic objects Level 4: Asymmetric nested organic objects
slide-47
SLIDE 47

Training

  • Measure of spatial abilities
  • By category
  • For cross-section task : Survey
  • Training tool
  • Guided skill set introduction
  • Customized help/instruction based on failure mode
  • Completed study (30 control, 30 train)
  • Increases spatial abilities in identified areas
  • Doesn’t in other area (paper folding)

24

slide-48
SLIDE 48

Where from here?

  • Haven’t shown that it improves segmentation task
  • Give training tool to our original expertise study participants
  • Different domain: Mechanics of solids
  • Verbal + mathematical + spatial
  • Lots of expertise modeling (textbooks, instructors)
  • Make interactive and 3D
  • Modeling students skill-set (verbal, spatial, mathematical)
  • Option 1: Use interactivity & 3D to reduce spatial skills needed
  • Option 2: Specific training to increase spatial skills
  • Capture expert decision making and use it to evaluate/guide student

25