Multitouch Puppetry Creating coordinated 3D motion for an - PowerPoint PPT Presentation

Multitouch Puppetry Creating coordinated 3D motion for an articulated arm Michael Kipp Quan Nguyen DFKI Embodied Agents Research Group Cluster of Excellence Multimodal Computing and Interaction Saarbrücken, Germany

Motivation • Non-experts perform 3D character animation in realtime... from their desktops • Why? ➡ animations for games / online worlds / fun ➡ „pose“ scenes for movie / theater ➡ dance choreography ➡ produce sign language ➡ teleoperate robots

Character Animation • Two ways: ➡ Pose-to-pose: precise, final production ➡ Straight ahead: creative, improvisational • Straight-ahead animation ideally has parallel realtime control of all degrees of freedom

Performance Animation • Use physical widgets, recognized by camera, to create motions [Oore et al. 02, Dontcheva et al. 03] ➡ special hardware ➡ fatigue • Use simple mouse input, exploit correlations between body parts [Neff et al. 07] • Shortcomings ➡ „layering“ of motion ➡ no solutions for hand shape

Problem & Scope shoulder elbow wrist hand shape

Problem & Scope Arm swivel Move wrist and use (1 DOF) Inverse Kinematics (IK) for resolving other joints => 3 DOFs hand hand shape orientation (many DOFs) (1 DOF)

Multitouch Interface • As many input DOFs as possible ➡ bimanual, multi-finger • Easy to learn ➡ standard interaction techniques (mostly) • Directness of control ➡ touch & output co-located ➡ no direct manipulation (occlusion of hand shape)

Dominant Hand: Arm x/y z swivel

Nondominant Hand: Hand • Index finger: hand shape ➡ High-dimensional space ➡ Few hand shapes suffice (open, fist, thumbs up...) ➡ 2D morph map • 2-Finger rotation: hand orientation

Nondominant Hand: Hand shape hand orient. 7 DOF bimanual control

closed spread out Technically: max. 3 shapes for interpolation similar shapes close base shapes Design guidelines: good „pass-through“ in center memorable „themes“ specific

User Study

User Study • Long-term study: 7 sessions (2 weeks) • Participants: 6 students (3+3 male/female), age 21-30 • Hardware: 15.4“ screen • Conditions: multitouch vs. mouse • Three layered design to capture performance & user experience • Within-subject: participants compared mouse and multitouch (measures, questionnaire)

Control condition • Mouse interface • Motion → frontal plane • Wheel → depth • Keyboard key → wheel mode ➡ arm swivel ➡ hand rotation

Task 1: Match Pose • Test precision of interface with matching task (docking) • Target pose shown with ghost arm (10 poses per session) • Bimanual control Hand orientation Hand position Hand shape Arm swivel

Task 1: Match Pose

Task 2: Trace Trajectory • Encourage fluid, coordinated 3D motion • Traces from motion captured gestures (10 per session) • Moving highlight indicates direction • Unimanual control Hand position Arm swivel

Task 2: Trace Trajectory

Task 3: Creative Exploration • Qualitative test of complete interface • No „fair“ control condition • Task: Create „appropriate“ motion for ➡ song (strong beat) ➡ song (slow ballad) ➡ voice track (male harsh) ➡ voice track (female melodic)

Non-Expert Subjects in the Creative Task Full multitouch interface Hand shape Hand position Hand orientation Arm swivel

Results Multitouch • Measure completion time (tasks 1+2) • Pilot studies P i l o t s t u d y ➡ Mouse outperformed MT • Hypothesis ➡ Mouse beats MT in first sessions but both converge after... Mouse

Final Results pose matching tracing

Measuring Coordination • Existing measures ➡ efficiency [Zhai, Milgram 98] and parallelism [Balakrishnan, Hinckley 99] require an optimal z target path optimum ➡ integrality [Jacob et al. 94] ignores quantity • Suggestion ➡ assume that maximum coordination happens if xy distance travelled along each dimension is equal N − 1 min(d xy ,d z ) ➡ penalize digression from this optimum 1 ∑ ➡ compare distance travelled in each dimension max(d xy ,d z ) N 0

Coordination in Session 7 Multitouch yields significantly higher coordination for both tasks 1 + 2 (p < .05)

Questionnaires • 16 questions • 5-point differential scale: -2 Mouse ... +2 MT • Bonferroni correction: alpha of .003 • For 6 Q significant differences from 0, all in favour of MT: ➡ Which interface was more useful ? ➡ Which interface allowed a faster solution of the tasks? ➡ With which interface were you more satisfied ? ➡ Which one would you recommend to others? ➡ Which device was more fun to use? ➡ Which interface do you prefer ?

Discussion • Performance & user experience ➡ Big leap from session 1 to 2 => multi-session studies ➡ MT feels faster, although objectively equal, probably due to higher coordination ➡ Comfortable only in session 2 • MT yields higher coordination ➡ < 0.1 bad coordination ➡ > 0.2 already good

Conclusion • Goal: Continuous, parallel control of high-dimensional spaces • Multitouch interface for coordinated arm motion Try it! • Create & eval complex interaction techniques for concrete application (3D) tasks • Contributions ➡ bimanual 7-DOF input arm animation interface ➡ 3-layered study design ➡ Novel coordination measure • Future [Neff et al. 06] ! u o y ➡ prototyping tool (dataflow) k n a h T ➡ scale up to whole body