Manipulation of 1D and 2D Deformable Objects Without Modeling - - PowerPoint PPT Presentation

Dmitry Berenson

Autonomous and Human-Robot Collaborative Manipulation of 1D and 2D Deformable Objects Without Modeling Deformation

Assistant Professor Robotics Engineering Computer Science

Manipulation of deformable objects

• Deformable objects are ubiquitous in the home and medical settings
• Present a method for manipulation deformable objects that does not

require modeling or simulating deformation

• Method is tested in simulation but has no knowledge of underlying

models or simulation methods

Assumptions on Sensing

• The geometry of the deformable object

can be perceived

• Though perception can be noisy
• There are one or more grippers

attached rigidly to the object

• Gripper configuration is known
• The geometry of obstacles is known

Gripper

Deformable

• bject

Target points Obstacle

• Compute a displacement

𝑟𝑢 that moves current points of deformable object 𝑄𝑢 as much as possible toward targets points 𝑈: argmin

𝑟𝑢

𝑒𝑗𝑡𝑢(𝑈, 𝑄𝑢+1)

• 𝑟𝑢 should also compensate for excessive

stretching

• 𝑟𝑢 should not bring the gripper(s) into

collision

Problem Statement

Obstacle Target points Deformable

• bject points

𝑟𝑢 𝑟𝑢+1 𝑟𝑢 𝑄𝑢 𝑈

Related Work: Modeling Deformable Objects

• Can create models of deformable objects by probing
• Online modeling is an open problem

[Lang et al., IJRR, 2002] [Cretu et al. ITIM, 2008]

Related Work: Simulating Deformable Objects

• Many methods are very sensitive to model discretization (e.g. FEM)
• Accurate simulation requires significant computation time
• Our approach avoids modeling and simulating deformation

Mass-spring model simulation [Desbrun et al., Graphics Interface, 1999] [Pezzementi et al., HIVETS, 2008] FEM simulation [Kaufmann et al., SIGGRAPH, 2008] Mesh-less model simulation [Faure et al., SIGGRAPH, 2011]

Related Work: Motion Planning for Deformable Objects

• These methods all rely on access to a deformation simulator

and accurate models

PRM for knot-tying [Saha et al., ISER, 2006] RRT in fully deformable environments [Rodriguez et al., ICRA, 2006] Pre-computing deformation to enable fast planning [Frank et al., IROS, 2011]

Related Work: Visual Servoing for Deformable Objects

• Most methods require mesh-less or spring models

Uses Jacobian of mesh-less reproducing kernel particle model [Smolen and Patriciu, ACHI, 2009] Control law based on lattice-of-springs model [Hirai and Wada, Robotica, 2000] [Wada et al., ICRA, 2001] Adaptive update to Jacobian [Navarro-Alarcon, ICRA, 2013]

• Adaptive Jacobian method

requires initial guess for Jacobian

• Number of interest points is limited

Outline

• Controller formulation
• Computing the diminishing-rigidity Jacobian
• Compensating for excessive stretching
• Avoiding Collision
• Simulation Results
• Wrapping a string around a cylinder
• Spreading a table cloth on a table
• Human-robot Collaborative Cloth Folding

Using the Jacobian

• Use Jacobian to compute displacement, given desired

displacement of points 𝑄 𝑄 = 𝐾 𝑟 𝑟

• Estimate the Jacobian 𝐾 𝑟 numerically requires simulation
• Scales poorly with number of DOF
• Computationally expensive
• Potentially inaccurate

Heuristic for Approximating Deformation

• Object exhibits diminishing rigidity near gripped points:
• Not true in all cases, but method based on this is surprisingly

effective

• Parts of the object near gripped

point move similarly to the gripper (i.e. as if rigidly attached)

• Parts that are farther away from the

gripper have smaller displacements

Approximating the Jacobian

• Approximate 𝐾 𝑟 with the diminishing rigidity Jacobian

𝐾 𝑟

1. Write Jacobian of all points of the deformable object assuming they are rigidly attached to gripper(s) 2. Scale magnitude of each point’s row in the Jacobian proportional to distance from nearest gripper

• For point 𝑗, scaling is 𝑥 = 𝑓−𝑙 𝐸(𝑗,𝑕𝑠𝑗𝑞𝑞𝑓𝑠)
• Then apply the pseudo-inverse to get

𝑟 : 𝑟 = 𝐾 𝑟 + 𝑄

Geodesic distance along object

Compensating for excessive stretching

• Assume we have knowledge of the object in its fully stretched-out state
• Pre-compute geodesics along object between each pair of points
• Online, compute separation between each pair of points
• If a pair’s separation is approaching geodesic distance, move this pair closer together
• Group all such movements in

𝑄

s

• Combine with servoing movement:

𝑟 = 𝐾 𝑟 +( 𝑄 + 𝑄

s)

Prevent collisions of gripper with obstacles

• Combine “repulsion” of grippers from obstacles with servoing motion
• Smoothly vary magnitude of repulsion relative to magnitude of servoing
• Allow servoing in null-space of collision avoidance
• For each gripper 𝑕:

𝑟𝑕

′ = 𝛿𝑕 𝐾𝑞𝑕 +

𝑦𝑞𝑕 + I − 𝐾𝑞𝑕

+ 𝐾𝑞𝑕

𝑟𝑕 + (1 − 𝛿𝑕) 𝑟𝑕

𝛿𝑕 ∈ 0,1 : proportional to gripper g distance to nearest obstacle 𝐾𝑞𝑕: Jacobian of closest point 𝑞𝑕 of gripper g to nearest obstacle 𝑦𝑞𝑕: Movement of 𝑞𝑕 away from nearest obstacle 𝐾 𝑟 +( 𝑄 + 𝑄

s)

Results

• Demonstrated the controller on 3 example tasks in Bullet simulator
• Method has no knowledge of underlying model or simulation method

used by simulator

Winding string around cylinder Spreading a tablecloth

• n a table

Human-robot collaborative cloth folding

Winding around cylinder

4x

String winding: Sensitivity to noise

• Tested with Gaussian random noise injected into sensing of

points of object

• Method degrades gracefully with noise in this example

Error

Spreading a tablecloth on a table

4x

Spreading tablecloth: Sensitivity to noise

• Tested with Gaussian random noise injected into sensing of points of
• bject
• Method can handle significant error, but diverges with 𝜏≥2.5cm (cloth

is 10mx10m)

Error

Human-robot collaborative cloth folding

• Controller matches its side of the object to user’s side
• Targets for servoing determined by reflecting points on user’s side

across plane of symmetry at each time step

User-controlled grippers Autonomous grippers Plane of Symmetry

Human-robot collaborative cloth folding

4x

Compensating for excessive stretching

4x

Conclusions

• Present a method for manipulation deformable objects that does not require

modeling or simulating deformation

• Method is tested in simulation but has no knowledge of underlying models or

simulation methods

• Future work
• Test on PR2 robot
• Make Jacobian adaptive
• Test on 3D deformable objects