Direct Skinning Methods and Deformation Primitives Ladislav Kavan - - PowerPoint PPT Presentation

โ–ถ
direct skinning methods and
SMART_READER_LITE
LIVE PREVIEW

Direct Skinning Methods and Deformation Primitives Ladislav Kavan - - PowerPoint PPT Presentation

May 31, 2023 48 likes โ€ข1.09k views

Skinning: Real-time Shape Deformation Direct Skinning Methods and Deformation Primitives Ladislav Kavan University of Pennsylvania 1 Variational vs. direct methods Variational Direct (numerical optimization) (closed-form) =

slide-1
SLIDE 1

Ladislav Kavan

University of Pennsylvania Skinning: Real-time Shape Deformation

Direct Skinning Methods and Deformation Primitives

1

slide-2
SLIDE 2

Variational vs. direct methods

๐ฐโ€ฒ = arg min

๐ฒ

E( ๐ฒ )

Variational (numerical optimization) Direct (closed-form)

๐ฐ๐‘—

โ€ฒ = ๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜๐ฐ๐‘—

2

slide-3
SLIDE 3

Variational vs. direct methods

Variational (numerical optimization) Direct (closed-form)

๐ฐ๐‘—

โ€ฒ = ๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜๐ฐ๐‘—

๐ฐโ€ฒ = arg min

๐ฒ

E( ๐ฒ )

3

slide-4
SLIDE 4

Linear blend skinning: basic setup

4

slide-5
SLIDE 5

1) Rest pose

๐ฐ๐‘—

5

slide-6
SLIDE 6

2) Skinning transformations

๐”

1

๐”2

6

slide-7
SLIDE 7

3) Skinning weights

1

๐‘ฅ๐‘—,1

7

slide-8
SLIDE 8

3) Skinning weights

1

๐‘ฅ๐‘—,2

8

slide-9
SLIDE 9

Linear blend skinning (LBS)

๐ฐ๐‘—

โ€ฒ = ๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜๐ฐ๐‘— = ๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜

๐ฐ๐‘—

9

slide-10
SLIDE 10

Linear blend skinning (LBS)

๐ฐ๐‘—

โ€ฒ = ๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜๐ฐ๐‘— = ๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜

๐ฐ๐‘—

10

slide-11
SLIDE 11

Linear blend skinning (LBS)

๐ฐ๐‘—

โ€ฒ = ๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜๐ฐ๐‘— = ๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜

๐ฐ๐‘—

11

slide-12
SLIDE 12

LBS is used widely in the industry

Halo 3 Bolt

12

slide-13
SLIDE 13

LBS: candy-wrapper artifact

13

slide-14
SLIDE 14

LBS: candy-wrapper artifact

14

slide-15
SLIDE 15

Advanced skinning methods

  • 1. Multi-linear methods
  • 2. Nonlinear methods
  • 3. Advanced deformation primitives

(deformers)

15

slide-16
SLIDE 16

Multi-linear skinning techniques

Multi-weight enveloping [Wang and Phillips 2002] Animation Space [Merry et al. 2006]

16

slide-17
SLIDE 17

General linear skinning model

๐ฐโ€ฒ = ๐˜ ๐ฎ

17

slide-18
SLIDE 18

General linear skinning model

๐ฐโ€ฒ = ๐˜ ๐ฎ

18

slide-19
SLIDE 19

General linear skinning model

๐ฐโ€ฒ = ๐˜ ๐ฎ ๐ฎ โˆˆ ๐‘†12๐‘› ๐ฎ stacks skinning transformations [ T1, โ€ฆ , Tm ]

19

slide-20
SLIDE 20

General linear skinning model

๐ฐโ€ฒ = ๐˜ ๐ฎ

20

slide-21
SLIDE 21

General linear skinning model

๐ฐโ€ฒ = ๐˜ ๐ฎ

21

slide-22
SLIDE 22

General linear skinning model

๐ฐโ€ฒ = ๐˜ ๐ฎ ๐˜ โˆˆ ๐‘†3๐‘œ ร— 12๐‘› Aggregates rest pose & weights

22

slide-23
SLIDE 23

General linear skinning model

๐ฐโ€ฒ = ๐˜ ๐ฎ

23

slide-24
SLIDE 24

General linear skinning model

๐ฐโ€ฒ = ๐˜ ๐ฎ

24

slide-25
SLIDE 25

The structure of matrix X

๐˜ = ๐˜1,1 โ‹ฏ ๐˜1,๐‘› โ‹ฎ โ‹ฑ โ‹ฎ ๐˜๐‘œ,1 โ‹ฏ ๐˜๐‘œ,๐‘› ๐˜ โˆˆ ๐‘†3๐‘œ ร— 12๐‘› ๐˜๐‘—,๐‘˜ โˆˆ ๐‘†3 ร— 12

25

slide-26
SLIDE 26

LBS in matrix form

๐˜๐‘—,๐‘˜

LBS = ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,1๐‰3

๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,2๐‰3 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,3๐‰3 ๐‘ฅ๐‘—,๐‘˜๐‰3 ๐‰3 = 1 1 1 ๐ฐ๐‘— = ๐‘ค๐‘—,1 ๐‘ค๐‘—,2 ๐‘ค๐‘—,3

26

slide-27
SLIDE 27

LBS in matrix form

๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,1 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,1 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,1 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,2 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,2 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,2 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,3 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,3 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,3 ๐‘ฅ๐‘—,๐‘˜ ๐‘ฅ๐‘—,๐‘˜ ๐‘ฅ๐‘—,๐‘˜

27

slide-28
SLIDE 28

LBS in matrix form

๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,1 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,1 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,1 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,2 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,2 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,2 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,3 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,3 ๐‘ฅ๐‘—,๐‘˜๐‘ค๐‘—,3 ๐‘ฅ๐‘—,๐‘˜ ๐‘ฅ๐‘—,๐‘˜ ๐‘ฅ๐‘—,๐‘˜

There is just 1 weight (per vertex/bone pair)

28

slide-29
SLIDE 29

Multi-weight Enveloping [Wang and Phillips 2002]

๐‘ฅ๐‘—,๐‘˜

1 ๐‘ค๐‘—,1

๐‘ฅ๐‘—,๐‘˜

2 ๐‘ค๐‘—,1

๐‘ฅ๐‘—,๐‘˜

3 ๐‘ค๐‘—,1

๐‘ฅ๐‘—,๐‘˜

4 ๐‘ค๐‘—,2

๐‘ฅ๐‘—,๐‘˜

5 ๐‘ค๐‘—,2

๐‘ฅ๐‘—,๐‘˜

6 ๐‘ค๐‘—,2

๐‘ฅ๐‘—,๐‘˜

7 ๐‘ค๐‘—,3

๐‘ฅ๐‘—,๐‘˜

8 ๐‘ค๐‘—,3

๐‘ฅ๐‘—,๐‘˜

9 ๐‘ค๐‘—,3

๐‘ฅ๐‘—,๐‘˜

10

๐‘ฅ๐‘—,๐‘˜

11

๐‘ฅ๐‘—,๐‘˜

12 29

slide-30
SLIDE 30

Multi-weight Enveloping [Wang and Phillips 2002]

๐‘ฅ๐‘—,๐‘˜

1 ๐‘ค๐‘—,1

๐‘ฅ๐‘—,๐‘˜

2 ๐‘ค๐‘—,1

๐‘ฅ๐‘—,๐‘˜

3 ๐‘ค๐‘—,1

๐‘ฅ๐‘—,๐‘˜

4 ๐‘ค๐‘—,2

๐‘ฅ๐‘—,๐‘˜

5 ๐‘ค๐‘—,2

๐‘ฅ๐‘—,๐‘˜

6 ๐‘ค๐‘—,2

๐‘ฅ๐‘—,๐‘˜

7 ๐‘ค๐‘—,3

๐‘ฅ๐‘—,๐‘˜

8 ๐‘ค๐‘—,3

๐‘ฅ๐‘—,๐‘˜

9 ๐‘ค๐‘—,3

๐‘ฅ๐‘—,๐‘˜

10

๐‘ฅ๐‘—,๐‘˜

11

๐‘ฅ๐‘—,๐‘˜

12

12 different weights (per vertex/bone pair)

30

slide-31
SLIDE 31

Multi-weight Enveloping [Wang and Phillips 2002]

More powerful LBS MWE

31

Weights trained from examples

slide-32
SLIDE 32

Animation Space [Merry et al. 2006]

MWE is โ€œtoo generalโ€ if we want world-space rotation invariance.

32

slide-33
SLIDE 33

Rotation-invariant linear skinning

๐˜๐‘—,๐‘˜ = ๐‘ง๐‘—,๐‘˜,1๐‰3 ๐‘ง๐‘—,๐‘˜,2๐‰3 ๐‘ง๐‘—,๐‘˜,3๐‰3 ๐‘ง๐‘—,๐‘˜,4๐‰3

4 weights per vertex/bone pair

๐‘˜=1 ๐‘›

๐‘ง๐‘—,๐‘˜,4 = 1

Translation invariance:

33

slide-34
SLIDE 34

Rotation-invariant linear skinning

๐‘ง๐‘—,๐‘˜,1 ๐‘ง๐‘—,๐‘˜,1 ๐‘ง๐‘—,๐‘˜,1 ๐‘ง๐‘—,๐‘˜,2 ๐‘ง๐‘—,๐‘˜,2 ๐‘ง๐‘—,๐‘˜,2 ๐‘ง๐‘—,๐‘˜,3 ๐‘ง๐‘—,๐‘˜,3 ๐‘ง๐‘—,๐‘˜,3 ๐‘ง๐‘—,๐‘˜,4 ๐‘ง๐‘—,๐‘˜,4 ๐‘ง๐‘—,๐‘˜,4

Animation Space [Merry et al. 2006]

34

slide-35
SLIDE 35

Animation Space [Merry et al. 2006]

LBS Animation space

35

slide-36
SLIDE 36

Summary of multi-linear methods

Improve quality by introducing more weights

36

slide-37
SLIDE 37

Nonlinear methods

๐ฐ๐‘—

โ€ฒ = ๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜๐ฐ๐‘— = ๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜

๐ฐ๐‘— The LBS formula: Assume the transformations ๐”

๐‘˜ are rigid

37

slide-38
SLIDE 38

Geometry of linear blending

SE(3)

38

slide-39
SLIDE 39

Geometry of linear blending

๐”

1

๐”2 SE(3)

39

slide-40
SLIDE 40

Geometry of linear blending

๐”

1

๐”2 ๐”blend SE(3)

40

slide-41
SLIDE 41

Geometry of linear blending

๐”

1

๐”2 ๐”blend SE(3)

41

slide-42
SLIDE 42

Geometry of linear blending

๐‘บ1 = 1 1 1 ๐‘บ2 = โˆ’1 โˆ’1 1

42

slide-43
SLIDE 43

Intrinsic blending

๐”

1

๐”2 ๐”blend

43

slide-44
SLIDE 44

Intrinsic blending

๐”

1

๐”2 ๐”blend

44

slide-45
SLIDE 45

Intrinsic blending

๐”

1

๐”2 ๐”blend

45

slide-46
SLIDE 46

Intrinsic blending using Lie algebras

[Buss and Fillmore 2001, Alexa 2002, Govindu 2004, Rossignac and Vinacua 2011]

arg min

๐˜ ๐‘˜=1 ๐‘›

๐‘ฅ

๐‘˜๐‘’(๐˜, ๐” ๐‘˜)

๐‘’ ๐˜, ๐™ = log(๐™๐˜โˆ’1) 2

46

Karcher / Frechet mean

slide-47
SLIDE 47

Intrinsic blending using Lie algebras

[Buss and Fillmore 2001, Alexa 2002, Govindu 2004, Rossignac and Vinacua 2011]

arg min

๐˜ ๐‘˜=1 ๐‘›

๐‘ฅ

๐‘˜๐‘’(๐˜, ๐” ๐‘˜)

๐‘’ ๐˜, ๐™ = log(๐™๐˜โˆ’1) 2

47

Karcher / Frechet mean

slide-48
SLIDE 48

Dual quaternions: approximation of intrinsic averages in SE(3)

[Kavan et al. 2008]

๐”

1 โˆˆ SE(3) can be converted to a unit dual

quaternion ๐ซ โˆˆ Q1

48

slide-49
SLIDE 49

Dual quaternions and rigid motion

49

slide-50
SLIDE 50

Dual quaternion skinning

[Kavan et al. 2008]

One to one correspondence between SE(3) and Q1?

No!

50

slide-51
SLIDE 51

Dual quaternion skinning

[Kavan et al. 2008]

๐ซ and โˆ’ ๐ซ represent the same transformation Q1 is a double cover of SE(3)

51

slide-52
SLIDE 52

Double cover visualized

๐”

1

๐”2 SE(3) Q1 ๐ซ1 ๐ซ2

52

slide-53
SLIDE 53

The advantages of dual quaternions

1. Q1 is โ€œless curvedโ€ than SE(3)

  • 2. Fast, closed-form projection on

Q1 ๐ซ1 ๐ซ2 ๐ซblend ๐ซprojected

53

slide-54
SLIDE 54

Dual quaternion skinning

[Kavan et al. 2008]

54

slide-55
SLIDE 55

Limitations of dual quaternions

Non-rigid ๐”

๐‘˜ not directly supported

Open problem: double cover of SL(3)

55

slide-56
SLIDE 56

Non-rigid transformations can be done with two-phase skinning

  • 1. Scaling
  • 2. Rotations

56

slide-57
SLIDE 57

Two-phase skinning

Enhanced for production use by Disney

[Lee et al. 2013]

57

slide-58
SLIDE 58

Neither matrices nor quaternions remember multiple revolutions

LBS & DQS: shortest path interpolation

58

Not always desired

slide-59
SLIDE 59

Linear blend skinning

59

slide-60
SLIDE 60

Dual quaternion skinning

60

slide-61
SLIDE 61

Differential blending [Oztireli et al. 2013]

root ๐ซ๐‘˜ ๐ซ๐‘˜,1 ๐ซ๐‘˜,2 ๐ซ๐‘˜,3

61

slide-62
SLIDE 62

Differential blending [Oztireli et al. 2013]

DiffBlend ๐‘ฅ

๐‘˜; ๐ซ๐‘˜ = ๐‘™=1 ๐‘š

Blend(๐‘ฅ

๐‘˜; ๐ซ๐‘˜,๐‘™)

62

slide-63
SLIDE 63

Differential blending [Oztireli et al. 2013]

63

slide-64
SLIDE 64

Dual quaternion bulging artifact

64

slide-65
SLIDE 65

Dual quaternion bulging artifact

65

slide-66
SLIDE 66

Dual quaternion bulging artifact

Not always undesirable (knuckles) Artists want control

66

Autodesk Maya: blend LBS and DQS Finer control is desirable

slide-67
SLIDE 67

Advanced deformation primitives (deformers)

Basic building block of deformations

67

slide-68
SLIDE 68

In LBS & DQS: deformers are skinning transformations

๐”

1

๐”2

68

slide-69
SLIDE 69

Curve-based deformers

Wires [Singh and Fiume 1998]

[Kalra et al. 1998] [Hyun et al. 2005]

69

slide-70
SLIDE 70

Spline-based skinning [Forstmann et al. 2007]

Linear blending Dual quaternions Spline skinning

70

slide-71
SLIDE 71

71

Deformers blended linearly (like LBS)

Spline-based skinning [Forstmann et al. 2007]

slide-72
SLIDE 72

Offset curve deformation

[Sederberg 2012]

Bind skin to offset curves

[Gregory and Weston 2008]

72

slide-73
SLIDE 73

Stretchable and twistable bones

Standard skinning transforms donโ€™t stretch well

73

[Jacobson and Sorkine 2011]

slide-74
SLIDE 74

Stretchable and twistable bones

Standard skinning transforms donโ€™t stretch well

74

[Jacobson and Sorkine 2011]

slide-75
SLIDE 75

Stretching results in shape explosion

75

slide-76
SLIDE 76

Twisting must be packed at joints

76

slide-77
SLIDE 77

Solution: endpoint weights

77

1

[Jacobson and Sorkine 2011]

slide-78
SLIDE 78

Stretchable and twistable bones

78

[Jacobson and Sorkine 2011]

slide-79
SLIDE 79

Stretchable and twistable bones

79

[Jacobson and Sorkine 2011]

slide-80
SLIDE 80

How to fix dual quaternion bulging?

80

slide-81
SLIDE 81

Joint-based deformers [Kavan and Sorkine 2012]

x y z

81

slide-82
SLIDE 82

๐‘†3 โ†’ ๐‘†3

82

Joint-based deformers [Kavan and Sorkine 2012]

slide-83
SLIDE 83

๐‘‡๐‘ƒ 3 ร— ๐‘†3 โ†’ ๐‘†3

83

Joint-based deformers [Kavan and Sorkine 2012]

slide-84
SLIDE 84

ฮ“: ๐‘‡๐‘ƒ 3 ร— ๐‘†3 โ†’ ๐‘†3 ฮ“ ๐‘น, ๐’š = ๐’šโ€ฒ

84

Joint-based deformers [Kavan and Sorkine 2012]

slide-85
SLIDE 85

Individual deformers blended linearly

x y z

ฮ“

๐‘“๐‘š๐‘๐‘๐‘ฅ

85

slide-86
SLIDE 86

Individual deformers blended linearly

x y z

ฮ“

๐‘“๐‘š๐‘๐‘๐‘ฅ

86

slide-87
SLIDE 87

Individual deformers blended linearly

x y z

ฮ“

๐‘กโ„Ž๐‘๐‘ฃ๐‘š๐‘’๐‘“๐‘ 

87

slide-88
SLIDE 88

Weights of bone-based deformers

88

slide-89
SLIDE 89

Weights of bone-based deformers

89

slide-90
SLIDE 90

Weights of joint-based deformers

90

slide-91
SLIDE 91

Weights of joint-based deformers

91

slide-92
SLIDE 92

Swing/twist deformer [Kavan and Sorkine 2012]

LBS DQS

92

slide-93
SLIDE 93

Swing/twist deformer [Kavan and Sorkine 2012]

ฮ“ ๐‘น, ๐’š ๐‘น = ๐‘น๐‘ก๐‘ฅ๐‘—๐‘œ๐‘•๐‘น๐‘ข๐‘ฅ๐‘—๐‘ก๐‘ข

x y z

twist

93

x y z

swing

zโ€™

slide-94
SLIDE 94

94

LBS DQS Swing/twist def.

Swing/twist deformer [Kavan and Sorkine 2012]

slide-95
SLIDE 95

Skinning normals

95

๐จ

๐

๐โˆ’๐‘ˆ๐จ

slide-96
SLIDE 96

Classical solution

96

๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜

Inverse transpose the linear part of the blended transformation

slide-97
SLIDE 97

This leads to inaccurate normals

97

[Merry et al. 2006; Tarini et al. 2014]

Skinned normal True normal

slide-98
SLIDE 98

We can re-compute the normalsโ€ฆ

โ€ฆ but thatโ€™s not ideal, esp. on the GPUs

98

slide-99
SLIDE 99

Correct skinned normals [LBS&DQS]

Take weight gradients into account

99

[Merry et al. 2006; Tarini et al. 2014]

๐‘˜=1 ๐‘›

๐‘ฅ๐‘—,๐‘˜ ๐”

๐‘˜ + ๐‘˜=1 ๐‘›

๐”

๐‘˜๐ฐ๐‘— ๐›ผ๐‘ฅ๐‘—,๐‘˜ ๐‘ˆ

Inverse transposition can be avoided

slide-100
SLIDE 100

Correct skinned normals [LBS&DQS]

100

[Tarini et al. 2014]

slide-101
SLIDE 101

Correct skinned normals [LBS&DQS]

101

[Tarini et al. 2014]

slide-102
SLIDE 102

Summary of direct skinning methods

  • 1. Multi-linear methods
  • 2. Nonlinear methods
  • 3. Advanced deformation primitives

102

slide-103
SLIDE 103

New trends: implicit skinning

[Vaillant et al. 2013]

103

LBS Implicit skinning

slide-104
SLIDE 104

Course notes: http://www.skinning.org

104