Lock-free algorithms for Kotlin coroutines It is all about - - PowerPoint PPT Presentation

▶
lock free algorithms for kotlin coroutines
SMART_READER_LITE
LIVE PREVIEW

Lock-free algorithms for Kotlin coroutines It is all about - - PowerPoint PPT Presentation

Dec 16, 2023 121 likes •1.42k views

Lock-free algorithms for Kotlin coroutines It is all about scalability Presented at SPTCC 2017 /Roman Elizarov @ JetBrains Speaker: Roman Elizarov 16+ years experience Previously developed high-perf trading software @ Devexperts

slide-1
SLIDE 1

Lock-free algorithms for Kotlin coroutines

It is all about scalability Presented at SPTCC 2017 /Roman Elizarov @ JetBrains

slide-2
SLIDE 2

Speaker: Roman Elizarov

  • 16+ years experience
  • Previously developed high-perf

trading software @ Devexperts

  • Teach concurrent & distributed

programming @ St. Petersburg ITMO University

  • Chief judge @ Northeastern

European Region of ACM ICPC

  • Now work on Kotlin @ JetBrains
slide-3
SLIDE 3

Agenda

  • Kotlin coroutines overview & motivation for lock-

free algorithms

  • Lock-free doubly linked list
  • Lock-free multi-word compare-and-swap
  • Combining them to get more complex atomic
  • perations (without STM)
slide-4
SLIDE 4

Kotlin basic facts

  • Kotlin is a JVM language developed by JetBrains
  • General purpose and statically-typed
  • Object-oriented and functional paradigms
  • Open source under Apache 2.0
  • Reached version 1.0 in 2016
  • Compatibility commitment
  • Now at version 1.1
  • Officially supported by Google on Android
slide-5
SLIDE 5

Kotlin is …

  • Modern
  • Concise
  • Safe
  • Extensible
  • Pragmatic
  • Fun to work with!
slide-6
SLIDE 6

Kotlin is pragmatic

… and easy to learn

slide-7
SLIDE 7

Coroutines

Asynchronous programming made easy

slide-8
SLIDE 8

How do we write code that waits for something most of the time?

slide-9
SLIDE 9

Blocking threads

Kotlin

fun postItem(item: Item) { val token = requestToken() val post = submitPost(token, item) processPost(post) }

slide-10
SLIDE 10

fun postItem(item: Item) { requestToken { token -> submitPost(token, item) { post -> processPost(post) } } }

Callbacks

Kotlin

slide-11
SLIDE 11

fun postItem(item: Item) { requestToken() .thenCompose { token -> submitPost(token, item) } .thenAccept { post -> processPost(post) } }

Futures/Promises/Rx

Kotlin

slide-12
SLIDE 12

Coroutines

Kotlin

fun postItem(item: Item) { launch(CommonPool) { val token = requestToken() val post = submitPost(token, item) processPost(post) } }

slide-13
SLIDE 13

CSP & Actor models

  • A style of programming for modern systems
  • Lots of concurrent tasks / jobs
  • Waiting most of the time
  • Communicating all the time

Share data by communicating

slide-14
SLIDE 14

Kotlin coroutines primitives

  • Jobs/Deferreds (futures)
  • join/await
  • Channels
  • send & receive
  • synchronous & buffered channels
  • Select/alternatives
  • Atomically wait on multiple events
  • Cancellation
  • Parent-child hierarchies
slide-15
SLIDE 15

Implementation challenges

  • Coroutines are like light-weight threads
  • All the low-level scheduling & communication

mechanisms have to scale to lots of coroutines

slide-16
SLIDE 16

Lock-free algorithms

slide-17
SLIDE 17

Building blocks

  • Single-word CAS (that’s all we have on JVM)
  • Automatic memory management (GC)
  • Practical lock-free algorithms
  • Lock-Free and Practical Doubly Linked List-Based

Deques Using Single-Word Compare-and-Swap by Sundell and Tsigas

  • A Practical Multi-Word Compare-and-Swap Operation

by Timothy L. Harris, Keir Fraser and Ian A. Pratt.

slide-18
SLIDE 18

Doubly linked list

S N 1 N P S P H T sentinel sentinel Use same node in practice next links form logical list contents prev links are auxiliary

slide-19
SLIDE 19

Insert

PushRight (like in queue)

slide-20
SLIDE 20

Doubly linked list (insert 0)

S N 1 N P S P H T 2 N P create & init 1 2

slide-21
SLIDE 21

Doubly linked list (insert 1)

S N 1 N P S P H T 2 N P CAS Retry insert on CAS failure

slide-22
SLIDE 22

Doubly linked list (insert 2)

S N 1 N P S P H T 2 N P CAS Ignore CAS failure ”finish insert”

slide-23
SLIDE 23

Remove

PopLeft (like in queue)

slide-24
SLIDE 24

Doubly linked list (remove 1)

S N 1 N P S P H T Mark removed node’s next link Use wrapper object for mark in practice Cache wrappers in pointed-to nodes CAS Retry remove on CAS failure 1 2 Don’t use AtomicMarkableReference

slide-25
SLIDE 25

CAS

Doubly linked list (remove 2)

S N 1 N P S P H T Mark removed node’s prev link Retry marking on CAS failure ”finish remove”

slide-26
SLIDE 26

Doubly linked list (remove 3)

S N 1 N P S P H T CAS ”help remove” – fixup next links

slide-27
SLIDE 27

Doubly linked list (remove 4)

S N 1 N P S P H T CAS ”correct prev” – fixup prev links

slide-28
SLIDE 28

State transitions

slide-29
SLIDE 29

Node states

Init next: Ok prev: Ok prev.next: -- next.prev: -- Insert 1 next: Ok prev: Ok prev.next: me next.prev: -- Insert 2 next: Ok prev: Ok prev.next: me next.prev: me Remove 1 next: Rem prev: Ok prev.next: me next.prev: me Remove 2 next: Rem prev: Rem prev.next: me next.prev: me Remove 3 next: Rem prev: Rem prev.next: ++ next.prev: me Remove 4 next: Rem prev: Rem prev.next: ++ next.prev: ++ help remove correct prev correct prev 1 2 3 4 5 6 7

slide-30
SLIDE 30

Helping

slide-31
SLIDE 31

Concurrent insert

slide-32
SLIDE 32

Concurrent insert (0)

S N 1 N P S P H T 2 N P 3 N P I2 I3

slide-33
SLIDE 33

Concurrent insert (1)

S N 1 N P S P H T 2 N P 3 N P CAS fail CAS ok I2 I3

slide-34
SLIDE 34

Concurrent insert (2)

S N 1 N P S P H T 2 N P 3 N P detect wrong prev (t.prev.next != t) I2 I3

slide-35
SLIDE 35

Concurrent insert (3)

S N 1 N P S P H T 2 N P 3 N P correct prev I2 I3

slide-36
SLIDE 36

Concurrent insert (4)

S N 1 N P S P H T 2 N P 3 N P reinit & repeat I2 I3

slide-37
SLIDE 37

Concurrent remove

slide-38
SLIDE 38

Concurrent remove (0)

S N 1 N P S P H T 2 N P R1 R2

slide-39
SLIDE 39

Concurrent remove (1)

S N 1 N P S P H T 2 N P R1 R1 R2

slide-40
SLIDE 40

Concurrent remove (2)

S N 1 N P S P H T 2 N P R1 R2 Finds already removed R1 R2

slide-41
SLIDE 41

Concurrent remove (3)

S N 1 N P S P H T 2 N P R1 R2 help remove mark prev R1 R2

slide-42
SLIDE 42

Concurrent remove (4)

S N 1 N P S P H T 2 N P R1 R2 Retry with corrected next R1 R2

slide-43
SLIDE 43

Concurrent remove (5)

S N 1 N P S P H T 2 N P R1 R2 help remove R1 R2

slide-44
SLIDE 44

Concurrent remove (6)

S N 1 N P S P H T 2 N P R1 R2 correct prev R1 R2

slide-45
SLIDE 45

Concurrent remove & insert

When remove wins

slide-46
SLIDE 46

Concurrent remove & insert (0)

S N 1 N P S P H T 2 N P create & init R1 R1 I2

slide-47
SLIDE 47

Concurrent remove & insert (1)

S N 1 N P S P H T 2 N P remove first R1 R1 I2

slide-48
SLIDE 48

Concurrent remove & insert (2)

S N 1 N P S P H T 2 N P CAS fail R1 R1 I2

slide-49
SLIDE 49

Concurrent remove & insert (3)

S N 1 N P S P H T 2 N P detect wrong prev (t.prev.next -- removed) do “correct prev” R1 R1 I2

slide-50
SLIDE 50

Concurrent remove & insert (4)

S N 1 N P S P H T 2 N P mark prev fixup next R1 R1 I2

slide-51
SLIDE 51

Concurrent remove & insert (5)

S N 1 N P S P H T 2 N P R1 R1 I2 update prev

slide-52
SLIDE 52

Concurrent remove & insert (6)

S N 1 N P S P H T 2 N P R1 reinit & repeat R1 I2

slide-53
SLIDE 53

Concurrent remove & insert

When insert wins

slide-54
SLIDE 54

Concurrent remove & insert (0)

S N 1 N P S P H T 2 N P create & init R1 R1 I2

slide-55
SLIDE 55

Concurrent remove & insert (1)

S N 1 N P S P H T 2 N P R1 CAS R1 I2

slide-56
SLIDE 56

Concurrent remove & insert (2)

S N 1 N P S P H T 2 N P R1 R1 I2 will succeed marking on remove retry

slide-57
SLIDE 57

Concurrent remove & insert (3)

S N 1 N P S P H T 2 N P R1 help remove mark prev R1 I2

slide-58
SLIDE 58

Concurrent remove & insert (4)

S N 1 N P S P H T 2 N P R1 correct prev R1 I2 Remove is over!

slide-59
SLIDE 59

Concurrent remove & insert (5)

S N 1 N P S P H T 2 N P R1 correct prev R1 I2

slide-60
SLIDE 60

Takeaways

  • A kind of algo you need a paper for
  • Hard to improve w/o writing another paper
  • Good news: stress tests uncover most impl bugs
  • Bad news: when stress test fails, you up to long

hours

  • More bad news: hard to find bugs that violate lock-

freedomness of algorithm

slide-61
SLIDE 61

Summary: what we can do

  • Insert items (at the end of the queue)
  • Remove items (at the front of the queue)
  • Traverse the list
  • Remove items at arbitrary locations
  • In O(1)
slide-62
SLIDE 62

Linearizability

  • Insert last
  • Linearizes at CAS of next
  • Remove first / arbitrary
  • Success – at CAS of next
  • Fail – at read of head.next
slide-63
SLIDE 63

More about algorithm

  • Sundell & Tsigas algo supports deque operations
  • Can PushLeft & PopRight
  • PopLeft is simple – read head.next & remove
  • But cannot linearize them all at cas points
  • PushLeft, PushRight, PopRight - Ok
  • PopLeft linearizes at head.next read (!!!)
slide-64
SLIDE 64

Summary of impl notes

  • Use GC (drop all memory management details)
  • Merge head & tail into a single sentinel node
  • Empty list is just one object (prev & next onto itself)
  • One item += one object
  • Reuse “remove mark” objects
  • One-element lists reuse of ptrs to sentinel all the time
  • Encapsulate!

S N P 1 N P Q

slide-65
SLIDE 65
slide-66
SLIDE 66

Mods

More complex atomic operations

slide-67
SLIDE 67

Basic mods (1)

  • Insert item conditionally on prev tail value

S N 1 N P S P H T 2 N P check & bailout before CAS

slide-68
SLIDE 68

Basic mods (2)

  • Remove head conditionally on prev head value

S N 1 N P S P H T R1 check & bailout before CAS

slide-69
SLIDE 69

Practical use-case: synchronous channels

val channel = Channel<Int>() // coroutine #1 for (x in 1..5) { channel.send(x * x) } // coroutine #2 repeat(5) { println(channel.receive()) } 1 2 3

slide-70
SLIDE 70

Senders wait

Sender #1 H Sender #2 Sender #3 T More senders Incoming receivers Receiver removes first if it is a sender node Sender inserts last if it is not a receiver node

slide-71
SLIDE 71

Receivers wait

Receiver #1 H Receiver #2 Receiver #3 T More receivers Incoming senders Sender removes first if it is a receiver node Receiver inserts last if it is not a sender node

slide-72
SLIDE 72

Send function sketch

fun send(element: T) { while (true) { // try to add sender, unless prev is receiver if (enqueueSend(element)) break // try to remove first receiver val receiver = removeFirstReceiver() if (receiver != null) { receiver.resume(element) // resume receiver break } } } 1 2 3 4

slide-73
SLIDE 73

Channel use-case recap

  • Uses insert/remove ops conditional on tail/head

node

  • Can abort (cancel) wait to receive/send at any time

by using remove

  • Full removal -- no garbage is left
  • Pretty efficient in practice
  • One item lists – one “garbage” object
slide-74
SLIDE 74

Multi-word compare and swap (CASN)

Build even bigger atomic operations

slide-75
SLIDE 75

Use-case: select expression

val channel1 = Channel<Int>() val channel2 = Channel<Int>() select { channel1.onReceive { e -> ... } channel2.onReceive { e -> ... } }

slide-76
SLIDE 76

Impl summary: register (1)

Select status: NS Channel1 Queue Channel2 Queue

  • 1. Not selected
  • 2. Selected
slide-77
SLIDE 77

Impl summary: register (2)

Select status: NS Channel1 Queue Channel2 Queue Add node to channel1 queue if not selected (NS) yet N1

slide-78
SLIDE 78

Impl summary: register (3)

Select status: NS Channel1 Queue Channel2 Queue Add node to channel2 queue if not selected (NS) yet N1 N2

slide-79
SLIDE 79

Impl summary: wait

Select status: NS Channel1 Queue Channel2 Queue N1 N2

slide-80
SLIDE 80

Impl summary: select (resume)

Select status: S Channel1 Queue Channel2 Queue N1 Make selected and remove node from queue

slide-81
SLIDE 81

Impl summary: clean up rest

Select status: S Channel1 Queue Channel2 Queue Remove non-selected waiters from queue

slide-82
SLIDE 82

Double-Compare Single-Swap (DCSS)

Building block for CASN

slide-83
SLIDE 83

DCSS spec in pseudo-code

A B fun <A,B> dcss( a: Ref<A>, expectA: A, updateA: A, b: Ref<B>, expectB: B) = atomic { if (a.value == expectA && b.value == expectB) { a.value = updateA } } 1 2 3 4

slide-84
SLIDE 84

DCSS: init descriptor

DCSS Descriptor (a, expectA, updateA, b, expectB) A B expectA expectB updateA

slide-85
SLIDE 85

DCSS: prepare

DCSS Descriptor (a, expectA, updateA, b, expectB) A B CAS ptr to descriptor if a.value == expectA expectA expectB updateA

slide-86
SLIDE 86

DCSS: read b.value

DCSS Descriptor (a, expectA, updateA, b, expectB) A B CAS ptr to descriptor if a.value == expectA expectA expectB updateA

slide-87
SLIDE 87

DCSS: complete (when success)

DCSS Descriptor (a, expectA, updateA, b, expectB) A B expectA expectB updateA CAS to updated value if a still points to descriptor

slide-88
SLIDE 88

DCSS: complete (alternative)

DCSS Descriptor (a, expectA, updateA, b, expectB) A B expectA !expectB updateA

slide-89
SLIDE 89

DCSS: complete (when fail)

DCSS Descriptor (a, expectA, updateA, b, expectB) A B expectA !expectB updateA CAS to original value if a still points to descriptor

slide-90
SLIDE 90

DCSS: States

Init A: ??? (desc created) A: desc A was expectA prep ok A: ??? A was !expectA prep fail

  • ne tread

1 2 A: updateA B was expectB success A: expectA B was !expectB 4 5 fail Any other thread encountering descriptor helps complete Originator cannot learn what was the

  • utcome

Lock-free algorithm without loops!

3

slide-91
SLIDE 91

Caveats

  • A & B locations must be totally ordered
  • or risk stack-overflow while helping
  • One way to look at it: Restricted DCSS (RDCSS)
slide-92
SLIDE 92

DCSS Mod: learn outcome

A B fun <A,B> dcssMod( a: Ref<A>, expectA: A, updateA: A, b: Ref<B>, expectB: B): Boolean = atomic { if (a.value == expectA && b.value == expectB) { a.value = updateA true } else false }

slide-93
SLIDE 93

DCSS Mod: init descriptor

DCSS Descriptor (a, expectA, updateA, b, expectB) A B expectA expectB updateA Outcome: UNDECIDED Consensus

slide-94
SLIDE 94

DCSS Mod: prepare

DCSS Descriptor (a, expectA, updateA, b, expectB) A B expectA expectB updateA Outcome: UNDECIDED

slide-95
SLIDE 95

DCSS Mod: read b.value

DCSS Descriptor (a, expectA, updateA, b, expectB) A B expectA expectB updateA Outcome: UNDECIDED

slide-96
SLIDE 96

DCSS Mod: reach consensus

DCSS Descriptor (a, expectA, updateA, b, expectB) A B expectA expectB updateA Outcome: SUCCESS CAS(UNDECIDED, DECISION)

slide-97
SLIDE 97

DCSS Mod: complete

DCSS Descriptor (a, expectA, updateA, b, expectB) A B expectA expectB updateA Outcome: SUCCESS

slide-98
SLIDE 98

DCSS Mod: States

Init A: ??? Outcome: UND (desc created) A: desc Outcome: UND A was expectA prep ok A: ??? Outcome: FAIL A was !expectA prep fail

  • ne tread

1 2 A: desc Outcome: SUCC B was expectB success A: desc Outcome: FAIL 6 fail A: expectA A: updateA 5 7

Still no loops!

3 4

slide-99
SLIDE 99

Compare-And-Swap N-words (CASN)

The ultimate atomic update

slide-100
SLIDE 100

CASN spec in pseudo-code

A B fun <A,B> cas2( a: Ref<A>, expectA: A, updateA: A, b: Ref<B>, expectB: B, updateB: B): Boolean = atomic { if (a.value == expectA && b.value == expectB) { a.value = updateA b.value = updateB true } else false } 1 2 3 4 5 For two words, for simplicity

slide-101
SLIDE 101

CASN: init descriptor

DCSS Descriptor (a, expectA, updateA, b, expectB, updateB) A B expectA expectB updateA Outcome: UNDECIDED updateB

slide-102
SLIDE 102

CASN: prepare (1)

DCSS Descriptor (a, expectA, updateA, b, expectB, updateB) A B expectA expectB updateA Outcome: UNDECIDED updateB CAS

slide-103
SLIDE 103

CASN: prepare (2)

DCSS Descriptor (a, expectA, updateA, b, expectB, updateB) A B expectA expectB updateA Outcome: UNDECIDED updateB Use DCSS to update B if Outcome == UNDECIDED DCSS

slide-104
SLIDE 104

CASN: decide

DCSS Descriptor (a, expectA, updateA, b, expectB, updateB) A B expectA expectB updateA Outcome: SUCCESS updateB CAS outcome

slide-105
SLIDE 105

CASN: complete (1)

DCSS Descriptor (a, expectA, updateA, b, expectB, updateB) A B expectA expectB updateA Outcome: SUCCESS updateB CAS

slide-106
SLIDE 106

CASN: complete (2)

DCSS Descriptor (a, expectA, updateA, b, expectB, updateB) A B expectA expectB updateA Outcome: SUCCESS updateB CAS

slide-107
SLIDE 107

CASN: States

A: ??? B: ??? O: UND A: desc B: ??? O: UND A: desc B: desc O: UND A: updateA B: desc O: SUCC Init A: updateA B: updateB O: SUCC 1 2 3 5 A: desc B: desc O: SUCC 4 6 A: ??? B: ??? O: FAIL A != expectA A: desc B: ??? O: FAIL B != expectB

  • ne tread

A: expectA B: ??? O: FAIL 7 8 9 DCSS Prevents from going back in this SM descriptor is known to other (helping) threads

slide-108
SLIDE 108

Using it in practice

All the little things that matter

slide-109
SLIDE 109

It is easy to combine multiple operations with DCSS/CASN that linearize on a CAS with a descriptor parameters that are known in advance

slide-110
SLIDE 110

Trivial example: Treiber stack

1 TOP 2 3 New node CAS expect update

slide-111
SLIDE 111

Let’s go deeper

Into unpublished territory

slide-112
SLIDE 112

Doubly linked list: insert last

slide-113
SLIDE 113

Operation Descriptor A ref: ??? expectA: Sentinel updateA: Node #2 … Outcome: UNDECIDED

Doubly linked list: insert (0)

S N 1 N P S P H T 2 N P CAS here We know expected value for CAS in advance We know updated value for CAS in advance ??? can fill in A before CAS & update on retry DCSS here is needed (always!)

slide-114
SLIDE 114

Doubly linked list: insert (1)

S N 1 N P S P H T 2 N P Operation Descriptor A ref: ??? expectA: Sentinel updateA: Node #2 … Outcome: UNDECIDED DCSS Descriptor affected node: #1

  • peration ref
slide-115
SLIDE 115

Doubly linked list: insert (2)

S N 1 N P S P H T 2 N P Operation Descriptor DCSS Descriptor Helpers are a bound to stumble upon the same descriptor CAS can only succeed on last node Competing inserts will complete (help) us first affected node: #1

  • peration ref

A ref: ??? expectA: Sentinel updateA: Node #2 … Outcome: UNDECIDED

slide-116
SLIDE 116

Doubly linked list: insert (3)

S N 1 N P S P H T 2 N P Operation Descriptor desc is updated after successful DCSS A ref: Node #1 expectA: Sentinel updateA: Node #2 … Outcome: UNDECIDED DCSS Descriptor affected node: #1

  • peration ref
slide-117
SLIDE 117

Doubly linked list: insert (4)

S N 1 N P S P H T 2 N P Operation Descriptor Stays pointed until operation

  • utcome is decided

A ref: Node #1 expectA: Sentinel updateA: Node #2 … Outcome: UNDECIDED

slide-118
SLIDE 118

Doubly linked list: remove first

slide-119
SLIDE 119

Doubly linked list: remove (0)

S N 1 N P S P H T R1 CAS here 2 N P Operation Descriptor A ref: ??? expectA: ??? updateA: Rem[???] … Outcome: UNDECIDED Both not known in advance Deterministic f(expectA)

slide-120
SLIDE 120

Doubly linked list: remove (1)

S N 1 N P S P H T R1 2 N P Operation Descriptor A ref: ??? expectA: ??? updateA: Rem[???] … Outcome: UNDECIDED DCSS Descriptor affected node: #1

  • peration ref
  • ld value: #2
slide-121
SLIDE 121

Doubly linked list: remove (2)

S N 1 N P S P H T R1 2 N P Operation Descriptor A ref: ??? expectA: ??? updateA: Rem[???] … Outcome: UNDECIDED It locks what node we are to remove Cannot change w/o removal of #1 We don’t support PushLeft!!! DCSS Descriptor affected node: #1

  • peration ref
  • ld value: #2
slide-122
SLIDE 122

Doubly linked list: remove (3)

S N 1 N P S P H T R1 2 N P Operation Descriptor A ref: Node #1 expectA: Node #2 updateA: Rem[#2] … Outcome: UNDECIDED desc is updated after successful DCSS DCSS Descriptor affected node: #1

  • peration ref
  • ld value: #2
slide-123
SLIDE 123

Doubly linked list: remove (4)

S N 1 N P S P H T R1 2 N P Operation Descriptor A ref: Node #1 expectA: Node #2 updateA: Rem[#2] … Outcome: UNDECIDED Stays pointed until operation

  • utcome is decided
slide-124
SLIDE 124

Closing notes

  • All we care about is CAS that linearizes operation
  • Subsequent updates are helper moves
  • Invoke regular help/correct functions
  • Perfect algorithm to combine with optional

Hardware Transactional Memory (HTM)

slide-125
SLIDE 125

Let’s enjoy what we’ve accomplished

slide-126
SLIDE 126

References

  • Kotlin language
  • http://kotlinlang.org
  • Kotlin coroutines support library
  • http://github.com/kotlin/kotlinx.coroutines
slide-127
SLIDE 127

Thank you

Any questions?

email me to elizarov at gmail relizarov