[ARRAYDEQUE] [ARRAYDEQUE] Scala 2.13 Collections * - - PowerPoint PPT Presentation

▶

arraydeque arraydeque

[ARRAYDEQUE] [ARRAYDEQUE] Scala 2.13 Collections * - - PowerPoint PPT Presentation

Jul 25, 2023 387 likes •819 views

[ARRAYDEQUE] [ARRAYDEQUE] Scala 2.13 Collections * https://english.stackexchange.com/questions/226954/how-is-deque-commonly-pronounced ABOUT ME ABOUT ME Pathikrit Bhowmick github.com/pathikrit Coatue Management Scala for 5 years Data

slide-1

SLIDE 1

[ARRAYDEQUE] [ARRAYDEQUE]

Scala 2.13 Collections

* https://english.stackexchange.com/questions/226954/how-is-deque-commonly-pronounced

slide-2

SLIDE 2

ABOUT ME ABOUT ME

Pathikrit Bhowmick Scala for 5 years

❤

github.com/pathikrit Coatue Management Data structures FP

slide-3

SLIDE 3

SCALA 2.13 COLLECTIONS SCALA 2.13 COLLECTIONS

https://www.scala-lang.org/blog/2018/06/13/scala-213-collections.html

slide-4

SLIDE 4

slide-5

SLIDE 5

slide-6

SLIDE 6

CANBUILDFROM CANBUILDFROM

def map[B, That](f: A => B)(implicit bf: CanBuildFrom[Repr, B, That]): That

https://www.scala-lang.org/blog/2017/05/30/tribulations-canbuildfrom.html

slide-7

SLIDE 7

CANBUILDFROM CANBUILDFROM

def map[B, That](f: A => B)(implicit bf: CanBuildFrom[Repr, B, That]): That trait IterableOps[A, CC[_]] { def map[B](f: A => B): CC[B] }

https://www.scala-lang.org/blog/2017/05/30/tribulations-canbuildfrom.html

slide-8

SLIDE 8

NEW APIS NEW APIS

slide-9

SLIDE 9

NEW APIS NEW APIS

def namesByAge(users: Seq[User]): Map[Int, Seq[String]] = users.groupBy(_.age).mapValues(users => users.map(_.name))

slide-10

SLIDE 10

NEW APIS NEW APIS

def namesByAge(users: Seq[User]): Map[Int, Seq[String]] = users.groupBy(_.age).mapValues(users => users.map(_.name)) def namesByAge(users: Seq[User]): Map[Int, Seq[String]] = users.groupMap(_.age)(_.name)

slide-11

SLIDE 11

IN PLACE MUTABLE API IN PLACE MUTABLE API

slide-12

SLIDE 12

IN PLACE MUTABLE API IN PLACE MUTABLE API

val users: mutable.ArrayBuffer[User] = ??? users .filterInPlace(user => !user.name.startsWith("J")) .mapInPlace(user => user.copy(age = user.age + 1))

slide-13

SLIDE 13

MORE MORE

Stream vs LazyList Better Views Iterable vs. Traversable/Iterator New collections

slide-14

SLIDE 14

ARRAYDEQUE ARRAYDEQUE

slide-15

SLIDE 15

ARRAYDEQUE ARRAYDEQUE

New collection in Scala 2.13 Also known as CircularBuffer

slide-16

SLIDE 16

ARRAYDEQUE ARRAYDEQUE

New collection in Scala 2.13 Also known as CircularBuffer Replacement for most mutable collections Faster than ArrayBuffer when used as an array Faster than LinkedList when used as a linked list

slide-17

SLIDE 17

ARRAYDEQUE ARRAYDEQUE

New collection in Scala 2.13 Also known as CircularBuffer Replacement for most mutable collections Faster than ArrayBuffer when used as an array Faster than LinkedList when used as a linked list My first contribution to Scala!

slide-18

SLIDE 18

DATA STRUCTURES 101 DATA STRUCTURES 101

Data Structures get(idx) update(idx, e) append(e) prepend(e) deleteFirst() deleteLast() insertAt(idx, e) deleteAt(idx) mutable.ArrayDeque O(1) O(1) O(1) O(1) O(1) O(1) O(min(i, n-i)) O(min(i, n-i)) mutable.ArrayBuffer O(1) O(1) O(1) O(n) O(n) O(1) O(n) O(n) mutable.Stack O(n) O(n) O(1) (push) O(n) O(n) O(1) (pop) O(n) O(n) mutable.Queue O(n) O(n) O(1) (enque) O(n) O(1) (deque) O(n) O(n) O(n) mutable.LinkedList O(n) O(n) O(1) O(1) O(1) O(1) O(n) O(n) java.util.ArrayList O(1) O(1) O(1) O(n) O(n) O(1) O(n) O(n) java.util.ArrayDeque N/A O(1) O(1) O(1) O(1) O(1) O(n) O(n)

slide-19

SLIDE 19

ARRAY ARRAY DOUBLY OUBLY ENDED NDED QUE QUEUE UE

slide-20

SLIDE 20

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) }

ARRAY ARRAY DOUBLY OUBLY ENDED NDED QUE QUEUE UE

slide-21

SLIDE 21

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) }

ARRAY ARRAY DOUBLY OUBLY ENDED NDED QUE QUEUE UE

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } }

slide-22

SLIDE 22

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) }

ARRAY ARRAY DOUBLY OUBLY ENDED NDED QUE QUEUE UE

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } }

slide-23

SLIDE 23

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) }

ARRAY ARRAY DOUBLY OUBLY ENDED NDED QUE QUEUE UE

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } def deleteLast(): Unit = end = (end - 1)%n }

slide-24

SLIDE 24

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) }

ARRAY ARRAY DOUBLY OUBLY ENDED NDED QUE QUEUE UE

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } def deleteLast(): Unit = end = (end - 1)%n } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } def deleteLast(): Unit = end = (end - 1)%n def deleteFirst(): Unit = start = (start + 1)%n }

slide-25

SLIDE 25

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) }

ARRAY ARRAY DOUBLY OUBLY ENDED NDED QUE QUEUE UE

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } def deleteLast(): Unit = end = (end - 1)%n } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } def deleteLast(): Unit = end = (end - 1)%n def deleteFirst(): Unit = start = (start + 1)%n } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } def deleteLast(): Unit = end = (end - 1)%n def deleteFirst(): Unit = start = (start + 1)%n def clear(): Unit = start = end }

slide-26

SLIDE 26

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) }

ARRAY ARRAY DOUBLY OUBLY ENDED NDED QUE QUEUE UE

class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } def deleteLast(): Unit = end = (end - 1)%n } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } def deleteLast(): Unit = end = (end - 1)%n def deleteFirst(): Unit = start = (start + 1)%n } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } def deleteLast(): Unit = end = (end - 1)%n def deleteFirst(): Unit = start = (start + 1)%n def clear(): Unit = start = end } class ArrayDeque[A] { val n = 64 val array = Array.ofDim[A](n) var start, end = 0 def update(i: Int, a: A): Unit = array((start + i)%n) = a def apply(i: Int): A = array((start + i)%n) def append(a: A): Unit = { array(end) = a end = (end + 1)%n } def prepend(a: A): Unit = { start = (start - 1)%n array(start) = a } def deleteLast(): Unit = end = (end - 1)%n def deleteFirst(): Unit = start = (start + 1)%n def clear(): Unit = start = end def size: Int = (end - start)%n }

slide-27

SLIDE 27

DEMO DEMO

slide-28

SLIDE 28

SCALA 2.13 SCALA 2.13

scala.collection.mutable.ArrayDeque

slide-29

SLIDE 29

PERFORMACE PERFORMACE

slide-30

SLIDE 30

PERFORMACE PERFORMACE

Array.copy (memcpy) insertAt(idx), deleteAt(idx), remove(idx) clone() slice() Pre-emptive allocations: insertAll(), prependAll()

slide-31

SLIDE 31

PERFORMACE PERFORMACE

Array.copy (memcpy) insertAt(idx), deleteAt(idx), remove(idx) clone() slice() Pre-emptive allocations: insertAll(), prependAll() Bit hack if n (array.length) = 2^k i%n == i&(n 1)

slide-32

SLIDE 32

BENCHMARKS BENCHMARKS

Operation ArrayBuffer ArrayDeque Speedup Insert lots of items 2473.36 ms 956.76 ms 2.5x Drop some items from an head index 7.65 ms 1.25 ms 5x Drop some items from a tail index 2.54 ms 0.28 ms 10x Append lots of items one by one 3576.63 ms 2222.13 ms 1.5x Prepend few items one by one 8699.13 ms 1.33 ms O(n) Prepend lots of items at once 2124.02 ms 462.76 ms 5x Random indexing 81.62 ms 84.02 ms

Insert items near head

2980.46 ms 1429.52 ms 2x Reversal 491.46 ms 378.69 ms 1.5x Insert items near tail 8588.98 ms 2504.20 ms 3x Sliding 1591.47 ms 157.25 ms 10x toArray 194.55 ms 181.07 ms

Clear lots of items

48.34 ms 28.62 ms 2x

slide-33

SLIDE 33

GITHUB.COM/STANCH/REFTREE GITHUB.COM/STANCH/REFTREE

slide-34

SLIDE 34

GITHUB.COM/STANCH/REFTREE GITHUB.COM/STANCH/REFTREE

import reftree.core._ import reftree.render._ import reftree.diagram._ import reftree.util.Reflection._ implicit def renderArrayDeque: ToRefTree[ArrayDeque[Char]] = ToRefTree {ds => val array = ds.privateField[Array[AnyRef]]("array") val start = ds.privateField[Int]("start") val end = ds.privateField[Int]("end") val arrayRef = { val arrayFields = array.zipWithIndex map { case (a, i) => val fieldName = { var s = i.toString if (i == start) s = '↳' + s if (i == end) s = s + '↲' s } val refTree = Option(a) match { case Some(c) => RefTree.Val(c.asInstanceOf[Char]).withHighlight(true) case None => RefTree.Null().withHighlight(i == end) } refTree.toField.withName(fieldName) } RefTree.Ref(array, arrayFields).rename(s"char[${array.length}]") } RefTree.Ref(ds, Seq( start.refTree.withHighlight(true).toField.withName("start"), end.refTree.withHighlight(true).toField.withName("end"), arrayRef.toField.withName("array") ) ++ ds.toArray.zipWithIndex.map({case (a, i) => a.refTree.toField.withName(i.toString)})) }

slide-35

SLIDE 35

TAKE AWAYS TAKE AWAYS

Please contribute

slide-36

SLIDE 36

TAKE AWAYS TAKE AWAYS

Please contribute Contributing to Scala is not scary

slide-37

SLIDE 37

TAKE AWAYS TAKE AWAYS

ArrayDeques are cool Please contribute Contributing to Scala is not scary Data structures > Algorithms Rope, SkipList, Zipper, Heap

slide-38

SLIDE 38

TAKE AWAYS TAKE AWAYS

ArrayDeques are cool Please contribute Contributing to Scala is not scary Data structures > Algorithms Rope, SkipList, Zipper, Heap Visualize your data structures

slide-39

SLIDE 39

TAKE AWAYS TAKE AWAYS

ArrayDeques are cool Please contribute Contributing to Scala is not scary Data structures > Algorithms Rope, SkipList, Zipper, Heap Visualize your data structures Scala 2.13 awaits!

slide-40

SLIDE 40

THANK YOU THANK YOU

github.com/pathikrit/arraydequetalk

slide-41

SLIDE 41

WE ARE HIRING WE ARE HIRING

Who we are: What we do: Quant Trading Data Science NLP What we love: Functional Programming Algorithms Statistics Data Coatue Management pbhowmick@coatue.com Tech stack: Scala Spark PostgreSQL Tableau R Python Docker AWS Where are we: NYC SF