2 Years of Real World FP at REA @KenScambler Scala Developer at - - PowerPoint PPT Presentation

▶
2 years of real world fp at rea
SMART_READER_LITE
LIVE PREVIEW

2 Years of Real World FP at REA @KenScambler Scala Developer at - - PowerPoint PPT Presentation

Dec 20, 2022 342 likes •1.95k views

2 Years of Real World FP at REA @KenScambler Scala Developer at Me 14 years 5 years 5 years when possible when bored when forced - 3 teams @ - 17 codebases - 43K LOC Jul 13 Jan 14 Jul 14 Jan 15 Why Functional Programming?

slide-1
SLIDE 1

2 Years of Real World FP at REA

@KenScambler Scala Developer at

λ

slide-2
SLIDE 2

Me

14 years 5 years 5 years when possible when bored when forced

slide-3
SLIDE 3

@

Jul 13 Jan 14 Jul 14 Jan 15

  • 3 teams
  • 17 codebases
  • 43K LOC
slide-4
SLIDE 4

Why Functional Programming?

slide-5
SLIDE 5

Compelling, tangible software engineering benefits

slide-6
SLIDE 6

Modularity Abstraction Composability

slide-7
SLIDE 7

Modular, abstract, composable programs are simple programs

slide-8
SLIDE 8

Modularity

slide-9
SLIDE 9

Can you reason about something in isolation?

slide-10
SLIDE 10

Or do you need to fit everything in your head at

  • nce?
slide-11
SLIDE 11

A B C K Is the cost of replacing B with K just writing K?

slide-12
SLIDE 12

A B C K Is the cost of replacing B with K just writing K?

slide-13
SLIDE 13

AK B CK K …or rewriting half the program?

glue glue

slide-14
SLIDE 14

A pure function is just input  output; no side effects You can tell what it does without Looking at surrounding context.

slide-15
SLIDE 15

Consider: Let’s parse a string like “Richmond, VIC 3121”

def def parseLocation(str: String): Location = { val val parts = str.split(“,”) val val secondStr = parts(1) val val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) }

slide-16
SLIDE 16

Could be null

Possible errors: 1

def def parseLocation(str: String): Location = { val val parts = str.split(“,”) val val secondStr = parts(1) val val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) }

slide-17
SLIDE 17

def def parseLocation(str: String): Location = { val val parts = str.split(“,”) val val secondStr = parts(1) val val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) }

Might not have enough elements

Possible errors: 2

slide-18
SLIDE 18

def def parseLocation(str: String): Location = { val val parts = str.split(“,”) val val secondStr = parts(1) val val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) }

Might not have enough elements

Possible errors: 3

slide-19
SLIDE 19

def def parseLocation(str: String): Location = { val val parts = str.split(“,”) val val secondStr = parts(1) val val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) }

Might not have enough elements

Possible errors: 4

slide-20
SLIDE 20

def def parseLocation(str: String): Location = { val val parts = str.split(“,”) val val secondStr = parts(1) val val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) }

Might not have enough elements

Possible errors: 5

slide-21
SLIDE 21

def def parseLocation(str: String): Location = { val val parts = str.split(“,”) val val secondStr = parts(1) val val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) }

Might not be an int

Possible errors: 6

slide-22
SLIDE 22

def ef doSomethingElse(): Unit = { // // . ...Do ..Do o

  • th

ther s er stu tuff ff parseLocation(“Melbourne, VIC 3000”) }

Possible errors: 6 + 3 = 9 6 3

slide-23
SLIDE 23

def ef anotherThing(): Unit = { // // . ...Do ..Do e eve ven mo n more re s stuff tuff doSomethingElse() }

Possible errors: 9 + 4 = 13 4 9

slide-24
SLIDE 24

Code inherits all the errors and side-effects of code it calls. Local reasoning becomes impossible; modularity is lost.

slide-25
SLIDE 25

def def parseLocation(str: String): Option[Location] = { val val parts = str.split(”,") for for { locality <- parts.optGet(0) theRestStr <- parts.optGet(1) theRest = theRestStr.split(" ") subdivision <- theRest.optGet(0) postcodeStr <- theRest.optGet(1) postcode <- postcodeStr.optToInt } yield yield Location(locality, subdivision, postcode) }

Possible errors: 0

slide-26
SLIDE 26

All possibilities have been elevated into the type system Local reasoning is possible! Local reasoning is possible about things that call it, etc…

slide-27
SLIDE 27

Abstraction

slide-28
SLIDE 28

Know as little as you need

slide-29
SLIDE 29

Know as little as you need Produce as little as you can

slide-30
SLIDE 30

def def sumInts( list: List[Int]): Int = { var var result = 0 for for (x <- list) { result = result + x } return return result }

slide-31
SLIDE 31

def def flatten[A]( list: List[List[A]]): List[A] = { var var result = List() for for (x <- list) { result = result ++ x } return return result }

slide-32
SLIDE 32

def def all[A]( list: List[Boolean]): Boolean = { var var result = tru rue for for (x <- list) { result = result && x } return return result }

slide-33
SLIDE 33

def def sumInts( list: List[Int]): Int = { var var result = 0 for for (x <- list) { result = result + x } return return result }

slide-34
SLIDE 34

def def flatten[A]( list: List[List[A]]): List[A] = { var var result = List() for for (x <- list) { result = result ++ x } return return result }

slide-35
SLIDE 35

def def all[A]( list: List[Boolean]): Boolean = { var var result = tru rue for for (x <- list) { result = result && x } return return result }

slide-36
SLIDE 36

trait trait Monoid[M] { def zero: M def append(m1: M, m2: M): M }

Extract the essence!

slide-37
SLIDE 37

def def fold[M](list: List[M]) (im impl plic icit it m: Monoid[M]): M = { var var result = m.z .zer ero for for (x <- list) { result = m.append(result,x) } result }

slide-38
SLIDE 38

def def fold[M](list: List[M]) (im impl plic icit it m: Monoid[M]): M = list.foldLeft(m.zero)(m.append)

slide-39
SLIDE 39

fold(List(1, 2, 3, 4))  10 fold(List(List("a", "b"), List("c")))  List("a", "b", "c") fold(List(true, true, false, true))  false

slide-40
SLIDE 40

Abstraction is always a good thing!

Less repetition More reuse

Less decay, because code can’t grow tumours around unnecessary detail

slide-41
SLIDE 41

Composability

slide-42
SLIDE 42

Functions compose.

A => B A => B B => C B => C C => D C => D D => E D => E

slide-43
SLIDE 43

A => E A => E

Functions compose.

slide-44
SLIDE 44

Sideways too.

A => E A => E X => Y X => Y

slide-45
SLIDE 45

(A,X) => (E,Y) (A,X) => (E,Y)

Sideways too.

slide-46
SLIDE 46

This works in the large, as well as the small! Entire systems can be composable like functions…. without side effects

slide-47
SLIDE 47

Truly composable systems can accrue more and more stuff without getting more complex!

slide-48
SLIDE 48

Simplicity

  • Modularity – Reason locally
  • Abstraction – say only what you need, hide

everything you don’t

  • Composability – scale without accruing complexity
slide-49
SLIDE 49

The human process

slide-50
SLIDE 50

Technical excellence isn’t enough! Software development is a human process

Coffee Tea Quiet time GSD Ponies Reconsider!

slide-51
SLIDE 51
slide-52
SLIDE 52
slide-53
SLIDE 53
slide-54
SLIDE 54
slide-55
SLIDE 55

Xi’an offshore team

Team Team Team Team

slide-56
SLIDE 56

Xi’an offshore team

  • Many teams are partly based in Xi’an.
  • They’re very good, but…
  • Communication is hard!
  • It works, but requires great investment of time and

money

slide-57
SLIDE 57

Bottom-up tech decisions

slide-58
SLIDE 58
slide-59
SLIDE 59
slide-60
SLIDE 60
slide-61
SLIDE 61
slide-62
SLIDE 62
slide-63
SLIDE 63
slide-64
SLIDE 64
slide-65
SLIDE 65
slide-66
SLIDE 66
slide-67
SLIDE 67
slide-68
SLIDE 68

GET /foo/bar PUT {"mode": "banana"} POST {"partyTime": "5:00"} GET /buzz/5/

slide-69
SLIDE 69

Architect Mountain

slide-70
SLIDE 70

Architect Mountain

slide-71
SLIDE 71

Architect Mountain

slide-72
SLIDE 72

Architect Mountain Just needs more Agile Don’t forget your velocity More meetings, but littler

NO

No no no no no no no no no no no no no no no no no no no no no no no no no no no no no no no no not like

  • this. Wake up it’s a

school day

slide-73
SLIDE 73

Bottom-up tech decisions

You have to win

slide-74
SLIDE 74

Bottom-up tech decisions

You have to win

slide-75
SLIDE 75

Bottom-up tech decisions

You have to win

slide-76
SLIDE 76

Software Paleontology

Everyone’s got a history…

slide-77
SLIDE 77

Scriptozoic era 1995 – 2010 Mostly Perl

slide-78
SLIDE 78

Monolithocene epoch 2010 – 2012 Ruby, Java Scriptozoic era 1995 – 2010 Mostly Perl

slide-79
SLIDE 79

AWS

Monolithocene epoch 2010 – 2012 Ruby, Java Scriptozoic era 1995 – 2010 Mostly Perl Microservices 2012 – Ruby, Scala, JS

slide-80
SLIDE 80

Adoption

slide-81
SLIDE 81

June, 2013

slide-82
SLIDE 82
slide-83
SLIDE 83
slide-84
SLIDE 84
slide-85
SLIDE 85

λ

slide-86
SLIDE 86
slide-87
SLIDE 87

λ λ

slide-88
SLIDE 88

λ λ

λ

slide-89
SLIDE 89

λ λ

λ

slide-90
SLIDE 90

λ λ

slide-91
SLIDE 91

λ λ

slide-92
SLIDE 92

λ

slide-93
SLIDE 93

Language choice

Object Oriented Powerful static types Functional JVM

slide-94
SLIDE 94

Object Oriented Powerful static types Functional JVM

slide-95
SLIDE 95

Functional JVM

slide-96
SLIDE 96

Functional JVM

slide-97
SLIDE 97

Functional JVM

slide-98
SLIDE 98

Functional JVM

slide-99
SLIDE 99

Whatever works for you!

slide-100
SLIDE 100

The journey

slide-101
SLIDE 101

Jul 13 Jan 14

1

slide-102
SLIDE 102

LOC Web JSON DB Dep Inj Play2 Play2 6K Squeryl / Play2 Constructors Type API

#1

slide-103
SLIDE 103

LOC Web JSON DB Dep Inj Play2 Play2 6K Squeryl / Play2 Constructors Type API

  • Mentor
  • Code reviews

Learning Investment

#1

slide-104
SLIDE 104

LOC Web JSON DB Dep Inj Play2 Play2 6K Squeryl / Play2 Constructors Type API

  • Mentor
  • Code reviews

Learning Investment Report card

Learning curve Technical result Productivity Sentiment Steep but ok OK; slight dip Great; but FWs too heavy

#1

slide-105
SLIDE 105

Jul 13 Jan 14

1 2

Some infrastructure

slide-106
SLIDE 106

LOC Web JSON DB Dep Inj Play2 Argonaut 3K Squeryl / Play2 Constructors Type API

#2

slide-107
SLIDE 107

LOC Web JSON DB Dep Inj Play2 Argonaut 3K Squeryl / Play2 Constructors Type API

  • Almost none

Learning Investment

#2

slide-108
SLIDE 108

LOC Web JSON DB Dep Inj Play2 Argonaut 3K Squeryl / Play2 Constructors Type API

  • Almost none

Learning Investment Report card

Learning curve Technical result Productivity Sentiment Learning? Meh Needed rework; OK in the end

#2

!!!

slide-109
SLIDE 109

Lesson #1

New tech, mindset requires investment in learning Invest in your people!

slide-110
SLIDE 110

λ

slide-111
SLIDE 111

Another team! λ

slide-112
SLIDE 112

“We’ll have some of that!” λ

slide-113
SLIDE 113

Jul 13 Jan 14

1 2 4 6

5

New team; new ideas!

3

slide-114
SLIDE 114

LOC Web JSON DB Dep Inj Unfinagled Argonaut 2K, 3K, 4K, 1K Slick Constructors Type Web app, libs, API x 2

  • 2 x Mentor

Learning Investment Report card

Learning curve Technical result Productivity Sentiment Not bad Great Great

#3,4, 5,6

slide-115
SLIDE 115

Jul 13 Jan 14

1 2 4 6

5 3

7

Theft & innovation

Jul 14

slide-116
SLIDE 116

Lesson #2

Having multiple teams is great, because you can steal from each other

slide-117
SLIDE 117

LOC Web JSON DB Dep Inj Unfinagled Argonaut 4K Slick Monad Transformers Type API

#7

slide-118
SLIDE 118

LOC Web JSON DB Dep Inj Unfinagled Argonaut 4K Slick Monad Transformers Type API

  • 2 x Mentor

Learning Investment

#7

slide-119
SLIDE 119

LOC Web JSON DB Dep Inj Unfinagled Argonaut 4K Slick Monad Transformers Type API

  • 2 x Mentor

Learning Investment Report card

Learning curve Technical result Productivity Sentiment Vertical Great Great

#7

slide-120
SLIDE 120

Monad Transformers

Good technical benefits, but… Only 2 people could understand the code Experienced engineers felt totally helpless Learning curve way too steep

slide-121
SLIDE 121

Devops

All-rounders Gurus

slide-122
SLIDE 122

All-rounders Gurus

JS / CSS / HTML

slide-123
SLIDE 123

AWS

All-rounders Gurus

slide-124
SLIDE 124

Scala (originally!)

Gurus

slide-125
SLIDE 125

All-rounders

Scala (now)

Gurus

  • Smooth learning curve

is utterly essential

  • We need more all-

rounders

  • We can’t leave people

behind

slide-126
SLIDE 126

Lesson #3

Familiar, but technically unsound concepts have limited value. However… if we can’t make a concept learnable, then we can’t use it.

slide-127
SLIDE 127

λ A Ruby team considers its options…

slide-128
SLIDE 128

λ

slide-129
SLIDE 129

λ

slide-130
SLIDE 130

Jul 13 Jan 14

1 2 4 6

5 3

7

A 3rd team dips its toe in the water

8

Jul 14

slide-131
SLIDE 131

LOC Web JSON DB Dep Inj Play2 Play2 2K Anorm Constructors Type Web app

#8

slide-132
SLIDE 132

LOC Web JSON DB Dep Inj Play2 Play2 2K Anorm Constructors Type Web app

  • Trial and error
  • Code Katas

Learning Investment

#8

slide-133
SLIDE 133

LOC Web JSON DB Dep Inj Play2 Play2 2K Anorm Constructors Type Web app

  • Trial and error
  • Code Katas

Learning Investment Report card

Learning curve Technical result Productivity Sentiment Steep Meh OK

#8

slide-134
SLIDE 134

Lesson #4

It’s really hard learning from scratch Be prepared for pain up front

slide-135
SLIDE 135

λ

slide-136
SLIDE 136

Jul 13 Jan 14

1 2 4 6

5 3

7

Latest iteration

8

Jul 14 Jan 15

17

slide-137
SLIDE 137

Design trends

Inheritance/mixins Static functions

slide-138
SLIDE 138

Design trends

Inheritance/mixins Static functions Partial functions Total functions

slide-139
SLIDE 139

Design trends

Inheritance/mixins Static functions Partial functions Total functions Exceptions Sum types

slide-140
SLIDE 140

Design trends

Inheritance/mixins Static functions Partial functions Total functions Strings/primitives Wrapper types Exceptions Sum types

slide-141
SLIDE 141

FP Guild

Every Thursday, in work hours 7 – 12 people each week Reading, exercises, talks, live coding

slide-142
SLIDE 142

LOC Web JSON DB Dep Inj Unfiltered Argonaut 3K Slick Free Monads Type API

#17

slide-143
SLIDE 143

LOC Web JSON DB Dep Inj Unfiltered Argonaut 3K Slick Free Monads Type API

  • 2 x Mentors
  • Pull Requests
  • Code reviews
  • Pairing
  • FP Guild

Learning Investment

#17

slide-144
SLIDE 144

LOC Web JSON DB Dep Inj Unfiltered Argonaut 3K Slick Free Monads Type API

  • 2 x Mentors
  • Pull Requests
  • Code reviews
  • Pairing
  • FP Guild

Learning Investment Report card

Learning curve Technical result Productivity Sentiment Smooth Great Brilliant

#17

slide-145
SLIDE 145

Pure core

Routes Controllers Logic Script

Interpreter

Actual DB “Authenticate” “Use config” “Get from DB” “Update DB” “Log result” Web Framework Server

App runtime

slide-146
SLIDE 146

Pure core

Routes Controllers Logic Script

Pure Interpreter

Pure tests

Input Output Assert “Authenticate” “Use config” “Get from DB” “Update DB” “Log result”

slide-147
SLIDE 147

Pure core

Interpreter

Actual DB Web Framework Server

App runtime

Wafer thin E2E tests

Input Output Assert

slide-148
SLIDE 148
  • bject SearchCon

SearchController troller { def def getList(uid: UserId): Script[Response] = { for for { searches <- getSearches(uid) cfg <- getConfig results <- addLinks(searches, cfg) } yield yield Ok ~> Response(result.toJson) } }

slide-149
SLIDE 149

Learning curve:

slide-150
SLIDE 150

Learning curve:

Smooooth

slide-151
SLIDE 151

Mocks

slide-152
SLIDE 152

Stubs

slide-153
SLIDE 153

Flaky pretend servers just to check a response code

slide-154
SLIDE 154

Flaky tests that take 30 min to run

slide-155
SLIDE 155

Side effects everywhere

slide-156
SLIDE 156

Exceptions

slide-157
SLIDE 157

Intrusive DI frameworks

slide-158
SLIDE 158

You shouldn’t be dealing with all that complexity and crap!

Most business software isn’t rocket science.

slide-159
SLIDE 159

There is a better way

FP is a tall tree But there is so much low hanging fruit!

slide-160
SLIDE 160

Thank you!