The Epistemology of Software Engineering Nathan Marz @nathanmarz - - PowerPoint PPT Presentation

▶
the epistemology of software engineering
SMART_READER_LITE
LIVE PREVIEW

The Epistemology of Software Engineering Nathan Marz @nathanmarz - - PowerPoint PPT Presentation

Sep 28, 2023 614 likes •2.12k views

The Epistemology of Software Engineering Nathan Marz @nathanmarz 1 My personal philosophies on software development Agenda 1. Limits of human knowledge 2. E ff ect of the limits of knowledge on software development 3. Embracing those limits

slide-1
SLIDE 1

The Epistemology of Software Engineering

Nathan Marz @nathanmarz

1
slide-2
SLIDE 2

My personal philosophies on software development

slide-3
SLIDE 3
slide-4
SLIDE 4

Agenda

  • 1. Limits of human knowledge
  • 2. Effect of the limits of knowledge on software development
  • 3. Embracing those limits enables you to build better software
slide-5
SLIDE 5

How do I know my software is correct?

slide-6
SLIDE 6

How do I know a proposition is true?

slide-7
SLIDE 7

Epistfmology

slide-8
SLIDE 8

How do I know my software is correct? PREVIEW

slide-9
SLIDE 9

You don’t

slide-10
SLIDE 10

Your code is wrong

slide-11
SLIDE 11

PREVIEW How do I know a proposition is true?

slide-12
SLIDE 12

You don’t

slide-13
SLIDE 13

True knowledge is unattainable

slide-14
SLIDE 14
slide-15
SLIDE 15
slide-16
SLIDE 16

But wait... philosophy?

slide-17
SLIDE 17

Fallacies

Strawman Appeal to authority Circular reasoning Appeal to emotion False dilemma Argument to moderation Moral highground Ad hominem attack Shotgun argumentation Correlation vs causation Equivocation Burden of proof

slide-18
SLIDE 18
slide-19
SLIDE 19

Your code is wrong

slide-20
SLIDE 20

Your code is literally wrong

slide-21
SLIDE 21

Your code is wrong

slide-22
SLIDE 22
slide-23
SLIDE 23

Why do you believe your code is correct?

slide-24
SLIDE 24

Your code Dependency 1 Dependency 2 Dependency 3

slide-25
SLIDE 25

Dependency 1 Dependency 4 Dependency 5

slide-26
SLIDE 26

Dependency 4 Dependency 6 Dependency 9 Dependency 7 Dependency 8

slide-27
SLIDE 27

Dependency 3,000,000 Hardware

slide-28
SLIDE 28

Electronics

slide-29
SLIDE 29

Chemistry

slide-30
SLIDE 30

Atomic physics

slide-31
SLIDE 31

Quantum mechanics

slide-32
SLIDE 32

I think I can safely say that nobody understands quantum mechanics.

Richard Feynman

slide-33
SLIDE 33

Your code is wrong

slide-34
SLIDE 34 Your code

...

slide-35
SLIDE 35

Infinite regress

slide-36
SLIDE 36

Epistemological “solutions”

  • 1. Infinitism
  • 2. Foundationalism
  • 3. Coherentism
slide-37
SLIDE 37

Coherentism

slide-38
SLIDE 38

Foundationalism

Axioms

slide-39
SLIDE 39

René Descartes

slide-40
SLIDE 40

Cogito ergo sum

slide-41
SLIDE 41

I think, therefore I am

slide-42
SLIDE 42

Codito ergo sum

slide-43
SLIDE 43

I code, therefore I am

slide-44
SLIDE 44

Cartesian foundationalism

  • 1. Limited axioms
  • 2. Knowledge through deduction
slide-45
SLIDE 45

Cartesian programming

  • 1. Axioms = rules of programming language
  • 2. Programs = deductions from those axioms
slide-46
SLIDE 46
slide-47
SLIDE 47
slide-48
SLIDE 48
slide-49
SLIDE 49
  • > OutOfMemoryException
slide-50
SLIDE 50
  • > Hallo welt!
slide-51
SLIDE 51

All the software you’ve used has had bugs in it

slide-52
SLIDE 52

Including the software you’ve written

slide-53
SLIDE 53
slide-54
SLIDE 54
slide-55
SLIDE 55
slide-56
SLIDE 56
slide-57
SLIDE 57

Inductjon

slide-58
SLIDE 58

f(0) and (f(n) → f(n+1)) ⇒ ∀n≥0, f(n)

slide-59
SLIDE 59

Inductjon

slide-60
SLIDE 60
slide-61
SLIDE 61
slide-62
SLIDE 62

<sidenote>

slide-63
SLIDE 63

David Hume

slide-64
SLIDE 64

“Why is inductive reasoning valid?”

slide-65
SLIDE 65

</sidenote>

slide-66
SLIDE 66
slide-67
SLIDE 67

Skeptjcism

slide-68
SLIDE 68
slide-69
SLIDE 69

perfect code

slide-70
SLIDE 70

value to users

slide-71
SLIDE 71

“My software is correct”

slide-72
SLIDE 72

“My software is sometimes correct”

slide-73
SLIDE 73

How do you minimize imperfection?

slide-74
SLIDE 74

Storm’s “reportError” method

slide-75
SLIDE 75

(Storm is a realtime computation system, like Hadoop but for realtime)

slide-76
SLIDE 76

Storm architecture

slide-77
SLIDE 77

Storm architecture

Master node (similar to Hadoop JobTracker)

slide-78
SLIDE 78

Storm architecture

Used for cluster coordination

slide-79
SLIDE 79

Storm architecture

Run worker processes

slide-80
SLIDE 80

Storm’s “reportError” method

slide-81
SLIDE 81

Used to show errors in the Storm UI

slide-82
SLIDE 82

Error info is stored in Zookeeper

slide-83
SLIDE 83

What happens when a user deploys code like this?

slide-84
SLIDE 84

Denial-of-service on Zookeeper and cluster goes down

slide-85
SLIDE 85 Robust! Designed input space Actual input space Failures! Bad performance! Security holes! Irrelevant!
slide-86
SLIDE 86

Implement self-throttling to avoid overloading Zookeeper

slide-87
SLIDE 87 Robust! Designed input space Actual input space
slide-88
SLIDE 88 Robust! Designed input space Actual input space
slide-89
SLIDE 89

Epistfmology

slide-90
SLIDE 90

Trth Truh Trut Tuth Tru

slide-91
SLIDE 91

Foundation of modern science

slide-92
SLIDE 92
  • 1. When viewed in an inertial reference frame,
an object either is at rest or moves at a constant velocity, unless acted upon by an external force.
  • 2. The acceleration of a body is directly proportional to,
and in the same direction as, the net force acting on the body, and inversely proportional to its mass. Thus, F = ma, where F is the net force acting on the object, m is the mass
  • f the object and a is the acceleration of the object.
  • 3. When one body exerts a force on a second body, the
second body simultaneously exerts a force equal in magnitude and opposite in direction to that of the first body.

Newton’s laws of motion

slide-93
SLIDE 93 Cambridge, we have a problem...

Orbit of Mercury problem

slide-94
SLIDE 94

Einstein’s theory of relativity

Sorry, Newton, you’ve been PWNED:
slide-95
SLIDE 95

limit

n → ∞

approximation (truth)

n

= truth

slide-96
SLIDE 96

Science algorithm

  • 1. Make observations
  • 2. Find theories consistent with those observations
  • 3. Falsify theories by making more observations
slide-97
SLIDE 97

Foundationalism Coherentism +

slide-98
SLIDE 98

Empiricism

slide-99
SLIDE 99

John Locke

slide-100
SLIDE 100
slide-101
SLIDE 101
slide-102
SLIDE 102
slide-103
SLIDE 103
slide-104
SLIDE 104
slide-105
SLIDE 105
slide-106
SLIDE 106

Occam’s Razor

slide-107
SLIDE 107

Software Use cases

slide-108
SLIDE 108

Software gets messy

slide-109
SLIDE 109

Refactoring

slide-110
SLIDE 110 Robust! Designed input space Actual input space
slide-111
SLIDE 111 Robust! Designed input space Actual input space
slide-112
SLIDE 112
slide-113
SLIDE 113

TESTING

slide-114
SLIDE 114
slide-115
SLIDE 115
slide-116
SLIDE 116

Unit testing Load testing Stress testing Fuzz testing

slide-117
SLIDE 117

TDD?

slide-118
SLIDE 118

Review

  • 1. Cannot perfectly reason about software
  • Infinite regress problem
  • Deduction is fundamentally flawed
  • Evidence shows programmers are not good at deductive reasoning
  • 2. Best you can do is minimize wrongness
  • Truth can only be approximate
  • Observe/theorize/falsify cycle minimizes wrongness over time
  • Testing = empiricism applied to software development
  • Make programs less wrong by testing more
slide-119
SLIDE 119

Does any of this matter?

slide-120
SLIDE 120

YES

slide-121
SLIDE 121

Embrace “your code is wrong” to design better software

slide-122
SLIDE 122
slide-123
SLIDE 123
slide-124
SLIDE 124

Redundancy Fault-tolerance > Perfection

slide-125
SLIDE 125

An example

slide-126
SLIDE 126

Learning from Hadoop

Jobtracker

Job Job Job

slide-127
SLIDE 127

Learning from Hadoop

Jobtracker

Job Job Job

slide-128
SLIDE 128

Learning from Hadoop

Jobtracker

Job Job Job

slide-129
SLIDE 129

Your code is wrong

slide-130
SLIDE 130

So your processes will crash

slide-131
SLIDE 131

Storm’s daemons are process fault-tolerant

slide-132
SLIDE 132

Storm

Nimbus

Topology Topology Topology

slide-133
SLIDE 133

Storm

Nimbus

Topology Topology Topology

slide-134
SLIDE 134

Storm

Nimbus

Topology Topology Topology

slide-135
SLIDE 135

Storm

Nimbus

Topology Topology Topology

slide-136
SLIDE 136

Storm

Nimbus

Topology Topology Topology

slide-137
SLIDE 137 Robust! Designed input space Actual input space
slide-138
SLIDE 138 Robust! Designed input space Actual input space
slide-139
SLIDE 139

Reasoning is fundamentally hard

slide-140
SLIDE 140

So program in ways that require less of it

slide-141
SLIDE 141
slide-142
SLIDE 142
slide-143
SLIDE 143

Pure function

slide-144
SLIDE 144

Mutability is hard to reason about

slide-145
SLIDE 145

Minimize state mutation

slide-146
SLIDE 146

Functional programming

slide-147
SLIDE 147

Clojure

slide-148
SLIDE 148

skepticism(skepticism)

slide-149
SLIDE 149

perfect software

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

slide-150
SLIDE 150

Thank you