CS107: Computing for Math CS107: Computing for Math and Science - - PowerPoint PPT Presentation

▶
cs107 computing for math cs107 computing for math and
SMART_READER_LITE
LIVE PREVIEW

CS107: Computing for Math CS107: Computing for Math and Science - - PowerPoint PPT Presentation

May 07, 2023 172 likes •871 views

CS107: Computing for Math CS107: Computing for Math and Science and Science Lecture 15: Real Numbers Lecture 15 CS107, Prof. Steinberg, f10 1 Sorting Sorting There are many ways to sort data Insertion Sort Selection Sort

slide-1
SLIDE 1

CS107, Prof. Steinberg, f10

1

Lecture 15

CS107: Computing for Math CS107: Computing for Math and Science and Science

Lecture 15: Real Numbers

slide-2
SLIDE 2

CS107, Prof. Steinberg, f10

2

Lecture 15

Sorting Sorting

  • There are many ways to sort data

– Insertion Sort – Selection Sort – Quick Sort – Merge Sort – (Many others)

slide-3
SLIDE 3

CS107, Prof. Steinberg, f10

3

Lecture 15

Insertion Sort Insertion Sort

  • One by one add numbers to the sorted

vector

– Moving numbers already there to make room Unsorted: Sorted:

20 10 30 5

slide-4
SLIDE 4

CS107, Prof. Steinberg, f10

4

Lecture 15

Insertion Sort Insertion Sort

  • One by one add numbers to the sorted

vector

– Moving numbers already there to make room Unsorted: Sorted:

20 10 30 5 20

slide-5
SLIDE 5

CS107, Prof. Steinberg, f10

5

Lecture 15

Insertion Sort Insertion Sort

  • One by one add numbers to the sorted

vector

– Moving numbers already there to make room Unsorted: Sorted:

20 10 30 5 20

slide-6
SLIDE 6

CS107, Prof. Steinberg, f10

6

Lecture 15

Insertion Sort Insertion Sort

  • One by one add numbers to the sorted

vector

– Moving numbers already there to make room Unsorted: Sorted:

20 10 30 5 10 20

slide-7
SLIDE 7

CS107, Prof. Steinberg, f10

7

Lecture 15

Insertion Sort Insertion Sort

  • One by one add numbers to the sorted

vector

– Moving numbers already there to make room Unsorted: Sorted:

20 10 30 5 10 20 30

slide-8
SLIDE 8

CS107, Prof. Steinberg, f10

8

Lecture 15

Insertion Sort Insertion Sort

  • One by one add numbers to the sorted

vector

– Moving numbers already there to make room Unsorted: Sorted:

20 10 30 5 10 20 30

slide-9
SLIDE 9

CS107, Prof. Steinberg, f10

9

Lecture 15

Insertion Sort Insertion Sort

  • One by one add numbers to the sorted

vector

– Moving numbers already there to make room Unsorted: Sorted:

20 10 30 5 10 20 30

slide-10
SLIDE 10

CS107, Prof. Steinberg, f10

10

Lecture 15

Insertion Sort Insertion Sort

  • One by one add numbers to the sorted

vector

– Moving numbers already there to make room Unsorted: Sorted:

20 10 30 5 10 20 30

slide-11
SLIDE 11

CS107, Prof. Steinberg, f10

11

Lecture 15

Insertion Sort Insertion Sort

  • One by one add numbers to the sorted

vector

– Moving numbers already there to make room Unsorted: Sorted:

20 10 30 5 5 10 20 30

slide-12
SLIDE 12

CS107, Prof. Steinberg, f10

12

Lecture 15

In Place Insertion Sort In Place Insertion Sort

  • We don’t really need two vectors,

unsorted and sorted - we can keep all the data in one vector

slide-13
SLIDE 13

CS107, Prof. Steinberg, f10

13

Lecture 15

In Place Insertion Sort In Place Insertion Sort

20 10 30 5 Sorted | unsorted

slide-14
SLIDE 14

CS107, Prof. Steinberg, f10

14

Lecture 15

In Place Insertion Sort In Place Insertion Sort

10 20 10 30 5 Sorted | unsorted

slide-15
SLIDE 15

CS107, Prof. Steinberg, f10

15

Lecture 15

In Place Insertion Sort In Place Insertion Sort

10 20 20 30 5 Sorted | unsorted

slide-16
SLIDE 16

CS107, Prof. Steinberg, f10

16

Lecture 15

In Place Insertion Sort In Place Insertion Sort

10 10 20 30 5 Sorted | unsorted

slide-17
SLIDE 17

CS107, Prof. Steinberg, f10

17

Lecture 15

In Place Insertion Sort In Place Insertion Sort

10 20 30 5 Sorted | unsorted

slide-18
SLIDE 18

CS107, Prof. Steinberg, f10

18

Lecture 15

In Place Insertion Sort In Place Insertion Sort

10 20 30 5 Sorted | unsorted

slide-19
SLIDE 19

CS107, Prof. Steinberg, f10

19

Lecture 15

In Place Insertion Sort In Place Insertion Sort

10 20 30 5 Sorted | unsorted

slide-20
SLIDE 20

CS107, Prof. Steinberg, f10

20

Lecture 15

In Place Insertion Sort In Place Insertion Sort

5 10 20 30 5 Sorted | unsorted

slide-21
SLIDE 21

CS107, Prof. Steinberg, f10

21

Lecture 15

In Place Insertion Sort In Place Insertion Sort

5 10 20 30 30 Sorted | unsorted

slide-22
SLIDE 22

CS107, Prof. Steinberg, f10

22

Lecture 15

In Place Insertion Sort In Place Insertion Sort

5 10 20 20 30 Sorted | unsorted

slide-23
SLIDE 23

CS107, Prof. Steinberg, f10

23

Lecture 15

In Place Insertion Sort In Place Insertion Sort

5 10 10 20 30 Sorted | unsorted

slide-24
SLIDE 24

CS107, Prof. Steinberg, f10

24

Lecture 15

In Place Insertion Sort In Place Insertion Sort

5 5 10 20 30 Sorted | unsorted

slide-25
SLIDE 25

CS107, Prof. Steinberg, f10

25

Lecture 15

How Much Work is it? How Much Work is it?

  • Approximately length squared
slide-26
SLIDE 26

CS107, Prof. Steinberg, f10

26

Lecture 15

How Much Work is it? How Much Work is it?

  • Length2 is worst case
  • We might find the right place early
  • If vector already close to sorted, takes

much less than length2

  • Note the cost of moving things in addition

to comparing

slide-27
SLIDE 27

CS107, Prof. Steinberg, f10

27

Lecture 15

Selection Sort Selection Sort

  • Repeatedly

– Find smallest in unsorted – Move it to end of sorted Unsorted: 20 10 5 30 Sorted:

slide-28
SLIDE 28

CS107, Prof. Steinberg, f10

28

Lecture 15

Selection Sort Selection Sort

  • Repeatedly

– Find smallest in unsorted – Move it to end of sorted Unsorted: 10 20 5 30 Sorted:

slide-29
SLIDE 29

CS107, Prof. Steinberg, f10

29

Lecture 15

Selection Sort Selection Sort

  • Repeatedly

– Find smallest in unsorted – Move it to end of sorted Unsorted: 10 20 5 7 Sorted

slide-30
SLIDE 30

CS107, Prof. Steinberg, f10

30

Lecture 15

Selection Sort Selection Sort

  • Repeatedly

– Find smallest in unsorted – Move it to end of sorted Unsorted: 10 20 5 7 Sorted: 5 How can we delete it from unsorted?

slide-31
SLIDE 31

CS107, Prof. Steinberg, f10

31

Lecture 15

Selection Sort Selection Sort

  • Repeatedly

– Find smallest in unsorted – Move it to end of sorted Unsorted: 10 20 7 7 Sorted: 5 How can we delete it from unsorted? `

ignore

slide-32
SLIDE 32

CS107, Prof. Steinberg, f10

32

Lecture 15

Selection Sort Selection Sort

  • Repeatedly

– Find smallest in unsorted – Move it to end of sorted Unsorted: 10 20 7 7 Sorted: 5

ignore

slide-33
SLIDE 33

CS107, Prof. Steinberg, f10

33

Lecture 15

Selection Sort Selection Sort

  • Repeatedly

– Find smallest in unsorted – Move it to end of sorted Unsorted: 10 20 7 7 Sorted: 5 7

ignore

slide-34
SLIDE 34

CS107, Prof. Steinberg, f10

34

Lecture 15

Selection Sort Selection Sort

  • Repeatedly

– Find smallest in unsorted – Move it to end of sorted Unsorted: 10 20 7 7 Sorted: 5 7

ignore

slide-35
SLIDE 35

CS107, Prof. Steinberg, f10

35

Lecture 15

Selection Sort Selection Sort

  • Repeatedly

– Find smallest in unsorted – Move it to end of sorted Unsorted: 20 20 7 7 Sorted: 5 7 10

ignore

slide-36
SLIDE 36

CS107, Prof. Steinberg, f10

36

Lecture 15

Selection Sort Selection Sort

  • Repeatedly

– Find smallest in unsorted – Move it to end of sorted Unsorted: 20 20 7 7 Sorted: 5 7 10

ignore

slide-37
SLIDE 37

CS107, Prof. Steinberg, f10

37

Lecture 15

Selection Sort Selection Sort

  • Repeatedly

– Find smallest in unsorted – Move it to end of sorted Unsorted: 20 20 7 7 Sorted: 5 7 10 20 See selectSort.m

ignore

slide-38
SLIDE 38

CS107, Prof. Steinberg, f10

38

Lecture 15

In Place Selection Sort In Place Selection Sort

  • Key idea: swap smallest unsorted and first

unsorted - see inplaceSelectSort.m and minPlace.m

|20 10 5 8 5|10 20 8 5 8| 20 10 5 8 10| 20

slide-39
SLIDE 39

CS107, Prof. Steinberg, f10

39

Lecture 15

How Much Work is it? How Much Work is it?

  • Length2
  • Same for all data
slide-40
SLIDE 40

CS107, Prof. Steinberg, f10

40

Lecture 15

Selection Selection vs vs Insertion Insertion

Selection sort:

  • Simpler to write
  • Same worst case:

O(n2)

  • Cannot make use of

partial pre-sorting: worse average case Insertion sort

  • Harder to write
  • Same worst case:

O(n2)

  • Can make use of

partially sorted list: better average case

slide-41
SLIDE 41

CS107, Prof. Steinberg, f10

41

Lecture 15

Quick Sort Quick Sort

  • Divide and conquer

– Divide numbers into two groups – Sort each group separately – Combine sorted groups

  • Quick sort:

– Choose one number, call it pivot – Groups are numbers > pivot, < pivot – Combine: just append

slide-42
SLIDE 42

CS107, Prof. Steinberg, f10

42

Lecture 15

Quick Sort Quick Sort

2 5 1 8 3 9 4 Pivot -> 4 2 1 3 5 8 9 1 2 3 5 8 9 1 2 3 4 5 8 9 How do we sort the groups? Quick sort! If group length = 1, already sorted

slide-43
SLIDE 43

CS107, Prof. Steinberg, f10

43

Lecture 15

Partitioning Partitioning

How to separate by the pivot

  • Scan from small indexes looking for a

number > pivot

  • Scan from large indexes looking for a

number < pivot

  • Swap the two numbers
slide-44
SLIDE 44

CS107, Prof. Steinberg, f10

44

Lecture 15

Partitioning Partitioning

2 2 1 1 9 9 8 8 7 7 4 4 5 5 10 10

Pivot = 6

slide-45
SLIDE 45

CS107, Prof. Steinberg, f10

45

Lecture 15

Partitioning Partitioning

2 2 1 1 9 9 8 8 7 7 4 4 5 5 10 10

Pivot = 6

slide-46
SLIDE 46

CS107, Prof. Steinberg, f10

46

Lecture 15

Partitioning Partitioning

2 2 1 1 5 5 8 8 7 7 4 4 9 9 10 10

Pivot = 6

slide-47
SLIDE 47

CS107, Prof. Steinberg, f10

47

Lecture 15

Partitioning Partitioning

2 2 1 1 5 5 8 8 7 7 4 4 9 9 10 10

Pivot = 6

slide-48
SLIDE 48

CS107, Prof. Steinberg, f10

48

Lecture 15

Partitioning Partitioning

2 2 1 1 5 5 4 4 7 7 8 8 9 9 10 10

Pivot = 6

slide-49
SLIDE 49

CS107, Prof. Steinberg, f10

49

Lecture 15

Partitioning Partitioning

2 2 1 1 5 5 4 4 7 7 8 8 9 9 10 10

Pivot = 6

slide-50
SLIDE 50

CS107, Prof. Steinberg, f10

50

Lecture 15

Quick Sort Quick Sort

1 12 5 11 3 7 6 8 15 9 10 13 2 4 14

1

5 3 7 6 2 4 8 12 11 15 9 10 13 14 1 2 3 4 5 7 6 8 9 11 10 12 15 13 14

1

2 3 4 5 6 7 8 9 10 11 12 13 14 15

slide-51
SLIDE 51

CS107, Prof. Steinberg, f10

51

Lecture 15

How Much Work? How Much Work?

  • Assuming each split is into halves:

Log(n) sizes

  • If pieces are of size k we have n/k of them
  • Work to split or join pieces is proportional

to size of the piece

  • So work = n/1 +2*n/2 + 4*n/4 + …

= log(n) * n

slide-52
SLIDE 52

CS107, Prof. Steinberg, f10

52

Lecture 15

Quicksort Quicksort Caveat Caveat

  • For quicksort we assumed each split was

in halves, but this depends on pivot

  • Suppose each pivot is max

1 12 5 11 3 7 6 8 15 9 10 13 2 4 14 1 12 5 11 3 7 6 8 9 10 13 2 4 14 15 1 12 5 11 3 7 6 8 9 10 13 2 4 14 15 1 12 5 11 3 7 6 8 9 10 2 4 13 14 15

. . .

slide-53
SLIDE 53

CS107, Prof. Steinberg, f10

53

Lecture 15

QuickSort vs QuickSort vs Insert & Select Insert & Select

Fastest Much faster Longest Time to write Slowest Much slower n log(n) - fastest Typical O(n2) O(n2) O(n2) - but very rare Worst case Select Insert Quick Sort

slide-54
SLIDE 54

CS107, Prof. Steinberg, f10

54

Lecture 15

On tests On tests

Yes Yes Yes speeds Yes Yes ?? Do by hand Yes ?? No Write Select Insert Quick Sort

slide-55
SLIDE 55

CS107, Prof. Steinberg, f10

55

Lecture 15

Merge Sort Merge Sort

  • Worst case n log(n)
  • Typical case slower than quick sort
  • Nothing more about merge sort on tests
slide-56
SLIDE 56

CS107, Prof. Steinberg, f10

56

Lecture 15

N N2

2 vs N log N

vs N log N

10,000 1,000,000 1000 600 10,000 100 30 100 10 N*log(N) N*N N

slide-57
SLIDE 57

CS107, Prof. Steinberg, f10

57

Lecture 15

Representing Numbers Representing Numbers

  • Problem:

– Infinite real numbers, even between 0 and 1 – Finite number of bits

  • Solution:

– Fixed-precision scientific notation – “Floating Point”

slide-58
SLIDE 58

CS107, Prof. Steinberg, f10

58

Lecture 15

Fixed-precision Scientific Fixed-precision Scientific Notation Notation

+ .141 x 10 + 01 Exponent Exponent sign Fraction Sign

+ 1 4 1 + 0 1

slide-59
SLIDE 59

CS107, Prof. Steinberg, f10

59

Lecture 15

Representable Numbers Representable Numbers

  • . 999 e 99 0 . 999 e 99

. 997 e99 . 998 e99 .999 e99

  • . 001 e-99 0 .001e-99 ,002e-99
slide-60
SLIDE 60

CS107, Prof. Steinberg, f10

60

Lecture 15

Approximation Approximation

How does this solve the problem?

  • Problem: infinite numbers, finite # bits
  • Solution:

– Use the bits to represent a finite number of numbers exactly – When you want to represent any other number, use the closest number you can represent

slide-61
SLIDE 61

CS107, Prof. Steinberg, f10

61

Lecture 15

Approximation Approximation

  • Eg, suppose you want to represent 1/3
  • Represent it as +.333e+00

.331e00 .332e00 .333e00 .334e00 1/3

slide-62
SLIDE 62

CS107, Prof. Steinberg, f10

62

Lecture 15

Round Off error Round Off error

  • Represent a real, X, by closest

representable number, X#

  • Absolute error: | X - X# |
  • Relative error: | X - X# | / | X |

.0005 .0005e00 .999e00 .9995e00 .0005 .0005e99 .999e99 .9985e99 .333 .0005e-99 .001e-99 .0015e-99 Rel err Abs err X# X

slide-63
SLIDE 63

CS107, Prof. Steinberg, f10

63

Lecture 15

Relative Error Relative Error

  • Often what you are more concerned with

– Eg error of 1 ft in measuring this room for a carpet is a big deal, but error of 1 ft in measuring distance from here to GW Bridge is not a big deal – Note that this representation gives roughly constant relative error, as long as number is not too close to 0 or too far from 0.

slide-64
SLIDE 64

CS107, Prof. Steinberg, f10

64

Lecture 15

Sources of Round Off Sources of Round Off

  • Irrationals: √2, ∏
  • Repeating decimals: .3333…
  • Operations not closed

– Division: 1/3 – Addition: 100+.01 – Subtraction: 123 - 122

slide-65
SLIDE 65

CS107, Prof. Steinberg, f10

65

Lecture 15

Sources of Large Relative Sources of Large Relative Error Error

  • Overflow: number too big, e.g. 10100
  • Underflow: number too close to 0,

e.g. 10 -100

  • Cancellation
  • Poor conditioning
slide-66
SLIDE 66

CS107, Prof. Steinberg, f10

66

Lecture 15

Cancellation Cancellation

A = 1.234567, so A# = 1.23, relative error = .004567 / 1.234567 ≈ .0037 B = 1.234111, so B# = 1.23 relative error = .004111 / 1.234111 ≈ .0033 A-B = 0.000456 = .456 x 10 -3 A# - B# = 0, abs error = .456 x 10 -3, rel. error = 1

slide-67
SLIDE 67

CS107, Prof. Steinberg, f10

67

Lecture 15

Poor Conditioning Poor Conditioning

  • Condition number = d result / d input

Given x -> Want y dy/dx = -x / (r2 - x2)1/2

slide-68
SLIDE 68

CS107, Prof. Steinberg, f10

68

Lecture 15

Consequences Consequences

  • Computer addition not associative
  • Therefore algebraic form matters

h a

slide-69
SLIDE 69

CS107, Prof. Steinberg, f10

69

Lecture 15

  • a = .102e3, h = .123e1
  • (a - 2h)2 = .100e3 2 = .100e5
  • a2 - (a -2 h)2 = .104e5 - .100e5 = .004e5=.4e3
  • 4 a h - 4 h2 = .500e3 - .604e1 = .494e3
  • More exactly a2 - (a - 2 h)2 = 495.7884