Interval Analysis Grading of On-Line Homework John L. Orr - - PowerPoint PPT Presentation

▶
interval analysis grading of on line homework
SMART_READER_LITE
LIVE PREVIEW

Interval Analysis Grading of On-Line Homework John L. Orr - - PowerPoint PPT Presentation

May 16, 2023 432 likes •981 views

Interval Analysis Grading of On-Line Homework John L. Orr jorr@math.unl.edu University of NebraskaLincoln Interval Analysis Grading of On-Line Homework p.1 Introduction Joint work with Stephen Scott (UNL) and Travis Fisher (UNL &

slide-1
SLIDE 1

Interval Analysis Grading of On-Line Homework

John L. Orr

jorr@math.unl.edu

University of Nebraska–Lincoln

Interval Analysis Grading of On-Line Homework – p.1

slide-2
SLIDE 2

Introduction

Joint work with Stephen Scott (UNL) and Travis Fisher (UNL & PSU).

Interval Analysis Grading of On-Line Homework – p.2

slide-3
SLIDE 3

Introduction

Joint work with Stephen Scott (UNL) and Travis Fisher (UNL & PSU). Lecture plan:

  • Describe practical problem from

mathematical software development

Interval Analysis Grading of On-Line Homework – p.2

slide-4
SLIDE 4

Introduction

Joint work with Stephen Scott (UNL) and Travis Fisher (UNL & PSU). Lecture plan:

  • Describe practical problem from

mathematical software development

  • Introduce interval analysis concepts

Interval Analysis Grading of On-Line Homework – p.2

slide-5
SLIDE 5

Introduction

Joint work with Stephen Scott (UNL) and Travis Fisher (UNL & PSU). Lecture plan:

  • Describe practical problem from

mathematical software development

  • Introduce interval analysis concepts
  • Describe our solution to the problem

Interval Analysis Grading of On-Line Homework – p.2

slide-6
SLIDE 6

Introduction

Joint work with Stephen Scott (UNL) and Travis Fisher (UNL & PSU). Lecture plan:

  • Describe practical problem from

mathematical software development

  • Introduce interval analysis concepts
  • Describe our solution to the problem
  • Describe a solution to a related mathematical

problem

Interval Analysis Grading of On-Line Homework – p.2

slide-7
SLIDE 7

Introduction

Joint work with Stephen Scott (UNL) and Travis Fisher (UNL & PSU). Lecture plan:

  • Describe practical problem from

mathematical software development

  • Introduce interval analysis concepts
  • Describe our solution to the problem
  • Describe a solution to a related mathematical

problem

  • Evaluation and results

Interval Analysis Grading of On-Line Homework – p.2

slide-8
SLIDE 8

Statement of the Problem

Computer grading of students’ answers to mathematical questions. Example: In response to “Differentiate y = xex” the student enters xex + ex but the stored answer in the question bank is (1 + x)ex Are the two answers equivalent?

Interval Analysis Grading of On-Line Homework – p.3

slide-9
SLIDE 9

Context of the Problem

We were developing software for on-line delivery

  • f student assessment.
  • Support multiple choice, fill-in-blank,

interactive Flash questions, etc

  • Multiple, reworkable assignments with

different algorithmically-generated parameters

  • Vital to be able to grade mathematical

questions on content

Interval Analysis Grading of On-Line Homework – p.4

slide-10
SLIDE 10

Context of the Problem

Our algorithms are now used in

  • Brownstone’s EDU,
  • Wiley’s eGrade,
  • Prentice Hall’s PHGA,
  • McGraw-Hill’s Netgrade and MHHM,
  • Freeman’s iSolve,
  • Maplesoft’s Maple T.A.

Interval Analysis Grading of On-Line Homework – p.5

slide-11
SLIDE 11

The Zero-Equivalence Problem

Problem: Given two functions f and g, determine whether f(x) = g(x) ∀x ∈ R. Equivalently, given an expression f, determine whether f(x) = 0 ∀x ∈ R.

Interval Analysis Grading of On-Line Homework – p.6

slide-12
SLIDE 12

The Zero-Equivalence Problem

Problem: Given two functions f and g, determine whether f(x) = g(x) ∀x ∈ R. Equivalently, given an expression f, determine whether f(x) = 0 ∀x ∈ R. Possible solutions:

  • Symbolic manipulation

Interval Analysis Grading of On-Line Homework – p.6

slide-13
SLIDE 13

The Zero-Equivalence Problem

Problem: Given two functions f and g, determine whether f(x) = g(x) ∀x ∈ R. Equivalently, given an expression f, determine whether f(x) = 0 ∀x ∈ R. Possible solutions:

  • Symbolic manipulation
  • Numerical evaluation

Interval Analysis Grading of On-Line Homework – p.6

slide-14
SLIDE 14

The Zero-Equivalence Problem

Problem: Given two functions f and g, determine whether f(x) = g(x) ∀x ∈ R. Equivalently, given an expression f, determine whether f(x) = 0 ∀x ∈ R. Possible solutions:

  • Symbolic manipulation
  • Numerical evaluation
  • Caviness, 1970: Undecidable for functions

built from 1, π, +, −, ×, ÷, x, sin(x), |x|.

Interval Analysis Grading of On-Line Homework – p.6

slide-15
SLIDE 15

Monte-Carlo Methods

Method: Evaluate f(x) and g(x) at a set of random points and compare.

  • Not always correct if f = g
  • Always correct if f = g

Interval Analysis Grading of On-Line Homework – p.7

slide-16
SLIDE 16

Monte-Carlo Methods

Method: Evaluate f(x) and g(x) at a set of random points and compare.

  • Not always correct if f = g
  • Always correct if f = g. . . or is it?

Interval Analysis Grading of On-Line Homework – p.7

slide-17
SLIDE 17

Monte-Carlo Methods

Method: Evaluate f(x) and g(x) at a set of random points and compare.

  • Not always correct if f = g
  • Always correct if f = g. . . or is it?
  • Rounding error problems

Interval Analysis Grading of On-Line Homework – p.7

slide-18
SLIDE 18

Monte-Carlo Methods

Method: Evaluate f(x) and g(x) at a set of random points and compare.

  • Not always correct if f = g
  • Always correct if f = g. . . or is it?
  • Rounding error problems

One-sided error would be acceptable in this appli- cation, so we wanted to overcome rounding errors

Interval Analysis Grading of On-Line Homework – p.7

slide-19
SLIDE 19

Review Floating Point Arithmetic

IEEE-754 64 bit floating point number

Interval Analysis Grading of On-Line Homework – p.8

slide-20
SLIDE 20

Review Floating Point Arithmetic

IEEE-754 64 bit floating point number

  • Sign bit, S = {0, 1}

Interval Analysis Grading of On-Line Homework – p.8

slide-21
SLIDE 21

Review Floating Point Arithmetic

IEEE-754 64 bit floating point number

  • Sign bit, S = {0, 1}
  • Exponent, 0 ≤ E ≤ 211 − 1 = 2047

Interval Analysis Grading of On-Line Homework – p.8

slide-22
SLIDE 22

Review Floating Point Arithmetic

IEEE-754 64 bit floating point number

  • Sign bit, S = {0, 1}
  • Exponent, 0 ≤ E ≤ 211 − 1 = 2047
  • Mantissa, 0 ≤ M ≤ 252 − 1

Interval Analysis Grading of On-Line Homework – p.8

slide-23
SLIDE 23

Review Floating Point Arithmetic

IEEE-754 64 bit floating point number

  • Sign bit, S = {0, 1}
  • Exponent, 0 ≤ E ≤ 211 − 1 = 2047
  • Mantissa, 0 ≤ M ≤ 252 − 1
  • I = 0 if E = 0; I = 1 otherwise

Interval Analysis Grading of On-Line Homework – p.8

slide-24
SLIDE 24

Review Floating Point Arithmetic

IEEE-754 64 bit floating point number

  • Sign bit, S = {0, 1}
  • Exponent, 0 ≤ E ≤ 211 − 1 = 2047
  • Mantissa, 0 ≤ M ≤ 252 − 1
  • I = 0 if E = 0; I = 1 otherwise

x = (−1)S(I + 2−52M) × 2E−1022−I

Interval Analysis Grading of On-Line Homework – p.8

slide-25
SLIDE 25

Review Floating Point Arithmetic

For example,

24.5 = 49 × 2−1 = 110001 × 10−1 = 1.10001 × 105 = (−1)S(1 + 2−52M) × 2E−1023 →

sign exponent

10000000100

mantissa

1000100000000000000000000000000000000000000000000000

Interval Analysis Grading of On-Line Homework – p.9

slide-26
SLIDE 26

Review Floating Point Arithmetic

Rounding errors, e.g. 0.1 + 0.2 = 0.3

0.1 →

sign exponent

01111111011

mantissa

1001100110011001100110011001100110011001100110011010 = 0.00011001100110011001100110011001100110011001100110011010

(base 2)

0.2 →

sign exponent

01111111100

mantissa

1001100110011001100110011001100110011001100110011010 = 0.0011001100110011001100110011001100110011001100110011010

(base 2)

So, adding 0.00011001100110011001100110011001100110011001100110011010 + 0.0011001100110011001100110011001100110011001100110011010 = 0.01001100110011001100110011001100110011001100110011001110 which is rounded to 0.010011001100110011001100110011001100110011001100110100

Interval Analysis Grading of On-Line Homework – p.10

slide-27
SLIDE 27

Review Floating Point Arithmetic

Rounding errors, e.g. 0.1 + 0.2 = 0.3

(0.1 + 0.2) →

sign exponent

01111111101

mantissa

0011001100110011001100110011001100110011001100110100 = 0.010011001100110011001100110011001100110011001100110100

(base 2)

0.3 →

sign exponent

01111111101

mantissa

0011001100110011001100110011001100110011001100110011 = 0.010011001100110011001100110011001100110011001100110011

(base 2)

These differ by one ULP .

Interval Analysis Grading of On-Line Homework – p.11

slide-28
SLIDE 28

Interval Analysis

Moore, 1966: Replace numbers with intervals

Interval Analysis Grading of On-Line Homework – p.12

slide-29
SLIDE 29

Interval Analysis

Moore, 1966: Replace numbers with intervals x = [x−, y+] y = [y−, y+]

Interval Analysis Grading of On-Line Homework – p.12

slide-30
SLIDE 30

Interval Analysis

Moore, 1966: Replace numbers with intervals x = [x−, y+] y = [y−, y+] x + y = [x− + y−, x+ + y+]

Interval Analysis Grading of On-Line Homework – p.12

slide-31
SLIDE 31

Interval Analysis

Moore, 1966: Replace numbers with intervals x = [x−, y+] y = [y−, y+] x + y = [x− + y−, x+ + y+] x × y = [x−y−, x+y+] (x−, y− ≥ 0)

Interval Analysis Grading of On-Line Homework – p.12

slide-32
SLIDE 32

Interval Analysis

Moore, 1966: Replace numbers with intervals x = [x−, y+] y = [y−, y+] x + y = [x− + y−, x+ + y+] x × y = [x−y−, x+y+] (x−, y− ≥ 0) f(x1, . . . , xn) = {f(x1, . . . , xn) : xi ∈ xi}

Interval Analysis Grading of On-Line Homework – p.12

slide-33
SLIDE 33

Interval Analysis

In practice, use rounded machine arithmetic

Interval Analysis Grading of On-Line Homework – p.13

slide-34
SLIDE 34

Interval Analysis

In practice, use rounded machine arithmetic x = [x−, y+] y = [y−, y+]

Interval Analysis Grading of On-Line Homework – p.13

slide-35
SLIDE 35

Interval Analysis

In practice, use rounded machine arithmetic x = [x−, y+] y = [y−, y+] x + y = [x− +

M y−, x+ + M y+]

Interval Analysis Grading of On-Line Homework – p.13

slide-36
SLIDE 36

Interval Analysis

In practice, use rounded machine arithmetic x = [x−, y+] y = [y−, y+] x + y = [x− +

M y−, x+ + M y+]

x × y = [x− ×

M y−, x+ × M y+]

(x−, y− ≥ 0)

Interval Analysis Grading of On-Line Homework – p.13

slide-37
SLIDE 37

Interval Analysis

In practice, use rounded machine arithmetic x = [x−, y+] y = [y−, y+] x + y = [x− +

M y−, x+ + M y+]

x × y = [x− ×

M y−, x+ × M y+]

(x−, y− ≥ 0) etc

Interval Analysis Grading of On-Line Homework – p.13

slide-38
SLIDE 38

Interval Analysis

Moral: Using rounded machine interval arithmetic, the true result is always contained in the computed interval. So, if the intervals U = f(x) and V = f(y) are disjoint, then f and g are guaranteed different.

Interval Analysis Grading of On-Line Homework – p.14

slide-39
SLIDE 39

Interval Arithmetic Solution

Algorithm 1:

start with TRIALS equal to 0 repeat until TRIALS > MAXTRIALS assign random values to each variable in f and g let U be the rounded interval evaluation of f under those assignments let V be the rounded interval evaluation of g under those assignments if U ∩ V = ∅ return FALSE (the functions cannot be equal) increment TRIALS return TRUE (if cannot demonstrate that f and g differ, assume they are equal)

Interval Analysis Grading of On-Line Homework – p.15

slide-40
SLIDE 40

Using Interval Arithmetic

c f g b a

Interval Analysis Grading of On-Line Homework – p.16

slide-41
SLIDE 41

A Related Problem

Determine whether f(x) and g(x) differ by a constant. E.g. The student is asked to integrate sin(2x).

Interval Analysis Grading of On-Line Homework – p.17

slide-42
SLIDE 42

Integrate sin(2x)

One possible route is:

  • sin(2x) dx = −1

2 cos(2x) + C

Interval Analysis Grading of On-Line Homework – p.18

slide-43
SLIDE 43

Integrate sin(2x)

One possible route is:

  • sin(2x) dx = −1

2 cos(2x) + C but another valid approach is

  • sin(2x) dx =
  • 2 sin(x) cos(x) dx

=

  • 2s ds = sin2(x) + C

Interval Analysis Grading of On-Line Homework – p.18

slide-44
SLIDE 44

Integrate sin(2x)

Moral: Even neglecting the constant of integra- tion, two different approaches to integration can give answers that differ by a constant (1, in this case).

Interval Analysis Grading of On-Line Homework – p.19

slide-45
SLIDE 45

Interval Arithmetic Solution

Algorithm 2:

start with TRIALS equal to 0 and INTERSECTION equal to [−∞, ∞] repeat until TRIALS > MAXTRIALS assign random values to each variable in f and g let U be the rounded interval evaluation of f under those assignments let V be the rounded interval evaluation of g under those assignments let INTERSECTION equal INTERSECTION ∩ (U −

M V )

if INTERSECTION = ∅ return FALSE (we have found a miss) increment TRIALS return TRUE (there is still a range of constants by which f and g might differ)

Interval Analysis Grading of On-Line Homework – p.20

slide-46
SLIDE 46

Interval Arithmetic Solution

b a c f - g

Interval Analysis Grading of On-Line Homework – p.21

slide-47
SLIDE 47

Evaluation

  • Evaluated using 8,000 responses to Gateway

Exam questions.

  • Compared student responses with “correct”

responses.

  • Performed same comparison using Maple’s

evalb(simplify(f-g)=0)

Interval Analysis Grading of On-Line Homework – p.22

slide-48
SLIDE 48

Evaluation

Maple IA Monte-Carlo simplify Algorithm f = g can be wrong always right f = g always right can be wrong

Interval Analysis Grading of On-Line Homework – p.23

slide-49
SLIDE 49

Evaluation

Maple IA Monte-Carlo simplify Algorithm f = g can be wrong always right f = g always right can be wrong Moral: If the two checks agree, then they must be right!

Interval Analysis Grading of On-Line Homework – p.23

slide-50
SLIDE 50

Evaluation

  • We found only 4 discrepancies out of 8,000
  • Hand-verified these and found IA

Monte-Carlo method was correct in all cases

Interval Analysis Grading of On-Line Homework – p.24

slide-51
SLIDE 51

Summary

IA Monte-Carlo solution to Zero Equivalence problem is:

Interval Analysis Grading of On-Line Homework – p.25

slide-52
SLIDE 52

Summary

IA Monte-Carlo solution to Zero Equivalence problem is:

  • very fast

Interval Analysis Grading of On-Line Homework – p.25

slide-53
SLIDE 53

Summary

IA Monte-Carlo solution to Zero Equivalence problem is:

  • very fast
  • very accurate

Interval Analysis Grading of On-Line Homework – p.25

slide-54
SLIDE 54

Summary

IA Monte-Carlo solution to Zero Equivalence problem is:

  • very fast
  • very accurate
  • with guaranteed one-sided error

Interval Analysis Grading of On-Line Homework – p.25