Linear classifiers : prediction eq u ations L IN E AR C L ASSIFIE R - - PowerPoint PPT Presentation

▶

Sep 02, 2022 285 likes •514 views

Linear classifiers : prediction eq u ations L IN E AR C L ASSIFIE R S IN P YTH ON Michael ( Mike ) Gelbart Instr u ctor , The Uni v ersit y of British Col u mbia Dot Prod u cts x = np.arange(3) np.sum(x*y) x 14 array([0, 1, 2]) x@y y =

SLIDE 1

Linear classifiers: prediction equations

L IN E AR C L ASSIFIE R S IN P YTH ON

Michael (Mike) Gelbart

Instructor, The University of British Columbia

SLIDE 2

LINEAR CLASSIFIERS IN PYTHON

Dot Products

x = np.arange(3) x array([0, 1, 2]) y = np.arange(3,6) y array([3, 4, 5]) x*y array([0, 4, 10]) np.sum(x*y) 14 x@y 14

x@y is called the dot

product of x and y , and is wrien x ⋅ y.

SLIDE 3

LINEAR CLASSIFIERS IN PYTHON

Linear classifier prediction

raw model output = coefficients ⋅ features + intercept

Linear classier prediction: compute raw model output, check the sign if positive, predict one class if negative, predict the other class This is the same for logistic regression and linear SVM

fit is dierent but predict is the same

SLIDE 4

LINEAR CLASSIFIERS IN PYTHON

How LogisticRegression makes predictions

raw model output = coefficients ⋅ features + intercept

lr = LogisticRegression() lr.fit(X,y) lr.predict(X)[10] lr.predict(X)[20] 1

SLIDE 5

LINEAR CLASSIFIERS IN PYTHON

How LogisticRegression makes predictions (cont.)

lr.coef_ @ X[10] + lr.intercept_ # raw model output array([-33.78572166]) lr.coef_ @ X[20] + lr.intercept_ # raw model output array([ 0.08050621])

SLIDE 6

LINEAR CLASSIFIERS IN PYTHON

The raw model output

SLIDE 7

LINEAR CLASSIFIERS IN PYTHON

The raw model output

SLIDE 8

LINEAR CLASSIFIERS IN PYTHON

The raw model output

SLIDE 9

Let's practice!

L IN E AR C L ASSIFIE R S IN P YTH ON

SLIDE 10

What is a loss function?

L IN E AR C L ASSIFIE R S IN P YTH ON

Michael Gelbart

Instructor, The University of British Columbia

SLIDE 11

LINEAR CLASSIFIERS IN PYTHON

Least squares: the squared loss

scikit-learn's LinearRegression minimizes a loss:

(true ith target value − predicted ith target value)

Minimization is with respect to coecients or parameters of the model. Note that in scikit-learn model.score() isn't necessarily the loss function.

i=1

∑

n 2

SLIDE 12

LINEAR CLASSIFIERS IN PYTHON

Classification errors: the 0-1 loss

Squared loss not appropriate for classication problems (more on this later). A natural loss for classication problem is the number of errors. This is the 0-1 loss: it's 0 for a correct prediction and 1 for an incorrect prediction. But this loss is hard to minimize!

SLIDE 13

LINEAR CLASSIFIERS IN PYTHON

Minimizing a loss

from scipy.optimize import minimize minimize(np.square, 0).x array([0.]) minimize(np.square, 2).x array([-1.88846401e-08])

SLIDE 14

Let's practice!

L IN E AR C L ASSIFIE R S IN P YTH ON

SLIDE 15

Loss function diagrams

L IN E AR C L ASSIFIE R S IN P YTH ON

Michael (Mike) Gelbart

Instructor, The University of British Columbia

SLIDE 16

LINEAR CLASSIFIERS IN PYTHON

The raw model output

SLIDE 17

LINEAR CLASSIFIERS IN PYTHON

0-1 loss diagram

SLIDE 18

LINEAR CLASSIFIERS IN PYTHON

Linear regression loss diagram

SLIDE 19

LINEAR CLASSIFIERS IN PYTHON

Logistic loss diagram

SLIDE 20

LINEAR CLASSIFIERS IN PYTHON

Hinge loss diagram

SLIDE 21

LINEAR CLASSIFIERS IN PYTHON

Hinge loss diagram

SLIDE 22

Let's practice!

L IN E AR C L ASSIFIE R S IN P YTH ON