Int Introductio ion t n to Deep Deep Lea earn rning Prof. - - PowerPoint PPT Presentation

▶
int introductio ion t n to deep deep lea earn rning
SMART_READER_LITE
LIVE PREVIEW

Int Introductio ion t n to Deep Deep Lea earn rning Prof. - - PowerPoint PPT Presentation

Jun 17, 2023 271 likes •864 views

Int Introductio ion t n to Deep Deep Lea earn rning Prof. Leal-Taix and Prof. Niessner 1 The The Te Team Lecturers Prof. Dr. Laura Prof. Dr. Matthias Leal-Taix Niessner Tutors Patrick Andreas Dendorfer Rssler Prof.

slide-1
SLIDE 1

Int Introductio ion t n to Deep Deep Lea earn rning

  • Prof. Leal-Taixé and Prof. Niessner

1

slide-2
SLIDE 2

Lecturers

  • Prof. Dr. Laura

Leal-Taixé

  • Prof. Dr. Matthias

Niessner Patrick Dendorfer

Tutors

The The Te Team

Andreas Rössler

  • Prof. Leal-Taixé and Prof. Niessner

2

slide-3
SLIDE 3

Wh What at is Com

  • mputer

er Vi Vision

  • n?
  • First defined in the 60s in artificial intelligence groups
  • “Mimic the human visual system”
  • Center block of robotic intelligence
  • Prof. Leal-Taixé and Prof. Niessner

3

slide-4
SLIDE 4
  • Prof. Leal-Taixé and Prof. Niessner

4

slide-5
SLIDE 5

Computer Vision

So Some de decade des l later…

  • Prof. Leal-Taixé and Prof. Niessner

5

slide-6
SLIDE 6

Computer Vision Physics Psychology Biology Mathematics Engineering Computer science Artificial Intelligence ML Neuroscience Algorithms Optimization NLP Speech Robotics Optics Image processing

  • Prof. Leal-Taixé and Prof. Niessner

6

slide-7
SLIDE 7

Computer Vision Physics Psychology Biology Engineering Computer science Artificial Intelligence ML Algorithms Optimization NLP Speech Robotics Optics Image processing Mathematics Neuroscience

  • Prof. Leal-Taixé and Prof. Niessner

7

slide-8
SLIDE 8

Computer Vision Physics Psychology Biology Engineering Computer science Artificial Intelligence ML Algorithms Optimization NLP Speech Robotics Optics Image processing Mathematics Neuroscience

  • Prof. Leal-Taixé and Prof. Niessner

8

slide-9
SLIDE 9

Computer Vision Physics Psychology Biology Engineering Computer science Artificial Intelligence ML Algorithms Optimization NLP Speech Robotics Optics Image processing Mathematics Neuroscience

  • Prof. Leal-Taixé and Prof. Niessner

9

slide-10
SLIDE 10

Pre 2012

Im Image classification

  • Prof. Leal-Taixé and Prof. Niessner

10

slide-11
SLIDE 11

A

Awesome magic box

Become magicians

Post 2012

Open the box

Im Image classification

  • Prof. Leal-Taixé and Prof. Niessner

11

slide-12
SLIDE 12

Why Why Deep Le Lear arning ning?

  • Prof. Leal-Taixé and Prof. Niessner

12

slide-13
SLIDE 13

De Deep p Le Learning g Histo tory

  • Prof. Leal-Taixé and Prof. Niessner

13

slide-14
SLIDE 14

The The empire str trike kes back ck

  • Prof. Leal-Taixé and Prof. Niessner

14

slide-15
SLIDE 15
  • MNIST digit

recognition dataset

  • 107 pixels used in

training

  • ImageNet image

recognition dataset

  • 1014 pixels used in

training 1988 LeCun et al. 2012 Krizhevsky et al.

Wh What at has as chan anged? ed?

  • Prof. Leal-Taixé and Prof. Niessner

15

slide-16
SLIDE 16

Big Data

Models know where to learn from

Hardware

Models are trainable

Deep

Models are complex

Wh What at made ade this pos

  • ssible?

e?

  • Prof. Leal-Taixé and Prof. Niessner

16

slide-17
SLIDE 17

AlphaGo Machine translation Emoticon suggestion

De Deep p Le Learning g nowadays ys

  • Prof. Leal-Taixé and Prof. Niessner

18

slide-18
SLIDE 18

Self-driving cars

De Deep p Le Learning g nowadays ys

  • Prof. Leal-Taixé and Prof. Niessner

19

slide-19
SLIDE 19

Healthcare, cancer detection

De Deep p Le Learning g nowadays ys

  • Prof. Leal-Taixé and Prof. Niessner

20

slide-20
SLIDE 20

De Deep p Le Learning g market

  • […]market research report Deep Learning Market […] Global

Forecasts to 2022", the deep learning market is expected to be worth USD D 1,7 ,722.9 Million by y 2022.

  • Prof. Leal-Taixé and Prof. Niessner

26

slide-21
SLIDE 21
  • S. Caelles, K.K. Maninis, J. Pont-Tuset, L. Leal-Taixé, D. Cremers, and L. Van Gool.

One-Shot Video Object Segmentation, CVPR 2017.

De Deep p Le Learning g at t TU TUM

  • Prof. Leal-Taixé and Prof. Niessner

27

slide-22
SLIDE 22

De Deep p Le Learning g at t TU TUM

  • Prof. Leal-Taixé and Prof. Niessner

28

slide-23
SLIDE 23

De Deep p Le Learning g at t TU TUM

CC3 CC2 CC1 Reshape Conv+BN+ReLU Pooling Upsample Concat Score

DDFF

  • Prof. Leal-Taixé and Prof. Niessner

29

slide-24
SLIDE 24

Comp Computer er Vis ision ion at TUM

ScanNet: Dai, Chang, Savva, Halber, Funkhouser, Niessner., CVPR 2017.

ScanNet Stats:

  • Kinect-style RGB-D

sensors

  • 1513 scans of 3D

environments

  • 2.5 Mio RGB-D frames
  • Dense 3D, crowd-source

MTurk labels

  • Annotations projected to

2D frames

  • Prof. Leal-Taixé and Prof. Niessner

30

slide-25
SLIDE 25

De Deep p Le Learning g at t TU TUM

Map Photo

  • Prof. Leal-Taixé and Prof. Niessner

31

slide-26
SLIDE 26

Int Introductio ion t n to D Deep Le Lear arning ning

  • Prof. Leal-Taixé and Prof. Niessner

32

slide-27
SLIDE 27

Ab About t the the lectu ture

  • Theory: 11 lectures
  • Every Thursday 18-20h (MI HS 1)
  • Practice: 4 exercises, practical sessions
  • Every Tuesday 18-20h (Interim HS1)
  • January 31st: guest lecture by tba

https://dvl.in.tum.de/lectures/i2dl-ws18.html

  • Prof. Leal-Taixé and Prof. Niessner

33

slide-28
SLIDE 28

Gra Grading syst g system

  • Exam: tba

tba

  • Review: 2 review sessions
  • Important: no retake exam
  • Practice: 4 exercises (Tuesdays)
  • Bonus 0.3 + questions in the final exam

https://dvl.in.tum.de/lectures/i2dl-ws18.html

  • Prof. Leal-Taixé and Prof. Niessner

34

slide-29
SLIDE 29

Ex Exer ercis ise e lec ectures es

  • Tuesday lecture 1: Exercise submission system will

be explained, no not to be missed!

  • Tuesday lecture 2: DL math background
  • Tuesday lecture 3: Python introduction
  • Prof. Leal-Taixé and Prof. Niessner

35

slide-30
SLIDE 30

Introduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back- propagation RNN

  • Prof. Leal-Taixé and Prof. Niessner

36

slide-31
SLIDE 31

Sli Slides

  • All material will be uploaded on Moodle
  • Questions regarding the syllabus, exercises or contents
  • f the lecture, use Moodle!
  • Questions regarding organization of the course:
  • Emails to our individual addresses will not be answered.

i2dl@dvl.in.tum.de

  • Prof. Leal-Taixé and Prof. Niessner

37

slide-32
SLIDE 32

De Deep p Le Learning g at t TU TUM

Intro to Deep Learning DL for Physics

(Th Thuerey)

DL for Vision

(Ni Niessner, , Le Leal al-Ta Taixe)

DL for Medical Applicat.

(Me Menze)

DL in Robotics

(Bä Bäuml)

Machine Learning

(Gü Günnema mann)

  • Prof. Leal-Taixé and Prof. Niessner

38

slide-33
SLIDE 33

Ma Machine chine Le Learning ning

  • Prof. Leal-Taixé and Prof. Niessner

39

slide-34
SLIDE 34

Mac Machine e lear earning

Task

  • Prof. Leal-Taixé and Prof. Niessner

40

slide-35
SLIDE 35

Im Image classification

  • Prof. Leal-Taixé and Prof. Niessner

41

slide-36
SLIDE 36

Pose Appearance Illumination

  • Prof. Leal-Taixé and Prof. Niessner

42

slide-37
SLIDE 37

Im Image classification

Occlusions

  • Prof. Leal-Taixé and Prof. Niessner

43

slide-38
SLIDE 38

Im Image classification

Background clutter

  • Prof. Leal-Taixé and Prof. Niessner

44

slide-39
SLIDE 39

Representation

Im Image classification

  • Prof. Leal-Taixé and Prof. Niessner

45

slide-40
SLIDE 40

Task Image classification Experience Data

Mac Machine e lear earning

  • How can we learn to perform image classification?
  • Prof. Leal-Taixé and Prof. Niessner

46

slide-41
SLIDE 41

Unsupervised learning Supervised learning

Mac Machine e lear earning

  • No label or target class
  • Find out properties of

the structure of the data

  • Clustering (k-means,

PCA)

  • Prof. Leal-Taixé and Prof. Niessner

47

slide-42
SLIDE 42

Mac Machine e lear earning

Unsupervised learning Supervised learning

  • Prof. Leal-Taixé and Prof. Niessner

48

slide-43
SLIDE 43

Mac Machine e lear earning

  • Labels or target

classes

Unsupervised learning Supervised learning

  • Prof. Leal-Taixé and Prof. Niessner

49

slide-44
SLIDE 44

DOG DOG DOG CAT CAT CAT

Mac Machine e lear earning

Unsupervised learning Supervised learning

  • Prof. Leal-Taixé and Prof. Niessner

50

slide-45
SLIDE 45

Experience Data Training data Test data Underlying assumption that train and test data come from the same distribution

Mac Machine e lear earning

  • How can we learn to perform image classification?
  • Prof. Leal-Taixé and Prof. Niessner

51

slide-46
SLIDE 46

Reinforcement learning

Agents Environment interaction

Mac Machine e lear earning

Unsupervised learning Supervised learning

  • Prof. Leal-Taixé and Prof. Niessner

52

slide-47
SLIDE 47

Reinforcement learning

Agents Environment reward

Mac Machine e lear earning

Unsupervised learning Supervised learning

  • Prof. Leal-Taixé and Prof. Niessner

53

slide-48
SLIDE 48
  • How can we learn to perform image classification?

Task Image classification Experience Data Performance measure Accuracy

Mac Machine e lear earning

  • Prof. Leal-Taixé and Prof. Niessner

54

slide-49
SLIDE 49

A simple le cla lassifier

  • Prof. Leal-Taixé and Prof. Niessner

55

slide-50
SLIDE 50

Ne Neare rest st Ne Neigh ghbo bor

?

  • Prof. Leal-Taixé and Prof. Niessner

56

slide-51
SLIDE 51

Ne Neare rest st Ne Neigh ghbo bor

distance

NN classifier = dog

  • Prof. Leal-Taixé and Prof. Niessner

57

slide-52
SLIDE 52

Ne Neare rest st Ne Neigh ghbo bor

distance

k-NN classifier = cat

  • Prof. Leal-Taixé and Prof. Niessner

58

slide-53
SLIDE 53

Ne Neare rest st Ne Neigh ghbo bor

Courtesy of Stanford course cs231n

What is the performance on training data for NN classifier? What classifier is more likely to perform best on test data?

  • Prof. Leal-Taixé and Prof. Niessner

59

slide-54
SLIDE 54

Ne Neare rest st Ne Neigh ghbo bor

  • Hyperpar

aram ameters

  • These parameters are problem dependent.
  • How do we choose these hyperparameters?

Distance (L1, L2) k (number of neighbors)

  • Prof. Leal-Taixé and Prof. Niessner

60

slide-55
SLIDE 55

Cr Cros

  • ss valid

idation ion

train validation Run 1 Run 2 Run 3 Run 4 Run 5 Split the trai aini ning ng dat ata a into N folds

  • Prof. Leal-Taixé and Prof. Niessner

61

slide-56
SLIDE 56

Cr Cros

  • ss valid

idation ion

train test train test validation 20% Find your hyperparameters

  • Prof. Leal-Taixé and Prof. Niessner

62

slide-57
SLIDE 57

Thi This le lectu cture: improvi ving ng our cla classifier

  • Beyond linear classification
  • How to train complex models à deep networks
  • What is happening behind the scenes: optimization,

CNN, regularization.

  • Prof. Leal-Taixé and Prof. Niessner

63

slide-58
SLIDE 58

Up Upcomin ming l lecture

  • Next Thursday: Lecture 2: Machine Learning basics
  • Next Tuesday: 1st practical lecture – if you want the

bonus do not miss it!

  • Prof. Leal-Taixé and Prof. Niessner

64