CS 4803 / 7643: Deep Learning Website: - - PowerPoint PPT Presentation

▶
cs 4803 7643 deep learning
SMART_READER_LITE
LIVE PREVIEW

CS 4803 / 7643: Deep Learning Website: - - PowerPoint PPT Presentation

Feb 12, 2024 16 likes •1.12k views

CS 4803 / 7643: Deep Learning Website: http://www.cc.gatech.edu/classes/AY2020/cs7643_spring/ Piazza: https://piazza.com/gatech/spring2020/cs4803dl7643a/ Staff mailing list (personal questions): cs4803-7643-staff@lists.gatech.edu Gradescope:

slide-1
SLIDE 1

CS 4803 / 7643: Deep Learning

Zsolt Kira School of Interactive Computing Georgia Tech

Website: http://www.cc.gatech.edu/classes/AY2020/cs7643_spring/ Piazza: https://piazza.com/gatech/spring2020/cs4803dl7643a/ Staff mailing list (personal questions): cs4803-7643-staff@lists.gatech.edu Gradescope: https://www.gradescope.com/courses/78537

Canvas: https://gatech.instructure.com/courses/94450/

Course Access Code (Piazza): MWXKY8

slide-2
SLIDE 2

Outline

  • What is Deep Learning, the field, about?
  • What is this class about?
  • What to expect?

– Logistics

  • FAQ

(C) Dhruv Batra & Zsolt Kira 2

slide-3
SLIDE 3

Outline

  • What is Deep Learning, the field, about?
  • What is this class about?
  • What to expect?

– Logistics

  • FAQ

(C) Dhruv Batra & Zsolt Kira 3

slide-4
SLIDE 4

What is Deep Learning?

Some of the most exciting developments in Machine Learning, Vision, NLP, Speech, Robotics & AI in general in the last 8 years!

(C) Dhruv Batra & Zsolt Kira 4

slide-5
SLIDE 5

Proxy for public interest

(C) Dhruv Batra & Zsolt Kira 5

slide-6
SLIDE 6

1000 object classes 1.4M/50k/100k images

Person Dalmatian

http://image-net.org/challenges/LSVRC/{2010,…,2015}

(C) Dhruv Batra & Zsolt Kira 6

ImageNet Large Scale Visual Recognition Challenge (ILSVRC)

Image Classification

slide-7
SLIDE 7

Image Classification

(C) Dhruv Batra & Zsolt Kira 7

slide-8
SLIDE 8

(C) Dhruv Batra & Zsolt Kira 8

https://qz.com/1034972/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world/

slide-9
SLIDE 9

AlphaGo vs Lee Sedol

(C) Dhruv Batra & Zsolt Kira 9

slide-10
SLIDE 10

Tasks are getting bolder

(C) Dhruv Batra & Zsolt Kira 10

A group of young people playing a game of Frisbee

Antol et al., 2015 Vinyals et al., 2015 Das et al., 2017

slide-11
SLIDE 11

Exit the bedroom and go towards the table. Go to the stairs on the left of the couch. Wait on the third step.

Textual reasoning Visual reasoning Decision making Sequence modeling Backtracking

new

Self-Monitoring

new

Vision-and-Language Navigation (VLN)

Goal – Backtracking Navigation Agent Motivation – Goal-Driven Navigation

slide-12
SLIDE 12

Evaluation Task

 Vision-and-Language Navigation (Room-to-Room) Leverages Matterport3D dataset Realistic perception  Combines perception and action Instructions to navigate to a goal No explicit target given  Our hypothesis: Reasoning about goal progress is important part of carrying

  • ut task
slide-13
SLIDE 13

(C) Dhruv Batra and Zsolt Kira 13  Attention-based grounding of both language and images  Tracked state using LSTM with inputs from previous action, previous hidden state, and grounded textual and visual input  Progress monitor estimates distance to goal

Approach

slide-14
SLIDE 14

Example

Step: 0 / 6 Progress Monitor: -0.19 Step: 0 / 6 Progress Monitor: -0.19 Step: 1 / 6 Progress Monitor: 0.20 Step: 1 / 6 Progress Monitor: 0.20 Step: 0 / 6 Progress Monitor: -0.19 Step: 0 / 6 Progress Monitor: -0.19 Step: 1 / 6 Progress Monitor: 0.20 Step: 1 / 6 Progress Monitor: 0.20 Step: 2 / 6 Progress Monitor: 0.24 Step: 2 / 6 Progress Monitor: 0.24 Step: 1 / 6 Progress Monitor: 0.20 Step: 1 / 6 Progress Monitor: 0.20 Step: 2 / 6 Progress Monitor: 0.24 Step: 2 / 6 Progress Monitor: 0.24 Step: 3 / 6 Progress Monitor: 0.44 Step: 3 / 6 Progress Monitor: 0.44 Step: 2 / 6 Progress Monitor: 0.24 Step: 2 / 6 Progress Monitor: 0.24 Step: 3 / 6 Progress Monitor: 0.44 Step: 3 / 6 Progress Monitor: 0.44 Step: 4 / 6 Progress Monitor: 0.70 Step: 4 / 6 Progress Monitor: 0.70 Step: 3 / 6 Progress Monitor: 0.44 Step: 3 / 6 Progress Monitor: 0.44 Step: 4 / 6 Progress Monitor: 0.70 Step: 4 / 6 Progress Monitor: 0.70 Step: 5 / 6 Progress Monitor: 0.92 Step: 5 / 6 Progress Monitor: 0.92 Step: 4 / 6 Progress Monitor: 0.70 Step: 4 / 6 Progress Monitor: 0.70 Step: 5 / 6 Progress Monitor: 0.92 Step: 5 / 6 Progress Monitor: 0.92 Step: 6 / 6 Progress Monitor: 0.95 Step: 6 / 6 Progress Monitor: 0.95 Step: 6 / 6 Progress Monitor: 0.95 Step: 6 / 6 Progress Monitor: 0.95 Step: 5 / 6 Progress Monitor: 0.92 Step: 5 / 6 Progress Monitor: 0.92

 Next steps:  Additional mechanisms for goal reasoning  Failure detection, inhibition

slide-15
SLIDE 15

Visual Dialog

[CVPR ‘17]

Abhishek Das (Georgia Tech) Satwik Kottur (CMU) Avi Singh (UC Berkeley) Khushi Gupta (CMU) Deshraj Yadav (Virginia Tech) José Moura (CMU) Devi Parikh (Georgia Tech / FAIR) Dhruv Batra (Georgia Tech / FAIR)

slide-16
SLIDE 16

(C) Dhruv Batra & Zsolt Kira 16

slide-17
SLIDE 17

(C) Dhruv Batra & Zsolt Kira 17

slide-18
SLIDE 18

(C) Dhruv Batra & Zsolt Kira 18

A man and a woman are holding umbrellas

slide-19
SLIDE 19

(C) Dhruv Batra & Zsolt Kira 19

A man and a woman are holding umbrellas What color is his umbrella?

slide-20
SLIDE 20

(C) Dhruv Batra & Zsolt Kira 20

man his

slide-21
SLIDE 21

(C) Dhruv Batra & Zsolt Kira 21

umbrella

slide-22
SLIDE 22

(C) Dhruv Batra & Zsolt Kira 22

A man and a woman are holding umbrellas What color is his umbrella? His umbrella is black

slide-23
SLIDE 23

(C) Dhruv Batra & Zsolt Kira 23

A man and a woman are holding umbrellas What color is his umbrella? His umbrella is black What about hers?

slide-24
SLIDE 24

(C) Dhruv Batra & Zsolt Kira 24

woman her

slide-25
SLIDE 25

(C) Dhruv Batra & Zsolt Kira 25

umbrella umbrella hers

slide-26
SLIDE 26

(C) Dhruv Batra & Zsolt Kira 26

slide-27
SLIDE 27

(C) Dhruv Batra & Zsolt Kira 27

slide-28
SLIDE 28
  • Deep Learning works! What are the implications?
  • Some issues:

– It’s still not perfect (or human-level for more complex things)

  • Does not work as well with limited or out of distribution data

– Visualization/interpretability – Security and adversaries – Theory and guarantees – Purely data-driven method => Dataset bias

  • Fairness, bias, privacy, …

(C) Dhruv Batra and Zsolt Kira 28

Some Implications

slide-29
SLIDE 29
  • Deep Learning is not the
  • nly technique in

machine learning!

  • It is powerful and general,

but has strengths and weaknesses, just like any

  • ther method

– Still important to try other methods if makes sense (random forests, etc.) – Sometimes overkill – Learn how to choose when to apply it, in addition to how

(C) Dhruv Batra and Zsolt Kira 29

A Warning

http://businessoverbroadway.com/wp-content/uploads/2018/01/data_science_methods_used_2017.png

slide-30
SLIDE 30

So what is Deep (Machine) Learning?

  • Representation Learning
  • Neural Networks
  • Deep Unsupervised/Reinforcement/Structured/

<insert-qualifier-here> Learning

  • Simply: Deep Learning

(C) Dhruv Batra & Zsolt Kira 30

slide-31
SLIDE 31

So what is Deep (Machine) Learning?

  • A few different ideas:
  • (Hierarchical) Compositionality

– Cascade of non-linear transformations – Multiple layers of representations

  • End-to-End Learning

– Learning (goal-driven) representations – Learning to feature extraction

  • Distributed Representations

– No single neuron “encodes” everything – Groups of neurons work together

(C) Dhruv Batra & Zsolt Kira 31

slide-32
SLIDE 32

Data: Object Classification Example

  • We want to learn what is in the image
  • Input:

– Lots and lots of examples with data and their labels

slide-33
SLIDE 33

Coming up with Features

  • Raw data from sensors is a product of a large

number of factors becoming entangled to produce the result

– They are almost never useful by themselves – Data => Model?

  • What are useful features you might be using?
  • Typical examples

– Color – Shape/contour – Parts – Attributes (furry, smooth, colorful, empty) – Spatial context – Temporal context/movement

You see this Camera see this

slide-34
SLIDE 34

34

\ˈd ē p\

fixed learned

your favorite classifier hand-crafted features SIFT/HOG

“car” “+”

This burrito place is yummy and fun!

VISION SPEECH NLP

Traditional Machine Learning

fixed learned

your favorite classifier hand-crafted features MFCC

fixed learned

your favorite classifier hand-crafted features Bag-of-words Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-35
SLIDE 35

35

VISION SPEECH NLP pixels edge texton motif part

  • bject

sample spectral band formant motif phone word character NP/VP/.. clause sentence story word

Hierarchical Compositionality

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-36
SLIDE 36

(C) Dhruv Batra & Zsolt Kira 36

Building A Complicated Function

Given a library of simple functions Compose into a complicate function

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-37
SLIDE 37

(C) Dhruv Batra & Zsolt Kira 37

Building A Complicated Function

Given a library of simple functions Compose into a complicate function

Idea 1: Linear Combinations

  • Boosting
  • Kernels
  • …

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-38
SLIDE 38

(C) Dhruv Batra & Zsolt Kira 38

Building A Complicated Function

Given a library of simple functions Compose into a complicate function

Idea 2: Compositions

  • Deep Learning
  • Grammar models
  • Scattering transforms…

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-39
SLIDE 39

(C) Dhruv Batra & Zsolt Kira 39

Building A Complicated Function

Given a library of simple functions Compose into a complicate function

Idea 2: Compositions

  • Deep Learning
  • Grammar models
  • Scattering transforms…

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-40
SLIDE 40

Deep Learning = Hierarchical Compositionality

“car”

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-41
SLIDE 41

Trainable Classifier Low-Level Feature Mid-Level Feature High-Level Feature

Feature visualization of convolutional net trained on ImageNet from [Zeiler & Fergus 2013]

“car”

Deep Learning = Hierarchical Compositionality

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-42
SLIDE 42

So what is Deep (Machine) Learning?

  • A few different ideas:
  • (Hierarchical) Compositionality

– Cascade of non-linear transformations – Multiple layers of representations

  • End-to-End Learning

– Learning (goal-driven) representations – Learning to feature extraction

  • Distributed Representations

– No single neuron “encodes” everything – Groups of neurons work together

(C) Dhruv Batra & Zsolt Kira 43

slide-43
SLIDE 43

44

\ˈd ē p\

fixed learned

your favorite classifier hand-crafted features SIFT/HOG

“car” “+”

This burrito place is yummy and fun!

VISION SPEECH NLP

Traditional Machine Learning

fixed learned

your favorite classifier hand-crafted features MFCC

fixed learned

your favorite classifier hand-crafted features Bag-of-words Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-44
SLIDE 44

Feature Engineering

(C) Dhruv Batra & Zsolt Kira 45

SIFT Spin Images HoG Textons and many many more….

slide-45
SLIDE 45

fixed unsupervised supervised

classifier Mixture of Gaussians MFCC

\ˈd ē p\

fixed unsupervised supervised

classifier K-Means/ pooling SIFT/HOG

“car”

fixed unsupervised supervised

classifier n-grams Parse Tree Syntactic

“+”

This burrito place is yummy and fun!

VISION SPEECH NLP

Traditional Machine Learning (more accurately)

“Learned”

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

(C) Dhruv Batra & Zsolt Kira 46

slide-46
SLIDE 46

fixed unsupervised supervised

classifier Mixture of Gaussians MFCC

\ˈd ē p\

fixed unsupervised supervised

classifier K-Means/ pooling SIFT/HOG

“car”

fixed unsupervised supervised

classifier n-grams Parse Tree Syntactic

“+”

This burrito place is yummy and fun!

VISION SPEECH NLP

Deep Learning = End-to-End Learning

“Learned”

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

(C) Dhruv Batra & Zsolt Kira 47

slide-47
SLIDE 47
  • “Shallow” models
  • Deep models

Trainable Feature- Transform / Classifier Trainable Feature- Transform / Classifier Trainable Feature- Transform / Classifier Learned Internal Representations

“Shallow” vs Deep Learning

“Simple” Trainable Classifier hand-crafted Feature Extractor

fixed learned

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-48
SLIDE 48

So what is Deep (Machine) Learning?

  • A few different ideas:
  • (Hierarchical) Compositionality

– Cascade of non-linear transformations – Multiple layers of representations

  • End-to-End Learning

– Learning (goal-driven) representations – Learning to feature extraction

  • Distributed Representations

– No single neuron “encodes” everything – Groups of neurons work together

(C) Dhruv Batra & Zsolt Kira 49

slide-49
SLIDE 49

Distributed Representations Toy Example

  • Local vs Distributed

(C) Dhruv Batra & Zsolt Kira 50

Slide Credit: Moontae Lee

slide-50
SLIDE 50

Distributed Representations Toy Example

  • Can we interpret each dimension?

(C) Dhruv Batra & Zsolt Kira 51

Slide Credit: Moontae Lee

slide-51
SLIDE 51

Ideal Feature Extractor

(C) Dhruv Batra & Zsolt Kira 52

slide-52
SLIDE 52

Power of distributed representations!

(C) Dhruv Batra & Zsolt Kira 53

Local Distributed

Slide Credit: Moontae Lee

slide-53
SLIDE 53

Power of distributed representations!

  • United States:Dollar :: Mexico:?

(C) Dhruv Batra & Zsolt Kira 54

Slide Credit: Moontae Lee

slide-54
SLIDE 54

ThisPlusThat.me

(C) Dhruv Batra & Zsolt Kira 55

Image Credit: http://insightdatascience.com/blog/thisplusthat_a_search_engine_that_lets_you_add_words_as_vectors.html

slide-55
SLIDE 55

So what is Deep (Machine) Learning?

  • A few different ideas:
  • (Hierarchical) Compositionality

– Cascade of non-linear transformations – Multiple layers of representations

  • End-to-End Learning

– Learning (goal-driven) representations – Learning to feature extraction

  • Distributed Representations

– No single neuron “encodes” everything – Groups of neurons work together

(C) Dhruv Batra & Zsolt Kira 56

slide-56
SLIDE 56

Benefits of Deep/Representation Learning

  • (Usually) Better Performance

– “Because gradient descent is better than you” Yann LeCun

  • New domains without “experts”

– RGBD – Multi-spectral data – Gene-expression data – Unclear how to hand-engineer

(C) Dhruv Batra & Zsolt Kira 57

slide-57
SLIDE 57

“Expert” intuitions can be misleading

  • “Every time I fire a linguist, the performance of our

speech recognition system goes up”

– Fred Jelinik, IBM ’98

(C) Dhruv Batra & Zsolt Kira 58

slide-58
SLIDE 58

Benefits of Deep/Representation Learning

  • Modularity!
  • Plug and play architectures!

(C) Dhruv Batra & Zsolt Kira 59

slide-59
SLIDE 59

Any DAG of differentialble modules is allowed!

Differentiable Computation Graph

Slide Credit: Marc'Aurelio Ranzato

(C) Dhruv Batra & Zsolt Kira 60

slide-60
SLIDE 60

(C) Dhruv Batra & Zsolt Kira 61

slide-61
SLIDE 61

Logistic Regression as a Cascade

(C) Dhruv Batra & Zsolt Kira 62

Given a library of simple functions Compose into a complicate function

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-62
SLIDE 62

Logistic Regression as a Cascade

(C) Dhruv Batra & Zsolt Kira 63

Given a library of simple functions Compose into a complicate function

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-63
SLIDE 63

Key Computation: Forward-Prop

(C) Dhruv Batra & Zsolt Kira 64

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-64
SLIDE 64

Key Computation: Back-Prop

(C) Dhruv Batra & Zsolt Kira 65

Slide Credit: Marc'Aurelio Ranzato, Yann LeCun

slide-65
SLIDE 65

Any DAG of differentialble modules is allowed!

Differentiable Computation Graph

Slide Credit: Marc'Aurelio Ranzato

(C) Dhruv Batra & Zsolt Kira 66

slide-66
SLIDE 66

Visual Dialog Model #1

Late Fusion Encoder

Slide Credit: Abhishek Das

slide-67
SLIDE 67

Visual Dialog Model #1

Late Fusion Encoder

Slide Credit: Abhishek Das

slide-68
SLIDE 68

Visual Dialog Model #1

Late Fusion Encoder

Slide Credit: Abhishek Das

slide-69
SLIDE 69

Visual Dialog Model #1

Late Fusion Encoder

Slide Credit: Abhishek Das

slide-70
SLIDE 70

Visual Dialog Model #1

Late Fusion Encoder

Slide Credit: Abhishek Das

slide-71
SLIDE 71

Visual Dialog Model #1

Late Fusion Encoder

Slide Credit: Abhishek Das

slide-72
SLIDE 72

Visual Dialog Model #1

Late Fusion Encoder

Slide Credit: Abhishek Das

slide-73
SLIDE 73

Visual Dialog Model #1

Late Fusion Encoder

Slide Credit: Abhishek Das

slide-74
SLIDE 74
slide-75
SLIDE 75
slide-76
SLIDE 76
slide-77
SLIDE 77
slide-78
SLIDE 78
slide-79
SLIDE 79
slide-80
SLIDE 80
slide-81
SLIDE 81
slide-82
SLIDE 82

Yes it works, but how?

(C) Dhruv Batra & Zsolt Kira 89

slide-83
SLIDE 83

Outline

  • What is Deep Learning, the field, about?
  • What is this class about?
  • What to expect?

– Logistics

  • FAQ

(C) Dhruv Batra & Zsolt Kira 90

slide-84
SLIDE 84

Outline

  • What is Deep Learning, the field, about?
  • What is this class about?
  • What to expect?

– Logistics

  • FAQ

(C) Dhruv Batra & Zsolt Kira 91

slide-85
SLIDE 85

What is this class about?

(C) Dhruv Batra & Zsolt Kira 92

slide-86
SLIDE 86

What is this class about?

  • Introduction to Deep Learning
  • Goal:

– After finishing this class, you should be ready to get started on your first DL research project.

  • Convolutional Neural Networks (CNNs)
  • Recurrent Neural Networks (RNNs)
  • Deep Reinforcement Learning
  • Generative Models (VAEs, GANs)
  • Target Audience:

– Senior undergrads, MS-ML, and new PhD students

  • Note: Materials largely follows those developed by Dhruv Batra

but with slight modifications

(C) Dhruv Batra & Zsolt Kira 93

slide-87
SLIDE 87

What this class is NOT

  • NOT the target audience:

– Advanced grad-students already working in ML/DL areas – People looking to understand latest and greatest cutting- edge research (e.g. GANs, AlphaGo, etc) – Undergraduate/Masters students looking to graduate with a DL class on their resume.

  • NOT the goal:

– Teaching a toolkit. “Intro to TensorFlow/PyTorch” – Intro to Machine Learning

(C) Dhruv Batra & Zsolt Kira 94

slide-88
SLIDE 88

Caveat

  • This is an ADVANCED Machine Learning class

– This should NOT be your first introduction to ML – You will need a formal class; not just self-reading/courser – Taking these concurrently does not count! – If you took CS 7641/ISYE 6740/CSE 6740 @GT, you’re in the right place – If you took an equivalent class elsewhere, see list of topics taught in CS 7641 to be sure.

(C) Dhruv Batra & Zsolt Kira 95

slide-89
SLIDE 89

Prerequisites

  • Intro Machine Learning

– Classifiers, regressors, loss functions, MLE, MAP

  • Linear Algebra

– Matrix multiplication, eigenvalues, positive semi-definiteness…

  • Calculus

– Multi-variate gradients, hessians, jacobians…

(C) Dhruv Batra & Zsolt Kira 96

If you do not have these pre-requisite, consider dropping!

  • This is for your benefit, as well as benefit of others
slide-90
SLIDE 90

Prerequisites

  • Intro Machine Learning

– Classifiers, regressors, loss functions, MLE, MAP

  • Linear Algebra

– Matrix multiplication, eigenvalues, positive semi-definiteness…

  • Calculus

– Multi-variate gradients, hessians, jacobians…

(C) Dhruv Batra & Zsolt Kira 97

slide-91
SLIDE 91

Prerequisites

  • Intro Machine Learning

– Classifiers, regressors, loss functions, MLE, MAP

  • Linear Algebra

– Matrix multiplication, eigenvalues, positive semi-definiteness…

  • Calculus

– Multi-variate gradients, hessians, jacobians…

  • Programming!

– Homeworks will require Python, C++! – Libraries/Frameworks: PyTorch – HW1 (pure python + PyTorch), HW2-4 (PyTorch) – Your language of choice for project

(C) Dhruv Batra & Zsolt Kira 98

slide-92
SLIDE 92

Course Information

  • Instructor: Zsolt Kira

– zkira@gatech – Location: 222 CCB

(C) Dhruv Batra and Zsolt Kira 99

  • I will always be available; just contact me or come to
  • ffice hours
  • My job is to:

– Teach the course such that you learn a lot – Provide any support needed towards that – Have fun and develop a passion for these topics

slide-93
SLIDE 93

Course Information

  • Instructor: Zsolt Kira

– zkira@gatech – Location: CODA room S1181B

(C) Dhruv Batra & Zsolt Kira 100

  • Zubair Irshad
  • Ben Wilson
  • James Smith

Incoming Ph.D.

slide-94
SLIDE 94

Current TAs

(C) Dhruv Batra & Zsolt Kira 101

More TAs coming soon!

Rahul Duggal 2nd year CS PhD student http://www.rahulduggal.com/ Patrick Grady 2nd year Robotics PhD student https://www.linkedin.com/in/patrick-grady Sameer Dharur MS-CS student https://www.linkedin.com/in/sameerdharur/ Jiachen Yang 2nd year ML PhD https://www.cc.gatech.edu/~jyang462/ Yinquan Lu 2nd year MSCSE student https://www.cc.gatech.edu/~jyang462/ Anishi Mehta MSCS student https://www.linkedin.com/in/anishimehta

slide-95
SLIDE 95

Organization & Deliverables

  • PS0 (2%) + 4 homeworks (78%)

– PS0 is warm-up graded pass/fail – Do it! – In general PS/HWs a mix of theory and implementation – First real one goes out next week

  • Start early, Start early, Start early, Start early, Start early, Start early,

Start early, Start early, Start early, Start early

  • Final project (20%)

– Projects done in groups of 3-4

  • (Bonus) Class Participation (up to 3%)

– Top contributors to discussions (mainly on Piazza) – Ask questions, answer questions

(C) Dhruv Batra & Zsolt Kira 102

slide-96
SLIDE 96

New Element: FB Co-Teaching!

  • Several elements including:

– Guest Lectures – 6 in-class lectures by FB

  • Data wrangling
  • Embeddings and world2vec
  • Self-attention and transformers
  • Language modeling and translation
  • Large-scale systems
  • Fairness, privacy, ethics

– Assignments – Volunteers developing some new elements for assignments – Project ideas – Instructors will provide ideas for real-world projects and possible (surrogate/public) data sources that mirror some of the challenges they are working on

(C) Dhruv Batra & Zsolt Kira 103

slide-97
SLIDE 97

Late Days

  • “Free” Late Days

– 7 late days for the semester

  • Use for HWs
  • Cannot use for project related deadlines

– After free late days are used up:

  • 25% penalty for each late day

(C) Dhruv Batra & Zsolt Kira 104

slide-98
SLIDE 98

PS0

  • Out today; due 01/14

– Available on website (will show up on Canvas today)

  • Grading: pass/fail

– <=80% means that you might not be prepared for the class – Consider dropping or talk to me if that’s the case!

  • Topics

– Probability, calculus, convexity, proving things

(C) Dhruv Batra & Zsolt Kira 105

slide-99
SLIDE 99

Project

  • Goal

– Chance to try Deep Learning – Encouraged to apply to your research (computer vision, NLP, robotics,…) – Must be done this semester. – Can combine with other classes with separated thrusts

  • get permission from both instructors; delineate different parts

– Extra credit for shooting for a publication – Teams of 3-4 people

  • Undergraduate and graduates on separate teams
  • Contributions of each member must be explained and cannot just be report

writing, etc.

  • Main categories

– Application/Survey

  • Compare a bunch of existing algorithms on a new application domain of your

interest

– Formulation/Development

  • Formulate a new model or algorithm for a new or old problem

– Theory

  • Theoretically analyze an existing algorithm

(C) Dhruv Batra & Zsolt Kira 106

slide-100
SLIDE 100

Computing

  • Major bottleneck

– GPUs

  • Options

– Your own / group / advisor’s resources – Google Cloud Credits

  • $50 credits to every registered student courtesy Google

– Google colaboratory allows free TPU access!!

  • https://colab.research.google.com/notebooks/welcome.ipynb

– Minsky cluster in IC

(C) Dhruv Batra & Zsolt Kira 107

slide-101
SLIDE 101

4803 vs 7643

  • Level differentiation
  • HWs

– Extra credit questions for 4803 students, necessary for 7643

  • Project

– Pre-proposal and poster session from grad students – Higher expectations from 7643

(C) Dhruv Batra & Zsolt Kira 108

slide-102
SLIDE 102

Mental Health and Well-being

  • Above all else take care of your well-being!
  • Lots of resources at GT

(C) Dhruv Batra & Zsolt Kira 109

slide-103
SLIDE 103

Outline

  • What is Deep Learning, the field, about?
  • What is this class about?
  • What to expect?

– Logistics

  • FAQ

(C) Dhruv Batra & Zsolt Kira 110

slide-104
SLIDE 104

Waitlist / Audit / Sit in

  • Waitlist

– Enrollment on-going and being increased. – Do PS0. Come to first few classes. – Hope people drop.

  • Audit or Pass/Fail

– We will give preference to people taking class for credit.

  • Sitting in

– Talk to instructor.

(C) Dhruv Batra & Zsolt Kira 111

slide-105
SLIDE 105

Re-grading Policy

  • Homework assignments

– Within 1 week of receiving grades: see the TAs

  • This is an advanced grad class.

– The goal is understanding the material and making progress towards our research.

(C) Dhruv Batra & Zsolt Kira 112

slide-106
SLIDE 106

Collaboration Policy

  • Collaboration

– Only on HWs and project (not allowed in PS0). – You may discuss the questions – Each student writes their own answers independently – Write on your homework anyone with whom you collaborate – Each student must write their own code for the programming part

  • Zero tolerance on plagiarism

– Neither ethical nor in your best interest – Always credit your sources – Don’t cheat. We will find out.

(C) Dhruv Batra & Zsolt Kira 113

slide-107
SLIDE 107

(C) Dhruv Batra & Zsolt Kira 114

slide-108
SLIDE 108

How to Succeed

  • Do the readings and show up to class

– Ask lots of questions!

  • Start problem sets and homeworks early
  • Start project early
  • Ask when you need help

(C) Dhruv Batra & Zsolt Kira 115

slide-109
SLIDE 109

Communication Channels

  • Primary means of communication -- Piazza

– No direct emails to Instructor unless private information – Instructor/TAs can provide answers to everyone on forum – Class participation credit for answering questions! – No posting answers. We will monitor.

  • Staff Mailing List (for personal questions)

– cs4803-7643-staff@lists.gatech.edu

  • Gradescope for submissions – will be sync’d to roll
  • Links:

(C) Dhruv Batra & Zsolt Kira 116

Website: http://www.cc.gatech.edu/classes/AY2020/cs7643_spring/ Piazza: https://piazza.com/gatech/spring2020/cs4803dl7643a/ Canvas: https://gatech.instructure.com/courses/94450/ Gradescope: https://www.gradescope.com/courses/78537

Course Access Code (Piazza): MWXKY8

slide-110
SLIDE 110

Todo

  • PS0

– Due Tues. Jan 14 11:55pm

(C) Dhruv Batra & Zsolt Kira 117