Neural Networks: Prediction (i.e. the forward pass) Machine - - PowerPoint PPT Presentation
Neural Networks: Prediction (i.e. the forward pass) Machine Learning Based on slides and material from Geoffrey Hinton, Richard Socher, Dan Roth, 1 Yoav Goldberg, Shai Shalev-Shwartz and Shai Ben-David, and others Neural Networks What is a
1
Based on slides and material from Geoffrey Hinton, Richard Socher, Dan Roth, Yoav Goldberg, Shai Shalev-Shwartz and Shai Ben-David, and others
2
3
4
We will use this example network as to introduce the general principle of how to make predictions with a neural network.
5
Naming conventions for this example
6
Bias feature, always 1
Naming conventions for this example
7
Sigmoid activations
Bias feature, always 1 Naming conventions for this example
8
Sigmoid activations
Bias feature, always 1 Naming conventions for this example
Linear activation
9
Naming Convention for Weights
10
Naming Convention for Weights
11
12
# + 𝑥!! # 𝑦! + 𝑥$! # 𝑦$)
13
# + 𝑥!! # 𝑦! + 𝑥$! # 𝑦$)
# + 𝑥!$ # 𝑦! + 𝑥$$ # 𝑦$)
14
# + 𝑥!! # 𝑦! + 𝑥$! # 𝑦$)
# + 𝑥!$ # 𝑦! + 𝑥$$ # 𝑦$)
% + 𝑥!! % 𝑨! + 𝑥$! % 𝑨$
15
# + 𝑥!! # 𝑦! + 𝑥$! # 𝑦$)
# + 𝑥!$ # 𝑦! + 𝑥$$ # 𝑦$)
% + 𝑥!! % 𝑨! + 𝑥$! % 𝑨$
16
# + 𝑥!! # 𝑦! + 𝑥$! # 𝑦$)
# + 𝑥!$ # 𝑦! + 𝑥$$ # 𝑦$)
% + 𝑥!! % 𝑨! + 𝑥$! % 𝑨$
In general, before visiting (i.e. computing) the value of a node, visit all nodes that serve as inputs to it.
17
# + 𝑥!! # 𝑦! + 𝑥$! # 𝑦$)
# + 𝑥!$ # 𝑦! + 𝑥$$ # 𝑦$)
% + 𝑥!! % 𝑨! + 𝑥$! % 𝑨$
In general, before visiting (i.e. computing) the value
Questions?