7.- Non supervised Neural Networks: Self-organizing Maps by - - PDF document

▶
7 non supervised neural networks self organizing maps
SMART_READER_LITE
LIVE PREVIEW

7.- Non supervised Neural Networks: Self-organizing Maps by - - PDF document

Apr 25, 2023 383 likes •518 views

CVG-UPM COMPUTER VISION Machine Learning & Neural Networks 7.- Non supervised Neural Networks: Self-organizing Maps by Pascual Campoy Grupo de Visin por Computador U.P.M. - DISAM P. Campoy P. Campoy Machine Learning and Neural

slide-1
SLIDE 1

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

Machine Learning & Neural Networks

7.- Non supervised Neural Networks: Self-organizing Maps

by Pascual Campoy Grupo de Visión por Computador U.P.M. - DISAM

3

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

Unsupervised learning

Feature space

Unsupervised learning concept

?

area length

Working structure y1 . . ym . . xn x1

Clustering

slide-2
SLIDE 2

4

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

Self organizing Maps (SOM)

  • Bio-inspired idea:

Similar inputs map onto neighbor outputs.

  • SOM objective:

Neighbor inputs map onto neighbor outputs and vice versa Rn → → R2, R1 D.R. into a pattern space

5

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

recent paper

slide-3
SLIDE 3

6

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM working principle

  • Objective:

To obtain a bijective application Rn ⇔ R2, such as neighborhood in the input space ⇔ neighborhood in the output space

  • Procedure:

To distribute an elastic 2D lattice into the nD input space, where the every cross represent a neuron that has:

  • a position in the input space

(defined by its weights)

  • a position in the output space

(defined by its coordinates in the lattice)

w

7

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM: viability

is it possible in this cubic example that any two neighbor input sample are represented by neighbor neurons? and in this Swiss roll example? concept of Intrinsic Dimensionality

  • f the data
slide-4
SLIDE 4

8

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM: running and learning

the neurons whose weight vector is the closest to this input data Learning: How are weights updated for every train input in order to fulfill the SOM objectives?

  • The weights of which neurons are updated?
  • How are they updated?

Running: Which neuron is activated by every input data?

9

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM: learning procedure

  • The neuron whose weights are the closest to

the present train sample x, called the winning neuron wb (also the best matching unit), and its neighbors are the ones that learn (i.e. update their weights)

α ds(wi-wb) α k

where α = α(dos(wi-wb),k) is function of:

  • the distance to the winning neuron in the output space dos(wi-wb),
  • the training instant k (e.g. epoch)
  • Learning rule: Δkwi = α (x-wi)
slide-5
SLIDE 5

10

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM: neural implementation

  • Training and running imply distance calculation, that

can be implemented by scalar product in a one dimensional incremented space . . . . . x1 xI xi . . . . . .

11

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM: discussion on

  • bjective fulfillment

feature 1 feature 2 feature 1 feature 2 examples R2 → R1

slide-6
SLIDE 6

13

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM: examples 1

14

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM: example 2

slide-7
SLIDE 7

15

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

σv=0.25

α0=0.1

σv=0.5 σv=0.75

α0=2.1

SOM results: influence of learning parameters

16

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

different instances SOM result different order different # neurons

SOM: influence of training samples and # of neurons

slide-8
SLIDE 8

17

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

Matlab commands

som1=newsom(minmax(psom),[10 1]); som1=train(som1,psom) plotsom(som1.iw{1,1},som1.layers{1}.distances) ynt=sim(som1,tsom); yntind=vec2ind(ynt);

18

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

Example 7.1: SOM as classifier

p.valor t.valor load datos_D2_C2

slide-9
SLIDE 9

19

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

Solution example 7.1

SOM 8x1 SOM 8x8

182 7 C2 11 100 C1 C2 C1 187 6 C2 6 101 C1 C2 C1

20

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

Exercise 7.1: SOM as classifier

Using the data of the previous example: Discuss the influence of the following factors (plot the results and quantify the test error and the training error):

  • 1. # of training samples
  • 2. # order of the training samples
  • 3. # of neurons
  • 4. # of epoches
slide-10
SLIDE 10

21

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM example: Transfos state

 5D input: % de H2 , CH4 C2H2 C2H4 C2H6  2D U-matrix output  Supervised manual semantic

22

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM example: temperature profile classification for pig iron control

24D input 10x10 output map manual labeling into 3 classes

95% confidence for pig iron temperature prediction (8h)

slide-11
SLIDE 11

23

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM example: video compression …

  • riginal sequence

training data

 1D output map  256 neurons

24

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

… SOM example: video compression …

Training: weights update

slide-12
SLIDE 12

25

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

… SOM example: video compression

√MSE=13,47 H=0.4375

compression factor: 1:16 bits/pixel: 0.5

Testing

27

CVG-UPM

COMPUTER VISION

Machine Learning and Neural Networks

  • P. Campoy
  • P. Campoy

SOM: concerns and limitations

  • Concerns:
  • output map dimension?
  • # of neurons?
  • learning rate? neighborhood?
  • order of the training samples?
  • Limitations:
  • neighbor inputs may activate distant neurons
  • distant inputs may activate neighbor neurons