] [ ( ) 1 = g u , 0 1 + 2 u e 1 Funzione di - - PowerPoint PPT Presentation

Hopfield Model – Continuous Case

The Hopfield model can be generalized using continuous activation functions. More plausible model. In this case: where is a continuous, increasing, non linear function. Examples

( )

      + = =

∑

j i j ij i i

I V W g u g V

β β

β

g

( ) ] [

1 1,

e e e e u tanh

u u u u

− ∈ + − =

− − β β β β

β

( ) ] [

1 1 1

2

, e u g

u

∈ + =

− β β

Funzione di attivazione

ß > 1 ß = 1 ß < 1

• 1

+1

( ) ( )

x tanh x f β =

Updating Rules

Several possible choices for updating the units : Asynchronous updating: one unit at a time is selected to have its output set Synchronous updating: at each time step all units have their output set Continuous updating: all units continuously and simultaneously change their outputs

Continuous Hopfield Models

Using the continuous updating rule, the network evolves according to the following set of (coupled) differential equations: where are suitable time constants ( > 0). Note When the system reaches a fixed point ( / = 0 ) we get Indeed, we study a very similar dynamics

( )

      + + − = + − =

∑

j i j ij i i i i i

I V w g V u g V dt dV

β β

τ

i

τ

i

τ

i

dV dt

i ∀ ( )

i i

u g V

β

=

( )

i j j ij i i i

I u g w u dt du + + − =

∑

β

τ

Modello di Hopfield continuo (energia)

Perché è monotona crescente e . N.B. cioè è un punto di equilibrio

( ) ( )

2 1 2 1

2 1

≤       ′ − = − =       + − − = − + − − =

∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑

−

dt du u g dt du dt dV I u V T dt dV dt dV I dt dV V g dt dV V T V dt dV T dt dE

i i i i i i i i i i j j ij i i i i i i i i j i ij ij j i ij ij β β

τ τ

β

g >

i

τ

= ⇔ = dt du dt dE

i

i

u

The Energy Function

As the discrete model, the continuous Hopfield network has an “energy” function, provided that W = WT : Easy to prove that with equality iff the net reaches a fixed point.

( )

∑ ∑∑ ∑ ∫

− + − =

− i i i i j i V j i ij

V I dV V g V V w E

i

1

2 1

β

≤ dt dE

Modello di Hopfield continuo (relazione con il modello discreto)

Esiste una relazione stretta tra il modello continuo e quello discreto. Si noti che : quindi : Il 2o termine in E diventa : L’integrale è positivo (0 se Vi=0). Per il termine diventa trascurabile, quindi la funzione E del modello continuo diventa identica a quello del modello discreto

( ) ( ) ( )

i i i i

u g u g u g V β β

β

≡ = =

1

( )

i i

V g u

1

1

−

= β

( ) dV

V g

i V i

i

∑ ∫

−

1

1

β

∞ → β

Optimization Using Hopfield Network

§

Energy function of Hopfield network

§

The network will evolve into a (locally / globally) minimum energy state

§

Any quadratic cost function can be rewritten as the Hopfield network Energy

• function. Therefore, it can be minimized using Hopfield network.

§

Classical Traveling Salesperson Problem (TSP)

§

Many other applications

• 2-D, 3-D object recognition
• Image restoration
• Stereo matching
• Computing optical flow

i i i j i i j ij

V I V V w E

∑ ∑ ∑

− − =

2 1