Learning Multi-Sensory Integration with Self- Organization and - - PowerPoint PPT Presentation

http://www.informatik.uni-hamburg.de/WTM/

Learning Multi-Sensory Integration with Self- Organization and Statistics

Johannes Bauer, Stefan Wermter

The Superior Colliculus

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 2

The Superior Colliculus

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 3

inverse effectiveness[1]

sub-additive super-additive additive

spatial principle[1]

• ptimal integration[2]

coincident close disparate

uni-sensory baseline

neural response

What’s Interesting About That?

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 4

?

Place in SC Meaning of Input

The Algorithm[3]

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 5

𝜍: input stimulus 𝑏𝑙: activity of 𝑗𝑙 𝜍𝑚: preferred value of 𝑝𝑚

The Algorithm[3]

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 6

𝜍: input stimulus 𝑏𝑙: activity of 𝑗𝑙 𝜍𝑚: preferred value of 𝑝𝑚 𝑄 𝜍 = 𝜍𝑚 𝑏1, 𝑏2, … , 𝑏𝑛 ∼ 𝑄 𝑏1, 𝑏2, … , 𝑏𝑛 𝜍 = 𝜍𝑚) 𝑄 𝑏1, 𝑏2, … , 𝑏𝑛 𝑄(𝜍 = 𝜍𝑚)

The Algorithm[3]

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 6

𝜍: input stimulus 𝑏𝑙: activity of 𝑗𝑙 𝜍𝑚: preferred value of 𝑝𝑚 𝑄 𝜍 = 𝜍𝑚 𝑏1, 𝑏2, … , 𝑏𝑛 ∼ 𝑄 𝑏1, 𝑏2, … , 𝑏𝑛 𝜍 = 𝜍𝑚) 𝑄 𝑏1, 𝑏2, … , 𝑏𝑛 𝑄(𝜍 = 𝜍𝑚)

The Algorithm[3]

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 6

𝜍: input stimulus 𝑏𝑙: activity of 𝑗𝑙 𝜍𝑚: preferred value of 𝑝𝑚 𝑄 𝜍 = 𝜍𝑚 𝑏1, 𝑏2, … , 𝑏𝑛 ∼ 𝑄(𝑏𝑙 ∣ 𝜍 = 𝜍𝑚)

𝑙

𝑄(𝑏𝑙)

𝑙

𝑄(𝜍 = 𝜍𝑚) Noise independent.

The Algorithm[3]

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 6

𝜍: input stimulus 𝑏𝑙: activity of 𝑗𝑙 𝜍𝑚: preferred value of 𝑝𝑚 Noise independent. 𝜍 unif. dist. 𝑄 𝜍 = 𝜍𝑚 𝑏1, 𝑏2, … , 𝑏𝑛 ∼ 𝑄(𝑏𝑙 ∣ 𝜍 = 𝜍𝑚)

𝑙

𝑄(𝑏𝑙)

𝑙

The Algorithm[3]

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 6

𝜍: input stimulus 𝑏𝑙: activity of 𝑗𝑙 𝜍𝑚: preferred value of 𝑝𝑚 Noise independent. 𝜍 unif. dist. 𝑄 𝜍 = 𝜍𝑚 𝑏1, 𝑏2, … , 𝑏𝑛 ∼ 𝑄(𝑏𝑙 ∣ 𝜍 = 𝜍𝑚)

𝑙

The Algorithm[3]

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics

𝑄 𝜍 = 𝜍𝑚 𝑏1, 𝑏2, … , 𝑏𝑛 ∼ 𝑄(𝑏𝑙 ∣ 𝜍 = 𝜍𝑚)

𝑙

11

The Algorithm[3]

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics

Assume 𝑄 𝑏𝑙 ∣ 𝜍 = 𝜍𝑘 =

𝑝𝑜𝑘,𝑏[𝑏𝑙] ∑𝑝𝑜𝑘,𝑏 , then

𝑄 𝜍 = 𝜍𝑘 ∣ 𝑏1, 𝑏2, … , 𝑏𝑛 ∼ 𝑝𝑜𝑘,𝑚 𝑏𝑚 ∑ 𝑝𝑜𝑘,𝑚

𝑚

can be adapted SOM-like

12

Comparison to regular SOM

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 13

Comparison to regular SOM

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 13

The Network in Action

Johannes Bauer ‘visual’ input population-coded PDF ‘auditory’ input Multi-Sensory Integration through Self-Organization and Statistics 15

The Network in Action

Johannes Bauer ‘visual’ input population-coded PDF ‘auditory’ input Multi-Sensory Integration through Self-Organization and Statistics 16

Performance

Johannes Bauer 17

The network replicates biological phenomena.

Inverse Effectiveness Spatial Principle

The network integrates multi-sensory information.

errors given ‘visual’ input errors given ‘auditory’ input errors given multi-sensory input Multi-Sensory Integration through Self-Organization and Statistics

presented a novel self-organizing ANN algorithm which

• learns to combine information near-optimally
• shows spatial principle and MLE-like behavior
• shows benefit of multisensory integration
• learns to compute a PDF for latent variables
• is unsupervised
• has few inbuilt assumptions

Conclusion

Johannes Bauer Multi-Sensory Integration through Self-Organization and Statistics 18

The End

References:

[1]: Stanford, T. R., Quessy, S., Stein, B. E., Jul. 2005. Evaluating the operations underlying multisensory integration in the cat superior colliculus. The Journal of Neuroscience 25 (28), 6499–6508. [2]: Alais, D., Burr, D., Feb. 2004. The ventriloquist effect results from Near-Optimal bimodal integration. Current Biology 14 (3), 257–262. [3]: Bauer, J. and Wermter, S., Sept. 2013. Self-organized neural learning of statistical inference from high-dimensional data. In: Proceedings of the International Joint Conference on Artificial Intelligence 2013.

Johannes Bauer 14 Multi-Sensory Integration through Self-Organization and Statistics

Performance – Behavioral*

Johannes Bauer 20 Multi-Sensory Integration through Self-Organization and Statistics

auditory 𝜏𝑏 = 4.594 ∙ 10−4 𝑞𝑏 = 1.680 ∙ 10−1 visual 𝜏𝑤 = 1.061 ∙ 10−4 𝑞𝑤 = 8.272 ∙ 10−1 multi-sensory 𝜏𝑛 = 8.153 ∙ 10−5

• ptimal

𝜏𝑛,𝑝𝑞𝑢 =

1

1 𝜏𝑤 2+ 1 𝜏𝑏 2

≈ 4.185 ∙ 10−5 𝑞𝑏,𝑝𝑞𝑢 ≈ 1.876 ∙ 10−1 𝑞𝑤,𝑝𝑞𝑢 ≈ 8.124 ∙ 10−1

*simulation parameters differ from rest of talk.