Introduction to Brain Imaging: fMRI and MEG/EEG The Algonauts - - PowerPoint PPT Presentation

▶
introduction to brain imaging fmri and meg eeg the
SMART_READER_LITE
LIVE PREVIEW

Introduction to Brain Imaging: fMRI and MEG/EEG The Algonauts - - PowerPoint PPT Presentation

Sep 01, 2022 364 likes •737 views

Introduction to Brain Imaging: fMRI and MEG/EEG The Algonauts Workshop Yalda Mohsenzadeh Computer Science and Artificial Intelligence Lab. MIT, Cambridge, USA July 2019 Convnets: Brain Inspired Architectures V1 V2 V4 IT Zebra 2

slide-1
SLIDE 1

Introduction to Brain Imaging: fMRI and MEG/EEG The Algonauts Workshop

Yalda Mohsenzadeh

Computer Science and Artificial Intelligence Lab. MIT, Cambridge, USA July 2019

slide-2
SLIDE 2

Convnets: Brain Inspired Architectures

Zebra

V1 V4 IT V2

2

(Khaligh-Razavi et al. 2014, Yamins et al. 2014, Guclu et al. 2015, Cichy et al. 2016 )

slide-3
SLIDE 3

What is fMRI?

3

slide-4
SLIDE 4

Magnetic Resonance Imaging (MRI)

4

slide-5
SLIDE 5

Functional Magnetic Resonance Imaging (fMRI)

5

Time (sec)

0 5 10 15 20 25 4 3 2 1 BOLD %

Stimulus on Neurons fire Bold response

BOLD %

Time (sec)

slide-6
SLIDE 6

Data Structure

6

Experiment

Data Time S1 S2

…

slide-7
SLIDE 7

General Linear Model: Constructing BOLD signals

7

HRF Baseline signal Response to S1 Response to S2 ß1 ß2 ß3

error(t) = signal (t) – prediction (t)

slide-8
SLIDE 8

General Linear Model: Constructing BOLD signals

8

ß1 ß2 ß3 x x x + + = + err = + err

slide-9
SLIDE 9

General Linear Model: Constructing BOLD signals

9

x = ß1 ß2 ß3 Time + Design Matrix

Bold signal Y = X × B + e Find B such that Min

  • Task related

variations Noise variations

𝑼 𝟐 𝑼

slide-10
SLIDE 10

Visual Recognition in the Brain

  • What brain regions are engaged in visual processing?
  • What kind of representations are held in these regions?
  • What algorithms are being carried out by these regions?

10

slide-11
SLIDE 11

Visual Recognition in the Brain

11

IT EVC

slide-12
SLIDE 12

Stimulus set

12

slide-13
SLIDE 13

+

1/2 sec

+

2.5 – 6 seconds 2.5 – 6 seconds

+ +

2.5 – 6 seconds

1/2 sec

PRESS BUTTON

+ + + +

2.5 – 6 seconds

1/2 sec

N=15

13

slide-14
SLIDE 14

Representational Similarity Analysis

Compute dissimilarity

(e.g. 1 - correlation)

Representational pattern

(e.g. voxels, neurons, model units)

(Kriegeskorte & Kievit 2013, Diedrichsen et al. 2011, Laakso & Cottrell 2000, Kriegeskorte et al. 2008)

Stimulus

(e.g. images, sounds, other experimental conditions)

Brain representation

(e.g. fMRI pattern dissimilarities)

14

Representational dissimilarity matrix (RDM)

slide-15
SLIDE 15

fMRI Track RDMs

15

EVC IT

face Bodies

  • bject

scene Dissimilarity Dissimilarity

slide-16
SLIDE 16

What is MEG?

16

slide-17
SLIDE 17

Magnetoencephalography (MEG) / Electroencephalography (EEG)

17

306 Channel SQUID sensor array EEG

slide-18
SLIDE 18

+

500 m sec

+

700 to 1000 m sec 700 to 1000 m sec

+ +

700 to 1000 m sec

500 m sec

PRESS BUTTON

+ + +

700 to 1000 m sec

500 m sec

N=15

18

+

slide-19
SLIDE 19

19

Stimulus on

slide-20
SLIDE 20

MEG Neural Data Decoding

t 900 ms

  • 300 ms

MEG pattern vector at time t

(Carlson et al. 2013; Cichy et al. 2014; Isik et al. 2014; Clarke et al. 2014; Kaneshiro et al. 2015)

20

vs. SVM Classifier Train set Test set Pairwise classification (i,j) Representational dissimilarity matrix at time t

slide-21
SLIDE 21

Time-Resolved MEG RDMs

21

slide-22
SLIDE 22

Temporal Generalization

22

(tx, ty)

Time-Time decoding matrix

N

Nth raw pattern vector at time ty MEG pattern vector at time t

(Carlson et al. 2013; Cichy et al. 2014; Isik et al. 2014; Clarke et al. 2014; Kaneshiro et al. 2015; King et al. 2016)

SVM Classifier

1 2 N-1

Train a SVM classifier using N-1 raw vectors at tx

slide-23
SLIDE 23

Possible Neural Architectures

23

(King et al., 2016)

slide-24
SLIDE 24

24

Feedforward Recurrent

Architecture

Occluded 60% Un-occluded

A Neural Architecture with Recurrent Interactions

(Rajaei, Mohsenzadeh, Ebrahimpour, Khaligh-Razavi, 2019)

slide-25
SLIDE 25

Distance Measures

25

Decoding Distance Time (ms) Pearson Distance Time (ms) Euclidean Distance Time (ms)

slide-26
SLIDE 26

Reliability of Distance Measures

26

Pearson Euclidean Decoding

Time (ms) Reliability

slide-27
SLIDE 27

MEG/ ROI fMRI fusion

MEG fMRI t 900 ms

  • 300 ms

Calculate Spearman’s rho MEG-fMRI representational similarity in EVC Early Visual Cortex (EVC) MEG fMRI t 900 ms

  • 300 ms

Calculate Spearman’s rho MEG-fMRI representational similarity in IT Inferior Temporal (IT)

27

Cichy et al. 2014, 2016; Mohsenzadeh et al., (2018, 2019)

slide-28
SLIDE 28

ROI-based fMRI-MEG fusion

Early Visual Cortex (EVC) Inferior Temporal (IT)

28

Spearman’s rho EVC IT Time (ms)

Late Early

slide-29
SLIDE 29

MEG Track RDMs

29

Dissimilarity Dissimilarity

Early Late

face Bodies

  • bject

scene

slide-30
SLIDE 30

Summary

30

slide-31
SLIDE 31

Brain Imaging Methods

MRI

31

slide-32
SLIDE 32

Brain Imaging Methods

MRI fMRI

32

slide-33
SLIDE 33

Time (sec)

0 5 10 15 20 25

4 3 2 1

BOLD %

Brain Imaging Methods

MRI fMRI

33

EEG MEG

slide-34
SLIDE 34

Brain representation

(e.g. fMRI pattern dissimilarities)

Computational model representation

(e.g. face-detector model)

Stimulus description

(e.g. pixel-based dissimilarity)

Behavior

(e.g. dissimilarity judgments)

Representational Similarity Analysis

34

(Kriegeskorte & Kievit 2013, Diedrichsen et al. 2011, Laakso & Cottrell 2000, Kriegeskorte et al. 2008)

slide-35
SLIDE 35

35

Acknowledgments

slide-36
SLIDE 36

36