binocular cells Tues. Jan. 30, 2018 1 Recall last lecture: - PowerPoint PPT Presentation

COMP 546 Lecture 6 orientation 2: complex cells binocular cells Tues. Jan. 30, 2018 1

Recall last lecture: simple Cell Linear response half wave rectification + - - + - - + - - + 2

Recall last lecture: simple Cell Linear response half wave rectification + - - + - - + - - + - + + - + + - + + - 3

“Complex Cell” (Hubel and Wiesel) Responds to preferred orientation of line anywhere in receptive field. 4

How to construct a complex cell? (1) + + + - + + - - - - - + + + - + + - - - - - + + + - + + - - - - - + + + - Complex cell + Use several simple cells with common orientation and neighboring receptive field locations . If we sum up their rectified responses then we get a response to image 5 structure of that orientation anywhere in the overlapping receptive fields.

How to construct a complex cell? (2) even odd - + + + + + - - - - - - + + + + + - - - - - - + + + + + - - - - - - + + + Complex cell + Now suppose these even cell and odd cells have the same receptive field locations (perfect overlap) . Again sum up their rectified responses and the result is a 6 response anywhere in the receptive field.

How to construct a complex cell? (3) rectify and square even odd the response - + + + + + - - - - - - + + + + + - - - - - - + + + + + - - - - - - + + + Complex cell + This is the same as the last model but now we square the positive values. This model is more commonly used than model (2) and so we’ll use this one. 7

Unit circle (cos 𝜒, sin 𝜒) 𝑑𝑝𝑡 2 𝜒 + 𝑡𝑗𝑜 2 𝜒 = 1 8

Model of a Complex Cell (3) ( < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 > , < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 > ) The response to an image 𝐽 𝑦 , 𝑧 is modelled as the Euclidean length of the vector, i.e. L2 norm ∥ 2 ∥ ( < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 > , < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 > ) 9

Complex cell response modelled as the length of 2D vector: ( < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 > , < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 > ) 10

We can model complex cells of any orientation. Cosine Gabor Sine Gabor 11

Example: image cross correlated with four complex cells 12

COMP 546 Lecture 6 orientation 2: complex cells binocular cells Tues. Jan. 30, 2018 13

𝜚 = 90 𝜚 = 90 𝜚 = −90 𝜚 = −90 Left halves of Right halves of retina map to V1 in retina map to V1 in left hemisphere right hemisphere 𝜚 = 𝜚 = eccentricity eccentricity 90 −90 −90

Superimposed left and right eye images 𝑧 Negative disparity 𝑦 Zero disparity (eyes verged at this depth) Positive disparity 15

How to estimate binocular disparity ? Computer vision-ish approach: For each 𝑦 0 , 𝑧 0 , find disparity value 𝑒 that minimizes: ) 2 ( 𝐽 𝑚𝑓𝑔𝑢 𝑦 + 𝑒, 𝑧 − 𝐽 𝑠𝑗𝑕ℎ𝑢 𝑦 , 𝑧 𝑦,𝑧 where sum is over a neighborhood of 𝑦 0 , 𝑧 0 . i.e. Shift the left image to undo the disparity and register the left and right images. 16

How to build ‘disparity tuned’ binocular cells ? We use vertically oriented cells only. 17

Left eye Right eye 𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦 0 − 𝑒 , 𝑧 − 𝑧 0 𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦 0 , 𝑧 − 𝑧 0 𝑦 0 + 𝑒 , 𝑧 0 𝑦 0 , 𝑧 0 𝑒 𝑦 0 + 𝑒 , 𝑧 0 𝑦 0 , 𝑧 0 18

Idea 1: (analogous to computer vision) To compute disparity at 𝑦 0 , 𝑧 0 , find the 𝑒 that minimizes : ( < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦 0 − 𝑒, 𝑧 − 𝑧 0 , 𝐽 𝑚𝑓𝑔𝑢 (𝑦, 𝑧) > < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠(𝑦 − 𝑦 0 , 𝑧 − 𝑧 0 ), 𝐽 𝑠𝑗𝑕ℎ𝑢 𝑦, 𝑧 > ) 2 − + ( < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦 0 − 𝑒, 𝑧 − 𝑧 0 , 𝐽 𝑚𝑓𝑔𝑢 (𝑦, 𝑧) > < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦 0 , 𝑧 − 𝑧 0 , 𝐽 𝑠𝑗𝑕ℎ𝑢 (𝑦, 𝑧) > ) 2 − 19

Idea 1: (analogous to computer vision) To compute disparity at 𝑦 0 , 𝑧 0 , find the 𝑒 that minimizes : ( < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦 0 − 𝑒, 𝑧 − 𝑧 0 , 𝐽 𝑚𝑓𝑔𝑢 (𝑦, 𝑧) > < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠(𝑦 − 𝑦 0 , 𝑧 − 𝑧 0 ), 𝐽 𝑠𝑗𝑕ℎ𝑢 𝑦, 𝑧 > ) 2 − + ( < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦 0 − 𝑒, 𝑧 − 𝑧 0 , 𝐽 𝑚𝑓𝑔𝑢 (𝑦, 𝑧) > < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦 0 , 𝑧 − 𝑧 0 , 𝐽 𝑠𝑗𝑕ℎ𝑢 (𝑦, 𝑧) > ) 2 − If 𝐽 𝑚𝑓𝑔𝑢 𝑦 + 𝑒, 𝑧 = 𝐽 𝑠𝑗𝑕ℎ𝑢 (𝑦, 𝑧) for all (𝑦, 𝑧) in receptive fields, then the minimum should be 0. 20

Let y position of these cells be the same and x positions 𝑒 of left and right be separated by 𝑒 . This binocular cell + is tuned to disparity 𝑒 - - - + - - - . + - - - + - 𝑚𝑓𝑔𝑢 + - - - + - - - + 𝑠𝑗𝑕ℎ𝑢 - - - + ( ) + ( ) + - + - + - - + 𝑚𝑓𝑔𝑢 + - + - + - + 𝑠𝑗𝑕ℎ𝑢 21

Idea 1 (computer vision): find the disparity 𝑒 that minimizes the squared differences : (𝑑 𝑚 − 𝑑 𝑠 ) 2 + (𝑡 𝑚 − 𝑡 𝑠 ) 2 = 𝑑 𝑚 2 + 𝑑 𝑠 2 + 𝑡 𝑚 2 + 𝑡 𝑠 2 − 2 ( 𝑑 𝑚 𝑑 𝑠 + 𝑡 𝑚 𝑡 𝑠 ) where 𝑑 𝑚 and 𝑡 𝑚 depend on 𝑒. 22

Idea 2 (biological vision): find the shift 𝑒 that maximizes the squared sums : (𝑑 𝑚 + 𝑑 𝑠 ) 2 + (𝑡 𝑚 + 𝑡 𝑠 ) 2 = 𝑑 𝑚 2 + 𝑑 𝑠 2 + 𝑡 𝑚 2 + 𝑡 𝑠 2 + 2 ( 𝑑 𝑚 𝑑 𝑠 + 𝑡 𝑚 𝑡 𝑠 ) where 𝑑 𝑚 and 𝑡 𝑚 depend on 𝑒. 23

𝑒 + - - - + - - - + - - - + + 𝑚𝑓𝑔𝑢 + - - - + - - - + 𝑠𝑗𝑕ℎ𝑢 - - - + + + - + - + - + + 𝑚𝑓𝑔𝑢 + - + - + - + 𝑠𝑗𝑕ℎ𝑢 24

Q: What happens if you close an eye? A: The cell behaves like a monocular complex cell. 25

Recall (monocular) complex cell response to white line Response of complex cell 26

Response of binocular complex cell tuned to 𝑒 = 0 when disparity of white line is 2 pixels. 27 shift the line

Response of binocular complex cell tuned to 𝑒 = 0 when disparity of white line is 10 pixels. lower 28 shift the line

Response of binocular complex cell tuned to 𝑒 = 0 when disparity of white line is 18 pixels. lower 29 shift the line

I will finish this next lecture. 30

Each binocular cell has receptive field location centered at 𝑦 𝑚 , 𝑧 𝑚 𝑦 𝑠 , 𝑧 𝑠 in the two eyes. and Q: What disparity is each cell tuned for ? A: We just discussed this. Q: How to visualize the set (“population”) of cells ? 31

Disparity Space 𝑒 < 0 𝑒 = 0 𝑦 𝑠𝑗𝑕ℎ𝑢 𝑒 > 0 𝑦 𝑚𝑓𝑔𝑢 32

Disparity Tuned Cells 𝑒 < 0 𝑒 = 0 𝑦 𝑠𝑗𝑕ℎ𝑢 𝑒 > 0 𝑦 0 𝑦 𝑚𝑓𝑔𝑢 Previous examples of vertical lines 33

Disparity Tuned Cells 𝑒 < 0 𝑒 = 0 𝑦 𝑠𝑗𝑕ℎ𝑢 𝑒 > 0 𝑦 𝑚𝑓𝑔𝑢 34

Disparity Tuned Cells 𝑒 < 0 𝑒 = 0 𝑦 𝑠𝑗𝑕ℎ𝑢 𝑒 > 0 𝑦 𝑚𝑓𝑔𝑢 To estimating the disparity at an image point, the visual system must select which of these binocular complex cells gives the largest response? 35