Pr Prog ogrammin ramming g th the e T op opolo ology gy of - - PowerPoint PPT Presentation

Pr Prog

• grammin

ramming g th the e T

• p
• polo
• logy

gy of N f Netw etworks

• rks

T ec echno hnolo logy gy and nd Alg lgor

• rithms

ithms

Pierre Blanche Klaus-Tycho Foerster Jeff Cox Jamie Gaudette Nikhil Devanur Phillipa Gill Mark Filer Madeleine Glick Daniel Kilper Gireeja Ranade Janar nardhan dhan Kulkar karni i Houman Rastegarfar Ratul Mahajan Stefan Schmid Amar Phanishayee Rachee Singh

Manya Ghoba badi di

Th The e clou

• ud

d infras frastructu tructure

Inefficiencies and waste in the cloud infrastructure

Data centers Optical cables

2

Network topology Traffic engineering

Capacity provisioning T ec echn hnolo

• logy

gy and d algor gorithms ithms to to op

• pti

timize mize net etwor

• rk to

topol

• logy
• gy

3

High gh-lev level el ide dea Networks with programmable topologies A B D C

1

A B D C

1 1 1 1

Throughput: 2 units Throughput: 3 units

1 1 1

Dynamic topology Static topology

4

• Challenging

llenging: :

• Requires reconfigurable hardware te

technolo chnology gy

• Requires revisiting networking layer algori

rithms thms

• Imp

mpactful ctful:

• Cheaper networks
• Higher throughput

Pr Prog

• gramm

rammab able le to topol

• logie
• gies

5

T alk lk ou

• utl

tline ine

Data ta cente nter netw tworks

• rks

ProjecT

• R: Programming the

network topology [SIGCOMM’16]

T echnology and algorithms to enable programmable topologies in the cloud

Google data center

Wide de-area networks tworks

Programming the capacity of links

[SIGCOMM’18, HotNets’17, OFC’16]

Level3 global backbone

6

T

• day’s data center interconnects

A B C D

3 3 3 3 3 3 3 3 3 3 3 3

Ideal demand matrix: uniform and static Static capacity between T

• p-of-Rack (T
• R) pairs

6 6

Non-ideal demand matrix: skewed and dynamic A B C D A B C D A B C D A B C D

10Gbps 10Gbps 8 6 7 7 12 8 6 6

7

Nee eed for

• r a r

a rec econ

• nfigurab

figurable le inter erconne connect ct

Data:

• 200K servers across 4 production clusters
• Cluster sizes: 100 -- 2500 racks

Observation:

• Many rack pairs exchange little traffic
• Only some hot rack pairs are active

Implication:

• Static topology with uniform capacity:
• Over-provisioned for most rack pairs
• Under-provisioned for few others

Reconfigurable interconnect: T

• dynamically provide additional capacity between hot rack pairs

8

Ou Our r propo

• posal:

sal: Pr Proj

• jec

ecT

• R
• R inter

tercon connect nect

9

• Free-space topology (programmable)
• Digital micromirror device to redirect light
• Disco-ball shaped mirror assembly to magnify reachability

Laser Photodetector Static topology

9

Di Digital gital Micr cromirr

• mirror
• r De

Device ce (DMD) DMD)

Array of micromirrors (10 um) Memory cell

10

A 3-T

• R
• R Pr

Proj

• jec

ecT

• R
• R inte

terconnect connect pr prot

• tot
• type

ype

T

• R1

T

• R2

T

• R3

Source laser

DMD

Mirrors reflecting to T

• R2 and T
• R3

T

• R1

T

• R2

T

• R3

11

Rou

• uting

ting algor gorithm ithm

• We have a highly flexible topology allowing for millions of ways to

connect lasers to photodetectors

• Ideal solution: fast changing topology to adapt to demand change
• Challenge: It takes 12μs to reprogram a link

ToR1 ToR1 ToR2 ToR3 ToR2 ToR3

lasers photodetectors

12

Rou

• uting

ting algor gorithm ithm

• Two topology approach:
• Slow switching topology or dedicated topology
• Fast switching links or opportunistic links

ToR1 ToR2 ToR3 ToR1 ToR2 ToR3 ToR1 ToR1 T

• R2

ToR3 ToR2 ToR3 ToR1 ToR2 T

• R3

ToR1 ToR2 T

• R3

lasers photodetectors

dedicated topology

• pportunistic links

13

Rou

• uting

ting packets ets

ToR1 ToR2 ToR3 ToR1 ToR2 ToR3

dedicated topology

2

3 3 3

Virtual output queues

K-shortest paths routing

• pportunistic link

2 2 2 2 2 2 2

14

Scheduling eduling op

• ppor
• rtun

tunistic istic lin inks ks

• Given a set of potential links and current traffic demand, find a set
• f active opportunistic links

ToR1 ToR2 ToR3 ToR1 T

• R2

ToR3

100 100 s

• u

r c e

d e s t i n a t i o n

15

Scheduling eduling op

• ppor
• rtun

tunistic istic lin inks ks

• Standard switch scheduling problem
• Blossom matching
• BvN matrix decomposition
• Centralized scheduler
• Single tiered matching

100 100 100 s

• u

r c e

d e s t i n a t i o n

input

• utput

16

• Standard switch scheduling problem
• Blossom matching
• BvN matrix decomposition
• Centralized scheduler
• Single tiered matching

Scheduling eduling op

• ppor
• rtun

tunistic istic lin inks ks

input

• utput

Src T

• Rs

Dst T

• Rs

Two-tiered Decentralized Extended the Gale-Shapely algorithm for finding stable matches [GS-1962] Constant competitive against an offline optimal allocation

17

• Slow switching time

+ Reconfigurable + Switching time: 12μs

• T

ail flow completion time

• Different traffic matrices
• Impact of switching time
• Impact of fan-out

Simu mulation lation res esul ults ts

5 10 15 20 25 30 35 40

20 30 40 50 60 70 80

Average Flow Completion Time (ms)

Average Load (%)

• No reconfigurability

95%

ProjecT

• R

Fat tree

[SIGCOMM’08]

FireFly [SIGCOMM’14]

18

Th The e key ey ta takea eaway way from

• m th

this is ta talk Current assumption: Network topology is fixed New world: Network topology is dynamic

c3 c4

s w u v y t

c7 c8 Problems to solve:

• Scheduling
• Capacity provisioning
• Traffic engineering
• Load-balancing
• Exci

citing ting: Unusual wealth of algorithms

• Challe

lleng ngin ing: : Changes fundamental assumptions

• Imp

mpactfu ful: l: Better efficiency ($/Gbps)

19