Thoughts on system software for next-generation hardware - - PowerPoint PPT Presentation

▶
thoughts on system software for next generation hardware
SMART_READER_LITE
LIVE PREVIEW

Thoughts on system software for next-generation hardware - - PowerPoint PPT Presentation

Oct 05, 2022 22 likes •407 views

Thoughts on system software for next-generation hardware !"#"$%"&'()*$ $ +,-"&#.-/$01)2&)3"$4"&5*.3.67$)*8$9.(:;<*6$=*2<#;#"$>049,?$ $$$$$$@-6.**"$A)<.*)3$B)C.-)#.-7$

slide-1
SLIDE 1

Thoughts on system software for next-generation hardware

!"#"$%"&'()*$ $+,-"&#.-/$01)2&)3"$4"&5*.3.67$)*8$9.(:;<*6$=*2<#;#"$>049,?$

$$$$$$@-6.**"$A)<.*)3$B)C.-)#.-7$ $D"*,.-$E"33.F/$9.(:;#)<.*$=*2<#;#"$ $$$$$$G*,H"-2,#7$.I$95,&)6.$ $

slide-2
SLIDE 2

Pete Beckman Argonne National Laboratory 2

What’s Happening in Exascale? (do we care?)

slide-3
SLIDE 3

Pete Beckman Argonne National Laboratory 3

!"#$%&'()*%+*,'(*-+,(.+/01+/$*21".+/$*13*4%5'* !(.31.6/+7(*8169"0+5*:99$%7/01+&* $

;"%$)*/+*%+,(.+/01+/$*9$/+*31.* 711.)%+/0+5*.(&(/.7'*31.*,'(*+(<,* 5(+(./01+*19(+*&1".7(*&1=>/.(*31.* &7%(+0?7*'%5'@9(.31.6/+7(*7169"0+5*

slide-4
SLIDE 4

Pete Beckman Argonne National Laboratory 4

EU Announced Funding…

J$

slide-5
SLIDE 5

Pete Beckman Argonne National Laboratory 5

!! K.)3L$M;(:2#)-#$"1)2&)3"$:3)N.-(2$I.-$0;-.:"$ !! O.,*#$I;*8,*6L$09$P$>2.("?$("(C"-$2#)#"2$ !! =(("8,)#"$,*H"2#("*#$(.8"2#Q$RSTU$#.#)3$)&-.22$T$7")-2$>RVWUX7")-?$

Y! A1+,@;$/+7*!-.M"&#*>WJZ[U\$#.#)3?$

]! 0;-.:")*L$$@^U$>G_?/$D4U,&-.$>E-)*&"X=#)37?/$%GBB$>E-)*&"?$ ]! P$-"2")-&5$#")(2$I-.($3)C2$X$;*,H"-2,<"2$

Y! BCC!$!-.M"&#$>W`Z[U\$#.#)3?$

]! 0G$X$GDL$$0a4bBB>K"-()*?/$=*#"3$>GD?$ ]! P$-"2")-&5$#")(2$I-.($3)C2$X$;*,H"-2,<"2$

Y! 8DCEF:$!-.M"&#$>WVU\$#.#)3?$

]! c)(:,-$>K"-()*?/$9-)7$>G_?/$@33,*")$>G_?$ ]! P$-"2")-&5$#")(2$I-.($3)C2$X$;*,H"-2,<"2$

!! 00D=$!3)*$-"d;"2#2$2,6*,e&)*#/$2;2#),*"8$,*H"2#("*#2$,*$V$.-$T$#-)&'2$I.-$VfWV$

Y! [ffU\$g$WfffU\$.H"-$Wf$7")-2$

Three Exascale Platform Projects Started in Oct-2011 to Explore European Prototype Architectures

[$

slide-6
SLIDE 6

Pete Beckman Argonne National Laboratory 6

Kobe Japan: Advanced Instituted for Computational Science

S$

slide-7
SLIDE 7

Pete Beckman Argonne National Laboratory 7

  • 24 Boards /

Cabinet 2060 mm 864 Cabinets 10PFlops 1PB

The heart of the K computer consists of 80,000 Fujitsu’s SPARC64 VIIIfx CPUs

  • Japan: Current #1: The “K” Computer

D":#$VfWWL$A"F$&5,:$)**.;*&"8$

An amazing accomplishment, with unique and advanced system software$

slide-8
SLIDE 8

Pete Beckman Argonne National Laboratory 8

`$

slide-9
SLIDE 9

Pete Beckman Argonne National Laboratory 9

h$

slide-10
SLIDE 10

Pete Beckman Argonne National Laboratory 10

W $

slide-11
SLIDE 11

Pete Beckman Argonne National Laboratory 11

W $

slide-12
SLIDE 12

Pete Beckman Argonne National Laboratory 12

New at Argonne: BLUE GENE/Q

!! !"#$%&%%3;"$K"*"Xi$D72#"($

]! J`$-)&'2$ ]! J`_$WZS$Kjk$*.8"2$ ]! lS`_$&.-"2$m$l`S4%$^@U$ ]! T`J$=Xb$*.8"2$ ]! !")'L$Wf!E$

!! D#.-)6"$ Y! nT[$!%$&):)&,#7/$VJfK%X2$C)*8F,8#5$>K!ED?$ Y! +,2'$2#.-)6"$;:6-)8"$:3)**"8$,*$VfW[$ ]! +.;C3"$&):)&,#7$)*8$C)*8F,8#5$ !! A"F$c,2;)3,k)<.*$D72#"(2$

Y! =*,<)3$272#"($,*$VfWV$ Y! @8H)*&"8$H,2;)3,k)<.*$272#"($,*$VfWJ$

]! D#)#"g.Ig#5"g)-#$2"-H"-$&3;2#"-$$ F,#5$3)#"2#$K!G$)&&"3"-)#.-2$ ]! !-.H,2,.*"8$F,#5$#5"$C"2#$)H),3)C3"$:)-)33"3$)*)372,2$)*8$H,2;)3,k)<.*$2.oF)-"$

$

slide-13
SLIDE 13

Pete Beckman Argonne National Laboratory 13

BG/Q installed and running! A GREEN Solution: Co-Designed with IBM

W $

slide-14
SLIDE 14

Pete Beckman Argonne National Laboratory 14

USA: Exascale RFI: Deep NDAs with Companies to Explore Computing Technology for 2020

slide-15
SLIDE 15

Pete Beckman Argonne National Laboratory 15

What Did We Learn? Maybe the Obvious… CPUs are Changing…

!! !/./$$($%&6$F,#5,*$)$*.8"$,2$8-)()<&)337$,*&-")2,*6$

Y! D72#"($2.oF)-"$F,33$&5)*6"$

!! BG+/6%7*91>(.*6/+/5(6(+,$,2$&-,<&)3$#.$:"-I.-()*&"$

Y! D72#"($2.oF)-"$F,33$&5)*6"$

!! B%&,.%#",()*6(61.GL$&)&5"$&.5"-"*&"$*.#$:.F"-$"p&,"*#$

Y! D72#"($2.oF)-"$F,33$&5)*6"$

!! B((9*6(61.G*'%(./.7'%(&L$T+$3.&)3$^@U$)*8$Ac^@U$

Y! D72#"($2.oF)-"$F,33$&5)*6"$

!! H/"$,&$()7$,*&-")2"$

Y! D72#"($2.oF)-"$F,33$&5)*6"$

'()*+,%-+$./%.+,01)2,%3)--)45%

slide-16
SLIDE 16

Pete Beckman Argonne National Laboratory 16

Parallelism

slide-17
SLIDE 17

Pete Beckman Argonne National Laboratory 17

D.;-&"L$+@^!@$01)2&)3"$^":.-#$

I* JIKIII* LIKIII* MIKIII* NIKIII* OIKIII* PIKIII* QIKIII* RIKIII* SIKIII* JIIKIII*

R*T(/.&*13*;$%&&*

Parallelism Has Suddenly Exploded “The core is the new transistor” (new Moore’s law)

^)2:C"--7$!,L$RV[$ !! lffUjk$@^UWW$ !! RV[$

slide-18
SLIDE 18

Pete Beckman Argonne National Laboratory 18

With Intranode Parallelism Exploding, How Do We Write Programs?

slide-19
SLIDE 19

Pete Beckman Argonne National Laboratory 19

In-Socket Parallel Programming is a Mess:

EG&,(6*E1=>/.(*8'/$$(+5(&U* !! q"$8.$*.#$7"#$5)H"$)$6..8$,*g2.&'"#$ :)-)33"3$:-.6-)((,*6$(.8"3$ !! A"F$!-.6-)((,*6$U.8"32$m$B)*6;)6"2$ A""8"8$>b:"*U!$,2$)$("22?$ !! U"(.-7$(6(#$I.-$8"":"-$5,"-)-&5,"2$>T+$ 2&-)#&5:)8/$&)&5"/$("(.-7?$ !! bD$#5)#$&.*#-.32$#5-")82/$#)2'2/$)*8$:.F"-$ !! j.F$8.$F"$-":-"2"*#$5"#"-.6"*".;2$jqr$

slide-20
SLIDE 20

Pete Beckman Argonne National Laboratory 20

Rethinking the sequential abstract machine….

V $

slide-21
SLIDE 21

Pete Beckman Argonne National Laboratory 21

V* V* V*

Rethinking the parallel abstract machine….

slide-22
SLIDE 22

Pete Beckman Argonne National Laboratory 22

Returning to our Roots: Graphs

JSSS$

slide-23
SLIDE 23

Pete Beckman Argonne National Laboratory 23

Reinventing Programming Models?

!! =*$#5,2$*"F$F.-38/$F"$(;2#$-",*H"*#$.;-$)C2#-)&#$()&5,*"$

Y! !-.6-)(("-2$5)H"$I.&;2"8$.*$s&.-"2t/$8,H,8,*6$F.-'$)&-.22$&.-"2$

!! q"$&)*u#$:-.6-)($#.$)*$"1:.*"*<)337$&5)*6,*6$&.(:.*"*#ZZZ$>*;($&.-"2?$

Y! b*37$#-""2$5)*83"$"1:.*"*<)337$6-.F,*6$-"2.;-&"2v$

!! q"$(;2#$-"#;-*$#.$5,65"-g3"H"3$(.8"32$

Y! 9.5"-"*&"$8.(),*2/$2")$.I$@BG2$$

!! !-.6-)((,*6$(.8"3$&)**.#$C"$C)2"8$.*$:)-)33"3,2($)o"-$#5"$I)&#$>.:"*U!?$

Y! 95)-(PP/$9=B_r$9.*&;--"*#$9.33"&<.*2r$$E;*&<.*)3$!-.6-)((,*6r$

!! D72#"($D.oF)-"$95)33"*6"L$

Y! 01:3.-"$*"F$)C2#-)&#$()&5,*"$)*8$:-.6-)((,*6$3)*6;)6"2/$)*8$-;*g<("$272#"(2$

!* A(61.G* !* !* !* !* A(61.G* !* !* !*

slide-24
SLIDE 24

Pete Beckman Argonne National Laboratory 24

Intranode Power Constraints and Cache Coherence

slide-25
SLIDE 25

Pete Beckman Argonne National Laboratory 25

Within the Node, What Else is Changing? How Will System Software Manage CPUs?

How Will They Be Programmed?

  • +,($U*W+%5',X&*H(..G$

F%$(./U*YZ$ :)/90[/U*$

  • ;AU*;Y\]$
  • +,($U*E88$

!1>(.*81+&,./%+()*A(61.G*81+&%&,(+7G*$

^JR$

B/$$GU*C7'($1+$ ;1._/.*`*8'%(+$ a$

slide-26
SLIDE 26

Pete Beckman Argonne National Laboratory 26

Power, Parallelism, Coherence, Fault, Storage

EG&,(6*E1=>/.(*8'/$$(+5(&U* Y! !.F"-$(;2#$C"$)$()*)6"8$-"2.;-&"$

]! +)-'$D,3,&.*L$U.-"$I;*&<.*)3$;*,#2$#5)*$&)*$-;*$)#$I;33$2:""8$ ]! c)-,)C3"$2:""8$2;C&.(:.*"*#2$ ]! A"FL$b:<(,k"$:"-I$$I.-$45"-()3$+"2,6*$!.,*#$>4+!?$

Y! ^"2#-;&#;-"8$*.8"$)-&5,#"&#;-"$

]! U)22,H"$3"H"32$.I$,*g:)&')6"$:)-)33"3,2($ ]! c)-,)C3"$&.5"-"*&"$8.(),*2$)*8$,*#-)2.&'"#$("22)6,*6$ ]! j"#"-.6"*".;2$(;3<g&.-"$>6-):5,&2/$&.(:-"22,.*/$"#&?$ Y! !-.6-)((,*6$(.8"3$I.-$#5,2r$

Y! 9.(:3"1$I);3#$C"5)H,.-$

]! D,*63"$&.-"$&.;38$"1:"-,"*&"$I);3#$

]! A""8$I.-$I);3#$8.(),*2$

slide-27
SLIDE 27

Pete Beckman Argonne National Laboratory 27

BG/P & BG/Q Power Experiments

CNK Linux % KWatt 14.935 13.809 7.75

Comparison between CNK and Linux

  • n sleep()
slide-28
SLIDE 28

Pete Beckman Argonne National Laboratory 28

Exploring Power on Intel Knights Ferry

slide-29
SLIDE 29

Pete Beckman Argonne National Laboratory 29

Seeking to Isolate Components

!! E;#;-"$()*7&.-"$&5,:2$F,33$:"-(,#$()*7$:.F"-$(.8"2$)*8$2:""82$:"-$&.-"$ !! D72#"($2.oF)-"$*""8"8$#.$()*)6"$:.F"-$ !! K.)3L$

Y! 9-")#"$)C2#-)&#$()&5,*"$(.8"3$I.-$:.F"-$;2"$>&.(:,3"-/$-;*<("/$"#&?$ Y! 9-")#"$87*)(,&$:.F"-g)F)-"$-;*g<("$272#"($

slide-30
SLIDE 30

Pete Beckman Argonne National Laboratory 30

Near Future Technologies

slide-31
SLIDE 31

Pete Beckman Argonne National Laboratory 31

Intel: NTV (Near Threshold Voltage) circuits & variable precision floating point

!! =*$A4c$-)*6"/$[$#.$Wf$<("2$(.-"$"p&,"*#$ !! +"(.*2#-)#"8$&5,:$#5)#$&)*$6.$I-.($TU5k$#.$ hW[U5k$ !! !-.#.#7:"$H)-,)C3"$:-"&,2,.*$w.)<*6$:.,*#$ 272#"($

!! EG&,(6*E1=>/.(*8'/$$(+5(&U*

Y! A/+/5(*&9(()*1[(.*./+5(*13*bcJIId** Y! 81+,.1$*9.(7%&%1+*#/&()*1+*.(e"%.()* (..1.*#1"+)&*

T $

slide-32
SLIDE 32

Pete Beckman Argonne National Laboratory 32

3D Chip Stacking: Really Fast, Really Close, Really Small

=%U$ G*,H$.I$U,&5,6)*$ K".-6,)$4"&5$ !! b*g&5,:$^@U$6"x*6$2()33"-$q^4$:)-)33"3,2($ !! %)*8F,8#5$F,33$C"$"1&"33"*#$ !! @8H)*&"8$("(.-7$.:"-)<.*2$:.22,C3"$ !! =*#"6-)#"8$A=9$,2$#5"$*"1#$2#":$ !! 01:3,&,#$8)#)$(.H"("*#$F,#5,*$&5,:$ !! EG&,(6*E1=>/.(*8'/$$(+5(&*

Y! A(61.G*6/+/5(6(+,K*)/,/*61[(6(+,* Y! bE*,'/,*71+,.1$&*,'.(/)&K*,/&_&K*/+)*91>(.*

U,&-.*$jU9$

s0)-37$C"*&5()-'2$25.F$)$("(.-7$&;C"$C3)2<*6$8)#)$WV$<("2$I)2#"-$ #5)*$++^TgWTTT$D+^@U$F5,3"$;2,*6$.*37$)C.;#$Wf$:"-&"*#$.I$#5"$:.F"-Zy$$

slide-33
SLIDE 33

Pete Beckman Argonne National Laboratory 33

University of Michigan

T $

slide-34
SLIDE 34

Pete Beckman Argonne National Laboratory 34

Non-Volatile RAM May Replace Everything…

!! !5)2"$95)*6"$U"(.-7$>!9U?$)3-")87$C",*6$ :-.8;&"8$I.-$2.("$()-'"#2$

Y! U.-"$:.F"-$#.$!"#$%$8)#)$#5)*$#.$"%&'$8)#)$

!! U"(-"2,2#.-$,2$:-.C)C37$I;-#5"-$I-.($ 8":3.7("*#$ !! %.#5$()7$C"$8":3.7"8$",#5"-$,*$T+$.-$)2$)$ C;-2#$C;z"-$*")-$&5,:2$ !! D72#"($D.oF)-"$95)33"*6"2$

Y! A"F$)36.-,#5(2$#5)#$-")8$(.-"$#5)*$#5"7$2#.-"$ Y! U)*)6"("*#$.I$#5"$8"":"-$5,"-)-&57$

]! U"(.-7$:)6"2r$%3.&'$=Xbr$$$

slide-35
SLIDE 35

Pete Beckman Argonne National Laboratory 35

Fault Intranode and Internode

slide-36
SLIDE 36

Pete Beckman Argonne National Laboratory 36

CIFTS project: Exploring a Fault Tolerance Backplane

=b$*.8"$I),3;-"Z$$ E,3"$272#"($8.F*$

Parallel FS

$ E,3"$D72#"($$ 25)-"2$ $#5,2$,*I.-()<.*$ $

FT-Job Scheduler

$ B);*&5$M.C2$F,#5$$ AED$e3"$272#"($ $

MPI-IO

!-,*#2$)$&.5"-"*#$ "--.-$("22)6"$ 95"&':.,*#2$#.$$ )$8,z$ED$

FT-Application

95"&':.,*#2$#.$$ )$8,z$ED$

FT-Application

$ U,6-)#"2$"1,2<*6$ M.C2$ $

E);3#$$ 4.3"-)*&"$$ %)&':3)*"$

B,*")-$$ @36"C-)$$ B,C-)-,"2$ j!9$ U,883"F)-"$ G*,H"-2)3$ B.66"-$ $ @;#.()<&$ @&<.*2$ +,)6*.2<&2$ 4..32$ 0H"*#$ @*)372,2$ O.C$$ D&5"8;3"-X$ ^"2.;-&"$$ ()*)6"-$ E,3"$$ D72#"(2$ 95"&'g$ !.,*<*6$ D.oF)-"$

A"#F.-'2$ m$ A"#F.-',*6$$ 3,C-)-,"2$

D72#"($ U.*,#.-,*6$$ 2.oF)-"$ D72#"($ U6(#$ 5)-8F)-"$ b:"-)<*6$$ D72#"($ @::3,&)<.*2$ @8H)*&"8$ D72#"(2$ >9-)72/$%K2?$

45"$H/"$,*F1$(./+7(*;/7_9$/+(*:-.H,8"2$ !! @$2&)3)C3"$I-)("F.-'$#.$"1&5)*6"$I);3#g-"3)#"8$,*I.-()<.*$ !! 01:.2"2$)$2#)*8)-8$,*#"-I)&"$#5)#$&)*$C"$;2"8$C7$)*7$2.oF)-"$#.$&.**"&#$#.$#5"$E4%$ !! !-.H,8"2$)$&.((.*$;*,I.-($"H"*#$5)*83,*6$("&5)*,2($)*8$"H"*#$*.<e&)<.*$("&5)*,2($

slide-37
SLIDE 37

Pete Beckman Argonne National Laboratory 37

Exciting Times

!! !/./$$($%&6$F,#5,*$)$*.8"$,2$8-)()<&)337$,*&-")2,*6$

Y! D72#"($2.oF)-"$F,33$&5)*6"$

!! BG+/6%7*91>(.*6/+/5(6(+,$,2$&-,<&)3$#.$:"-I.-()*&"$

Y! D72#"($2.oF)-"$F,33$&5)*6"$

!! B%&,.%#",()*6(61.GL$&)&5"$&.5"-"*&"$*.#$:.F"-$"p&,"*#$

Y! D72#"($2.oF)-"$F,33$&5)*6"$

!! B((9*6(61.G*'%(./.7'%(&L$T+$3.&)3$^@U$)*8$Ac^@U$

Y! D72#"($2.oF)-"$F,33$&5)*6"$

!! H/"$,&$()7$,*&-")2"$

Y! D72#"($2.oF)-"$F,33$&5)*6"$

'()*+,%-+$./%.+,01)2,%3)--)45%

slide-38
SLIDE 38

Pete Beckman Argonne National Laboratory 38

What Does This Mean for Computer Science? (and System Software)

!! !)-)33"3,2(L$D"d;"*<)3$&.8"$,2$.C2.3"#"Z$9-)k7$)(.;*#2$.I$:)-)33"3,2($

]! D=U+/$c"&#.-/$U=U+/$"#&$ ]! q"$(;2#$-"H,2,#$:-.6-)((,*6$(.8"32/$3)*6;)6"2/$,*H"*#$*"F$F)72$#.$"1:-"22$:)-)33"3,2($ ]! @8H)*&"8$-;*g<("$272#"(2$#.$()*)6"$#)2'2$)*8$8":"*8"*&,"2$$

!! +7*)(,&$:.F"-$()*)6"("*#L$e-2#$&3)22$.CM"&#$,*$272#"($2.oF)-"$

]! !"-I.-()*&"$,2$3,(,#"8$C7$45"-()3$+"2,6*$!.,*#$>4+!?$ ]! A"F$)36.-,#5(2$#.$,(:-.H"$:"-I.-()*&"$F,#5,*$4+!v$A"F$)*)372,2$#"&5*,d;"2$ ]! !.F"-$>2:""8?$)*8$8)-'$2,3,&.*$(;2#$C"$"1:3,&,#37$()*)6"8$C7$272#"($2.oF)-"$

!! +,2#-,C;#"8$("(.-7L$,*#-)*.8"$:-.6-)((,*6$(;2#$)&&"22$-"(.#"$8)#)$

]! 9.(C,*"8$F,#5$:)-)33"3,2(/$:-.6-)((,*6$(.8"3$F,33$()*)6"$8)#)$(.H"("*#$

!! +"":$("(.-7$5,"-)-&5,"2L$T+$^@U/$Ac^@U$.*$*.8"$

]! =Xb$E.-F)-8,*6$,*2,8"$#5"$*.8"$ ]! A"F$(.8"32$I.-$8"":$("(.-7$5,"-)-&57$

!! E);3#2L$+,2#-,C;#"8$&.(:;<*6$)--,H"2$F,#5,*$#5"$*.8"$ !! c)-,)C3"$:-"&,2,.*$w.)<*6$:.,*#L$*"F$*;("-,&)3$)*)372,2$)*8$3,C-)-7$8"2,6*2$

]! i;)*<I7$:-"&,2,.*$)*8$;*&"-#),*#7$ ]! B,C-)-7$,*#"-I)&"2$#.$2:"&,I7$:-"&,2,.*$ ]! j7C-,8$)36.-,#5(2$C)2"8$.*$:-"&,2,.*/$2:""8/$:.F"-$