Scaling RAPIDS with Dask Matthew Rocklin, Systems Software Manager - - PowerPoint PPT Presentation

▶
scaling rapids with dask
SMART_READER_LITE
LIVE PREVIEW

Scaling RAPIDS with Dask Matthew Rocklin, Systems Software Manager - - PowerPoint PPT Presentation

Mar 03, 2024 344 likes •725 views

Scaling RAPIDS with Dask Matthew Rocklin, Systems Software Manager GTC San Jose 2019 PyData is Pragmatic, but Limited How do we accelerate an existing software stack? The PyData Ecosystem NumPy: Arrays Pandas: Dataframes

slide-1
SLIDE 1

Matthew Rocklin, Systems Software Manager GTC San Jose 2019

Scaling RAPIDS with Dask

slide-2
SLIDE 2

2

PyData is Pragmatic, but Limited

The PyData Ecosystem

  • NumPy: Arrays
  • Pandas: Dataframes
  • Scikit-Learn: Machine Learning
  • Jupyter: Interaction
  • … (many other projects)

Is well loved

  • Easy to use
  • Broadly taught
  • Community Governed

But sometimes slow

  • Single CPU core
  • In-memory data

How do we accelerate an existing software stack?

slide-3
SLIDE 3

3

95% of the time, PyData is great 5% of the time, you want more performance

(and you can ignore the rest of this talk)

slide-4
SLIDE 4

4

Scale up and out with RAPIDS and Dask

Accelerated on single GPU NumPy -> CuPy/PyTorch/.. Pandas -> cuDF Scikit-Learn -> cuML Numba -> Numba

RAPIDS and Others

NumPy, Pandas, Scikit-Learn and many more Single CPU core In-memory data

PyData

Multi-GPU On single Node (DGX) Or across a cluster

Dask + RAPIDS

Multi-core and Distributed PyData NumPy -> Dask Array Pandas -> Dask DataFrame Scikit-Learn -> Dask-ML … -> Dask Futures

Dask

Scale Up / Accelerate Scale out / Parallelize

slide-5
SLIDE 5

5

Scale up and out with RAPIDS and Dask

Accelerated on single GPU NumPy -> CuPy/PyTorch/.. Pandas -> cuDF Scikit-Learn -> cuML Numba -> Numba

RAPIDS and Others

NumPy, Pandas, Scikit-Learn and many more Single CPU core In-memory data

PyData

Scale Up / Accelerate Scale out / Parallelize

slide-6
SLIDE 6

6

RAPIDS: GPU variants of PyData libraries

  • NumPy -> CuPy, PyTorch, TensorFlow
  • Array computing
  • Mature due to deep learning boom
  • Also useful for other domains
  • Obvious fit for GPUs
  • Pandas -> cuDF
  • Tabular computing
  • New development
  • Parsing, joins, groupbys
  • Not an obvious fit for GPUs
  • Scikit-Learn -> cuML
  • Traditional machine learning
  • Somewhere in between
slide-7
SLIDE 7

7

RAPIDS: GPU variants of PyData libraries

  • NumPy -> CuPy, PyTorch, TensorFlow
  • Array computing
  • Mature due to deep learning boom
  • Also useful for other domains
  • Obvious fit for GPUs
  • Pandas -> cuDF
  • Tabular computing
  • New development
  • Parsing, joins, groupbys
  • Not an obvious fit for GPUs
  • Scikit-Learn -> cuML
  • Traditional machine learning
  • Somewhere in between
slide-8
SLIDE 8

8

RAPIDS: GPU variants of PyData libraries

  • NumPy -> CuPy, PyTorch, TensorFlow
  • Array computing
  • Mature due to deep learning boom
  • Also useful for other domains
  • Obvious fit for GPUs
  • Pandas -> cuDF
  • Tabular computing
  • New development
  • Parsing, joins, groupbys
  • Not an obvious fit for GPUs
  • Scikit-Learn -> cuML
  • Traditional machine learning
  • Somewhere in between
slide-9
SLIDE 9

9

RAPIDS: GPU variants of PyData libraries

  • NumPy -> CuPy, PyTorch, TensorFlow
  • Array computing
  • Mature due to deep learning boom
  • Also useful for other domains
  • Obvious fit for GPUs
  • Pandas -> cuDF
  • Tabular computing
  • New development
  • Parsing, joins, groupbys
  • Not an obvious fit for GPUs
  • Scikit-Learn -> cuML
  • Traditional machine learning
  • Somewhere in between
slide-10
SLIDE 10

10

Scale up and out with RAPIDS and Dask

Accelerated on single GPU NumPy -> CuPy/PyTorch/.. Pandas -> cuDF Scikit-Learn -> cuML Numba -> Numba

RAPIDS and Others

NumPy, Pandas, Scikit-Learn and many more Single CPU core In-memory data

PyData

Multi-GPU On single Node (DGX) Or across a cluster

Dask + RAPIDS

Multi-core and Distributed PyData NumPy -> Dask Array Pandas -> Dask DataFrame Scikit-Learn -> Dask-ML … -> Dask Futures

Dask

Scale Up / Accelerate Scale out / Parallelize

slide-11
SLIDE 11

11

Scale up and out with RAPIDS and Dask

NumPy, Pandas, Scikit-Learn and many more Single CPU core In-memory data

PyData

Multi-core and Distributed PyData NumPy -> Dask Array Pandas -> Dask DataFrame Scikit-Learn -> Dask-ML … -> Dask Futures

Dask

Scale Up / Accelerate Scale out / Parallelize

slide-12
SLIDE 12

12

  • PyData Native
  • Built on top of NumPy, Pandas Scikit-Learn, … (easy to migrate)
  • With the same APIs (easy to train)
  • With the same developer community (well trusted)
  • Scales
  • Scales out to thousand-node clusters
  • Easy to install and use on a laptop
  • Popular
  • Most common parallelism framework today at PyData and SciPy conferences
  • Deployable
  • HPC: SLURM, PBS, LSF, SGE
  • Cloud: Kubernetes
  • Hadoop/Spark: Yarn

Dask Parallelizes PyData

Natively

slide-13
SLIDE 13

13

Parallel NumPy

For imaging, simulation analysis, machine learning

  • Same API as NumPy
  • One Dask Array is built from

many NumPy arrays Either lazily fetched from disk Or distributed throughout a cluster

slide-14
SLIDE 14

14

Parallel Pandas

For ETL, time series, data munging

  • Same API as Pandas
  • One Dask DataFrame is built from many

Pandas DataFrames Either lazily fetched from disk Or distributed throughout a cluster

slide-15
SLIDE 15

15

  • Same API
  • Same exact code, just wrap with a decorator
  • Replaces default threaded execution with Dask

Allowing scaling onto clusters

  • Available in most Scikit-Learn algorithms where joblib is

used

Parallel Scikit-Learn

Thread Pool For Hyper-Parameter Optimization, Random Forests, ...

slide-16
SLIDE 16

16

  • Same API
  • Same exact code, just wrap with a decorator
  • Replaces default threaded execution with Dask

Allowing scaling onto clusters

  • Available in most Scikit-Learn algorithms where joblib is

used

Parallel Scikit-Learn

Thread Pool For Hyper-Parameter Optimization, Random Forests, ...

slide-17
SLIDE 17

17

Parallel Python

For custom systems, ML algorithms, workflow engines

  • Parallelize existing codebases
slide-18
SLIDE 18

18

Parallel Python

For custom systems, ML algorithms, workflow engines

  • Parallelize existing codebases

M Tepper, G Sapiro “Compressed nonnegative matrix factorization is fast and accurate”, IEEE Transactions on Signal Processing, 2016

slide-19
SLIDE 19

19

Dask Connects Python users to Hardware

User Execute on distributed hardware

slide-20
SLIDE 20

20

Dask Connects Python users to Hardware

User Writes high level code (NumPy/Pandas/Scikit-Learn) Turns into a task graph Executes on distributed hardware

slide-21
SLIDE 21

21

Example: Dask + Pandas on NYC Taxi

We see how well New Yorkers Tip

import dask.dataframe as dd df = dd.read_csv('gcs://bucket-name/nyc-taxi-*.csv', parse_dates=['pickup_datetime', 'dropoff_datetime']) df2 = df[(df.tip_amount > 0) & (df.fare_amount > 0)] df2['tip_fraction'] = df2.tip_amount / df2.fare_amount hour = df2.groupby(df2.pickup_datetime.dt.hour).tip_fraction.mean() hour.compute().plot(figsize=(10, 6), title='Tip Fraction by Hour')

slide-22
SLIDE 22

22

examples.dask.org

Try live

slide-23
SLIDE 23

23

Dask scales PyData libraries

(A good fit if you’re building a new data science platform)

But is compute-agnostic to those libraries

slide-24
SLIDE 24

24

Scale up and out with RAPIDS and Dask

Accelerated on single GPU NumPy -> CuPy/PyTorch/.. Pandas -> cuDF Scikit-Learn -> cuML Numba -> Numba

RAPIDS and Others

NumPy, Pandas, Scikit-Learn and many more Single CPU core In-memory data

PyData

Multi-core and Distributed PyData NumPy -> Dask Array Pandas -> Dask DataFrame Scikit-Learn -> Dask-ML … -> Dask Futures

Dask

Scale Up / Accelerate Scale out / Parallelize

slide-25
SLIDE 25

25

Scale up and out with RAPIDS and Dask

Accelerated on single GPU NumPy -> CuPy/PyTorch/.. Pandas -> cuDF Scikit-Learn -> cuML Numba -> Numba

RAPIDS and Others

NumPy, Pandas, Scikit-Learn and many more Single CPU core In-memory data

PyData

Multi-GPU On single Node (DGX) Or across a cluster

Dask + RAPIDS

Multi-core and Distributed PyData NumPy -> Dask Array Pandas -> Dask DataFrame Scikit-Learn -> Dask-ML … -> Dask Futures

Dask

Scale Up / Accelerate Scale out / Parallelize

slide-26
SLIDE 26

26

Combine Dask with cuDF

Many GPU DataFrames form a distributed DataFrame

slide-27
SLIDE 27

27

Combine Dask with cuDF

Many GPU DataFrames form a distributed DataFrame

cuDF

slide-28
SLIDE 28

28

Combine Dask with CuPy

Many GPU arrays form a Distributed GPU array

slide-29
SLIDE 29

29

Combine Dask with CuPy

Many GPU arrays form a Distributed GPU array

GPU

slide-30
SLIDE 30

30

Experiments

... SVD with Dask Array NYC Taxi with Dask DataFrame

slide-31
SLIDE 31

31

So what works in DataFrames?

Read CSV: Elementwise operations: Reductions: Groupby Aggregations: Joins (hash, sorted, large-to-small): Leverages Dask DataFrame algorithms (been around for years) API matches Pandas

Lots!

slide-32
SLIDE 32

32

So what doesn’t work?

Read Parquet/ORC Reductions: Groupby Aggregations: Rolling window operations Leverages Dask DataFrame algorithms (been around for years) API matches Pandas

Lots!

slide-33
SLIDE 33

33

So what doesn’t work?

  • When cuDF and Pandas match, existing Dask algorithms work seamlessly
  • But the APIs don’t always match

API Alignment

slide-34
SLIDE 34

34

So what doesn’t work?

  • When cuDF and Pandas match, existing Dask algorithms work seamlessly
  • But the APIs don’t always match

API Alignment

slide-35
SLIDE 35

35

So what works in Arrays?

  • This work is much younger, but moving quickly
  • CuPy has been around for a while, and is fairly mature
  • Most work today happening upstream in NumPy and Dask

Thanks Peter Entschev, Hameer Abbasi, Stephan Hoyer, Marten van Kerkwijk, Eric Wieser

  • Ecosystem approach benefits other NumPy-like arrays as well, sparse arrays, Xarray, ...

We genuinely don’t know yet

slide-36
SLIDE 36

36

So what’s next?

  • High Performance Communication
  • Today Dask uses in-memory or TCP
  • For Infiniband and NVLink, now integrating OpenUCX with ucx-py
  • Spilling to main memory
  • Today Dask spills from memory to disk
  • For GPUs, we’d like to spill from device, to host, to disk
  • Mixing CPU and GPU workloads
  • Today Dask has one thread per core, or one thread per GPU
  • For mixed systems we need to auto-annotate GPU vs CPU tasks
  • Better recipes for deployment
  • Today Dask deploys on Kubernetes, HPC job schedulers, YARN
  • Today these technologies also support GPU workloads
  • Need better examples using both together

Lots of issues with Dask, too!

slide-37
SLIDE 37

37

PyData: pydata.org RAPIDS: rapids.ai Dask: dask.org examples.dask.org

Learn More

Thank you for your time