Kubernetes for machine learning: productivity over primitives - - PowerPoint PPT Presentation

▶
kubernetes for machine learning productivity over
SMART_READER_LITE
LIVE PREVIEW

Kubernetes for machine learning: productivity over primitives - - PowerPoint PPT Presentation

Mar 22, 2023 218 likes •1.25k views

Kubernetes for machine learning: productivity over primitives Sophie Watson @sophwats sophie@redhat.com William Benton @willb willb@redhat.com Kubernetes for machine learning: productivity over primitives Sophie

slide-1
SLIDE 1

Kubernetes for machine learning: productivity over primitives

Sophie Watson • @sophwats • sophie@redhat.com William Benton • @willb • willb@redhat.com

slide-2
SLIDE 2

Kubernetes for machine learning: productivity over primitives

Sophie Watson • @sophwats • sophie@redhat.com William Benton • @willb • willb@redhat.com

slide-3
SLIDE 3

@sophwats @willb

feature engineering model training and tuning model validation model deployment monitoring, validation data collection and cleaning codifying problem 
 and metrics

codifying problem 
 and metrics

slide-4
SLIDE 4

@sophwats @willb

feature engineering model training and tuning model validation model deployment monitoring, validation data collection and cleaning codifying problem 
 and metrics

codifying problem 
 and metrics

slide-5
SLIDE 5

@sophwats @willb

feature engineering model training and tuning model validation model deployment monitoring, validation data collection and cleaning codifying problem 
 and metrics

codifying problem 
 and metrics

slide-6
SLIDE 6

@sophwats @willb

codifying problem 
 and metrics feature engineering model training and tuning model validation model deployment monitoring, validation codifying problem 
 and metrics data collection and cleaning

data collection and cleaning

slide-7
SLIDE 7

@sophwats @willb

numquam sit. Est sunt eum rerum cumque rerum quia numquam. Et necessitatibus quo blanditiis dolorem quia id qui. A delectus enim id accusamus quia. Omnis possimus enim ea velit repudiandae id. Et placeat ipsam reiciendis quia ipsam accusantium in. Minima velit ut eos doloremque pariatur provident voluptatem placeat. Enim ut recusandae eos ad sit architecto ut
  • et. Maiores nihil quae eius sit. Nobis
nobis sint sint sequi sed alias quod. Commodi autem neque numquam optio et aut id. Impedit quo id dolorum debitis qui omnis. Et ipsa animi ab ipsa blanditiis
  • consequatur. Est consequatur cumque minima
nesciunt sint. Illum rerum minus odit qui. In quia excepturi adipisci. Maxime aut est libero atque quod. Voluptatum quae quos
  • ccaecati expedita qui impedit sunt nisi.
Qui quod eligendi provident. Quia est sint ex officiis et non fugiat
  • nihil. Neque eum et doloremque et totam
quis ut explicabo. Ipsam animi consequatur rerum doloremque est amet et eaque. Ut voluptatibus nisi commodi.

codifying problem 
 and metrics feature engineering model training and tuning model validation model deployment monitoring, validation codifying problem 
 and metrics data collection and cleaning

data collection and cleaning

slide-8
SLIDE 8

@sophwats @willb

feature engineering model training and tuning model validation model deployment monitoring, validation data collection and cleaning codifying problem 
 and metrics codifying problem 
 and metrics

feature engineering

data collection and cleaning

slide-9
SLIDE 9

@sophwats @willb

feature engineering model training and tuning model validation model deployment monitoring, validation data collection and cleaning codifying problem 
 and metrics codifying problem 
 and metrics

feature engineering

data collection and cleaning

f( ) =

0.67 0.57 0.84 0.08 0.42 0.01

slide-10
SLIDE 10

@sophwats @willb

data collection and cleaning feature engineering model training and tuning model validation model deployment monitoring, validation codifying problem 
 and metrics codifying problem 
 and metrics feature engineering

model training and tuning

data collection and cleaning

slide-11
SLIDE 11

@sophwats @willb

data collection and cleaning feature engineering model training and tuning model validation model deployment monitoring, validation codifying problem 
 and metrics data collection and cleaning codifying problem 
 and metrics feature engineering model training and tuning

model validation

slide-12
SLIDE 12

@sophwats @willb

data collection and cleaning feature engineering model training and tuning model validation model deployment monitoring, validation codifying problem 
 and metrics data collection and cleaning codifying problem 
 and metrics feature engineering model training and tuning

model validation

slide-13
SLIDE 13

@sophwats @willb

feature engineering model training and tuning model validation data collection and cleaning model deployment monitoring, validation codifying problem 
 and metrics data collection and cleaning codifying problem 
 and metrics feature engineering model training and tuning model validation

model deployment monitoring, validation

slide-14
SLIDE 14

@sophwats @willb

feature engineering model training and tuning data collection and cleaning model validation model deployment monitoring, validation codifying problem 
 and metrics codifying problem 
 and metrics feature engineering model training and tuning model validation data collection and cleaning model deployment monitoring, validation

slide-15
SLIDE 15

@sophwats @willb

feature engineering model training and tuning data collection and cleaning model validation model deployment monitoring, validation codifying problem 
 and metrics codifying problem 
 and metrics feature engineering model training and tuning model validation data collection and cleaning model deployment monitoring, validation

slide-16
SLIDE 16

@sophwats @willb

feature engineering model training and tuning data collection and cleaning model validation model deployment monitoring, validation codifying problem 
 and metrics feature engineering model training and tuning model deployment monitoring, validation

slide-17
SLIDE 17

@sophwats @willb

slide-18
SLIDE 18

@sophwats @willb

slide-19
SLIDE 19

@sophwats @willb

slide-20
SLIDE 20

@sophwats @willb

slide-21
SLIDE 21

@sophwats @willb

configuration data collection feature extraction process management analysis tools monitoring serving infrastructure machine resource management data verification

(Adapted from Sculley et al., “Hidden Technical Debt in Machine Learning Systems.” NIPS 2015)

slide-22
SLIDE 22

@sophwats @willb

configuration data collection feature extraction process management analysis tools monitoring serving infrastructure machine resource management data verification

(Adapted from Sculley et al., “Hidden Technical Debt in Machine Learning Systems.” NIPS 2015)

slide-23
SLIDE 23

@sophwats @willb

configuration data collection feature extraction process management analysis tools monitoring serving infrastructure machine resource management data verification

(Adapted from Sculley et al., “Hidden Technical Debt in Machine Learning Systems.” NIPS 2015)

ML Code

slide-24
SLIDE 24

@sophwats @willb

What’s a container?

slide-25
SLIDE 25

@sophwats @willb

%

slide-26
SLIDE 26

@sophwats @willb

% pip install numpy

slide-27
SLIDE 27

@sophwats @willb pip install numpy /usr/bin/pip executable arguments virtual memory file handles / root filesystem environment LANG=en_US USER=willb ... process table network routes

slide-28
SLIDE 28

@sophwats @willb pip install numpy /usr/bin/pip executable arguments virtual memory file handles / root filesystem environment LANG=en_US USER=willb ... process table network routes

slide-29
SLIDE 29

@sophwats @willb pip install numpy /usr/bin/pip executable arguments virtual memory file handles / root filesystem environment LANG=en_US USER=willb ... process table network routes

slide-30
SLIDE 30

@sophwats @willb pip install numpy /usr/bin/pip executable arguments virtual memory file handles / root filesystem environment LANG=en_US USER=willb ... process table network routes

slide-31
SLIDE 31

@sophwats @willb pip install numpy /usr/bin/pip executable arguments virtual memory file handles / root filesystem environment LANG=en_US USER=willb ... process table network routes

slide-32
SLIDE 32

@sophwats @willb

pip install numpy /usr/bin/pip

executable arguments virtual memory file handles /var/lib/envs/main root filesystem environment

LANG=en_US USER=willb ...

process table network routes

slide-33
SLIDE 33

@sophwats @willb

pip install numpy /usr/bin/pip

executable arguments virtual memory file handles /var/lib/envs/main root filesystem environment

LANG=en_US USER=willb ...

process table network routes

slide-34
SLIDE 34

@sophwats @willb

pip install numpy /usr/bin/pip

executable arguments virtual memory file handles /var/lib/envs/main root filesystem environment

LANG=en_US USER=willb ...

process table network routes

SPEED LIMIT

55

slide-35
SLIDE 35

@sophwats @willb

What does Kubernetes build

  • n containers?
slide-36
SLIDE 36

@sophwats @willb

Immutable images

base image configuration and installation recipes user application code

979229b9 33721112 e8cae4f6 2bb6ab16 a8296f7e a6afd91e 6b8cad3e

slide-37
SLIDE 37

@sophwats @willb

Immutable images

base image configuration and installation recipes user application code

979229b9 33721112 e8cae4f6 2bb6ab16 a8296f7e a6afd91e 6b8cad3e

slide-38
SLIDE 38

@sophwats @willb

Immutable images

base image configuration and installation recipes user application code

979229b9 33721112 e8cae4f6 2bb6ab16 a8296f7e a6afd91e 6b8cad3e

slide-39
SLIDE 39

@sophwats @willb

Immutable images

base image configuration and installation recipes user application code

979229b9 33721112 e8cae4f6 2bb6ab16 a8296f7e a6afd91e 6b8cad3e

slide-40
SLIDE 40

@sophwats @willb

Immutable images

base image configuration and installation recipes user application code

979229b9 33721112 e8cae4f6 2bb6ab16 a8296f7e a6afd91e 6b8cad3e model in production

  • n 16 July 2019
slide-41
SLIDE 41

@sophwats @willb

Stateless microservices

slide-42
SLIDE 42

@sophwats @willb

Stateless microservices

slide-43
SLIDE 43

@sophwats @willb

Stateless microservices

slide-44
SLIDE 44

@sophwats @willb

Stateless microservices

slide-45
SLIDE 45

@sophwats @willb

Stateless microservices

slide-46
SLIDE 46

@sophwats @willb

Stateless microservices

slide-47
SLIDE 47

@sophwats @willb

Stateless microservices

slide-48
SLIDE 48

@sophwats @willb

Stateless microservices

slide-49
SLIDE 49

@sophwats @willb

Declarative app configuration

slide-50
SLIDE 50

@sophwats @willb

Integration and deployment

OK!

slide-51
SLIDE 51

@sophwats @willb

Integration and deployment

OK!

base image configuration and installation recipes application code

slide-52
SLIDE 52

@sophwats @willb

Integration and deployment

OK!

base image configuration and installation recipes application code

slide-53
SLIDE 53

@sophwats @willb

Integration and deployment

base image configuration and installation recipes application code

slide-54
SLIDE 54

@sophwats @willb

What containers offer for machine learning workflows

slide-55
SLIDE 55

@sophwats @willb

FROM centos:centos7 RUN yum install -y \ python python-pip \ java java-devel git ENTRYPOINT /bin/bash

slide-56
SLIDE 56

@sophwats @willb

FROM centos:centos7 RUN yum install -y \ python python-pip \ java java-devel git ENTRYPOINT /bin/bash

slide-57
SLIDE 57

@sophwats @willb

slide-58
SLIDE 58

@sophwats @willb

slide-59
SLIDE 59

@sophwats @willb

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0.13 0.13 0.06 0.07 0.07 0.06 0.02 0.08 0.17 0.11 0.11 0.09 0.04 0.18 0.13 0.04 0.13 0.21 0.14 0.03

*

slide-60
SLIDE 60

@sophwats @willb

Self-service environments

more storage sensitive data more CPUs better GPUs

slide-61
SLIDE 61
slide-62
SLIDE 62
slide-63
SLIDE 63

@sophwats @willb

slide-64
SLIDE 64

@sophwats @willb

slide-65
SLIDE 65

@sophwats @willb

O K ! O K !

slide-66
SLIDE 66

@sophwats @willb

No friction: mybinder.org

slide-67
SLIDE 67
slide-68
SLIDE 68
slide-69
SLIDE 69

@sophwats @willb

More flexible: source-to-image

%

slide-70
SLIDE 70

@sophwats @willb

More flexible: source-to-image

%

https://github.com/openshift/source-to-image

builder image application image

slide-71
SLIDE 71

@sophwats @willb

  • c new-app getwarped/s2i-minimal-notebook:latest~\

https://github.com/willb/probabilistic-structures \

  • e JUPYTER_NOTEBOOK_PASSWORD=developer
slide-72
SLIDE 72

@sophwats @willb

  • c new-app getwarped/s2i-minimal-notebook:latest~\

https://github.com/willb/probabilistic-structures \

  • e JUPYTER_NOTEBOOK_PASSWORD=developer
  • c new-app getwarped/s2i-minimal-notebook:latest
slide-73
SLIDE 73

@sophwats @willb

  • c new-app getwarped/s2i-minimal-notebook:latest~\

https://github.com/willb/probabilistic-structures \

  • e JUPYTER_NOTEBOOK_PASSWORD=developer
  • c new-app

https://github.com/willb/probabilistic-structures

slide-74
SLIDE 74

@sophwats @willb

  • c new-app getwarped/s2i-minimal-notebook:latest~\

https://github.com/willb/probabilistic-structures \

  • e JUPYTER_NOTEBOOK_PASSWORD=developer
  • c new-app
  • e JUPYTER_NOTEBOOK_PASSWORD=developer
slide-75
SLIDE 75

@sophwats @willb

slide-76
SLIDE 76

@sophwats @willb

slide-77
SLIDE 77

@sophwats @willb

slide-78
SLIDE 78
slide-79
SLIDE 79
slide-80
SLIDE 80

@sophwats @willb

feature engineering model training and tuning data collection and cleaning model validation model deployment monitoring, validation codifying problem 
 and metrics feature engineering model training and tuning model deployment monitoring, validation

slide-81
SLIDE 81

@sophwats @willb

feature engineering model training and tuning data collection and cleaning model validation model deployment monitoring, validation codifying problem 
 and metrics feature engineering model training and tuning model deployment monitoring, validation

slide-82
SLIDE 82
slide-83
SLIDE 83
slide-84
SLIDE 84

@sophwats @willb

mA

slide-85
SLIDE 85

@sophwats @willb

mA

slide-86
SLIDE 86

@sophwats @willb

mA

distribution of input data? distribution of predictions? distribution of number of multiplications while scoring? (joint)

slide-87
SLIDE 87
slide-88
SLIDE 88
slide-89
SLIDE 89

@sophwats @willb

slide-90
SLIDE 90

@sophwats @willb

Where from here?

slide-91
SLIDE 91

@sophwats @willb

federate train models events databases file, object storage management web and mobile reporting developer UI transform transform transform archive feature engineerin g model training and tuning data collection and model validation model deploymen t monitoring, validation codifying problem 
 and metrics

codifying problem 
 and metrics feature engineering model training and tuning model validation

data collection and cleaning

model deployment monitoring, validation

slide-92
SLIDE 92

@sophwats @willb

data scientists application developers data engineers

federate train models events databases file, object storage management web and mobile reporting developer UI transform transform transform archive

slide-93
SLIDE 93

@sophwats @willb

data scientists application developers data engineers

federate train models events databases file, object storage management web and mobile reporting developer UI transform transform transform archive

machine learning engineers

slide-94
SLIDE 94

@sophwats @willb

  • pendatahub.io
slide-95
SLIDE 95

@sophwats @willb

radanalytics.io

slide-96
SLIDE 96

@sophwats @willb

Kubeflow

slide-97
SLIDE 97

@sophwats @willb

What did we talk about today?

slide-98
SLIDE 98

@sophwats @willb

slide-99
SLIDE 99

@sophwats @willb

slide-100
SLIDE 100

@sophwats @willb

slide-101
SLIDE 101

@sophwats @willb

slide-102
SLIDE 102

@sophwats @willb

sophie@redhat.com • @sophwats willb@redhat.com • @willb

THANKS