M E S O S H T T P A P I @ v i n o d k o n e @ i j i m e n e - - PowerPoint PPT Presentation

▶
m e s o s h t t p a p i
SMART_READER_LITE
LIVE PREVIEW

M E S O S H T T P A P I @ v i n o d k o n e @ i j i m e n e - - PowerPoint PPT Presentation

Jan 28, 2023 128 likes •488 views

M E S O S H T T P A P I @ v i n o d k o n e @ i j i m e n e Mesos 1.0 IS COMING M E S O S A P I S Scheduler API Executor API Internal API S l a v e S c h e d u l e r M a s t e r E x e c u t o r Operator API D E P E N D E N C E O N

slide-1
SLIDE 1

M E S O S H T T P A P I

@ v i n o d k o n e @ i j i m e n e

slide-2
SLIDE 2

Mesos 1.0 IS COMING

slide-3
SLIDE 3

M E S O S A P I S

S c h e d u l e r M a s t e r S l a v e E x e c u t o r

Scheduler API Executor API Internal API Operator API

slide-4
SLIDE 4
slide-5
SLIDE 5

D E P E N D E N C E O N N A T I V E L I B R A RY

S c h e d u l e r M a s t e r

J a v a L i b r a r y N a t i v e L i b r a r y

mesos.jar libmesos.so

Hard to debug Not portable

slide-6
SLIDE 6

N O N - S TA N D A R D F R A M E W O R K A P I

POST /master/mesos.internal.LaunchTasksMessage HTTP/1.1 User-Agent: libprocess/scheduler-1234-23-23342@127.0.0.1:8081 Libprocess-From: scheduler-1234-23-23342@127.0.0.1:8081 Connection: Keep-Alive Host: Transfer-Encoding: chunked

slide-7
SLIDE 7

N E T W O R K I N G C O N S T R A I N T S

S c h e d u l e r M a s t e r

Firewall Containers :( Network partitions :(

slide-8
SLIDE 8

L A C K O F A P I V E R S I O N I N G

GET /metrics/snapshot

Version

GET /state.json

“version” : 0.23.0 JSON

slide-9
SLIDE 9

P I TA F O R M E S O S D E V S

Lot of boiler plate to add new calls/events Forced upgrade dependencies

slide-10
SLIDE 10

– H A R V E Y D E N T

“You either die a hero…

  • r live long enough to see yourself become the villain”

replaced by a better API

slide-11
SLIDE 11

1 . 0 G O A L S

Consistent APIs Versioning

slide-12
SLIDE 12

N E W H T T P A P I

  • Standard HTTP 1.1
  • Versioned !
  • Well documented
slide-13
SLIDE 13

N E W M E S O S A P I S

Endpoint API Hosted by /api/v1/scheduler Scheduler API Master /api/v1/executor Executor API Slave /api/v1/internal Internal API Master /api/v1/admin Operator API Master / Slave

slide-14
SLIDE 14

S C H E D U L E R H T T P A P I

  • Based on Calls and Events
  • Scheduler opens connections to the master
  • A persistent connection to receive events
  • One (or more) connection(s) to send calls
slide-15
SLIDE 15

R A T I O N A L E

Simplicity

Off-the-shelf HTTP client libraries No native dependencies

slide-16
SLIDE 16

R A T I O N A L E

Upgradability

Familiar to existing APIs killTask() Call.Kill

slide-17
SLIDE 17

R A T I O N A L E

Extensibility

Easy to add support for new features

slide-18
SLIDE 18

C A L L S

Old API

S U B S C R I B E

start()

T E A R D O W N

stop()

A C C E P T

acceptOffers()

D E C L I N E

declineOffer()

R E V I V E

reviveOffers()

K I L L

killTask()

S H U T D O W N

* Shutdown executor *

A C K N O W L E D G E

acknowledgeStatusUpdate()

R E C O N C I L E

reconcileTasks()

M E S S A G E

sendFrameworkMessage()

R E Q U E S T

requestResources()

slide-19
SLIDE 19

E V E N T S

Old API

S U B S C R I B E D

registered() / reregistered()

O F F E R S

resourceOffers()

R E S C I N D

  • fferRescinded()

U P D AT E

statusUpdate()

M E S S A G E

frameworkMessage()

FA I L U R E

slaveLost() / executorLost()

E R R O R

error()

H E A R T B E AT

* Periodic heartbeats *

slide-20
SLIDE 20

P R O T O C O L

  • Every call is a HTTP POST request
  • application/json or application/x-protobuf
  • SUBSCRIBE call results in a “200 OK” streaming response
  • Record-IO formatted events
  • chunked encoding
  • All successful non-SUBSCRIBE calls result in “202 Accepted”
slide-21
SLIDE 21

S U B S C R I P T I O N R E Q U E S T

POST /api/v1/scheduler HTTP/1.1 Host: masterhost:5050 Content-Type: application/json Accept: application/json Connection: close { “type” : “SUBSCRIBE”, “subscribe” : { “framework_info” : { “user” : “foo”, “name” : “Example HTTP Framework” }, “force” : true } }

slide-22
SLIDE 22

S U B S C R I P T I O N R E S P O N S E

HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked <Event Length> { “type” : “SUBSCRIBED”, “subscribed” : { “framework_id” : {“value”:“12220-3440-12532-2345”}, “heartbeat_interval_seconds” : 15 } } <more events>

slide-23
SLIDE 23

K I L L

POST /api/v1/scheduler HTTP/1.1 Host: masterhost:5050 Content-Type: application/son Accept: application/json Connection: close { “framework_id” : {“value” : “12220-3440-12532-2345”}, “type” : “KILL”, “kill” : { “task_id” : {“value” : “12220-3440-12532-my-task”}, “slave_id” : {“value” : “12220-3440-12532-S123345”} } } Response: HTTP/1.1 202 Accepted

slide-24
SLIDE 24

D I S C O N N E C T I O N S & PA R T I T I O N S

  • Master tracks the persistent subscription connection
  • Reconnect within failover timeout
  • Subscribe.force : Failover
  • Periodic HEARTBEATs sent by master
slide-25
SLIDE 25

V E R S I O N I N G

Explicit Simple Avoid version explosion /api/v1/scheduler /api/v1/executor /api/v1/admin /api/v1/internal

slide-26
SLIDE 26

A P I V E R S I O N V S R E L E A S E V E R S I O N

  • API version == Major release version
  • v1 API supported by 1.0.0, 1.4.0, 1.20.0
  • vN API released in N-1 release version
  • vN API considered stable in the last N-1 release
  • Version bumping
  • Major/API version bumped for backwards incompatible changes (> yearly)
  • Minor version bumped regularly (4-8 weeks)
slide-27
SLIDE 27

R O A D T O 1 . 0

v1 scheduler API v1 executor API 0.24.0 0.25.0 v1 operator API

1.0

v1 internal API

slide-28
SLIDE 28

D E M O

slide-29
SLIDE 29
slide-30
SLIDE 30

R A T I O N A L E

Simplicity

slide-31
SLIDE 31

R A T I O N A L E

Simplicity Upgradability

slide-32
SLIDE 32

R A T I O N A L E

Simplicity Upgradability Extensibility

slide-33
SLIDE 33

A C K N O W L E D G E M E N T S

slide-34
SLIDE 34

T H A N K Y O U

MESOS-2288