Clarinet: WAN-Aware Optimization for Analyt ytics Queries Raajay - - PowerPoint PPT Presentation

Clarinet: WAN-Aware Optimization for Analyt ytics Queries

Raajay Viswanathan, Ganesh Ananthanarayanan, Aditya Akella

1

Overview

2

• Web apps hosted on multiple DCs  Low latency access to end-user

Overview

2

• Web apps hosted on multiple DCs  Low latency access to end-user

Overview

2

• Web apps hosted on multiple DCs  Low latency access to end-user

Overview

2

• Web apps hosted on multiple DCs  Low latency access to end-user

Overview

2

• Web apps hosted on multiple DCs  Low latency access to end-user
• Need efficient methods to analyze data located in multiple data centers

Centralized Aggregation is Wasteful

3

Centralized Aggregation is Wasteful

3

Intra-data center Analytics Framework

SELECT * … FROM .. WHERE .. ;

Centralized Aggregation is Wasteful

3

Intra-data center Analytics Framework

SELECT * … FROM .. WHERE .. ;

• Available WAN bandwidth is limited  Aggregation latency overhead

50 100 150 200 250 300 350 400 450 500 1 11 21 31 41 51 61 71 81

Bandwidth (Mbps) Directional WAN links sorted by bandwidth Measured pairwise bandwidth between EC2 regions

450 Mbps 20 Mbps

Centralized Aggregation is Wasteful

3

Intra-data center Analytics Framework

SELECT * … FROM .. WHERE .. ;

• Available WAN bandwidth is limited  Aggregation latency overhead
• WAN links are expensive  High data transfer cost

$$$$ $$$$ $$$

Geo-distributed Analytics

4

Geo-distributed Analytics

4

Analytics framework

One Logical Datacenter

Geo-distributed Analytics

4

Analytics framework

One Logical Datacenter

Distributed Storage Layer

Geo-distributed Analytics

4

Analytics framework

One Logical Datacenter

Distributed Storage Layer Distributed Execution Layer

SELECT * … FROM .. WHERE .. ;

Query Optimizer

Multi-stage parallelizable jobs

Geo-distributed Analytics

4

Analytics framework

One Logical Datacenter

Distributed Storage Layer Distributed Execution Layer

SELECT * … FROM .. WHERE .. ;

Query Optimizer

Multi-stage parallelizable jobs Geo-distributed

Requires WAN-aware

• ptimization

Geo-distributed Analytics

4

Analytics framework

One Logical Datacenter

Distributed Storage Layer Distributed Execution Layer

SELECT * … FROM .. WHERE .. ;

Query Optimizer

Multi-stage parallelizable jobs Geo-distributed

Requires WAN-aware

• ptimization

Iridium [SIGCOMM 15] GeoDe [NSDI 15]

Geo-distributed Analytics

4

Analytics framework

One Logical Datacenter

Distributed Storage Layer Distributed Execution Layer

SELECT * … FROM .. WHERE .. ;

Query Optimizer

Multi-stage parallelizable jobs Geo-distributed

Requires WAN-aware

• ptimization

Clarinet

Geo-distributed Analytics

4

Analytics framework

One Logical Datacenter

Distributed Storage Layer Distributed Execution Layer

SELECT * … FROM .. WHERE .. ;

Query Optimizer

Multi-stage parallelizable jobs Geo-distributed

Requires WAN-aware

• ptimization

Clarinet

2.7x reduction in query runtime

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

5

T1, T2, T3: Tables storing click logs

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

T1, T2, T3: Tables storing click logs

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

QUERY SELECT T1.user, T1.latency, T2.latency, T3.latency FROM T1, T2, T3 WHERE T1.user == T2.user AND T1.user == T3.user AND T1.device == T2.device == T3.device == “mobile”;

T1, T2, T3: Tables storing click logs

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

⋈

T2 T3

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

QUERY SELECT T1.user, T1.latency, T2.latency, T3.latency FROM T1, T2, T3 WHERE T1.user == T2.user AND T1.user == T3.user AND T1.device == T2.device == T3.device == “mobile”;

T1, T2, T3: Tables storing click logs

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

⋈

T2 T3

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈

T1

𝜏𝑁𝑝𝑐𝑗𝑚𝑓

QUERY SELECT T1.user, T1.latency, T2.latency, T3.latency FROM T1, T2, T3 WHERE T1.user == T2.user AND T1.user == T3.user AND T1.device == T2.device == T3.device == “mobile”;

T1, T2, T3: Tables storing click logs

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

10 GB 200 GB 200 GB 200 GB

⋈

T2 T3

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈

T1

𝜏𝑁𝑝𝑐𝑗𝑚𝑓

QUERY SELECT T1.user, T1.latency, T2.latency, T3.latency FROM T1, T2, T3 WHERE T1.user == T2.user AND T1.user == T3.user AND T1.device == T2.device == T3.device == “mobile”;

T1, T2, T3: Tables storing click logs

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

10 GB 200 GB 200 GB 200 GB

⋈

T2 T3

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈

T1

𝜏𝑁𝑝𝑐𝑗𝑚𝑓

Plan running time: 41 s

QUERY SELECT T1.user, T1.latency, T2.latency, T3.latency FROM T1, T2, T3 WHERE T1.user == T2.user AND T1.user == T3.user AND T1.device == T2.device == T3.device == “mobile”;

T1, T2, T3: Tables storing click logs

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

10 GB 200 GB 200 GB 200 GB

⋈

T2 T3

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈

T1

𝜏𝑁𝑝𝑐𝑗𝑚𝑓

40 s 1 s

Plan running time: 41 s

QUERY SELECT T1.user, T1.latency, T2.latency, T3.latency FROM T1, T2, T3 WHERE T1.user == T2.user AND T1.user == T3.user AND T1.device == T2.device == T3.device == “mobile”;

T1, T2, T3: Tables storing click logs

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

10 GB 200 GB 200 GB 200 GB

⋈

T2 T3

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈

T1

𝜏𝑁𝑝𝑐𝑗𝑚𝑓

40 s 1 s

Plan running time: 41 s

QUERY SELECT T1.user, T1.latency, T2.latency, T3.latency FROM T1, T2, T3 WHERE T1.user == T2.user AND T1.user == T3.user AND T1.device == T2.device == T3.device == “mobile”;

T1, T2, T3: Tables storing click logs WAN-only bottleneck

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

10 GB 200 GB 200 GB 200 GB

⋈

T2 T3

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈

T1

𝜏𝑁𝑝𝑐𝑗𝑚𝑓

40 s 1 s

Plan running time: 41 s

QUERY SELECT T1.user, T1.latency, T2.latency, T3.latency FROM T1, T2, T3 WHERE T1.user == T2.user AND T1.user == T3.user AND T1.device == T2.device == T3.device == “mobile”;

T1, T2, T3: Tables storing click logs

⋈ ⋈

12 GB 200 GB 200 GB

T2 T1 T3

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈ ⋈

16 GB 200 GB 200 GB

T1 T3 T2

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

Plan A Plan B Plan C

Plan running time: 20.96 s Plan running time: 17.6 s WAN-only bottleneck

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

10 GB 200 GB 200 GB 200 GB

⋈

T2 T3

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈

T1

𝜏𝑁𝑝𝑐𝑗𝑚𝑓

40 s 1 s

Plan running time: 41 s

QUERY SELECT T1.user, T1.latency, T2.latency, T3.latency FROM T1, T2, T3 WHERE T1.user == T2.user AND T1.user == T3.user AND T1.device == T2.device == T3.device == “mobile”;

T1, T2, T3: Tables storing click logs

⋈ ⋈

12 GB 200 GB 200 GB

T2 T1 T3

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈ ⋈

16 GB 200 GB 200 GB

T1 T3 T2

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

Plan A Plan B Plan C

Plan running time: 20.96 s Plan running time: 17.6 s Chosen by network agnostic query

• ptimizer

WAN-only bottleneck

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

10 GB 200 GB 200 GB 200 GB

⋈

T2 T3

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈

T1

𝜏𝑁𝑝𝑐𝑗𝑚𝑓

40 s 1 s

Plan running time: 41 s

QUERY SELECT T1.user, T1.latency, T2.latency, T3.latency FROM T1, T2, T3 WHERE T1.user == T2.user AND T1.user == T3.user AND T1.device == T2.device == T3.device == “mobile”;

T1, T2, T3: Tables storing click logs

⋈ ⋈

12 GB 200 GB 200 GB

T2 T1 T3

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈ ⋈

16 GB 200 GB 200 GB

T1 T3 T2

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

Plan A Plan B Plan C

Plan running time: 20.96 s Plan running time: 17.6 s Chosen by network agnostic query

• ptimizer

WAN-only bottleneck

WAN Aware Query Optimization

T2 DC2 T3 DC3 T1 DC1

80 Gbps 40 Gbps 100 Gbps

5

10 GB 200 GB 200 GB 200 GB

⋈

T2 T3

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈

T1

𝜏𝑁𝑝𝑐𝑗𝑚𝑓

40 s 1 s

Plan running time: 41 s

QUERY SELECT T1.user, T1.latency, T2.latency, T3.latency FROM T1, T2, T3 WHERE T1.user == T2.user AND T1.user == T3.user AND T1.device == T2.device == T3.device == “mobile”;

T1, T2, T3: Tables storing click logs

⋈ ⋈

12 GB 200 GB 200 GB

T2 T1 T3

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

⋈ ⋈

16 GB 200 GB 200 GB

T1 T3 T2

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓

Plan A Plan B Plan C

Plan running time: 20.96 s Plan running time: 17.6 s Chosen by network agnostic query

• ptimizer

WAN-only bottleneck

WAN-aware query optimizer that uses network transfer duration to choose query plans

Outline

• 1. Motivation
• 2. Challenges in choosing query plan based on WAN transfer durations
• 3. Solution
• Single query
• Multiple simultaneous queries
• 4. Experimental Evaluation

6

Other factors also affect query plan run time

7

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

Other factors also affect query plan run time

7

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

⋈

200 GB 200 GB

T1 T2

𝜏𝐷 𝜏𝐷

Other factors also affect query plan run time

7

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

⋈

200 GB 200 GB

T1 T2

𝜏𝐷 𝜏𝐷

T1 T2

MAP: SELECT MAP: SELECT REDUCE: JOIN

200 GB 200 GB

Map Reduce Job

Other factors also affect query plan run time

7

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

⋈

200 GB 200 GB

T1 T2

𝜏𝐷 𝜏𝐷 20 s

T1 T2

MAP: SELECT MAP: SELECT REDUCE: JOIN

200 GB 200 GB

Map Reduce Job

Tasks placed in single DC

Other factors also affect query plan run time

7

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

⋈

200 GB 200 GB

T1 T2

𝜏𝐷 𝜏𝐷 10 s

T1 T2

MAP: SELECT MAP: SELECT REDUCE: JOIN

200 GB 200 GB

Map Reduce Job

Tasks are placed uniformly across DC1 and DC2

Other factors also affect query plan run time

7

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

⋈

200 GB 200 GB

T1 T2

𝜏𝐷 𝜏𝐷

T1 T2

MAP: SELECT MAP: SELECT REDUCE: JOIN

200 GB 200 GB

Map Reduce Job

• 1. Plan A:

41 s

• 2. Plan B:

20.96

• 3. Plan C:

17.6 s

Tasks are placed uniformly across DC1 and DC2

While evaluating different query plans

Other factors also affect query plan run time

7

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

⋈

200 GB 200 GB

T1 T2

𝜏𝐷 𝜏𝐷

T1 T2

MAP: SELECT MAP: SELECT REDUCE: JOIN

200 GB 200 GB

Map Reduce Job

• 1. Plan A:

41 s

• 2. Plan B:

20.96

• 3. Plan C:

17.6 s 20.5 s 11.2 s

Tasks are placed uniformly across DC1 and DC2

While evaluating different query plans

Other factors also affect query plan run time

7

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

⋈

200 GB 200 GB

T1 T2

𝜏𝐷 𝜏𝐷

T1 T2

MAP: SELECT MAP: SELECT REDUCE: JOIN

200 GB 200 GB

Map Reduce Job

• 1. Plan A:

41 s

• 2. Plan B:

20.96

• 3. Plan C:

17.6 s 20.5 s 11.2 s

Tasks are placed uniformly across DC1 and DC2

While evaluating different query plans

Used by high priority application

Other factors also affect query plan run time

7

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

⋈

200 GB 200 GB

T1 T2

𝜏𝐷 𝜏𝐷

T1 T2

MAP: SELECT MAP: SELECT REDUCE: JOIN

200 GB 200 GB

Map Reduce Job

• 1. Plan A:

41 s

• 2. Plan B:

20.96

• 3. Plan C:

17.6 s 20.5 s 11.2 s

Tasks are placed uniformly across DC1 and DC2

While evaluating different query plans

Used by high priority application

Other factors also affect query plan run time

7

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

⋈

200 GB 200 GB

T1 T2

𝜏𝐷 𝜏𝐷

T1 T2

MAP: SELECT MAP: SELECT REDUCE: JOIN

200 GB 200 GB

Map Reduce Job

• 1. Plan A:

41 s

• 2. Plan B:

20.96

• 3. Plan C:

17.6 s 20.5 s 11.2 s

Tasks are placed uniformly across DC1 and DC2

While evaluating different query plans

Used by high priority application

Choose query plan based on: 1. Best available task placements

Other factors also affect query plan run time

7

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

⋈

200 GB 200 GB

T1 T2

𝜏𝐷 𝜏𝐷

T1 T2

MAP: SELECT MAP: SELECT REDUCE: JOIN

200 GB 200 GB

Map Reduce Job

• 1. Plan A:

41 s

• 2. Plan B:

20.96

• 3. Plan C:

17.6 s 20.5 s 11.2 s

Tasks are placed uniformly across DC1 and DC2

While evaluating different query plans

Used by high priority application

Choose query plan based on: 1. Best available task placements 2. Schedule of network transfers

Joint plan selection, placement and scheduling

8

Joint plan selection, placement and scheduling

Query Optimizer Multiple query plans (join orders) per query

SELECT * FROM … WHERE.. ;

8

Joint plan selection, placement and scheduling

Query Optimizer Multiple query plans (join orders) per query

SELECT * FROM … WHERE.. ;

8

Logical plan to physical plan Assign parallelism for each stage

Joint plan selection, placement and scheduling

Clarinet Query Optimizer Network aware task placement and scheduling for each query plan Multiple query plans (join orders) per query Choose plan with smallest run time for execution

SELECT * FROM … WHERE.. ;

8

Logical plan to physical plan Assign parallelism for each stage

Joint plan selection, placement and scheduling

Clarinet Query Optimizer Network aware task placement and scheduling for each query plan Multiple query plans (join orders) per query Choose plan with smallest run time for execution

SELECT * FROM … WHERE.. ;

8

Logical plan to physical plan Assign parallelism for each stage

Clarinet binds query to plan lower in the stack

Network aware placement and scheduling

9

T1 T3 T2

SELECT SELECT SELECT JOIN JOIN

Network aware placement and scheduling

• Task placement decided greedily one stage at a time
• Minimize per stage run time

9

T1 T3 T2

SELECT SELECT SELECT JOIN JOIN

Network aware placement and scheduling

• Task placement decided greedily one stage at a time
• Minimize per stage run time
• Scheduling of network transfers
• Determines start times of inter-DC network transfers

9

T1 T3 T2

SELECT SELECT SELECT JOIN JOIN

Network aware placement and scheduling

• Task placement decided greedily one stage at a time
• Minimize per stage run time
• Scheduling of network transfers
• Determines start times of inter-DC network transfers
• Formulate a Binary Integer Linear Program to solve

scheduling

• Factors transfer dependencies

9

T1 T3 T2

SELECT SELECT SELECT JOIN JOIN

10

How to extend the late-binding strategy to multiple queries?

Queries affect each others’ run time

11

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1

Queries affect each others’ run time

11

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1 QUERY 1 SELECT … device == “mobile” …; QUERY 2 SELECT … genre == “pc” …;

Queries affect each others’ run time

11

⋈ ⋈

16 GB 200 GB 200 GB

T1 T3 T2

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝐷

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1 QUERY 1 SELECT … device == “mobile” …; QUERY 2 SELECT … genre == “pc” …;

⋈ ⋈

16 GB 200 GB 200 GB

T1 T3 T2

200 GB

𝜏𝑄𝐷 𝜏𝑄𝐷 𝜏𝑆 Same query plan (Plan C) for Query 1 and Query 2

Queries affect each others’ run time

11

⋈ ⋈

16 GB 200 GB 200 GB

T1 T3 T2

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝐷

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1 QUERY 1 SELECT … device == “mobile” …; QUERY 2 SELECT … genre == “pc” …;

⋈ ⋈

16 GB 200 GB 200 GB

T1 T3 T2

200 GB

𝜏𝑄𝐷 𝜏𝑄𝐷 𝜏𝑆 Same query plan (Plan C) for Query 1 and Query 2 Contention increases query run time

Queries affect each others’ run time

11

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1 QUERY 1 SELECT … device == “mobile” …; QUERY 2 SELECT … genre == “pc” …;

⋈ ⋈

16 GB 200 GB 200 GB

T1 T3 T2

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝐷

⋈ ⋈

12 GB 200 GB 200 GB

T2 T1 T3

200 GB

𝜏𝑄𝐷 𝜏𝑄𝐷 𝜏𝐷

Different query plans for Query 1 (Plan C) and Query 2 (Plan B)

Queries affect each others’ run time

11

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1 QUERY 1 SELECT … device == “mobile” …; QUERY 2 SELECT … genre == “pc” …;

⋈ ⋈

16 GB 200 GB 200 GB

T1 T3 T2

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝐷

⋈ ⋈

12 GB 200 GB 200 GB

T2 T1 T3

200 GB

𝜏𝑄𝐷 𝜏𝑄𝐷 𝜏𝐷

Different query plans for Query 1 (Plan C) and Query 2 (Plan B)

No contention of network links

Queries affect each others’ run time

11

80 Gbps 40 Gbps 100 Gbps

T2 DC2 T3 DC3 T1 DC1 QUERY 1 SELECT … device == “mobile” …; QUERY 2 SELECT … genre == “pc” …;

⋈ ⋈

16 GB 200 GB 200 GB

T1 T3 T2

200 GB

𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝑁𝑝𝑐𝑗𝑚𝑓 𝜏𝐷

⋈ ⋈

12 GB 200 GB 200 GB

T2 T1 T3

200 GB

𝜏𝑄𝐷 𝜏𝑄𝐷 𝜏𝐷

Different query plans for Query 1 (Plan C) and Query 2 (Plan B)

No contention of network links

Choosing execution plans jointly for multiple queries improves performance

Iterative Shortest Job First

QO

QUERY A QUERY B

QO

QUERY C

QO

• Best combination  minimize average

completion

• Computationally intractable

12

Iterative Shortest Job First

Clarinet QO

QUERY A QUERY B

QO

QUERY C

QO

• Best combination  minimize average

completion

• Computationally intractable
• Iterative Shortest Job First (SJF)

scheduling heuristic

1. Pick shortest physical query plan in each iteration

12

Iterative Shortest Job First

Clarinet QO

QUERY A QUERY B

QO

QUERY C

QO

• Best combination  minimize average

completion

• Computationally intractable
• Iterative Shortest Job First (SJF)

scheduling heuristic

1. Pick shortest physical query plan in each iteration

10 12 18 5 8 20 30 Iter 1:

12

Iterative Shortest Job First

Clarinet QO

QUERY A QUERY B

QO

QUERY C

QO

• Best combination  minimize average

completion

• Computationally intractable
• Iterative Shortest Job First (SJF)

scheduling heuristic

1. Pick shortest physical query plan in each iteration

10 12 18 5 8 20 30 Iter 1:

12

Iterative Shortest Job First

Clarinet QO

QUERY A QUERY B

QO

QUERY C

QO

• Best combination  minimize average

completion

• Computationally intractable
• Iterative Shortest Job First (SJF)

scheduling heuristic

1. Pick shortest physical query plan in each iteration

• Reserve bandwidth to guarantee

completion time

10 12 18 5 8 20 30 Iter 1: t

12

B1

Link 1 Link 2

5

Iterative Shortest Job First

Clarinet QO

QUERY A

QUERY B

QO

QUERY C

QO

• Best combination  minimize average

completion

• Computationally intractable
• Iterative Shortest Job First (SJF)

scheduling heuristic

1. Pick shortest physical query plan in each iteration

• Reserve bandwidth to guarantee

completion time

t

12

B1

Link 1 Link 2

5

Iterative Shortest Job First

Clarinet QO

QUERY A

QUERY B

QO

QUERY C

QO

• Best combination  minimize average

completion

• Computationally intractable
• Iterative Shortest Job First (SJF)

scheduling heuristic

1. Pick shortest physical query plan in each iteration

• Reserve bandwidth to guarantee

completion time

15 17 18 25 30 Iter 2: t

12

B1

Link 1 Link 2

5

Iterative Shortest Job First

Clarinet QO

QUERY A

QUERY B

QO

QUERY C

QO

• Best combination  minimize average

completion

• Computationally intractable
• Iterative Shortest Job First (SJF)

scheduling heuristic

1. Pick shortest physical query plan in each iteration

• Reserve bandwidth to guarantee

completion time

15 17 18 25 30 Iter 2: t

12

B1

Link 1 Link 2

5

Iterative Shortest Job First

Clarinet QO

QUERY A

QUERY B

QO

QUERY C

QO

• Best combination  minimize average

completion

• Computationally intractable
• Iterative Shortest Job First (SJF)

scheduling heuristic

1. Pick shortest physical query plan in each iteration

• Reserve bandwidth to guarantee

completion time

15 17 18 25 30 Iter 2: A1 A2 t

12

B1

Link 1 Link 2

5 15 7

Avoid fragmentation and improve completion time

13

Avoid fragmentation and improve completion time

• SJF & reservation leads to bandwidth fragmentation

13

Avoid fragmentation and improve completion time

• SJF & reservation leads to bandwidth fragmentation

13

A1 A2 t B1

Link 1 Link 2

12 10

Scheduled in SJF order

22

Avoid fragmentation and improve completion time

• SJF & reservation leads to bandwidth fragmentation

13

A1 A2 t B1

Link 1 Link 2

12 10

Scheduled in SJF order

22 Dominant transfers execute sequentially

Avoid fragmentation and improve completion time

• SJF & reservation leads to bandwidth fragmentation

13

Extended idling A1 A2 t B1

Link 1 Link 2

12 10

Scheduled in SJF order

22 Dominant transfers execute sequentially

Avoid fragmentation and improve completion time

• SJF & reservation leads to bandwidth fragmentation

13

Alternate schedule with same query plans

t B1 A1 A2 12 2 Extended idling A1 A2 t B1

Link 1 Link 2

12 10

Scheduled in SJF order

22 Dominant transfers execute sequentially

Avoid fragmentation and improve completion time

• SJF & reservation leads to bandwidth fragmentation

13

Alternate schedule with same query plans

t B1 A1 A2 12 2 Extended idling A1 A2 t B1

Link 1 Link 2

12 10

Scheduled in SJF order

22 Dominant transfers execute sequentially

Re-arranging transfers resulting in deviation from SJF schedule can help

k-Shortest Jobs First Heuristic

14

Link 1 Link 2 Link n Offline schedule

t

k-Shortest Jobs First Heuristic

14

Link 1 Link 2

• Identify transfers of k-shortest yet incomplete jobs

Link n Offline schedule

t

k-Shortest Jobs First Heuristic

14

Link 1 Link 2

• Identify transfers of k-shortest yet incomplete jobs
• Relax transfer schedule  Start as soon as link is free and task is available

Link n Offline schedule

t

k-Shortest Jobs First Heuristic

14

Link 1 Link 2

• Identify transfers of k-shortest yet incomplete jobs
• Relax transfer schedule  Start as soon as link is free and task is available
• Best ’k’ Prior observations (or) through offline simulations

Link n Offline schedule

t

Clarinet Implementation

15

QUERY 1 QUERY 2 QUERY 3

QO QO QO

Existing Query Optimizers Batch of queries

Clarinet Implementation

15

QUERY 1 QUERY 2 QUERY 3

QO QO QO

Existing Query Optimizers

• Modified Hive to generate multiple

plans Batch of queries

Clarinet Implementation

15

QUERY 1 QUERY 2 QUERY 3

QO QO QO

Existing Query Optimizers

• Modified Hive to generate multiple

plans

• QOs control set of generated plans
• Existing optimizations are applied
• Push down Select
• Partition pruning

Batch of queries

Clarinet Implementation

15

QUERY 1 QUERY 2 QUERY 3

QO QO QO

Existing Query Optimizers

• Modified Hive to generate multiple

plans

• QOs control set of generated plans
• Existing optimizations are applied
• Push down Select
• Partition pruning

Batch of queries Enforces Clarinet’s schedule

Clarinet

Execution framework

Clarinet Implementation

15

QUERY 1 QUERY 2 QUERY 3

QO QO QO

Existing Query Optimizers

• Modified Hive to generate multiple

plans

• QOs control set of generated plans
• Existing optimizations are applied
• Push down Select
• Partition pruning

Batch of queries Enforces Clarinet’s schedule

• Modified Tez’s DAGScheduler

Clarinet

Execution framework

Clarinet Implementation

15

QUERY 1 QUERY 2 QUERY 3

QO QO QO

Existing Query Optimizers

• Modified Hive to generate multiple

plans

• QOs control set of generated plans
• Existing optimizations are applied
• Push down Select
• Partition pruning

Batch of queries Online query arrivals Enforces Clarinet’s schedule

• Modified Tez’s DAGScheduler
• Fairness guarantees

Clarinet

Execution framework

16

Evaluation

Compare Clarinet with following GDA approaches:

16

Evaluation

• 1. Hive
• 2. Hive + Iridium
• 3. Hive + Reducers in single DC

Compare Clarinet with following GDA approaches:

16

Evaluation

• 1. Hive
• 2. Hive + Iridium
• 3. Hive + Reducers in single DC

: WAN agnostic task placement + scheduling Compare Clarinet with following GDA approaches:

16

Evaluation

• 1. Hive
• 2. Hive + Iridium
• 3. Hive + Reducers in single DC

: WAN agnostic task placement + scheduling : WAN aware task placement across DCs Compare Clarinet with following GDA approaches:

16

Evaluation

• 1. Hive
• 2. Hive + Iridium
• 3. Hive + Reducers in single DC

: WAN agnostic task placement + scheduling : WAN aware task placement across DCs : Distributed filtering + central aggregation Compare Clarinet with following GDA approaches:

16

Evaluation

• 1. Hive
• 2. Hive + Iridium
• 3. Hive + Reducers in single DC

: WAN agnostic task placement + scheduling : WAN aware task placement across DCs : Distributed filtering + central aggregation Compare Clarinet with following GDA approaches:

• Geo-Distributed Analytics stack across 10 EC2 regions

16

Evaluation

• 1. Hive
• 2. Hive + Iridium
• 3. Hive + Reducers in single DC

: WAN agnostic task placement + scheduling : WAN aware task placement across DCs : Distributed filtering + central aggregation Compare Clarinet with following GDA approaches:

• Geo-Distributed Analytics stack across 10 EC2 regions
• Workload:
• 30 batches of 12 randomly chosen TPC-DS queries

Evaluation: Reduction in average completion time

17

GDA Approach

• Vs. Hive

Average Gains

Clarinet 2.7x Hive + Iridium 1.5x Hive + Reducers in single DC 0.6x

Evaluation: Reduction in average completion time

17

GDA Approach

• Vs. Hive

Average Gains

Clarinet 2.7x Hive + Iridium 1.5x Hive + Reducers in single DC 0.6x

Clarinet chooses a different plan for 75% of queries

Evaluation: Reduction in average completion time

17

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 6 11 16 21 26 31 36 41 46 51 56

CDF

Link ID sorted by bandwidth

WAN bandwidth distribution Hive bytes distribution Clarinet bytes distribution

Data from a single batch 12 queries

GDA Approach

• Vs. Hive

Average Gains

Clarinet 2.7x Hive + Iridium 1.5x Hive + Reducers in single DC 0.6x

Clarinet chooses a different plan for 75% of queries

Evaluation: Reduction in average completion time

17

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 6 11 16 21 26 31 36 41 46 51 56

CDF

Link ID sorted by bandwidth

WAN bandwidth distribution Hive bytes distribution Clarinet bytes distribution

Data from a single batch 12 queries

GDA Approach

• Vs. Hive

Average Gains

Clarinet 2.7x Hive + Iridium 1.5x Hive + Reducers in single DC 0.6x

Clarinet chooses a different plan for 75% of queries

Evaluation: Reduction in average completion time

17

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 6 11 16 21 26 31 36 41 46 51 56

CDF

Link ID sorted by bandwidth

WAN bandwidth distribution Hive bytes distribution Clarinet bytes distribution

Data from a single batch 12 queries

GDA Approach

• Vs. Hive

Average Gains

Clarinet 2.7x Hive + Iridium 1.5x Hive + Reducers in single DC 0.6x

Clarinet chooses a different plan for 75% of queries

18

Evaluation: Optimization overhead

18

Evaluation: Optimization overhead

• 1. Generate multiple query plans
• 2. Iterative multi-query plan selection

18

Evaluation: Optimization overhead

• 1. Generate multiple query plans
• Up to 64 plans in less than 5 s
• 2. Iterative multi-query plan selection

18

Evaluation: Optimization overhead

• 1. Generate multiple query plans
• Up to 64 plans in less than 5 s
• 2. Iterative multi-query plan selection
• Max. 15 s for batches with 12 queries

18

Evaluation: Optimization overhead

• 1. Generate multiple query plans
• Up to 64 plans in less than 5 s
• 2. Iterative multi-query plan selection
• Max. 15 s for batches with 12 queries

Insignificant w.r.t. query running times (order of 10’s of minutes)

Summary

• WAN-awareness in QO +

cross-layer optimization

19

Distributed Storage Layer Distributed Execution Layer Query Optimizer

Clarinet

Summary

• WAN-awareness in QO +

cross-layer optimization

• Presented a scalable way to

implement multi-query

• ptimization with minimal
• verhead

19

Distributed Storage Layer Distributed Execution Layer Query Optimizer

Clarinet

Summary

• WAN-awareness in QO +

cross-layer optimization

• Presented a scalable way to

implement multi-query

• ptimization with minimal
• verhead

19

Distributed Storage Layer Distributed Execution Layer Query Optimizer

Clarinet

2.7x

Reduction in average completion time