On Delay-Storage Trade-o ff s in Content Download from Coded - - PowerPoint PPT Presentation

On Delay-Storage Trade-offs in Content Download from Coded Distributed Storage Systems

Gauri Joshi (MIT)

joint work with Yanpei Liu (UW-Madison) Emina Soljanin (Bell Labs)

DIMACS Workshop on Algorithms for Green Data Storage

Gauri Joshi (MIT) Delay-Storage Trade-offs 1 / 24

Why Use Coding in Distributed Storage

Data Centers Server clusters that store and process all the data in the Internet More than 500000 data centers worldwide Consume vast amounts of energy - more than 2% of US electricity

Power to run and repair servers, and for cooling systems

Gauri Joshi (MIT) Delay-Storage Trade-offs 2 / 24

Trade-offs in Coding for Distributed Storage

Reliability vs. Storage Replication is the most commonly used redundancy (n, k) MDS Codes - any k out of n sufficient for data recovery Repair Bandwidth vs. Storage Locally Repairable Codes[Dimakis, IT-Tran ’10] Regenerative codes for storage [Rashmi, IT-Tran ’12]

Gauri Joshi (MIT) Delay-Storage Trade-offs 3 / 24

Trade-offs in Coding for Distributed Storage

Accessibility vs. Storage Lower blocking probability than replication for the same storage (Energy Cost) [Ferner, Allerton ’12] Delay vs. Storage Our work - k out of n fork-join queues Packet Routing Diversity [Maxemchuk, 1991], [Kabatiansky, 2005] – do not consider queueing Redundant requests, MDS queue [Shah, Lee, 2013]

Gauri Joshi (MIT) Delay-Storage Trade-offs 4 / 24

How Coding Reduces Download Time

Single M/M/1 Queue Requests arrive at rate λ and served at rate µ Mean response time T1,1 =

1 µλ for Poisson arrivals and departures

µ λ

Gauri Joshi (MIT) Delay-Storage Trade-offs 5 / 24

How Coding Reduces Download Time

Multiple Copies give Diversity, but with More Storage Requests is sent to n disks storing copies of content Need to wait only for download of only one n copies Mean response time Tn,1 =

1 nµλ, but storage increases n-fold µ λ µ λ λ µ λ

Gauri Joshi (MIT) Delay-Storage Trade-offs 6 / 24

How Coding Reduces Download Time

Coding Gives Diversity with Lower Storage Content divided into k blocks and encoded to n blocks Each disk stores 1/k units, so service rate becomes µ0 = kµ Downloading any k blocks is sufficient to decode the file

kµ λ λ λ λ kµ kµ

Gauri Joshi (MIT) Delay-Storage Trade-offs 7 / 24

Definition: (n, k) Fork-Join System

Requests arrivals are Poisson with rate λ A request forked into n tasks ! enter FCFS queues at the n disks Time to download one block of content ⇠ exp(µ0), where µ0 = kµ Load factor ρ = λ/µ0 for each queue.

kµ λ λ λ λ kµ kµ

Gauri Joshi (MIT) Delay-Storage Trade-offs 8 / 24

Fork-Join Queues: Example

A content file of unit size is divided into k = 2 blocks, a and b Encoded into 3 blocks, a, b and a + b Downloading any 2 blocks is sufficient to decode the entire file Storage is 50% higher, but response time is reduced. a + b b a a + b b

Gauri Joshi (MIT) Delay-Storage Trade-offs 9 / 24

Fork-Join Queues: Example

A content file of unit size is divided into k = 2 blocks, a and b Encoded into 3 blocks, a, b and a + b Downloading any 2 blocks is sufficient to decode the entire file Storage is 50% higher, but response time is reduced.

F

Abandon

J

1 1 1 2 2 2 3 3 3 4 4 4

λ

λ λ kµ kµ kµ Gauri Joshi (MIT) Delay-Storage Trade-offs 10 / 24

Mean Response Time

Challenges Arrivals to the n queues are perfectly synchronized. Hence it is not the kth order statistic of exponential Previous work has attempted finding Tn,n, but only bounds are known

F

Abandon

J

1 1 1 2 2 2 3 3 3 4 4 4

λ

λ λ kµ kµ kµ Gauri Joshi (MIT) Delay-Storage Trade-offs 11 / 24

Our Contributions

Bounds on mean response time of the (n, k) fork-join system Delay-Storage Trade-offs

Fixed storage expansion k/n what is the best n? Fixed n disks what is the best k?

Extensions to correlated service times, (m, n, k) fork-join etc.

[1] G. Joshi, Y. Liu, E. Soljanin, ”Coding for Fast Content Download”, Allerton Conference 2012 [2] G. Joshi, Y. Liu, E. Soljanin, ”On Delay-Storage Trade-offs in Content Download from Coded Distributed Storage Systems”, to appear in JSAC 2014

Gauri Joshi (MIT) Delay-Storage Trade-offs 12 / 24

Upper Bound on Response Time

Comparison with a split-merge system Split-merge system - All n queues are blocked until k tasks finish Response time of split-merge is always greater than fork-join

F

Abandon

J

1 1 1 2 2 2 3 3 3 4 4 4

λ

λ λ kµ kµ kµ Gauri Joshi (MIT) Delay-Storage Trade-offs 13 / 24

Upper Bound on Response Time

Equivalent to an M/G/1 queue

Arrivals are Poisson with rate λ Departures according to S, kth order statistic of exp(µ0)

E[S] = Hn Hnk µ0 V[S] = Hn2 H(nk)2 µ02 . Mean Response time given by the Pollaczek-Khinchin formula, Tn,k  E[S] + λ

• V[S] + E[S]2

2(1 λE[S])

Gauri Joshi (MIT) Delay-Storage Trade-offs 14 / 24

Lower Bound on Response Time

Stages of Processing of a Job A job goes through k stages of processing, at stage j, 0  j  k 1 At stage j, the job has completed j tasks and waiting for the remaining k j The service rate of a job in stage j stage is at most (n j)µ0 [Varki]. Tn,k

k1

X

j=0

1 (n j)µ0 λ Sum of response times of k stages = 1 µ0

k1

X

j=0

h 1 n j + ρ (n j)(n j ρ) i = 1 µ0 ⇥ Hn Hnk + ρ · (Hn(nρ) H(nk)(nkρ)) ⇤

Gauri Joshi (MIT) Delay-Storage Trade-offs 15 / 24

Flexible Disks, Fixed Storage Expansion

Parameters: Expansion k/n = 1/2, λ = 1 More diversity ! Lower Response Time

2 4 6 8 10 12 14 16 18 20 10

−2

10

−1

10 10

1

10

2

n, n = 2k Mean response time µ = 5 upper bound µ = 5 lower bound µ = 1 upper bound µ = 1 lower bound µ = 0.51 upper bound µ = 0.51 lower bound Gauri Joshi (MIT) Delay-Storage Trade-offs 16 / 24

How Much Can Double Storage Improve Completion Time?

0.0 0.5 1.0 1.5 2.0 0.0 0.2 0.4 0.6 0.8 1.0 response time fraction of completed downloads single disk k=1 k=2 k=5

Gauri Joshi (MIT) Delay-Storage Trade-offs 17 / 24

Comparison to Power-of-d

For same storage fork-join gives much faster response

1 2 3 4 5 6 7 8 10

−2

10

−1

10 10

1

10

2

Average time to download one unit of content (1/µ) Mean Response Time (10, 1) fork−join system (20, 2) fork−join system Power−of−2 Power−of−10 (LWL job assignment) Gauri Joshi (MIT) Delay-Storage Trade-offs 18 / 24

Flexible Storage Expansion, Fixed Disks

Parameters: n = 10, λ = 1, µ = 1 More redundancy ! Lower Response Time

1 2 3 4 5 6 7 8 9 10 0.1 0.15 0.2 0.25 0.3 0.35 Mean response time k 1 2 3 4 5 6 7 8 9 10 2 4 6 8 10 Storage T(10, k) simulation Upper bound Lower bound Required storage

Gauri Joshi (MIT) Delay-Storage Trade-offs 19 / 24

Flexible Storage Expansion, Fixed Disks

0.0 0.5 1.0 1.5 2.0 0.0 0.2 0.4 0.6 0.8 1.0 response time fraction of completed downloads single disk k=1 k=5 k=10 0.00 0.05 0.10 0.15 0.20 0.0 0.2 0.4 0.6 0.8 1.0 response time fraction of completed downloads

M/M/1 λ = 1 µ = 3

single disk baseline – unit storage the same total storage double total storage 10⇥ increase in storage

Gauri Joshi (MIT) Delay-Storage Trade-offs 20 / 24

Correlated Service Times

Service time X = δXd + (1 δ)Xr,i, for i = 1, 2, · · · n More correlation ! lose the diversity advantage

2 4 6 8 10 0.03 0.1 0.4 k Mean response time δ = 1 δ = 0.5 δ = 0

Figure : λ = 1, µ = 3

Gauri Joshi (MIT) Delay-Storage Trade-offs 21 / 24

(m,n,k) fork-join system

Large number of disks m n Can be divided into m/n = g fork-join systems

. . . 1 . . . n . . . n + 1 . . . 2n . . . . . . . . . (g 1)n + 1 . . . gn = m λ λ1 λ2 λg

Gauri Joshi (MIT) Delay-Storage Trade-offs 22 / 24

(m,n,k) fork-join system

2 4 6 8 10 12 0.05 0.1 0.15 0.2 0.25 0.3 k Mean response time (12, 6, k) Exponential (12, 6, k) Pareto, α = 1.8 (12, 12, k) Pareto, α = 1.8 (12, 12, k) Exponential

(12, 6, k) system (12, 12, k) system

Figure : λ = 1, µ = 3

Gauri Joshi (MIT) Delay-Storage Trade-offs 23 / 24

Concluding Remarks

Major Implications Investigated the delay-storage trade-off in distributed storage Showed that diversity of more disks helps, for same storage space used Generalization of (n, n) fork-join systems to the (n, k) fork-join system Future Perspectives Percentile analysis from the CDF of response time Extension to parallel computing instead of storage

Gauri Joshi (MIT) Delay-Storage Trade-offs 24 / 24