Data Acquisition for Real-time Decision-making under Freshness - - PowerPoint PPT Presentation
Data Acquisition for Real-time Decision-making under Freshness Constraints Shaohan Hu , Shuochao Yao, Haiming Jin, Yiran Zhao, Yitao Hu, Xiaochen Liu, Nooreddin Naghibolhosseini, Shen Li, Akash Kapoor, William Dron, Lu Su, Amotz Bar-Noy, Pedro
Shaohan Hu, Shuochao Yao, Haiming Jin, Yiran Zhao, Yitao Hu, Xiaochen Liu, Nooreddin Naghibolhosseini, Shen Li, Akash Kapoor, William Dron, Lu Su, Amotz Bar-Noy, Pedro Szekely, Ramesh Govindan, Reginald Hobbs, Tarek Abdelzaher
2
3
Road not blocked Shelter has vacancy Disease controlled Medic team staffed Comm. center up Food store not flooded
4
OR AND AND
5
Road not blocked Shelter has vacancy Disease controlled Medic team staffed Comm. center up Food store not flooded
DECISION
Decision Maker
6
7
8
9
10
OR AND AND
11
12
13
OR AND A B C AND D E
14
OR AND A B C AND D E
15
Maximize probability of short-circuiting per unit cost First examine the course of action that’s most likely to succeed Within a course of action, first examine the condition that’s most likely to fail
OR AND A B C
AND D E
C P(True) 4 3 0.3 0.6
16
R (1-0.3)/4 =0.175 (1-0.6)/3 =0.133
C(D->E) = 4+0.3(3) = 4.9 C(E->D) = 3+0.6(4) = 5.4
OR AND B C
AND D E
C P(True) 4 3 0.3 0.6
17
R (1-0.3)/4 =0.175 (1-0.6)/3 =0.133
A B
OR AND
A B C
AND
D E
C P(True) 4.9 0.18 7 0.1
18
R 0.18/4.9=0.037 0.1/7=0.014
A B
OR AND
A B C
AND
D E
C P(True) 4.9 0.18 7 0.1 C(L->R) = 7+(1-0.1)x4.9 = 11.41 C(R->L) = 4.9+(1-0.18)x7 = 10.64
19
R 0.18/4.9=0.037 0.1/7=0.014
20
21
OR AND AND
23
24
25
retrieval latency freshness interval
26
27
Some random ordering
28
expired
Some random ordering
29
expired bad resolution
Some random ordering
30
31
Latest Deadline First (LDF)
32
good resolution
Latest Deadline First (LDF)
Inspired by EDF: data objects with later freshness deadlines are retrieved sooner Optimal: if LDF cannot avoid freshness deadline violation, no sequential order can
33
34
35
LDF satisfies data freshness constraints
36
This might have lower expected cost
37
This might have lower expected cost
38
Greedily rearrange LDF order to reduce the expected data retrieval cost
39
40
41
42
expired
43
parallel retrieval
44
good resolution
45
good resolution
46
Increment parallel retrieval level until freshness constraints are met
Compute LDF order Greedily rearrange LDF order to reduce the expected data retrieval cost Gradually increment parallel retrieval level until freshness constraints are met
47
48
Baselines
LCF - Lowest Cost Source First SCB - Shortcircuit Benefit only PbP - Probability based Prediction
Settings
% fast changing data: 40~100%, default 70% # Action size: 4~10, default 6 Data object size: 3~5 MB, default 3.45 MB Network bandwidth: 3.5~6.5 KBps, default 5 KBps Transmission latency fluctuation: -3~3 min, default 0
49
Varying % of fast changing data
50
40 50 60 70 80 90 100 50 100 150 Ratio of Fast−Changing Data Items (%) Retrieval Cost Ratio (%) vLDF SCB PbP LCF 40 50 60 70 80 90 100 20 40 60 80 100 Ratio of Fast−Changing Data Items (%) Request Resolution Rate (%) vLDF SCB PbP LCF
Varying action size
51
4 5 6 7 8 9 10 50 100 150 200 Number of Conditions per Action Retrieval Cost Ratio (%) vLDF SCB PbP LCF 4 5 6 7 8 9 10 20 40 60 80 100 Number of Conditions per Action Request Resolution Rate (%) vLDF SCB PbP LCF
Varying data object size
52
2000 2500 3000 3500 4000 4500 5000 50 100 150 Average Data Item Size (KB) Retrieval Cost Ratio (%) vLDF SCB PbP LCF 2000 2500 3000 3500 4000 4500 5000 20 40 60 80 100 Average Data Item Size (KB) Request Resolution Rate (%) vLDF SCB PbP LCF
Varying network bandwidth
53
3.5 4 4.5 5 5.5 6 6.5 50 100 150 Network Bandwidth (KBps) Retrieval Cost Ratio (%) vLDF SCB PbP LCF 3.5 4 4.5 5 5.5 6 6.5 20 40 60 80 100 Network Bandwidth (KBps) Request Resolution Rate (%) vLDF SCB PbP LCF
Varying network transmission fluctuation
54
−3 −2 −1 1 2 3 50 100 150 Network Transmission Delay Fluctuation − Mean Retrieval Cost Ratio (%) vLDF SCB PbP LCF −3 −2 −1 1 2 3 20 40 60 80 100 Network Transmission Delay Fluctuation − Mean Request Resolution Rate (%) vLDF SCB PbP LCF
Find routes for <src, dst> pairs Each candidate route: AND of its segments Routing result: OR of all candidate routes
Visual verification for route segment conditions
55
vLDF Cost (KB) PbP Cost (KB) vLDF Time (s) PbP Time (s) 516 685 164 255 343 598 150 206 319 485 160 248 506 1093 165 372 524 1042 175 206
Environment dynamics & resource limitations affect real-time decision-making Efficient data acquisition algorithm Promising results through simulations and concrete route finding application scenario
59