On the Impact of Clustering for IoT Analytics and Message Broker - - PowerPoint PPT Presentation

On the Impact of Clustering for IoT Analytics and Message Broker Placement across Cloud and Edge

Daniel Happ (TU Berlin, happ@tkn.tu-berlin.de) Suzan Bayhan (University of Twente)

EdgeSys 2020 April 27, 2020

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

1 / 15

Motivation

Internet of Things (IoT) becoming sharing economy Multiple Applications use same sensor data Sensor data processing needed for meaningful insights Storage, distribution, and processing usually in cloud Plentiful resources Flexible (on-demand, pay-as-you-go) Cloud has latency and privacy issues, preventing certain use-cases Moving more processing to the edge

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

2 / 15

The case for Publish/Subscribe in IoT

”Sense once, notify many” translates well to publish/subscribe pattern

Publisher Broker publish Subscriber notify Subscriber notify Subscriber notify subscribe

Decoupling properties:

• Time
• Synchronization
• Space

Enable seamless processing operator offloading scheme1

• 1D. Happ and A. Wolisz, ”Towards gateway to cloud offloading in IoT publish/subscribe systems,” in 2017 Second International Conference on Fog

and Mobile Edge Computing (FMEC), May 2017, pp. 101–106. IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

3 / 15

Research questions

• 1. How to place operators and brokers jointly across a cloud/fog/edge topology?
• 2. What is the impact of clustering of publishers and subscribers on the

placement?

• 3. What is the impact of the network size?

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

4 / 15

JOI deploys Operators & Message Brokers2

JOI

IoT application Network and application constraints

Broker deployment Application deployment

Fog-service provider

Cloud layer Fog layer Edge layer

Application users

Sensors

Input: network and application constraints & application graph Output: where to deploy

• perators and brokers

2Daniel Happ, Suzan Bayhan, and Vlado Handziski. 2020. JOI: Joint Placementof IoT Analytics Operators and Pub/Sub Message Brokers in Fog-centric IoT Platforms. Future Generation Comp. Sys. (2020). under review IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

5 / 15

JOI: Optimal Solution Sketch

Variables: xi,j: Operator i placed on node j yi,j: Operator i publishes to broker on node j Constraints: Node resources (CPU, memory) Node-to-node bandwidth Node access network bandwidth Objective: Minimize sum of subscriber delays

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

6 / 15

Greedy Heuristic for JOI

Preparation: Sort operators by hops to sink (stratum) Depth-first search: Place ops with low stratum first Place op optimally (with current knowledge) Place its broker optimally (with current knowledge) Finally: Join brokers until ”maximum number of brokers” constraint met

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

7 / 15

Tabu Heuristic for JOI

Starting-Point: Best solution of cloud and greedy In each step, try to improve:

• 1. Placement of one operator
• 2. Placement of one broker
• 3. Operator to broker association

Tabu: Short term memory

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

8 / 15

Degrees of Clustering in IoT

P1 S1 P2 S2 S2 S2

Cloud Layer Fog Layer Edge Layer

High degree of clustering for IoT publisher P1 Low degree of clustering for publisher P2

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

9 / 15

Von Mises Distribution models Clustering

Properties: uniform for β = 0, approaches the positive half of a normal distribution Nodes numbered according to delay to pivot node Fixed operators follow modified von Mises function

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

10 / 15

System-level Simulations

Application graphs: Random, fanout, sequence Fixed, but random, network topology: Edge, fog, cloud Realistic delays and bandwidth (public route servers) Metric: cloud gap

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

11 / 15

Impact of Clustering on Cloud Gap (Random)

3 6 9 12 15 18 Clustering Factor 0.0 0.5 1.0 1.5 2.0 2.5 Cloud Gap GREEDY TABU

clustering factor 0–9: cloud deployment best clustering factor 12: transition clustering factor 15+: improvement by greedy/tabu enables joint heuristic based on clustering factor

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

12 / 15

Placement of Operators (Clustering Factor 18)

CLOUD TABU GREEDY CLOUD TABU GREEDY CLOUD TABU GREEDY 0.0 0.2 0.4 0.6 0.8 1.0 Share

Random Fanout Sequence

Greedy places most (approx. 80%) operators in the fog Tabu improves by putting less operator on edge & mostly more in cloud hard to give easy ”rules of thumb”

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

13 / 15

Impact of Network Size (Greedy)

3 9 30 95 300 Topology Size 5 10 15 20 25 Cloud Gap Cluster Factor 0 Cluster Factor 18

Difference between greedy and cloud more pronounced in smaller topologies Observations hold true for all the sizes considered

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

14 / 15

Conclusion

Contributions: We proposed two heuristics for joint operator & broker placement We proposed modified von Mises distributions for modelling clustering We conducted simulations to evaluate impact of clustering Main Result: Increasing clustering leads to placement towards the edge Future Work: Cluster-aware placement heuristic Adaptive/dynamic scheme

IoT Analytics and Message Broker Placement

• D. Happ/S. Bayhan

15 / 15