NM
Uploaded byNitinder Mohan
515 views

Managing data in computational edge clouds

The document discusses the management of data in edge clouds, highlighting the deployment of small-scale servers at the network edge to enhance data processing with decreased latency and network traffic. It emphasizes the importance of caching strategies for data availability and introduces a grouping mechanism for edge resources based on locally cached data to improve task deployment efficiency. Additionally, the communication model and resource roles are outlined, detailing how data consistency and network traffic can be managed effectively.

Education◦
Related topics:
Fog Computing Insights•

Embed presentation

Managing Data in Computational Edge Clouds ACM SIGCOMM 2017 Workshop on Mobile Edge Communications (MECOMM’2017), Los Angeles Nitinder Mohan✼ Pengyuan Zhou✼ Keerthana Govindaraj△ Jussi Kangasharju✼ ✼University of Helsinki △Robert Bosch GmbH ✼{firstname.lastname@helsinki.fi} △{keerthana.govindaraj@de.bosch.com}
Rise of Edge Clouds Network DatacenterEdge Server User 2 Edge Cloud: Small-scale server(s) deployed at network edge to compute user data Motivation:
Rise of Edge Clouds Network DatacenterEdge Server User Edge Cloud: Small-scale server(s) deployed at network edge to compute user data Motivation: üDecreased latency and network traffic 2
Rise of Edge Clouds Network Edge Cloud: Small-scale server(s) deployed at network edge to compute user data Motivation: üDecreased latency and network traffic üComputing data of local relevance 2
Edge-Fog Cloud Nitinder Mohan and Jussi Kangasharju. "Edge-fog cloud: A distributed cloud for internet of things computations." Cloudification of the Internet of Things (CIoT). IEEE, 2016. • Voluntary compute devices • 1-2 network hops from users • Ad-hoc network connectivity • Dedicated compute servers • 5-6 network hops from users • Dense network connectivity • Permanent data archival • No compute capability 3
Data Availability in Edge Cache Query: Give me a route from point A to B without road disruptions 4
Data Availability in Edge Cache • Edge cloud proposals presume that the required data for processing is pre- available in Edge servers cache • However, Edge server may request Data Store for required data Data Request: Any construction/ accidents on the map? 4
Data Availability in Edge Cache • Edge cloud proposals presume that the required data for processing is pre- available in Edge servers cache • However, Edge server may request Data Store for required data • The computation is delayed by a round-trip-time to central Data Store • Reduces benefits of employing Edge clouds! Data Request: Any construction/ accidents on the map? Requirement: Caching strategy which predicts and stores required data before computation 4
Edge Cloud Cache Grouping Edge cloud caching mechanism ØGroup edge resources into cache groups based on their locally cached data ØPredict required data for computation based on cache grouping classification ØProvide coherence on multiple copies of shared data throughout a cache group 5
Resource Cache Grouping
Grouping Algorithm Task Deployment Populate Caches Grouping Classifiers 7
Grouping Algorithm Deployment: Select {RC}i resources for computing a task Tc 1. Algorithms such as LPCF* selects a subset from available resources which achieves least processing and network cost for collaboration. 2. Several deployments for different tasks can occur at same time. Task Deployment T1 T2 Nitinder Mohan and Jussi Kangasharju. "Edge-fog cloud: A distributed cloud for internet of things computations." Cloudification of the Internet of Things (CIoT). IEEE, 2016. 8
Grouping Algorithm 1. At t=0, {RC}i does not have data required for computation and it requests the required data from Data Store 2. The Data Store sends the required data to requesting resources populating their caches Task Deployment Populate Caches 9
Grouping Algorithm 1. At t=0, {RC}i does not have data required for computation and it requests the required data from Data Store 2. The Data Store sends the required data to requesting resources populating their caches Task Deployment Populate Caches 9
Grouping Algorithm 1. The task Tc is classified in a workload Wk where Wk is a set of similar tasks 2. The resources {RC}i caching data of workload Wk are grouped together as cache group CGk Task Deployment Populate Caches Grouping Classifiers W1 W2 10
Grouping Algorithm 1. The deployment algorithm prefers CGk ∈Wk for task Ti ∈Wk Task Deployment Populate Caches Grouping Classifiers W1 W2 T3 11
Grouping Algorithm 1. The deployment algorithm prefers CGk ∈Wk for task Ti ∈Wk 2. If the resources in group are not appropriate for deployment, the algorithm can select other available resources Task Deployment Populate Caches Grouping Classifiers W1 W2 11
Grouping Algorithm 1. The deployment algorithm prefers CGk ∈Wk for task Ti ∈Wk 2. If the resources in group are not appropriate for deployment, the algorithm can select other available resources 3. These resources then join their respective cache group Task Deployment Populate Caches Grouping Classifiers W1 W2 11
Grouping Algorithm 1. The deployment algorithm prefers CGk ∈Wk for task Ti ∈Wk 2. If the resources in group are not appropriate for deployment, the algorithm can select other available resources 3. These resources then join their respective cache group 4. A resource can be a part of more than one group Task Deployment Populate Caches Grouping Classifiers W1 W2 11
Group Communication
Requirements 1. Reduce unnecessary network traffic 2. Ensure consistent computation by avoiding stale data copies 13
Resource Roles 1. Group Member • Resource (RCx) part of a cache group {CG}i • The group member stores only data of its group workload 14
Resource Roles 1. Group Member • Resource (RCx) part of a cache group {CG}i • The group member stores only data of its group workload 2. Group Leader • Representative of a group • Responsible for maintaining coherent data within a group • Has consistent connectivity with group members • Elected via distributed election algorithm 14
Communication Flow The communication model is strictly pull-based to limit number of messages in the system The model operates over following assumptions: 1. The resources upload their updated data copy to Data Store after every successful computation 2. Data upload is in-parallel to ongoing local computation and does not impact the computation completion time 3. Data Store is persistent and immune to failures 4. Messages in the system are not lost 15
Cache Data Structures 1. Group member table Maps locally cached data with group leader 2. Group leader table Ensures that group leader has consistent state and address of resource caching data 3. Data Store table Maps data to groups Data Name Leader Address Tag Data Name Leader Address Classifier Type Data Name Resource Address Tag Timestamp 16
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario 1: Retrieving free data 17
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Scenario 1: Retrieving free data 17
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Scenario 1: Retrieving free data 17
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Scenario 1: Retrieving free data 17
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Scenario 1: Retrieving free data 17
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Scenario 1: Retrieving free data 17
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’ Scenario 1: Retrieving free data 17
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’ Scenario 1: Retrieving free data Content ’X’ 17
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Scenario 2: Resource failure 18
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Unavailable! Scenario 2: Resource failure 18
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Timeout Unavailable! Scenario 2: Resource failure 18
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’? Timeout Unavailable! Scenario 2: Resource failure 18
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’? Timeout Unavailable! Scenario 2: Resource failure 18
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’? Timeout Content ‘X’ Unavailable! Scenario 2: Resource failure 18
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’? Timeout Content ‘X’ Unavailable! Scenario 2: Resource failure Content ’X’ 18
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario 3: Data under update 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario 3: Data under update Content ‘X’? 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario 3: Data under update Content ‘X’? 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Content ‘X’? 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Content ‘X’? 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Content ‘X’? 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Content ‘X’? Free 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? Free 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? Free 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? Free Free 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? Free Free 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? GoTo: ’Resource 1’ Free Free 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Lock: ’Resource 1’ Content ‘X’? Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? GoTo: ’Resource 1’ Free Free 19
Content Retrival in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Lock: ’Resource 1’ Content ‘X’? Content ‘X’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? GoTo: ’Resource 1’ Free Free 19
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 20
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ 20
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ 20
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock 20
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Lock 20
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Lock 20
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Lock Lock 20 Resource 2
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Processing Lock Lock 20 Resource 2
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Processing Free Lock 20 Resource 2
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Processing Free: Content ‘X’ Lock Lock 20 Resource 2
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx1 Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Processing Free: Content ‘X’ Lock Free 20 Resource 2
Content Update in Group Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx1 Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Processing Free: Content ‘X’ Content ‘X’ Lock Free 20 Resource 2
Requirements 1. Reduce unnecessary network traffic Ø Communication model encourages data sharing within a group and removes a single point-of-failure 2. Ensure consistent computation by avoiding stale data copies Ø Model ensures causal coherence on shared data 21
Evaluation
Evaluation Evaluated on Icarus simulator* Parameters: 1. 320 Edge and Fog nodes + Data Store 2. Fog node acts as group leader 3. 96*104 content items divided in 32 workloads 4. Resource cache size capacity is 10% of overall content size *Lorenzo Saino, Ioannis Psaras, and George Pavlou. "ICARUS: a caching simulator for information centric networking (ICN)." Proceedings of the 7th International ICST conference on Simulation Tools and Techniques. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 2014. 23
Evaluation Q1. Does resource grouping offer any benefits to edge clouds? Q2. If yes, what is the appropriate size of a group? 24
Q1. Does resource grouping offer any benefits to edge clouds? a) Effect on cache hit rate 0.02 0.04 0.06 0.08 0.1 Cache Size 0 0.2 0.4 0.6 0.8 1 CacheHitRate 4-group 4-no_group 32-group 32-no_group 25
Q1. Does resource grouping offer any benefits to edge clouds? a) Effect on cache hit rate 0.02 0.04 0.06 0.08 0.1 Cache Size 0 0.2 0.4 0.6 0.8 1 CacheHitRate 4-group 4-no_group 32-group 32-no_group ≈2 times increase in hit rate 25
Q1. Does resource grouping offer any benefits to edge clouds? b) Effect on latency 0.02 0.04 0.06 0.08 0.1 Cache Size 20 30 40 50 60 70 80 Latency(ms) 4-group 4-no_group 32-group 32-no_group 26
Q1. Does resource grouping offer any benefits to edge clouds? b) Effect on latency 0.02 0.04 0.06 0.08 0.1 Cache Size 20 30 40 50 60 70 80 Latency(ms) 4-group 4-no_group 32-group 32-no_group 45% decrease in latency 26
Q2. If yes, what is the appropriate size of a group? 0 25 50 75 100 125 150 Number of Groups 0.2 0.4 0.6 0.8 1 CacheHitRate 4 workloads 8 workloads 16 workloads 32 workloads 0 25 50 75 100 125 150 Number of Groups 20 30 40 50 Latency(ms) 4 workloads 8 workloads 16 workloads 32 workloads Maximum gains are achieved when number of cache groups = number of workloads 27
Conclusion 1. Edge resource grouping strategy manages content in Edge caches 2. Strategy classifies content based on resource’s deployed workloads and cached content 3. The 3-step iterative algorithm maps future deployments on resources with required data locally cached 4. Our evaluations via simulation show i. Direct correlation between grouping and increased system performance ii. Number of cache groups and number of assigned workloads 28
Thank You! Questions? nitinder.mohan@helsinki.fi CleanSky - EU FP7 Marie Curie Initial Training Network
Backup Slides
Industry 4.0 on Edge Industry 4.0 proposes automation in manufacturing 1. Robots such as Bosch APAS can automate tasks and work along side humans while ensuring safety 2. IoT sensor data is used in augmented reality for collaborative support and manufacturing Requirement: Time-critical and fast data processing
Industry 4.0 on Edge Industry 4.0 proposes automation in manufacturing 1. Robots such as Bosch APAS can automate tasks and work along side humans while ensuring safety 2. IoT sensor data is used in augmented reality for collaborative support and manufacturing Requirement: Time-critical and fast data processing Solution: Implement industrial Edge clouds Network Datacenter Edge Servers
Grouping Classifier 1. Location: location of place while constructing AR and 3D models 2. Relevance: laptop and cellphone manufacturing tasks as casing 3. Time: sensor data generation and ‘freshness’ time 4. Warning signals: broadcast of information-only or safety-related signals
Q1. Does resource grouping offer any benefits to edge clouds? c) Effect of cache size 0.02 0.04 0.06 0.08 0.1 Cache Size 20 30 40 50 60 70 80 Latency(ms) 4-group 4-no_group 32-group 32-no_group 0.02 0.04 0.06 0.08 0.1 Cache Size 0 0.2 0.4 0.6 0.8 1 CacheHitRate 4-group 4-no_group 32-group 32-no_group Gains decrease as we increase cache size! 1. Grouping works best when cache sizes are limited and cannot store all content 2. With larger caches, resources can locally store all content!
Q1. Does resource grouping offer any benefits to edge clouds? d) Effect of number of workloads 0.02 0.04 0.06 0.08 0.1 Cache Size 20 30 40 50 60 70 80 Latency(ms) 4-group 4-no_group 32-group 32-no_group 0.02 0.04 0.06 0.08 0.1 Cache Size 0 0.2 0.4 0.6 0.8 1 CacheHitRate 4-group 4-no_group 32-group 32-no_group Gains decrease on increasing number of workloads! Ø More workloads ≈ more unique content requests which cannot be satisfied by local caches Workloads is a uniform distribution of similar content requests

More Related Content

PPTX
Accelerating Data-driven Discovery in Energy Science
byIan Foster
 
PDF
IRJET - A Secure Access Policies based on Data Deduplication System
byIRJET Journal
 
PDF
TriRODS Dec 2017
bymoughto1
 
PDF
E FFICIENT D ATA R ETRIEVAL F ROM C LOUD S TORAGE U SING D ATA M ININ...
byIJCI JOURNAL
 
PDF
IRJET- Cross User Bigdata Deduplication
byIRJET Journal
 
PPTX
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
byIan Foster
 
PPTX
Streamlined data sharing and analysis to accelerate cancer research
byIan Foster
 
PPTX
Applications of SOA and Web Services in Grid Computing
byyht4ever
 
Accelerating Data-driven Discovery in Energy Science
byIan Foster
 
IRJET - A Secure Access Policies based on Data Deduplication System
byIRJET Journal
 
TriRODS Dec 2017
bymoughto1
 
E FFICIENT D ATA R ETRIEVAL F ROM C LOUD S TORAGE U SING D ATA M ININ...
byIJCI JOURNAL
 
IRJET- Cross User Bigdata Deduplication
byIRJET Journal
 
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
byIan Foster
 
Streamlined data sharing and analysis to accelerate cancer research
byIan Foster
 
Applications of SOA and Web Services in Grid Computing
byyht4ever
 

What's hot

PPTX
Introduction: Linked Data and the Semantic Web
byNikolaos Konstantinou
 
PDF
3 ijece 1 ed iqbal qc
byIAESIJEECS
 
PDF
Key-Aggregate Searchable Encryption (KASE) for Group Data Sharing via Cloud S...
by1crore projects
 
PDF
An efficient, secure deduplication data storing in cloud storage environment
byeSAT Journals
 
PDF
Cloud-based Data Stream Processing
byZbigniew Jerzak
 
PDF
Secure Mining of Association Rules in Horizontally Distributed Databases
byIJSRD
 
PDF
Privacy Preserving Multi-keyword Top-K Search based on Cosine Similarity Clus...
byIRJET Journal
 
PDF
A Survey on Approaches for Frequent Item Set Mining on Apache Hadoop
byIJTET Journal
 
PPTX
Deploying Linked Open Data: Methodologies and Software Tools
byNikolaos Konstantinou
 
PDF
Integrity Privacy to Public Auditing for Shared Data in Cloud Computing
byIJERA Editor
 
PDF
Turn InSecure And High Speed Intra-Cloud and Inter-Cloud Communication
byRichard Jung
 
PDF
Dq36708711
byIJERA Editor
 
PDF
IRJET- Multiple Keyword Search over Encrypted Cloud Data
byIRJET Journal
 
PDF
A1803010105
byIOSR Journals
 
PPTX
Materializing the Web of Linked Data
byNikolaos Konstantinou
 
PDF
A Novel Additive Order Protocol in Cloud Storage and Avoiding the Trapdoors
byIRJET Journal
 
PDF
Duplicate File Analyzer using N-layer Hash and Hash Table
byAM Publications
 
PDF
A hybrid cloud approach for secure authorized deduplication
byTmks Infotech
 
PDF
DataXDay - Exploring graphs: looking for communities & leaders
byDataXDay Conference by Xebia
 
PDF
An effective classification approach for big data with parallel generalized H...
byriyaniaes
 
Introduction: Linked Data and the Semantic Web
byNikolaos Konstantinou
 
3 ijece 1 ed iqbal qc
byIAESIJEECS
 
Key-Aggregate Searchable Encryption (KASE) for Group Data Sharing via Cloud S...
by1crore projects
 
An efficient, secure deduplication data storing in cloud storage environment
byeSAT Journals
 
Cloud-based Data Stream Processing
byZbigniew Jerzak
 
Secure Mining of Association Rules in Horizontally Distributed Databases
byIJSRD
 
Privacy Preserving Multi-keyword Top-K Search based on Cosine Similarity Clus...
byIRJET Journal
 
A Survey on Approaches for Frequent Item Set Mining on Apache Hadoop
byIJTET Journal
 
Deploying Linked Open Data: Methodologies and Software Tools
byNikolaos Konstantinou
 
Integrity Privacy to Public Auditing for Shared Data in Cloud Computing
byIJERA Editor
 
Turn InSecure And High Speed Intra-Cloud and Inter-Cloud Communication
byRichard Jung
 
Dq36708711
byIJERA Editor
 
IRJET- Multiple Keyword Search over Encrypted Cloud Data
byIRJET Journal
 
A1803010105
byIOSR Journals
 
Materializing the Web of Linked Data
byNikolaos Konstantinou
 
A Novel Additive Order Protocol in Cloud Storage and Avoiding the Trapdoors
byIRJET Journal
 
Duplicate File Analyzer using N-layer Hash and Hash Table
byAM Publications
 
A hybrid cloud approach for secure authorized deduplication
byTmks Infotech
 
DataXDay - Exploring graphs: looking for communities & leaders
byDataXDay Conference by Xebia
 
An effective classification approach for big data with parallel generalized H...
byriyaniaes
 

Similar to Managing data in computational edge clouds

PPTX
Introducing Technologies for Handling Big Data by Jaseela
byStudent
 
PDF
Google Storage concepts and computing concepts.pdf
byashokchoppadandi685
 
PDF
Big data caching for networking : Moving from cloud to edge
byHicham HAMMOUCHI
 
PDF
Dynamic Adaptation of Software-defined Networks for IoT Systems: A Search-bas...
byLionel Briand
 
PPT
Caching in Information Centric Network (ICN)
byLippo Group Digital
 
PPT
Data storage in Cloud computing
byDong Yuan
 
PDF
Systems Bioinformatics Workshop Keynote
byDeepak Singh
 
PPTX
01 introduction to cloud computing technology
byNan Sheng
 
PPT
Computing Outside The Box June 2009
byIan Foster
 
PDF
Research portfolio
byMehdi Bennis
 
PPTX
Converged IT and Data Commons
bySimon Twigger
 
PPT
Waters Grid & HPC Course
byjimliddle
 
PDF
Efficient Cloud Caching
byIJERA Editor
 
PDF
Big data caching for networking
byHicham HAMMOUCHI
 
PPTX
Взгляд на облака с точки зрения HPC
byOlga Lavrentieva
 
PDF
Cloud Computing Big Data Is Future Of It
byAman Ghei
 
PPT
High Performance Computing - Cloud Point of View
byaragozin
 
PDF
北航云计算公开课01 introduction to cloud computing technology
byyhz87
 
PDF
Transport Layer Caching Mechanisms and Optimization
byNestor Michael Tiglao
 
PDF
Applications in the Cloud
byEberhard Wolff
 
Introducing Technologies for Handling Big Data by Jaseela
byStudent
 
Google Storage concepts and computing concepts.pdf
byashokchoppadandi685
 
Big data caching for networking : Moving from cloud to edge
byHicham HAMMOUCHI
 
Dynamic Adaptation of Software-defined Networks for IoT Systems: A Search-bas...
byLionel Briand
 
Caching in Information Centric Network (ICN)
byLippo Group Digital
 
Data storage in Cloud computing
byDong Yuan
 
Systems Bioinformatics Workshop Keynote
byDeepak Singh
 
01 introduction to cloud computing technology
byNan Sheng
 
Computing Outside The Box June 2009
byIan Foster
 
Research portfolio
byMehdi Bennis
 
Converged IT and Data Commons
bySimon Twigger
 
Waters Grid & HPC Course
byjimliddle
 
Efficient Cloud Caching
byIJERA Editor
 
Big data caching for networking
byHicham HAMMOUCHI
 
Взгляд на облака с точки зрения HPC
byOlga Lavrentieva
 
Cloud Computing Big Data Is Future Of It
byAman Ghei
 
High Performance Computing - Cloud Point of View
byaragozin
 
北航云计算公开课01 introduction to cloud computing technology
byyhz87
 
Transport Layer Caching Mechanisms and Optimization
byNestor Michael Tiglao
 
Applications in the Cloud
byEberhard Wolff
 

More from Nitinder Mohan

PDF
Edge Computing Platforms and Protocols - Ph.D. thesis
byNitinder Mohan
 
PDF
Anveshak: Placing Edge Servers In The Wild
byNitinder Mohan
 
PPTX
Edge-Fog Cloud
byNitinder Mohan
 
PPTX
Edge-Fog Cloud: Scaling IoT computations on the edge
byNitinder Mohan
 
PDF
MUTE: Multi-Tier Edge networks
byNitinder Mohan
 
PPTX
ExEC: Elastic Extensible Edge Cloud
byNitinder Mohan
 
PDF
ICON: Intelligent Container Overlays
byNitinder Mohan
 
PPTX
Redesigning MPTCP in Edge clouds
byNitinder Mohan
 
PDF
Open Infrastructure for Edge: A Distributed Ledger Outlook
byNitinder Mohan
 
PDF
Pruning Edge Research with Latency Shears
byNitinder Mohan
 
PDF
DeCloud: Truthful Decentralized Double Auction for Edge Clouds
byNitinder Mohan
 
PDF
Big Data Frameworks: A primer on Apache Spark and MapReduce
byNitinder Mohan
 
Edge Computing Platforms and Protocols - Ph.D. thesis
byNitinder Mohan
 
Anveshak: Placing Edge Servers In The Wild
byNitinder Mohan
 
Edge-Fog Cloud
byNitinder Mohan
 
Edge-Fog Cloud: Scaling IoT computations on the edge
byNitinder Mohan
 
MUTE: Multi-Tier Edge networks
byNitinder Mohan
 
ExEC: Elastic Extensible Edge Cloud
byNitinder Mohan
 
ICON: Intelligent Container Overlays
byNitinder Mohan
 
Redesigning MPTCP in Edge clouds
byNitinder Mohan
 
Open Infrastructure for Edge: A Distributed Ledger Outlook
byNitinder Mohan
 
Pruning Edge Research with Latency Shears
byNitinder Mohan
 
DeCloud: Truthful Decentralized Double Auction for Edge Clouds
byNitinder Mohan
 
Big Data Frameworks: A primer on Apache Spark and MapReduce
byNitinder Mohan
 

Recently uploaded

PDF
1. Doing Academic Research: Problems and Issues, 2. Academic Research Writing...
byProf. Vinod Kumar Kanvaria
 
PDF
Introductions Introductions Introductions
byaidanattema
 
PDF
Phytogeography- species distribution, vegetation in India
byANAKHA JACOB
 
PDF
Unit 1- Basics of Management Cha. of Magmt
byNileshKumbhar21
 
PPTX
Photography Pillar 1 The Subject PowerPoint
bymjb77ny
 
PPTX
G-Protein-Coupled Receptors (GPCRs): Structure, Mechanism, and Functions
byAnoja Kurian
 
PDF
Fundamental of Horticulture _HORTICULTURAL PLANT CLASSIFICATION.pdf
bybisensharad
 
PPTX
Unit 8- Immunochemistry.pptx............
byAkanksha Kotangale
 
PDF
the dynasty history of Chahmanas Early Medieval India
byPrachiSontakke5
 
PDF
Herniation Syndromes - Neuro Imaging Master Series
bySean M. Fox
 
PPTX
An Detailed Overview of Menus in Odoo 18
byCeline George
 
PDF
EVOLUTION - Theories, Pre-Darwinism, Darwinism
byANAKHA JACOB
 
PDF
Approaches and Methods in Political Science -EVOLUTION OF POLITICAL THOUGHT -...
byDr Raja Mohammed T
 
PPTX
TEACHING OF POETRY.pptx. The PPT helps and provides guidance on Lesson Planni...
byDr. Meeta Agrawal
 
PPTX
introduction to clinical pharmacology.pptx
byAneetaSharma15
 
PPTX
Q3 WEEK 1 _ LAYERS OF THE EARTH SCIENCE 8
byRoyoMel
 
PPTX
How to Enter Opening Stock in Odoo 18 Inventory
byCeline George
 
PPTX
Types_of_Volcanoes_Presentation.pptx Science 8 Matatag
byNekoChan623496
 
PPTX
The Ides Of March Throughout History-- not just Caesar but also . . .
byBob Mayer
 
PPTX
2. Classification of Drugs.pptx (Pharmacognosy department)
byBushraDeshpande
 
1. Doing Academic Research: Problems and Issues, 2. Academic Research Writing...
byProf. Vinod Kumar Kanvaria
 
Introductions Introductions Introductions
byaidanattema
 
Phytogeography- species distribution, vegetation in India
byANAKHA JACOB
 
Unit 1- Basics of Management Cha. of Magmt
byNileshKumbhar21
 
Photography Pillar 1 The Subject PowerPoint
bymjb77ny
 
G-Protein-Coupled Receptors (GPCRs): Structure, Mechanism, and Functions
byAnoja Kurian
 
Fundamental of Horticulture _HORTICULTURAL PLANT CLASSIFICATION.pdf
bybisensharad
 
Unit 8- Immunochemistry.pptx............
byAkanksha Kotangale
 
the dynasty history of Chahmanas Early Medieval India
byPrachiSontakke5
 
Herniation Syndromes - Neuro Imaging Master Series
bySean M. Fox
 
An Detailed Overview of Menus in Odoo 18
byCeline George
 
EVOLUTION - Theories, Pre-Darwinism, Darwinism
byANAKHA JACOB
 
Approaches and Methods in Political Science -EVOLUTION OF POLITICAL THOUGHT -...
byDr Raja Mohammed T
 
TEACHING OF POETRY.pptx. The PPT helps and provides guidance on Lesson Planni...
byDr. Meeta Agrawal
 
introduction to clinical pharmacology.pptx
byAneetaSharma15
 
Q3 WEEK 1 _ LAYERS OF THE EARTH SCIENCE 8
byRoyoMel
 
How to Enter Opening Stock in Odoo 18 Inventory
byCeline George
 
Types_of_Volcanoes_Presentation.pptx Science 8 Matatag
byNekoChan623496
 
The Ides Of March Throughout History-- not just Caesar but also . . .
byBob Mayer
 
2. Classification of Drugs.pptx (Pharmacognosy department)
byBushraDeshpande
 

Managing data in computational edge clouds

  • 1.
    Managing Data in ComputationalEdge Clouds ACM SIGCOMM 2017 Workshop on Mobile Edge Communications (MECOMM’2017), Los Angeles Nitinder Mohan✼ Pengyuan Zhou✼ Keerthana Govindaraj△ Jussi Kangasharju✼ ✼University of Helsinki △Robert Bosch GmbH ✼{firstname.lastname@helsinki.fi} △{keerthana.govindaraj@de.bosch.com}
  • 2.
    Rise of EdgeClouds Network DatacenterEdge Server User 2 Edge Cloud: Small-scale server(s) deployed at network edge to compute user data Motivation:
  • 3.
    Rise of EdgeClouds Network DatacenterEdge Server User Edge Cloud: Small-scale server(s) deployed at network edge to compute user data Motivation: üDecreased latency and network traffic 2
  • 4.
    Rise of EdgeClouds Network Edge Cloud: Small-scale server(s) deployed at network edge to compute user data Motivation: üDecreased latency and network traffic üComputing data of local relevance 2
  • 5.
    Edge-Fog Cloud Nitinder Mohanand Jussi Kangasharju. "Edge-fog cloud: A distributed cloud for internet of things computations." Cloudification of the Internet of Things (CIoT). IEEE, 2016. • Voluntary compute devices • 1-2 network hops from users • Ad-hoc network connectivity • Dedicated compute servers • 5-6 network hops from users • Dense network connectivity • Permanent data archival • No compute capability 3
  • 6.
    Data Availability inEdge Cache Query: Give me a route from point A to B without road disruptions 4
  • 7.
    Data Availability inEdge Cache • Edge cloud proposals presume that the required data for processing is pre- available in Edge servers cache • However, Edge server may request Data Store for required data Data Request: Any construction/ accidents on the map? 4
  • 8.
    Data Availability inEdge Cache • Edge cloud proposals presume that the required data for processing is pre- available in Edge servers cache • However, Edge server may request Data Store for required data • The computation is delayed by a round-trip-time to central Data Store • Reduces benefits of employing Edge clouds! Data Request: Any construction/ accidents on the map? Requirement: Caching strategy which predicts and stores required data before computation 4
  • 9.
    Edge Cloud CacheGrouping Edge cloud caching mechanism ØGroup edge resources into cache groups based on their locally cached data ØPredict required data for computation based on cache grouping classification ØProvide coherence on multiple copies of shared data throughout a cache group 5
  • 10.
  • 11.
  • 12.
    Grouping Algorithm Deployment: Select{RC}i resources for computing a task Tc 1. Algorithms such as LPCF* selects a subset from available resources which achieves least processing and network cost for collaboration. 2. Several deployments for different tasks can occur at same time. Task Deployment T1 T2 Nitinder Mohan and Jussi Kangasharju. "Edge-fog cloud: A distributed cloud for internet of things computations." Cloudification of the Internet of Things (CIoT). IEEE, 2016. 8
  • 13.
    Grouping Algorithm 1. Att=0, {RC}i does not have data required for computation and it requests the required data from Data Store 2. The Data Store sends the required data to requesting resources populating their caches Task Deployment Populate Caches 9
  • 14.
    Grouping Algorithm 1. Att=0, {RC}i does not have data required for computation and it requests the required data from Data Store 2. The Data Store sends the required data to requesting resources populating their caches Task Deployment Populate Caches 9
  • 15.
    Grouping Algorithm 1. Thetask Tc is classified in a workload Wk where Wk is a set of similar tasks 2. The resources {RC}i caching data of workload Wk are grouped together as cache group CGk Task Deployment Populate Caches Grouping Classifiers W1 W2 10
  • 16.
    Grouping Algorithm 1. Thedeployment algorithm prefers CGk ∈Wk for task Ti ∈Wk Task Deployment Populate Caches Grouping Classifiers W1 W2 T3 11
  • 17.
    Grouping Algorithm 1. Thedeployment algorithm prefers CGk ∈Wk for task Ti ∈Wk 2. If the resources in group are not appropriate for deployment, the algorithm can select other available resources Task Deployment Populate Caches Grouping Classifiers W1 W2 11
  • 18.
    Grouping Algorithm 1. Thedeployment algorithm prefers CGk ∈Wk for task Ti ∈Wk 2. If the resources in group are not appropriate for deployment, the algorithm can select other available resources 3. These resources then join their respective cache group Task Deployment Populate Caches Grouping Classifiers W1 W2 11
  • 19.
    Grouping Algorithm 1. Thedeployment algorithm prefers CGk ∈Wk for task Ti ∈Wk 2. If the resources in group are not appropriate for deployment, the algorithm can select other available resources 3. These resources then join their respective cache group 4. A resource can be a part of more than one group Task Deployment Populate Caches Grouping Classifiers W1 W2 11
  • 20.
  • 21.
    Requirements 1. Reduce unnecessarynetwork traffic 2. Ensure consistent computation by avoiding stale data copies 13
  • 22.
    Resource Roles 1. GroupMember • Resource (RCx) part of a cache group {CG}i • The group member stores only data of its group workload 14
  • 23.
    Resource Roles 1. GroupMember • Resource (RCx) part of a cache group {CG}i • The group member stores only data of its group workload 2. Group Leader • Representative of a group • Responsible for maintaining coherent data within a group • Has consistent connectivity with group members • Elected via distributed election algorithm 14
  • 24.
    Communication Flow The communicationmodel is strictly pull-based to limit number of messages in the system The model operates over following assumptions: 1. The resources upload their updated data copy to Data Store after every successful computation 2. Data upload is in-parallel to ongoing local computation and does not impact the computation completion time 3. Data Store is persistent and immune to failures 4. Messages in the system are not lost 15
  • 25.
    Cache Data Structures 1.Group member table Maps locally cached data with group leader 2. Group leader table Ensures that group leader has consistent state and address of resource caching data 3. Data Store table Maps data to groups Data Name Leader Address Tag Data Name Leader Address Classifier Type Data Name Resource Address Tag Timestamp 16
  • 26.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario 1: Retrieving free data 17
  • 27.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Scenario 1: Retrieving free data 17
  • 28.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Scenario 1: Retrieving free data 17
  • 29.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Scenario 1: Retrieving free data 17
  • 30.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Scenario 1: Retrieving free data 17
  • 31.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Scenario 1: Retrieving free data 17
  • 32.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’ Scenario 1: Retrieving free data 17
  • 33.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’ Scenario 1: Retrieving free data Content ’X’ 17
  • 34.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Scenario 2: Resource failure 18
  • 35.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Unavailable! Scenario 2: Resource failure 18
  • 36.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Timeout Unavailable! Scenario 2: Resource failure 18
  • 37.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’? Timeout Unavailable! Scenario 2: Resource failure 18
  • 38.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’? Timeout Unavailable! Scenario 2: Resource failure 18
  • 39.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’? Timeout Content ‘X’ Unavailable! Scenario 2: Resource failure 18
  • 40.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? GoTo: ’Resource 1’ Content ‘X’? Content ‘X’? Timeout Content ‘X’ Unavailable! Scenario 2: Resource failure Content ’X’ 18
  • 41.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario 3: Data under update 19
  • 42.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario 3: Data under update Content ‘X’? 19
  • 43.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario 3: Data under update Content ‘X’? 19
  • 44.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Content ‘X’? 19
  • 45.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Content ‘X’? 19
  • 46.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Content ‘X’? 19
  • 47.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Content ‘X’? Free 19
  • 48.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? Free 19
  • 49.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Lock Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? Free 19
  • 50.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? Free Free 19
  • 51.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? Free Free 19
  • 52.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Lock: ’Resource 1’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? GoTo: ’Resource 1’ Free Free 19
  • 53.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Lock: ’Resource 1’ Content ‘X’? Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? GoTo: ’Resource 1’ Free Free 19
  • 54.
    Content Retrival inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Lock Resource 2 Member Table Content ’X’ (stale) Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Content ‘X’? Lock: ’Resource 1’ Content ‘X’? Content ‘X’ Scenario 3: Data under update Time Interval Free: Content ‘X’ Content ‘X’? GoTo: ’Resource 1’ Free Free 19
  • 55.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 20
  • 56.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ 20
  • 57.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ 20
  • 58.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock 20
  • 59.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Lock 20
  • 60.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Free Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Lock 20
  • 61.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Lock Lock 20 Resource 2
  • 62.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Processing Lock Lock 20 Resource 2
  • 63.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Processing Free Lock 20 Resource 2
  • 64.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Processing Free: Content ‘X’ Lock Lock 20 Resource 2
  • 65.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx1 Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Processing Free: Content ‘X’ Lock Free 20 Resource 2
  • 66.
    Content Update inGroup Resource 1 Member Table Content ’X’ Content ‘Y’ Leader A Leader A Free Lock Resource 2 Member Table Content ’X’ Content ‘Q’ Leader A Leader A Free Leader A Leader Table Content ’X’ Content ‘Y’ Tx1 Ty Content ‘Q’ Free Lock Tq Resource 1 Resource 2Content ‘Z’ Leader B Free Data Store Data Store Table Content ’X’ Content ‘Y’ Leader A Leader B Wi WjContent ‘Z’ Content ‘Q’ Leader A Leader A Wi Wi Scenario: Update on Content ‘X’ by Resource 2 Resource 2: Retrieve updated Content ‘X’ Lock: Content ‘X’ Processing Free: Content ‘X’ Content ‘X’ Lock Free 20 Resource 2
  • 67.
    Requirements 1. Reduce unnecessarynetwork traffic Ø Communication model encourages data sharing within a group and removes a single point-of-failure 2. Ensure consistent computation by avoiding stale data copies Ø Model ensures causal coherence on shared data 21
  • 68.
  • 69.
    Evaluation Evaluated on Icarussimulator* Parameters: 1. 320 Edge and Fog nodes + Data Store 2. Fog node acts as group leader 3. 96*104 content items divided in 32 workloads 4. Resource cache size capacity is 10% of overall content size *Lorenzo Saino, Ioannis Psaras, and George Pavlou. "ICARUS: a caching simulator for information centric networking (ICN)." Proceedings of the 7th International ICST conference on Simulation Tools and Techniques. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 2014. 23
  • 70.
    Evaluation Q1. Does resourcegrouping offer any benefits to edge clouds? Q2. If yes, what is the appropriate size of a group? 24
  • 71.
    Q1. Does resourcegrouping offer any benefits to edge clouds? a) Effect on cache hit rate 0.02 0.04 0.06 0.08 0.1 Cache Size 0 0.2 0.4 0.6 0.8 1 CacheHitRate 4-group 4-no_group 32-group 32-no_group 25
  • 72.
    Q1. Does resourcegrouping offer any benefits to edge clouds? a) Effect on cache hit rate 0.02 0.04 0.06 0.08 0.1 Cache Size 0 0.2 0.4 0.6 0.8 1 CacheHitRate 4-group 4-no_group 32-group 32-no_group ≈2 times increase in hit rate 25
  • 73.
    Q1. Does resourcegrouping offer any benefits to edge clouds? b) Effect on latency 0.02 0.04 0.06 0.08 0.1 Cache Size 20 30 40 50 60 70 80 Latency(ms) 4-group 4-no_group 32-group 32-no_group 26
  • 74.
    Q1. Does resourcegrouping offer any benefits to edge clouds? b) Effect on latency 0.02 0.04 0.06 0.08 0.1 Cache Size 20 30 40 50 60 70 80 Latency(ms) 4-group 4-no_group 32-group 32-no_group 45% decrease in latency 26
  • 75.
    Q2. If yes,what is the appropriate size of a group? 0 25 50 75 100 125 150 Number of Groups 0.2 0.4 0.6 0.8 1 CacheHitRate 4 workloads 8 workloads 16 workloads 32 workloads 0 25 50 75 100 125 150 Number of Groups 20 30 40 50 Latency(ms) 4 workloads 8 workloads 16 workloads 32 workloads Maximum gains are achieved when number of cache groups = number of workloads 27
  • 76.
    Conclusion 1. Edge resourcegrouping strategy manages content in Edge caches 2. Strategy classifies content based on resource’s deployed workloads and cached content 3. The 3-step iterative algorithm maps future deployments on resources with required data locally cached 4. Our evaluations via simulation show i. Direct correlation between grouping and increased system performance ii. Number of cache groups and number of assigned workloads 28
  • 77.
    Thank You! Questions? nitinder.mohan@helsinki.fi CleanSky -EU FP7 Marie Curie Initial Training Network
  • 78.
  • 79.
    Industry 4.0 onEdge Industry 4.0 proposes automation in manufacturing 1. Robots such as Bosch APAS can automate tasks and work along side humans while ensuring safety 2. IoT sensor data is used in augmented reality for collaborative support and manufacturing Requirement: Time-critical and fast data processing
  • 80.
    Industry 4.0 onEdge Industry 4.0 proposes automation in manufacturing 1. Robots such as Bosch APAS can automate tasks and work along side humans while ensuring safety 2. IoT sensor data is used in augmented reality for collaborative support and manufacturing Requirement: Time-critical and fast data processing Solution: Implement industrial Edge clouds Network Datacenter Edge Servers
  • 81.
    Grouping Classifier 1. Location:location of place while constructing AR and 3D models 2. Relevance: laptop and cellphone manufacturing tasks as casing 3. Time: sensor data generation and ‘freshness’ time 4. Warning signals: broadcast of information-only or safety-related signals
  • 82.
    Q1. Does resourcegrouping offer any benefits to edge clouds? c) Effect of cache size 0.02 0.04 0.06 0.08 0.1 Cache Size 20 30 40 50 60 70 80 Latency(ms) 4-group 4-no_group 32-group 32-no_group 0.02 0.04 0.06 0.08 0.1 Cache Size 0 0.2 0.4 0.6 0.8 1 CacheHitRate 4-group 4-no_group 32-group 32-no_group Gains decrease as we increase cache size! 1. Grouping works best when cache sizes are limited and cannot store all content 2. With larger caches, resources can locally store all content!
  • 83.
    Q1. Does resourcegrouping offer any benefits to edge clouds? d) Effect of number of workloads 0.02 0.04 0.06 0.08 0.1 Cache Size 20 30 40 50 60 70 80 Latency(ms) 4-group 4-no_group 32-group 32-no_group 0.02 0.04 0.06 0.08 0.1 Cache Size 0 0.2 0.4 0.6 0.8 1 CacheHitRate 4-group 4-no_group 32-group 32-no_group Gains decrease on increasing number of workloads! Ø More workloads ≈ more unique content requests which cannot be satisfied by local caches Workloads is a uniform distribution of similar content requests