Cost-based Memory Partitioning and Management in Memcached Damiano - PowerPoint PPT Presentation

Cost-based Memory Partitioning and Management in Memcached Damiano Carra Pietro Michiardi Computer Science Department Network and Security Department University of Verona Eurecom – Sophia Antipolis

Background Design Evaluation Conclusions Outline Background Design Evaluation Conclusions 2

Background Design Evaluation Conclusions Outline Background Design Evaluation Conclusions 3

Background Design Evaluation Conclusions Memcached ! Key-value store – Simple APIs: set , get , delete , … ! Common component of Web architectures – Caching layer ! All data is kept in RAM – Fast response time 4

Background Design Evaluation Conclusions Memory management: Classes and Slabs ! Memcached divides the objects into classes – Based on their sizes ! Example: 3 classes of objects – Small – Medium – Large 5

Background Design Evaluation Conclusions Memory management: Classes and Slabs ! Memcached divides the ! Memory is divided into objects into classes blocks called slabs – Based on their sizes – Default size of a slab: 1 MB ! Example: 3 classes of ! A slab contains a variable objects number of objects – Small – Many small ones – Medium – Some medium – Large – Few large 6

Background Design Evaluation Conclusions Slabs are assigned to classes upon request available memory (divided into slabs) 7

Background Design Evaluation Conclusions Slabs are assigned to classes upon request available memory (divided into slabs) set( ) ! 8

Background Design Evaluation Conclusions Slabs are assigned to classes upon request available memory (divided into slabs) set( ) ! set( ) ! 9

Background Design Evaluation Conclusions Slabs are assigned to classes upon request available memory (divided into slabs) set( ) ! set( ) ! set( ) ! 10

Background Design Evaluation Conclusions Slabs are assigned to classes upon request available memory (divided into slabs) set( ) ! set( ) ! set( ) ! set( ) ! 11

Background Design Evaluation Conclusions Slabs are assigned to classes upon request ! After many requests… available memory (divided into slabs) ! What happens when the cache receives a new ? set( ) ! 12

Background Design Evaluation Conclusions Slabs are assigned to classes upon request ! After many requests… available memory (divided into slabs) ! What happens when the cache receives a new ? set( ) ! – LRU eviction within the class 13

Background Design Evaluation Conclusions Key challenges ! How the memory can be divided among the classes? – Static vs dynamic assignment ! What are the criteria used to assign the memory – Hit-ratio vs cost-hit-ratio 14

Background Design Evaluation Conclusions Outline Background Design Evaluation Conclusions 15

Background Design Evaluation Conclusions Miss-ratio curves (MRC) ! Given a trace and an eviction policy, we can 1 compute the MRC 0.95 Miss Ratio (%) 0.9 – What would be the miss ratio if the class had that 0.85 specific memory assigned? 0.8 Class 12 0.75 ! MRC can be found with a Class 8 Class 5 single pass of the traces 0.7 0 200 400 600 800 1000 – Mattson stack algorithm Memory Size (MB) 16

Background Design Evaluation Conclusions Optimal offline slab assignment ! Goal " Minimize the miss ratio ! Assumptions – MRC are concave – Memory is divided into finite number of blocks 17

Background Design Evaluation Conclusions Online slab assignment ! The offline optimal slab assignment can be used as a reference – It can be computed once we have the whole trace ! We provide a heuristic for the online assignment ! Before showing the heuristic, we consider the object costs 18

Background Design Evaluation Conclusions A new interface ! The interface set(k,v) implies that all the objects have the same cost or weight ! But some objects can be more difficult to obtain – E.g., simply because they are larger – or because it takes more time to retrieve them from the DB • complex query ! We have added a new interface: set(k,v,c) – The application can associate a cost (or weight) to the object 19

Background Design Evaluation Conclusions Slab Allocation Scheme (SAS) ! We define an observation epoch – E.g., when the system has experienced M misses ! During each epoch, we collect, for each classes: – The number of (weighted) misses – The number of (weighted) requests ! High number of misses " The class is suffering ! Low number of miss ratio per assigned slab " The class can lose a slab 20

Background Design Evaluation Conclusions Outline Background Design Evaluation Conclusions 21

Background Design Evaluation Conclusions Settings ! We use a representative Web architecture – Application server, cache, database ! Trace driven experiment – Traces collected by a major CDN operator – The objects in the trace are stored in the database – The cost of retrieving the objects is stored in the application server – The sequence of the requests is issued by a client 22

Background Design Evaluation Conclusions About the traces ! The costs of the objects are not correlated with the sizes – Correlation coefficient # 0.013 10 4 10 4 Number of requests Number of requests 10 3 10 3 Object cost Object cost 10 2 10 2 10 1 10 1 10 0 10 0 10 0 10 1 10 2 10 3 10 4 10 5 10 6 10 7 10 0 10 1 10 2 10 3 10 4 10 5 10 6 Object rank Object size (byte) 23

Background Design Evaluation Conclusions Results ! We compute the optimal cost hit ratio with the 100 offline algorithm 90 optimal ! Our scheme Cost hit ratio (%) 80 progressively adapts the slab assignment 70 – It converges to the 60 optimal 50 SAS – While Memcached (static Memcached 40 assignment) is far from 10 20 30 40 50 60 70 80 90 100 optimal Requests (%) 24

Background Design Evaluation Conclusions Results: slab assignment ! Snapshot taken when half of the requests has been processed by the client Optimal SAS ! Comparison with the Number of slabs 100 Memcached – Optimal assignment (offline) 10 – Static assignement (Memecached) 1 5 10 15 20 25 30 Class ID 25

Background Design Evaluation Conclusions Results: Different Costs ! All objects have the same ! The cost of the object is cost given by its size 100 100 optimal 90 90 optimal Byte hit ratio (%) Hit ratio (%) 80 80 70 70 60 60 50 SAS 50 SAS Memcached Memcached 40 40 10 20 30 40 50 60 70 80 90 100 10 20 30 40 50 60 70 80 90 100 Requests (%) Requests (%) 26

Background Design Evaluation Conclusions Outline Background Testbed Results Conclusions 27

Background Design Evaluation Conclusions Discussion ! Synthetic traces – We tested different traces derived from the literature ! Calcification – What happens if the statistical properties of the requested objects change? – The dynamic nature of the scheme solves a problem known in the literature as calcification 28

Background Design Evaluation Conclusions Conclusions ! Memcached: in-memory key value store largely adopted… – Facebook, Twitter ! .. with a specific memory allocation scheme – Slabs, Classes, LRU within the same class ! We provide a scheme for dynamically assigning the slabs to the different classes – Trying to minimize the number of (weighted) misses – Many solutions proposed in the literature not applicable to Memcached due to its memory allocation scheme 29

Thanks!