Hyrise-R: Scale-out and Hot-Standby through Lazy Master Replication - - PowerPoint PPT Presentation
Hyrise-R: Scale-out and Hot-Standby through Lazy Master Replication for Enterprise Applications David Schwalb, Jan Kossmann, Martin Faust, Stefan Klauck, Matthias Uflacker, Hasso Plattner Hasso Plattner Institute, University of Potsdam, Germany
Hyrise-R: Scale-out and Hot-Standby through Lazy Master Replication for Enterprise Applications
David Schwalb, Jan Kossmann, Martin Faust, Stefan Klauck, Matthias Uflacker, Hasso Plattner Hasso Plattner Institute, University of Potsdam, Germany IMDM 2015
Motivation
New enterprise applications .. □ Growing number of users □ Increasingly complex queries □ Interactive data exploration .. require scalability [1] Scale-up vs. scale-out (+ availability)
Chart 2 Hyrise-R Stefan Klauck
OLXP OLAP, Search and Read-Only Applications
OLTP Master Node Read-Only Replicas Data Entry Operational Reporting & New Applications Customers Sales Managers Decision Support < 1 Second
Related Work
Theoretical replication models and comparison [2] □ Eager vs. lazy □ Group vs. master Implementations □ Postgres-R – Eager group replication based on shadow copies [3] □ ScyPer – Lazy master replication with row layout for master node [4] □ ..
Chart 3 Hyrise-R Stefan Klauck
Hyrise
Storage engine developed at HPI for research and prototyping, initially focused on main memory processing and hybrid storage layouts of tables □ Dictionary and bit-vector compression □ Main/delta architecture with merge process □ Hybrid row and column layouts of tables □ Supports vertical and horizontal partitioning
Chart 4 Hyrise-R Stefan Klauck
2009 2011 2013 2015
A Common Database Approach for OLTP and OLAP MVCC HYRISE- A Main Memory Hybrid Storage Engine TAMEX: A Task-Based Query Execution Framework Merge Process Non volatile Memory Hyrise-NV Replication SGI installation Data Aging SSICLOPS Main Memory Optimized Index Structures Frontend SQL
Hyrise – Research History
Chart 5 Hyrise-R Stefan Klauck
Hyrise-R
Chart 6 Hyrise-R Stefan Klauck
Dispatcher Write workload Read workload Request Handler R Data Storage Cluster Interface Logger Primary Node Request Handler Data Storage Cluster Interface Logger Replica 1 R Request Handler Data Storage Cluster Interface Logger Replica n
R R R
Dispatcher Write workload Read workload Request Handler R Data Storage Cluster Interface Logger Primary Node Request Handler Data Storage Cluster Interface Logger Replica 1 R Request Handler Data Storage Cluster Interface Logger Replica n
R R R
Dispatcher
Redirect queries to cluster nodes □ Transactional workload -> master node □ Reads -> all cluster nodes
Chart 7 Hyrise-R Stefan Klauck
Replication Mechanism
Logs are written to file system + send to cluster interface Cluster interface sends (dictionary encoded) log information to replicas Frequency is configurable and based on □ Number of calls □ Exceeding buffer size □ Time since last transmission TCP with nanomsg □ Survey pattern allows replicas to acknowledge reception □ Heartbeat protocol for failover
Chart 8 Hyrise-R Stefan Klauck
Why Hyrise-R is a good fit for Enterprise Applications
[1]
Chart 9 Hyrise-R Stefan Klauck
OLXP OLAP, Search and Read-Only Applications
OLTP Master Node Read-Only Replicas Data Entry Operational Reporting & New Applications Customers Sales Managers Decision Support < 1 Second
Evaluation
5 machines with □ Intel Xeon E5-2666 v3 (36cCPUs; 10 cores @ 2.9GHz) □ 60 GiB main memory
Chart 10 Hyrise-R Stefan Klauck
Conclusion
Hyrise-R – a system to cluster Hyrise instances using lazy master replication □ Dictionary compressed logs for updating replicas □ Heartbeat protocol for failover □ Benchmarks on Amazon EC2 cluster Future Work □ Extend query dispatching and distribution □ Extend mixed workload measurements (ch-beCHmark)
Chart 11 Hyrise-R Stefan Klauck
References
[1] H. Plattner. The Impact of Columnar In-Memory Databases on Enterprise Systems. 2014 [2] J. Gray, P. Helland, P. O’Neil, and D. Shasha. The dangers of replication and a solution. 1996 [3] B. Kemme and G. Alonso. Don’t be lazy, be consistent: Postgres-r, a new way to implement database replication. 2000 [4] T. Mühlbauer, W. Rödiger, A. Reiser, A. Kemper, and T. Neumann. Scyper: Elastic olap throughput on transactional data. 2013
Chart 12 Hyrise-R Stefan Klauck
Stefan Klauck stefan.klauck@hpi.de http://epic.hpi.de