Hyrise-R: Scale-out and Hot-Standby through Lazy Master Replication - - PowerPoint PPT Presentation

hyrise r scale out and hot standby through lazy master
SMART_READER_LITE
LIVE PREVIEW

Hyrise-R: Scale-out and Hot-Standby through Lazy Master Replication - - PowerPoint PPT Presentation

Hyrise-R: Scale-out and Hot-Standby through Lazy Master Replication for Enterprise Applications David Schwalb, Jan Kossmann, Martin Faust, Stefan Klauck, Matthias Uflacker, Hasso Plattner Hasso Plattner Institute, University of Potsdam, Germany


slide-1
SLIDE 1

Hyrise-R: Scale-out and Hot-Standby through Lazy Master Replication for Enterprise Applications

David Schwalb, Jan Kossmann, Martin Faust, Stefan Klauck, Matthias Uflacker, Hasso Plattner Hasso Plattner Institute, University of Potsdam, Germany IMDM 2015

slide-2
SLIDE 2

Motivation

New enterprise applications .. □ Growing number of users □ Increasingly complex queries □ Interactive data exploration .. require scalability [1] Scale-up vs. scale-out (+ availability)

Chart 2 Hyrise-R Stefan Klauck

OLXP OLAP, Search and Read-Only Applications

  • n Transactional Schema

OLTP Master Node Read-Only Replicas Data Entry Operational Reporting & New Applications Customers Sales Managers Decision Support < 1 Second

slide-3
SLIDE 3

Related Work

Theoretical replication models and comparison [2] □ Eager vs. lazy □ Group vs. master Implementations □ Postgres-R – Eager group replication based on shadow copies [3] □ ScyPer – Lazy master replication with row layout for master node [4] □ ..

Chart 3 Hyrise-R Stefan Klauck

slide-4
SLIDE 4

Hyrise

Storage engine developed at HPI for research and prototyping, initially focused on main memory processing and hybrid storage layouts of tables □ Dictionary and bit-vector compression □ Main/delta architecture with merge process □ Hybrid row and column layouts of tables □ Supports vertical and horizontal partitioning

Chart 4 Hyrise-R Stefan Klauck

slide-5
SLIDE 5

2009 2011 2013 2015

A Common Database Approach for OLTP and OLAP MVCC HYRISE- A Main Memory Hybrid Storage Engine TAMEX: A Task-Based Query Execution Framework Merge Process Non volatile Memory Hyrise-NV Replication SGI installation Data Aging SSICLOPS Main Memory Optimized Index Structures Frontend SQL

Hyrise – Research History

Chart 5 Hyrise-R Stefan Klauck

slide-6
SLIDE 6

Hyrise-R

Chart 6 Hyrise-R Stefan Klauck

Dispatcher Write workload Read workload Request Handler R Data Storage Cluster Interface Logger Primary Node Request Handler Data Storage Cluster Interface Logger Replica 1 R Request Handler Data Storage Cluster Interface Logger Replica n

  • Cluster

R R R

slide-7
SLIDE 7

Dispatcher Write workload Read workload Request Handler R Data Storage Cluster Interface Logger Primary Node Request Handler Data Storage Cluster Interface Logger Replica 1 R Request Handler Data Storage Cluster Interface Logger Replica n

  • Cluster

R R R

Dispatcher

Redirect queries to cluster nodes □ Transactional workload -> master node □ Reads -> all cluster nodes

Chart 7 Hyrise-R Stefan Klauck

slide-8
SLIDE 8 Dispatcher Write workload Read workload Request Handler R Data Storage Cluster Interface Logger Primary Node Request Handler Data Storage Cluster Interface Logger Replica 1 R Request Handler Data Storage Cluster Interface Logger Replica n
  • Cluster
R R R

Replication Mechanism

Logs are written to file system + send to cluster interface Cluster interface sends (dictionary encoded) log information to replicas Frequency is configurable and based on □ Number of calls □ Exceeding buffer size □ Time since last transmission TCP with nanomsg □ Survey pattern allows replicas to acknowledge reception □ Heartbeat protocol for failover

Chart 8 Hyrise-R Stefan Klauck

slide-9
SLIDE 9

Why Hyrise-R is a good fit for Enterprise Applications

[1]

Chart 9 Hyrise-R Stefan Klauck

OLXP OLAP, Search and Read-Only Applications

  • n Transactional Schema

OLTP Master Node Read-Only Replicas Data Entry Operational Reporting & New Applications Customers Sales Managers Decision Support < 1 Second

slide-10
SLIDE 10

Evaluation

  • n Amazon EC2 cluster

5 machines with □ Intel Xeon E5-2666 v3 (36cCPUs; 10 cores @ 2.9GHz) □ 60 GiB main memory

Chart 10 Hyrise-R Stefan Klauck

slide-11
SLIDE 11

Conclusion

Hyrise-R – a system to cluster Hyrise instances using lazy master replication □ Dictionary compressed logs for updating replicas □ Heartbeat protocol for failover □ Benchmarks on Amazon EC2 cluster Future Work □ Extend query dispatching and distribution □ Extend mixed workload measurements (ch-beCHmark)

Chart 11 Hyrise-R Stefan Klauck

slide-12
SLIDE 12

References

[1] H. Plattner. The Impact of Columnar In-Memory Databases on Enterprise Systems. 2014 [2] J. Gray, P. Helland, P. O’Neil, and D. Shasha. The dangers of replication and a solution. 1996 [3] B. Kemme and G. Alonso. Don’t be lazy, be consistent: Postgres-r, a new way to implement database replication. 2000 [4] T. Mühlbauer, W. Rödiger, A. Reiser, A. Kemper, and T. Neumann. Scyper: Elastic olap throughput on transactional data. 2013

Chart 12 Hyrise-R Stefan Klauck

slide-13
SLIDE 13

Thanks

Stefan Klauck stefan.klauck@hpi.de http://epic.hpi.de