Runaway complexity in Big Data And a plan to stop it Nathan Marz - PowerPoint PPT Presentation

Apr 25, 2023 •109 likes •1.33k views

Runaway complexity in Big Data And a plan to stop it Nathan Marz @nathanmarz 1 Agenda Common sources of complexity in data systems Design for a fundamentally better data system What is a data system? A system that manages the storage

Batch views are optimized for the queries they serve
Batch views • Batch-writable from MapReduce • Fast random reads • Examples: ElephantDB, Voldemort
Batch view database No random writes required!
Properties All Batch data view Function ElephantDB is only a few thousand lines of code Simple
Properties All Batch data view Function Scalable
Properties All Batch data view Function Highly available
Properties All Batch data view Function Can be heavily optimized (b/c no random writes)
Properties All Batch data view Function Normalized

Recommend

Partnering with Foster Care and Runaway and Homeless Youth Programs October 1, 2019 October 1,

Partnering with Foster Care and Runaway and Homeless Youth Programs October 1, 2019 October 1, 2019 2 Runaway and Homeless Youth Programs October 1, 2019 3 Runaway and Homeless Youth (RHY) Definitions Runaway youth: a person under 18

529 views • 34 slides

Thermal Runaway Warning signs and preventive actions Presented by Peter DeMar 1 If thermal

Thermal Runaway Warning signs and preventive actions Presented by Peter DeMar 1 If thermal Thermal runaway runaway does not scare you, then you are 2 Thermal runaway Brand new to the stationary battery industry Ignorant of thermal

361 views • 20 slides

Complexity, Big Data Science, and Complexity Introduction Happiness Emergence Universality

Complexity, Big Data Science, and Happiness Complexity, Big Data Science, and Complexity Introduction Happiness Emergence Universality Symmetry Breaking Discrete Days, St. Michaels College, 2011 The Big Theory Revolution: Big Data

912 views • 74 slides

Machine Learning Anders Holst SICS Big Data Analytics Analysis Big Data Big Value Big Data

Machine Learning Anders Holst SICS Big Data Analytics Analysis Big Data Big Value Big Data Analytics Analysis Big Data Big Value Real world Question Data Model Conclusion Machine Learning Use real data to train a model, which can

626 views • 27 slides

Philosophical Ethics Philosophical Ethics an introduction George Matthews 2020 1 / 17 The

Philosophical Ethics Philosophical Ethics an introduction George Matthews 2020 1 / 17 The case of the runaway trolley The case of the runaway trolley You are standing near a switch next a railroad track when you notice a runaway trolley

572 views • 34 slides

Big Data Algorithms with Medical Applications Yixin Chen Outline Challenges to big data

Big Data Algorithms with Medical Applications Yixin Chen Outline Challenges to big data algorithms Clinical Big Data Our new algorithms Small data vs. Big data Small data vs. Big data VS Small data vs. Big

924 views • 57 slides

CS535 Big Data 1/22/2020 Sangmi Lee Pallickara CS535 Big Data | Computer Science Department

CS535 Big Data 1/22/2020 Sangmi Lee Pallickara CS535 Big Data | Computer Science Department | Colorado State University CS535 BIG DATA PART A. BIG DATA TECHNOLOGY 1. INTRODUCTION TO BIG DATA What is Big Data? Sangmi Lee Pallickara

569 views • 7 slides

COMP9313: Big Data Management Introduction to Big Data Management What is big data? Tweeted by

COMP9313: Big Data Management Introduction to Big Data Management What is big data? Tweeted by Prof. Dan Ariely, Duke University 2 What is big data? No standard definition! Wikipedia: Big data is a field that treats ways to

1.47k views • 53 slides

The DB Industry RDBMSs are a runaway success of simple theoretical ideas Big 3 Database

CS 5614: Database Management Systems 5 The DB Industry RDBMSs are a runaway success of simple theoretical ideas Big 3 Database companies are among the largest in the s/w industry Oracle, Informix, Sybase Other Market Forces: Microsoft,

688 views • 31 slides

Computability and Complexity Lecture 9 Big-O and small-o notation Time complexity class TIME( t (

Computability and Complexity Lecture 9 Big-O and small-o notation Time complexity class TIME( t ( n )) The class P and examples given by Jiri Srba Lecture 9 Computability and Complexity 1/15 Complexity Theory Question Assume that a problem

526 views • 15 slides

Hans Vangheluwe Modelling and Simulation Causes of Complexity Dealing with Complexity

Modelling and Simulation Causes of Complexity Dealing with Complexity Multi-Paradigm Modelling Modelling and Simulation to tackle Complexity Hans Vangheluwe Modelling and Simulation Causes of Complexity Dealing with Complexity

567 views • 39 slides

Hans Vangheluwe Modelling and Simulation Causes of Complexity Dealing with Complexity

607 views • 46 slides

Background Background Text Complexity Text Complexity Text Complexity Sowmya V.B., Sowmya

On Measures of On Measures of On Measures of Background Background Text Complexity Text Complexity Text Complexity Sowmya V.B., Sowmya V.B., Sowmya V.B., What do we mean by measures of text complexity ? Why would anyone want to do

45 views • 3 slides

Kolmogorov Complexity of Categories Complexity Programing Language Kolmogorov Noson S.

Kolmogorov Complexity of Categories Noson S. Yanofsky Kolmogorov Kolmogorov Complexity of Categories Complexity Programing Language Kolmogorov Noson S. Yanofsky Complexity of Categories Complexity Brooklyn College, CUNY Computability

952 views • 42 slides

IN 5210 Complexity Theory Complexity Complexity: Socio-technical (Internet, globalization)

IN 5210 Complexity Theory Complexity Complexity: Socio-technical (Internet, globalization) Complexity (-ies) = Number of types of components*number of types of links*speed of change Key issues: emergence, side-effects (=history),

822 views • 32 slides

Communication Complexity Lecture 23 Computing with remote inputs 1 Communication Complexity

Communication Complexity Lecture 23 Computing with remote inputs 1 Communication Complexity 2 Communication Complexity Setting 2 Communication Complexity Setting Alice wants to compute f(x,y) 2 Communication Complexity Setting

1.58k views • 105 slides

Lecture 10 Seasonal Arima 10/05/2018 1 Australian Wine Sales Example (Lecture 6) Australian

Lecture 10 Seasonal Arima 10/05/2018 1 Australian Wine Sales Example (Lecture 6) Australian total wine sales by wine makers in bottles <= 1 litre. Jan 1980 2 Aug 1994. wineind 40000 30000 20000 1980 1985 1990 1995 0.6 0.6

632 views • 38 slides

present with Google Trends - Hyunyoung Choi - Hal Varian Outline Problem Statement Goal

Predicting the present with Google Trends - Hyunyoung Choi - Hal Varian Outline Problem Statement Goal Methodology Analysis and Forecasting Evaluation Applications and Examples Summary and Future work Problem

649 views • 28 slides

Section 4.1: Time Series I Jared S. Murray The University of Texas at Austin McCombs School of

Section 4.1: Time Series I Jared S. Murray The University of Texas at Austin McCombs School of Business 1 Time Series Data and Dependence Time-series data are simply a collection of observations gathered over time. For example, suppose y 1 . .

558 views • 44 slides

On Deploying Property- Preserving Encryp4on Paul Grubbs Cornell

On Deploying Property- Preserving Encryp4on Paul Grubbs Cornell University/Skyhigh Networks Outline Encryp4on that trades leakage for func4onality Look at

676 views • 23 slides

Fundamentals of Programming Session 24 Instructor: Reza Entezari-Maleki Email:

Fundamentals of Programming Session 24 Instructor: Reza Entezari-Maleki Email: entezari@ce.sharif.edu 1 Fall 2013 These slides have been created using Deitels slides Sharif University of Technology Outlines Data Members, set Functions

831 views • 35 slides

Aggregations Wednesday, February 22, 2017 Agenda Announcements Reading Quiz

Aggregations Wednesday, February 22, 2017 Agenda Announcements Reading Quiz Aggregations Discussion 2 Practice Problems Group Bys Discussion 2 Practice Problems Announcements Heads-up on Lab 3

459 views • 24 slides

Stored Procedures and Functions Rose-Hulman Institute of Technology Curt Clifton Outline

Stored Procedures and Functions Rose-Hulman Institute of Technology Curt Clifton Outline Stored Procedures or Sprocs Functions Statements Reference Defining Stored Procedures Named Collections of Transact-SQL Statements

514 views • 33 slides

61A Extra Lecture 13 Announcements Prediction Regression 4 Regression Given a set of (x, y)

61A Extra Lecture 13 Announcements Prediction Regression 4 Regression Given a set of (x, y) pairs, find a function f(x) that returns good y values 4 Regression Given a set of (x, y) pairs, find a function f(x) that returns good y values

764 views • 41 slides

Download document

More recommend

Explore More Topics

Stay informed with curated content and fresh updates.

animals pets art culture automotive transportation business finance computer internet construction architecture education-career electronics communication

Runaway complexity in Big Data And a plan to stop it Nathan Marz - PowerPoint PPT Presentation

Runaway complexity in Big Data And a plan to stop it Nathan Marz @nathanmarz 1 Agenda Common sources of complexity in data systems Design for a fundamentally better data system What is a data system? A system that manages the storage

Partnering with Foster Care and Runaway and Homeless Youth Programs October 1, 2019 October 1,

Thermal Runaway Warning signs and preventive actions Presented by Peter DeMar 1 If thermal

Complexity, Big Data Science, and Complexity Introduction Happiness Emergence Universality

Machine Learning Anders Holst SICS Big Data Analytics Analysis Big Data Big Value Big Data

Philosophical Ethics Philosophical Ethics an introduction George Matthews 2020 1 / 17 The

Big Data Algorithms with Medical Applications Yixin Chen Outline Challenges to big data

CS535 Big Data 1/22/2020 Sangmi Lee Pallickara CS535 Big Data | Computer Science Department

COMP9313: Big Data Management Introduction to Big Data Management What is big data? Tweeted by

The DB Industry RDBMSs are a runaway success of simple theoretical ideas Big 3 Database

Computability and Complexity Lecture 9 Big-O and small-o notation Time complexity class TIME( t (

Hans Vangheluwe Modelling and Simulation Causes of Complexity Dealing with Complexity

Hans Vangheluwe Modelling and Simulation Causes of Complexity Dealing with Complexity

Background Background Text Complexity Text Complexity Text Complexity Sowmya V.B., Sowmya

Kolmogorov Complexity of Categories Complexity Programing Language Kolmogorov Noson S.

IN 5210 Complexity Theory Complexity Complexity: Socio-technical (Internet, globalization)

Communication Complexity Lecture 23 Computing with remote inputs 1 Communication Complexity

Lecture 10 Seasonal Arima 10/05/2018 1 Australian Wine Sales Example (Lecture 6) Australian

present with Google Trends - Hyunyoung Choi - Hal Varian Outline Problem Statement Goal

Section 4.1: Time Series I Jared S. Murray The University of Texas at Austin McCombs School of

On Deploying Property- Preserving Encryp4on Paul Grubbs Cornell

Fundamentals of Programming Session 24 Instructor: Reza Entezari-Maleki Email:

Aggregations Wednesday, February 22, 2017 Agenda Announcements Reading Quiz

Stored Procedures and Functions Rose-Hulman Institute of Technology Curt Clifton Outline

61A Extra Lecture 13 Announcements Prediction Regression 4 Regression Given a set of (x, y)

Explore More Topics

Sambuz

Useful Links

Newsletter

Mail Us