Ask and You Shall Receive: Empirically Evaluating Declarative - - PowerPoint PPT Presentation

▶

Apr 16, 2023 354 likes •1.35k views

Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps William F . Jones and Gregory M. Kapfhammer Allegheny College http://www.cs.allegheny.edu/ gkapfham/ 20th International

SLIDE 1

Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

William F . Jones and Gregory M. Kapfhammer Allegheny College

http://www.cs.allegheny.edu/∼gkapfham/

20th International Conference on Software Engineering and Data Engineering, June 20 - 22, 2011

SLIDE 2

Introduction Query Methods Empirical Study Conclusion

Overview of the Presentation

Finding Data in Unstructured Heaps

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 3

Introduction Query Methods Empirical Study Conclusion

Overview of the Presentation

Finding Data in Unstructured Heaps Challenges

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 4

Introduction Query Methods Empirical Study Conclusion

Overview of the Presentation

Finding Data in Unstructured Heaps Challenges Solutions

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 5

Introduction Query Methods Empirical Study Conclusion

Overview of the Presentation

Finding Data in Unstructured Heaps Challenges Solutions Benchmarking Framework

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 6

Introduction Query Methods Empirical Study Conclusion

Overview of the Presentation

Finding Data in Unstructured Heaps Challenges Solutions Benchmarking Framework JQL: Java Query Language

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 7

Introduction Query Methods Empirical Study Conclusion

Overview of the Presentation

Finding Data in Unstructured Heaps Challenges Solutions Benchmarking Framework JQL: Java Query Language JoSQL: Java Objects Struc- tured Query Language

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 8

Introduction Query Methods Empirical Study Conclusion

Overview of the Presentation

Finding Data in Unstructured Heaps Challenges Solutions Benchmarking Framework JQL: Java Query Language JoSQL: Java Objects Struc- tured Query Language HC: Hand Coded Iterative Methods

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 9

Introduction Query Methods Empirical Study Conclusion

Overview of the Presentation

Finding Data in Unstructured Heaps Challenges Solutions Benchmarking Framework JQL: Java Query Language JoSQL: Java Objects Struc- tured Query Language HC: Hand Coded Iterative Methods Comprehensive Empirical Study

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 10

Introduction Query Methods Empirical Study Conclusion

Overview of the Presentation

Finding Data in Unstructured Heaps Challenges Solutions Benchmarking Framework JQL: Java Query Language JoSQL: Java Objects Struc- tured Query Language HC: Hand Coded Iterative Methods Comprehensive Empirical Study Experiments Reveal Trade-offs in Performance and Overall Viability

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 11

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

The unstructured heap in a Java virtual machine stores

bjects that are connected in

complex and unpredictable ways (Xu and Rountev, ICSE 2008)

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 12

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

The unstructured heap in a Java virtual machine stores

bjects that are connected in

complex and unpredictable ways (Xu and Rountev, ICSE 2008) When is an Object Allocated to the Heap? LinkedList list = new LinkedList()

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 13

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

The unstructured heap in a Java virtual machine stores

bjects that are connected in

complex and unpredictable ways (Xu and Rountev, ICSE 2008) When is an Object Allocated to the Heap? LinkedList list = new LinkedList() Let’s Allocate Some Objects to the Heap!

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 14

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

uk−1 uk uk+1 wk−1 B wk B wk+1 B · · · xk−1 A xk A xk+1 · · · vk−1 H vk H vk+1 H zk−1 zk zk+1

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 15

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

uk−1 uk uk+1 wk−1 B wk B wk+1 B · · · xk−1 A xk A xk+1 · · · vk−1 H vk H vk+1 H zk−1 zk zk+1

S00 T01 S01 T02 S02 T03 S03 T04 S04 T05 S05 T06 S06 T07 S07 T08 S08 T09.0 T09.1 S09 T10 S10 T11 S11 T12 S12 T13 S13 T14 S14 T15 S15 T16 S16 T17 S17 T18 S18 T19 S19 T20 S20 T21 S21 T22.0 T22.1 S22

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 16

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

uk−1 uk uk+1 wk−1 B wk B wk+1 B · · · xk−1 A xk A xk+1 · · · vk−1 H vk H vk+1 H zk−1 zk zk+1

S00 T01 S01 T02 S02 T03 S03 T04 S04 T05 S05 T06 S06 T07 S07 T08 S08 T09.0 T09.1 S09 T10 S10 T11 S11 T12 S12 T13 S13 T14 S14 T15 S15 T16 S16 T17 S17 T18 S18 T19 S19 T20 S20 T21 S21 T22.0 T22.1 S22

55 119 5 238 94 ∅

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 17

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

uk−1 uk uk+1 wk−1 B wk B wk+1 B · · · xk−1 A xk A xk+1 · · · vk−1 H vk H vk+1 H zk−1 zk zk+1

S00 T01 S01 T02 S02 T03 S03 T04 S04 T05 S05 T06 S06 T07 S07 T08 S08 T09.0 T09.1 S09 T10 S10 T11 S11 T12 S12 T13 S13 T14 S14 T15 S15 T16 S16 T17 S17 T18 S18 T19 S19 T20 S20 T21 S21 T22.0 T22.1 S22

55 119 5 238 94 ∅ Root ... 22 5 19

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 18

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

uk−1 uk uk+1 wk−1 B wk B wk+1 B · · · xk−1 A xk A xk+1 · · · vk−1 H vk H vk+1 H zk−1 zk zk+1

S00 T01 S01 T02 S02 T03 S03 T04 S04 T05 S05 T06 S06 T07 S07 T08 S08 T09.0 T09.1 S09 T10 S10 T11 S11 T12 S12 T13 S13 T14 S14 T15 S15 T16 S16 T17 S17 T18 S18 T19 S19 T20 S20 T21 S21 T22.0 T22.1 S22

55 119 5 238 94 ∅ Root ... 22 5 19 Start 75 ...

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 19

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

uk−1 uk uk+1 wk−1 B wk B wk+1 B · · · xk−1 A xk A xk+1 · · · vk−1 H vk H vk+1 H zk−1 zk zk+1

S00 T01 S01 T02 S02 T03 S03 T04 S04 T05 S05 T06 S06 T07 S07 T08 S08 T09.0 T09.1 S09 T10 S10 T11 S11 T12 S12 T13 S13 T14 S14 T15 S15 T16 S16 T17 S17 T18 S18 T19 S19 T20 S20 T21 S21 T22.0 T22.1 S22

55 119 5 238 94 ∅ Root ... 22 5 19 Start 75 ... LinkedList Node(s) with Values Greater Than Those in the Trees

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 20

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

uk−1 uk uk+1 wk−1 B wk B wk+1 B · · · xk−1 A xk A xk+1 · · · vk−1 H vk H vk+1 H zk−1 zk zk+1

S00 T01 S01 T02 S02 T03 S03 T04 S04 T05 S05 T06 S06 T07 S07 T08 S08 T09.0 T09.1 S09 T10 S10 T11 S11 T12 S12 T13 S13 T14 S14 T15 S15 T16 S16 T17 S17 T18 S18 T19 S19 T20 S20 T21 S21 T22.0 T22.1 S22

55 119 5 238 94 ∅ Root ... 22 5 19 Start 75 ... LinkedList Node(s) with Values Greater Than Those in the Trees 238

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 21

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

uk−1 uk uk+1 wk−1 B wk B wk+1 B · · · xk−1 A xk A xk+1 · · · vk−1 H vk H vk+1 H zk−1 zk zk+1

S00 T01 S01 T02 S02 T03 S03 T04 S04 T05 S05 T06 S06 T07 S07 T08 S08 T09.0 T09.1 S09 T10 S10 T11 S11 T12 S12 T13 S13 T14 S14 T15 S15 T16 S16 T17 S17 T18 S18 T19 S19 T20 S20 T21 S21 T22.0 T22.1 S22

55 119 5 238 94 ∅ Root ... 22 5 19 Start 75 ... 238 How Do We Find These Nodes?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 22

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

uk−1 uk uk+1 wk−1 B wk B wk+1 B · · · xk−1 A xk A xk+1 · · · vk−1 H vk H vk+1 H zk−1 zk zk+1

S00 T01 S01 T02 S02 T03 S03 T04 S04 T05 S05 T06 S06 T07 S07 T08 S08 T09.0 T09.1 S09 T10 S10 T11 S11 T12 S12 T13 S13 T14 S14 T15 S15 T16 S16 T17 S17 T18 S18 T19 S19 T20 S20 T21 S21 T22.0 T22.1 S22

55 119 5 238 94 ∅ Root ... 22 5 19 Start 75 ... 238 How Do We Find These Nodes? Imperative - Give the Procedure

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 23

Introduction Query Methods Empirical Study Conclusion

Correctly and Efficiently Finding Objects in the Heap

uk−1 uk uk+1 wk−1 B wk B wk+1 B · · · xk−1 A xk A xk+1 · · · vk−1 H vk H vk+1 H zk−1 zk zk+1

S00 T01 S01 T02 S02 T03 S03 T04 S04 T05 S05 T06 S06 T07 S07 T08 S08 T09.0 T09.1 S09 T10 S10 T11 S11 T12 S12 T13 S13 T14 S14 T15 S15 T16 S16 T17 S17 T18 S18 T19 S19 T20 S20 T21 S21 T22.0 T22.1 S22

55 119 5 238 94 ∅ Root ... 22 5 19 Start 75 ... 238 How Do We Find These Nodes? Declarative - Give the Specification

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 24

Introduction Query Methods Empirical Study Conclusion

Object Query Languages and Bicycles

Efficiency - Bicycle: Low wind resistance and time to destination

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 25

Introduction Query Methods Empirical Study Conclusion

Object Query Languages and Bicycles

Efficiency - Query: Minimal space overhead and a low response time

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 26

Introduction Query Methods Empirical Study Conclusion

Object Query Languages and Bicycles

Effectiveness - Bicycle: Transports all item(s) with no break downs

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 27

Introduction Query Methods Empirical Study Conclusion

Object Query Languages and Bicycles

Effectiveness - Query: Always returns the correct result(s) to a query

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 28

Introduction Query Methods Empirical Study Conclusion

Object Query Languages and Bicycles

Cost - Bicycle: Frame material(s) and components cause price to vary

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 29

Introduction Query Methods Empirical Study Conclusion

Object Query Languages and Bicycles

Cost - Query: Must consider installation and development challenges

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 30

Introduction Query Methods Empirical Study Conclusion

JQL: Java Query Language

JQL Compiler

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 31

Introduction Query Methods Empirical Study Conclusion

JQL: Java Query Language

JQL Compiler JQL File

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 32

Introduction Query Methods Empirical Study Conclusion

JQL: Java Query Language

JQL Compiler JQL File Java Source Code

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 33

Introduction Query Methods Empirical Study Conclusion

JQL: Java Query Language

JQL Compiler JQL File Java Source Code Java Compiler

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 34

Introduction Query Methods Empirical Study Conclusion

JQL: Java Query Language

JQL Compiler JQL File Java Source Code Java Compiler Java Bytecodes

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 35

Introduction Query Methods Empirical Study Conclusion

JQL: Java Query Language

JQL Compiler JQL File Java Source Code Java Compiler Java Bytecodes Collection

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 36

Introduction Query Methods Empirical Study Conclusion

JQL: Java Query Language

JQL Compiler JQL File Java Source Code Java Compiler Java Bytecodes Collection Query Executor

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 37

Introduction Query Methods Empirical Study Conclusion

JQL: Java Query Language

JQL Compiler JQL File Java Source Code Java Compiler Java Bytecodes Collection Query Executor Results Cache

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 38

Introduction Query Methods Empirical Study Conclusion

JQL: Java Query Language

JQL Compiler JQL File Java Source Code Java Compiler Java Bytecodes Collection Query Executor Results Cache Features Pre-compilation AOP with AspectJ Method Queries Caching Optimizations

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 39

Introduction Query Methods Empirical Study Conclusion

JQL: Java Query Language

JQL Compiler JQL File Java Source Code Java Compiler Java Bytecodes Collection Query Executor Results Cache Features Pre-compilation AOP with AspectJ Method Queries Caching Optimizations References (Willis et al. ECOOP 2006) (Willis et al. OOPSLA 2008)

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 40

Introduction Query Methods Empirical Study Conclusion

JoSQL: Java Objects Structured Query Language

Parse SQL

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 41

Introduction Query Methods Empirical Study Conclusion

JoSQL: Java Objects Structured Query Language

Parse SQL SQL String Query Object

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 42

Introduction Query Methods Empirical Study Conclusion

JoSQL: Java Objects Structured Query Language

Parse SQL SQL String Query Object Executable Query Collection

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 43

Introduction Query Methods Empirical Study Conclusion

JoSQL: Java Objects Structured Query Language

Parse SQL SQL String Query Object Executable Query Collection Query Executor

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 44

Introduction Query Methods Empirical Study Conclusion

JoSQL: Java Objects Structured Query Language

Parse SQL SQL String Query Object Executable Query Collection Query Executor Query Results

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 45

Introduction Query Methods Empirical Study Conclusion

JoSQL: Java Objects Structured Query Language

Parse SQL SQL String Query Object Executable Query Collection Query Executor Query Results Features SQL Statements String Parsing Java Reflection Query Facilities

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 46

Introduction Query Methods Empirical Study Conclusion

JoSQL: Java Objects Structured Query Language

Parse SQL SQL String Query Object Executable Query Collection Query Executor Query Results Features SQL Statements String Parsing Java Reflection Query Facilities Reference http://josql.sf.net

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 47

Introduction Query Methods Empirical Study Conclusion

Comparison of Data Finding Methods

As the number of collections and

bjects increases, imperative

programming may lead to ap- plications that are complicated, error-prone, and hard to maintain (Xu and Rountev, ICSE 2008)

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 48

Introduction Query Methods Empirical Study Conclusion

Comparison of Data Finding Methods

As the number of collections and

bjects increases, imperative

programming may lead to ap- plications that are complicated, error-prone, and hard to maintain (Xu and Rountev, ICSE 2008) JQL: Compile-Time

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 49

Introduction Query Methods Empirical Study Conclusion

Comparison of Data Finding Methods

As the number of collections and

bjects increases, imperative

programming may lead to ap- plications that are complicated, error-prone, and hard to maintain (Xu and Rountev, ICSE 2008) JQL: Compile-Time JoSQL: Run-Time

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 50

Introduction Query Methods Empirical Study Conclusion

Comparison of Data Finding Methods

As the number of collections and

bjects increases, imperative

programming may lead to ap- plications that are complicated, error-prone, and hard to maintain (Xu and Rountev, ICSE 2008) JQL: Compile-Time JoSQL: Run-Time Performance Trade-Offs?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 51

Introduction Query Methods Empirical Study Conclusion

Comparison of Data Finding Methods

As the number of collections and

bjects increases, imperative

programming may lead to ap- plications that are complicated, error-prone, and hard to maintain (Xu and Rountev, ICSE 2008) JQL: Compile-Time JoSQL: Run-Time Performance Trade-Offs? Effectiveness Concerns?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 52

Introduction Query Methods Empirical Study Conclusion

Comparison of Data Finding Methods

As the number of collections and

bjects increases, imperative

programming may lead to ap- plications that are complicated, error-prone, and hard to maintain (Xu and Rountev, ICSE 2008) JQL: Compile-Time JoSQL: Run-Time Performance Trade-Offs? Effectiveness Concerns? Benchmarking Framework Helps to Answer These Questions

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 53

Introduction Query Methods Empirical Study Conclusion

Benchmarking Framework to Evaluate Query Methods

Random Collection Generator

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 54

Introduction Query Methods Empirical Study Conclusion

Benchmarking Framework to Evaluate Query Methods

Random Collection Generator Benchmark Initializer

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 55

Introduction Query Methods Empirical Study Conclusion

Benchmarking Framework to Evaluate Query Methods

Random Collection Generator Benchmark Initializer Configuration

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 56

Introduction Query Methods Empirical Study Conclusion

Benchmarking Framework to Evaluate Query Methods

Random Collection Generator Benchmark Initializer Configuration Collection Benchmark

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 57

Introduction Query Methods Empirical Study Conclusion

Benchmarking Framework to Evaluate Query Methods

Random Collection Generator Benchmark Initializer Configuration Collection Benchmark Benchmark Executor Evaluation Report

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 58

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Explored a wide variety of benchmark configurations

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 59

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods What operations do we run to evaluate the query methods?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 60

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods

Parameter Values

What operations do we run to evaluate the query methods?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 61

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods

Parameter Values Query Join

Sub-Query Others What operations do we run to evaluate the query methods?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 62

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods What objects will we allocate to the JVM’s heap?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 63

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods

Parameter Values

What objects will we allocate to the JVM’s heap?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 64

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods

Parameter Values Integers Strings

Graphs Complex Objects What objects will we allocate to the JVM’s heap?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 65

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods How big should we make the objects and the collections?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 66

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods

Parameter Values

How big should we make the objects and the collections?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 67

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods

Parameter Values Small Medium

Large How big should we make the objects and the collections?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 68

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Which methods should be part of the framework?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 69

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods

Parameter Values

Which methods should be part of the framework?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 70

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods

Parameter Values JQL JoSQL

HC HC-HJ Which methods should be part of the framework?

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 71

Introduction Query Methods Empirical Study Conclusion

Configuration of the Benchmarking Framework

Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods Operations Possible Configurations Objects Sizes Methods

Parameter Values JQL JoSQL

HC HC-HJ See the paper for further operator and configuration details

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 72

Introduction Query Methods Empirical Study Conclusion

Analysis Techniques: Regression Tree Models

Method: HC-HJ, JQL

Tree Models: Recursive partitioning creates hierarchical view of data

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 73

Introduction Query Methods Empirical Study Conclusion

Analysis Techniques: Regression Tree Models

Method: HC-HJ, JQL 247.40 CollectionSize < 2250

Tree Models: Recursive partitioning creates hierarchical view of data

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 74

Introduction Query Methods Empirical Study Conclusion

Analysis Techniques: Regression Tree Models

Method: HC-HJ, JQL 247.40 CollectionSize < 2250 3651.00 CollectionType: ArrayList, Vector

Tree Models: Recursive partitioning creates hierarchical view of data

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 75

Introduction Query Methods Empirical Study Conclusion

Analysis Techniques: Regression Tree Models

Method: HC-HJ, JQL 247.40 CollectionSize < 2250 3651.00 CollectionType: ArrayList, Vector Method CollectionSize CollectionType

Explanatory Variable: Configuration of the benchmarking framework

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 76

Introduction Query Methods Empirical Study Conclusion

Analysis Techniques: Regression Tree Models

Method: HC-HJ, JQL 247.40 CollectionSize < 2250 3651.00 CollectionType: ArrayList, Vector Method CollectionSize CollectionType Categorical

Non-parametric techniques that handles different variable types

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 77

Introduction Query Methods Empirical Study Conclusion

Analysis Techniques: Regression Tree Models

Method: HC-HJ, JQL 247.40 CollectionSize < 2250 3651.00 CollectionType: ArrayList, Vector Method CollectionSize CollectionType Categorical Numerical

Non-parametric techniques that handles different variable types

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 78

Introduction Query Methods Empirical Study Conclusion

Analysis Techniques: Regression Tree Models

Method: HC-HJ, JQL 247.40 CollectionSize < 2250 3651.00 CollectionType: ArrayList, Vector Method CollectionSize CollectionType Categorical Numerical Method CollectionSize CollectionType Categorical Numerical 247.40 3651.00

Response Variable: Response time of the benchmark

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 79

Introduction Query Methods Empirical Study Conclusion

Join Benchmark with Integers and Strings

Method: HC-HJ, JQL

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 80

Introduction Query Methods Empirical Study Conclusion

Join Benchmark with Integers and Strings

Method: HC-HJ, JQL 247.40 CollectionSize < 2250

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 81

Introduction Query Methods Empirical Study Conclusion

Join Benchmark with Integers and Strings

Method: HC-HJ, JQL 247.40 CollectionSize < 2250 3651.00 CollectionType: ArrayList, Vector

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 82

Introduction Query Methods Empirical Study Conclusion

Join Benchmark with Integers and Strings

Method: HC-HJ, JQL 247.40 CollectionSize < 2250 3651.00 CollectionType: ArrayList, Vector 8447.00

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 83

Introduction Query Methods Empirical Study Conclusion

Join Benchmark with Integers and Strings

Method: HC-HJ, JQL 247.40 CollectionSize < 2250 3651.00 CollectionType: ArrayList, Vector 8447.00 80720.00

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 84

Introduction Query Methods Empirical Study Conclusion

Join Benchmark with Integers and Strings

Method: HC-HJ, JQL 247.40 CollectionSize < 2250 3651.00 CollectionType: ArrayList, Vector 8447.00 80720.00

Reflection’s Impact: HC-HJ and JQL exhibit lower values than JoSQL

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 85

Introduction Query Methods Empirical Study Conclusion

Join Benchmark with Integers and Strings

Method: HC-HJ, JQL 247.40 CollectionSize < 2250 3651.00 CollectionType: ArrayList, Vector 8447.00 80720.00

Reflection’s Impact: LinkedList further degrades JoSQL ’s performance

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 86

Introduction Query Methods Empirical Study Conclusion

Impact of Object Size on Joining

Small Objects Collection Size Method Small Medium Large JQL 57.2 390.2 981.8 HC-HJ 69.3 378.1 923.5 JoSQL 997.3 3620.2 8823.1

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 87

Introduction Query Methods Empirical Study Conclusion

Impact of Object Size on Joining

Small Objects Collection Size Method Small Medium Large JQL 57.2 390.2 981.8 HC-HJ 69.3 378.1 923.5 JoSQL 997.3 3620.2 8823.1 Large Objects Collection Size Method Small Medium Large JQL 35.4 80.8 255.4 HC-HJ 11.4 63.3 217.8 JoSQL 930.3 3107.3 8165.9

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 88

Introduction Query Methods Empirical Study Conclusion

Impact of Object Size on Joining

Small Objects Collection Size Method Small Medium Large JQL 57.2 390.2 981.8 HC-HJ 69.3 378.1 923.5 JoSQL 997.3 3620.2 8823.1 Large Objects Collection Size Method Small Medium Large JQL 35.4 80.8 255.4 HC-HJ 11.4 63.3 217.8 JoSQL 930.3 3107.3 8165.9

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 89

Introduction Query Methods Empirical Study Conclusion

Impact of Object Size on Joining

Small Objects Collection Size Method Small Medium Large JQL 57.2 390.2 981.8 HC-HJ 69.3 378.1 923.5 JoSQL 997.3 3620.2 8823.1 Large Objects Collection Size Method Small Medium Large JQL 35.4 80.8 255.4 HC-HJ 11.4 63.3 217.8 JoSQL 930.3 3107.3 8165.9

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 90

Introduction Query Methods Empirical Study Conclusion

Impact of Object Size on Joining

Small Objects Collection Size Method Small Medium Large JQL 57.2 390.2 981.8 HC-HJ 69.3 378.1 923.5 JoSQL 997.3 3620.2 8823.1 Large Objects Collection Size Method Small Medium Large JQL 35.4 80.8 255.4 HC-HJ 11.4 63.3 217.8 JoSQL 930.3 3107.3 8165.9

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 91

Introduction Query Methods Empirical Study Conclusion

SubQuery Benchmark with Graphs Containing Strings

(LinkedList, ObjectSize = 50)

Time (ms) Collection Size

Large Medium Small 100 200 300 400

Technique

JQL HC JoSQL

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 92

Introduction Query Methods Empirical Study Conclusion

SubQuery Benchmark with Graphs Containing Strings

(LinkedList, ObjectSize = 50)

Time (ms) Collection Size

Large Medium Small 100 200 300 400

Technique

JQL HC JoSQL

JQL is Faster Than HC When the Collection Size is Small and Medium

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 93

Introduction Query Methods Empirical Study Conclusion

SubQuery Benchmark with Graphs Containing Strings

(LinkedList, ObjectSize = 50)

Time (ms) Collection Size

Large Medium Small 100 200 300 400

Technique

JQL HC JoSQL

HC is Faster Than JQL When the Collection Size is Large

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 94

Introduction Query Methods Empirical Study Conclusion

SubQuery Benchmark with Graphs Containing Strings

(LinkedList, ObjectSize = 50)

Time (ms) Collection Size

Large Medium Small 100 200 300 400

Technique

JQL HC JoSQL

Why? JQL Must Track All of the Objects in the Heap

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 95

Introduction Query Methods Empirical Study Conclusion

SubQuery Benchmark with Graphs Containing Strings

(LinkedList, ObjectSize = 100)

Time (ms) Collection Size

Large Medium Small 200 400 600 800

Technique

JQL HC JoSQL

Trend is Even More Pronounced as the Object Size Increases

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 96

Introduction Query Methods Empirical Study Conclusion

Conclusions and Future Work

Concluding Remarks Comprehensive empirical study of query methods Interesting trends concerning JQL, JoSQL, HC, and HC-HJ Refer to the paper for many more insights Future Work Integrate new benchmarks and object types Consider different sizes of objects and collections Incorporate different data finding methods Leverage additional statistical analysis techniques

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 97

Introduction Query Methods Empirical Study Conclusion

Conclusions and Future Work

Concluding Remarks Comprehensive empirical study of query methods Interesting trends concerning JQL, JoSQL, HC, and HC-HJ Refer to the paper for many more insights Future Work Integrate new benchmarks and object types Consider different sizes of objects and collections Incorporate different data finding methods Leverage additional statistical analysis techniques

Jones & Kapfhammer Allegheny College Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps

SLIDE 98

Ask and You Shall Receive: Empirically Evaluating Declarative Approaches to Finding Data in Unstructured Heaps Thank you for your attention! Questions?