A Study of Erlang ETS Table Implementations and Performance Or: - - PowerPoint PPT Presentation

A Study of Erlang ETS Table Implementations and Performance

Or: Judy Arrays Are Amazing Data Structures

Scott Lystig Fritchie

<slfritchie@snookles.com>

Snookles Music Consulting

A Study of Erlang ETS Table Implementations and Performance – p.1/19

Overview

ETS table data structures Judy arrays “Contiguous Key Problem” Solving the “Contiguous Key Problem” Performance results

A Study of Erlang ETS Table Implementations and Performance – p.2/19

Audience

Erlang community Using ETS directly Using ETS indirectly via Mnesia and other OTP applications C/C++ developers using hash tables and balanced trees Performance gains by using “Judy arrays” can be impressive Consider using Judy arrays in your applications

A Study of Erlang ETS Table Implementations and Performance – p.3/19

ETS Table Implementations

Types included in Erlang/OTP: AVL balanced binary tree: ordered_set Resizable linear hash table: set, bag, duplicate_bag New research types: In-memory B-tree: btree Judy arrays (based on tries): judysl, judyesl, judyeh

A Study of Erlang ETS Table Implementations and Performance – p.4/19

Judy Arrays

Invented by Doug Baskins, implemented by Hewlett-Packard. Named after Baskins’s sister. Source code now available under GNU LGPL license. Source & docs at http://judy.sourceforge.net/

A Study of Erlang ETS Table Implementations and Performance – p.5/19

Judy Arrays (continued)

Judy arrays are dynamic arrays Index = 1 word, 32- or 64-bit Value = 1 bit or 1 word Handles small & large populations, sparse & dense populations, no tuning parameters! Implemented as a logical 256-ary trie

A Study of Erlang ETS Table Implementations and Performance – p.6/19

Data Structures Review: The Trie

*ABCDEFGHIJKLMNOPQRSTUVWXYZ

A AIR AISLE ASKED ASKING Root Node

*ABC . . . Z *ABC . . . QRST . . . Z *ABC . . . JKL . . . Z *ABCDEFGHIJ . . . Z

ASK

A Study of Erlang ETS Table Implementations and Performance – p.7/19

Data Structures Review: The Trie

*ABCDEFGHIJKLMNOPQRSTUVWXYZ

A AIR AISLE ASKED ASKING Root Node

*ABC . . . Z *ABC . . . QRST . . . Z *ABC . . . JKL . . . Z *ABCDEFGHIJ . . . Z

ASK

A Study of Erlang ETS Table Implementations and Performance – p.7/19

Data Structures Review: The Trie

*ABCDEFGHIJKLMNOPQRSTUVWXYZ

A AIR AISLE ASKED ASKING Root Node

*ABC . . . Z *ABC . . . QRST . . . Z *ABC . . . JKL . . . Z *ABCDEFGHIJ . . . Z

ASK

A Study of Erlang ETS Table Implementations and Performance – p.7/19

Data Structures Review: The Trie

*ABCDEFGHIJKLMNOPQRSTUVWXYZ

A AIR AISLE ASKED ASKING Root Node

*ABC . . . Z *ABC . . . QRST . . . Z *ABC . . . JKL . . . Z *ABCDEFGHIJ . . . Z

ASK

A Study of Erlang ETS Table Implementations and Performance – p.7/19

Data Structures Review: The Trie

*ABCDEFGHIJKLMNOPQRSTUVWXYZ

A AIR AISLE ASKED ASKING Root Node

*ABC . . . Z *ABC . . . QRST . . . Z *ABC . . . JKL . . . Z *ABCDEFGHIJ . . . Z

ASK

A Study of Erlang ETS Table Implementations and Performance – p.7/19

JudySL: A Trie of JudyL Arrays

BEEHIVE \0 E−PAINT\0 \0 \0 BE BEAR BEEH HOUS I VE\0

Short−cut leaf JudyL JudyL Short−cut leaf

Words: BE, BEAR, BEEHOUSE−PAINT,

A Study of Erlang ETS Table Implementations and Performance – p.8/19

JudyESL: A Variation of JudySL

Words: BE, BEAR, BEEHIVE len=−2, data= len=0, data= len=3, data=IVE

JudyL

\0 \0 BE BEAR BEEH

Short−cut leaf

Short−cut leaf

Short−cut leaf

A Study of Erlang ETS Table Implementations and Performance – p.9/19

The Contiguous Key Problem

116

element 0 = atom #A element 1 element 2 element 3 {scott, "scott", <<"To">>, <<"scott">>}

Atom Table number name

foo scott bar A − 1 A A + 1 115 99 111 116

Example tuple

data = 84,111,58,115,99,111,116,116 Ref−counted binary: refcount = 2 size = 8

T : t c t

• s
• Binary:

Binary: start = 3 length = 5 val start = 0 length = 2 val

s c

• t

t

Tuple, size = 4:

A Study of Erlang ETS Table Implementations and Performance – p.10/19

Judy-Based Tables

judysl table type Serialized key = encode_NUL_bytes(term_to_binary(Key)) judyesl table type The JudyESL library uses explicit string length, not NUL termination. Serialized key = term_to_binary(Key)

NOTE: JudySL and JudyESL preserve lexigraphic sort

• rder of serialized keys, not of original Erlang key terms.

A Study of Erlang ETS Table Implementations and Performance – p.11/19

Judy-Based Tables (continued)

judyeh table type JudyL array for hash table:

• ✁✂

hash buckets! No serialization, unlike judysl and judyesl No meta-trie: search one JudyL array, not several Hash collision rate

• % for 7 million items

A Study of Erlang ETS Table Implementations and Performance – p.12/19

Experiment Design

Intentionally maximize time executing ETS-related code. Show table differences as much as possible. Benchmark time in ETS-related code: 35-70% SCCT time in ETS-related code: 18% All other parts of VM unchanged. Benchmark result graphs Overall, set is fastest “old” table type. All run times normalized against set’s time. Run time

1.0

better CPU cache size reflected between

and

keys.

A Study of Erlang ETS Table Implementations and Performance – p.13/19

Sequential Insertion Into Empty Table

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 102 103 104 105 106 107 108 Time relative to ’set’ type Number of keys set judyeh

• rdered_set

judysl judyesl btree

A Study of Erlang ETS Table Implementations and Performance – p.14/19

Sequential Insertion, Per 1K Keys

0.05 0.1 0.15 0.2 0.25 106 107 108 Seconds per thousand keys Number of keys set judyeh

• rdered_set

judysl judyesl btree

A Study of Erlang ETS Table Implementations and Performance – p.15/19

Random Lookup in Full Table

0.5 1 1.5 2 2.5 3 102 103 104 105 106 107 108 Time relative to ’set’ type Number of keys set judyeh

• rdered_set

judysl judyesl btree

A Study of Erlang ETS Table Implementations and Performance – p.16/19

Forward Traversal of Full Table

0.5 1 1.5 2 2.5 3 102 103 104 105 106 107 108 Time relative to ’set’ type Number of keys set judyeh

• rdered_set

judysl judyesl

A Study of Erlang ETS Table Implementations and Performance – p.17/19

Memory Utilization

Table type Memory used Memory used Difference by 70K keys by 21M keys from set

btree 10.4MB 1,055MB 7.7% judyeh 10.4MB 1,036MB 5.7% judysl 10.4MB 1,033MB 5.4% judyesl 11.3MB 1,324MB 35%

• rdered set

10.7MB 1,129MB 15% set 10.2MB 980MB —

A Study of Erlang ETS Table Implementations and Performance – p.18/19

Conclusion

Judy array-based ETS tables perform very well for ETS table sizes that exceed CPU cache size. Table traversal performance is probably fixable. Performance gain of Judy-based tables far exceeds extra memory consumption. JudySL- or JudyESL-based technique could perform better than set and still preserve key sort order. Using Judy arrays in a “real world” application can improve performance. Your application can probably benefit, too.

A Study of Erlang ETS Table Implementations and Performance – p.19/19