Motivation Data-intensive applications need large machines with - - PowerPoint PPT Presentation

▶
motivation
SMART_READER_LITE
LIVE PREVIEW

Motivation Data-intensive applications need large machines with - - PowerPoint PPT Presentation

Nov 24, 2023 125 likes •241 views

Motivation Data-intensive applications need large machines with plenty of NumaGiC: cores and memory A garbage collector for big-data on big NUMA machines Lokesh Gidra , Gal Thomas , Julien Sopena , Marc Shapiro , Nhan

slide-1
SLIDE 1

NumaGiC: A garbage collector for big-data

  • n big NUMA machines

Lokesh Gidra‡, Gaël Thomas, Julien Sopena‡, Marc Shapiro‡, Nhan Nguyen♀

‡ LIP6/UPMC-INRIA Telecom SudParis ♀Chalmers University

Motivation

◼ Data-intensive applications need large machines with plenty of cores and memory

Lokesh Gidra 2

Motivation

◼ Data-intensive applications need large machines with plenty of cores and memory ◼ But, for large heaps, GC is inefficient on such machines

Lokesh Gidra 3

GC Throughput (GB collected per second)

Baseline PS

# of cores

Ideal Scalability

Page rank computation of 100million edge Friendster dataset with Spark on Hotspot/Parallel Scavenge with 40GB on a 48-core machine

Motivation

◼ Data-intensive applications need large machines with plenty of cores and memory ◼ But, for large heaps, GC is inefficient on such machines

Lokesh Gidra 4

GC Throughput (GB collected per second)

Baseline PS

# of cores

Ideal Scalability GC takes roughly 60% of the total time

Page rank computation of 100million edge Friendster dataset with Spark on Hotspot/Parallel Scavenge with 40GB on a 48-core machine

slide-2
SLIDE 2

Outline

◼ Why GC doesn’t scale? ◼ Our Solution: NumaGiC ◼ Evaluation

Lokesh Gidra 5

GCs don’t scale because machines are NUMA

Hardware hides the distributed memory application silently creates inter-node references

Node 0 Node 1 Node 2 Node 3

Memory Memory Memory Memory

Lokesh Gidra 6

GCs don’t scale because machines are NUMA

But memory distribution is also hidden to the GC threads when they traverse the object graph

Node 0 Node 1 Node 2 Node 3

Memory Memory Memory Memory

GC thread

Lokesh Gidra 7

GCs don’t scale because machines are NUMA

But memory distribution is also hidden to the GC threads when they traverse the object graph

Node 0 Node 1 Node 2 Node 3

Memory Memory Memory Memory

GC thread

Lokesh Gidra 8

slide-3
SLIDE 3

GCs don’t scale because machines are NUMA

A GC thread thus silently traverses remote references

Node 0 Node 1 Node 2 Node 3

Memory Memory Memory Memory

GC thread

Lokesh Gidra 9

GCs don’t scale because machines are NUMA

A GC thread thus silently traverses remote references and continues its graph traversal on any node

Node 0 Node 1 Node 2 Node 3

Memory Memory Memory Memory

GC thread

Lokesh Gidra 10

GCs don’t scale because machines are NUMA

A GC thread thus silently traverses remote references and continues its graph traversal on any node

Node 0 Node 1 Node 2 Node 3

Memory Memory Memory Memory

GC thread

Lokesh Gidra 11

GCs don’t scale because machines are NUMA

When all GC threads access any memory nodes, the inter-connect potentially saturates => high memory access latency

Node 0 Node 1 Node 2 Node 3

Memory Memory Memory Memory

Lokesh Gidra 12

slide-4
SLIDE 4

Outline

◼ Why GC doesn’t scale? ◼ Our Solution: NumaGiC ◼ Evaluation

Lokesh Gidra 13

How can we fix the memory locality issue? Simply by preventing any remote memory access

Lokesh Gidra 14

Prevent remote access using messages

Enforces memory access locality by trading remote memory accesses by messages

Memory Memory

Node 0 Node 1

Thread 0 Thread 1

Lokesh Gidra 15

Prevent remote access using messages

Enforces memory access locality by trading remote memory accesses by messages

Memory Memory

Node 0 Node 1

Thread 0 Thread 1

Lokesh Gidra 16

slide-5
SLIDE 5

Prevent remote access using messages

Enforces memory access locality by trading remote memory accesses by messages

Memory Memory

Node 0 Node 1

Thread 0 Thread 1

Remote reference sends it to its home-node

Lokesh Gidra 17

Prevent remote access using messages

Enforces memory access locality by trading remote memory accesses by messages

Remote reference sends it to its home-node

Memory Memory

Node 0 Node 1

Thread 0 Thread 1

Lokesh Gidra 18

Prevent remote access using messages

Enforces memory access locality by trading remote memory accesses by messages

And continue the graph traversal locally

Memory Memory

Node 0 Node 1

Thread 0 Thread 1

Lokesh Gidra 19

Prevent remote access using messages

Enforces memory access locality by trading remote memory accesses by messages

And continue the graph traversal locally

Memory Memory

Node 0 Node 1

Thread 0 Thread 1

Lokesh Gidra 20

slide-6
SLIDE 6

Using messages enforces local access… …but opens up other performance challenges

Lokesh Gidra 21

Problem1: a msg is costlier than a remote access

Node 0 Node 1

Too many messages

Lokesh Gidra 22

Inter-node messages must be minimized

Problem1: a msg is costlier than a remote access

Node 0 Node 1

Too many messages

Lokesh Gidra 23

Inter-node messages must be minimized

  • Observation: app threads naturally create clusters of new allocated objs
  • 99% of recently allocated objects are clustered

Node 0 Node 1

Problem1: a msg is costlier than a remote access

Node 0 Node 1

Too many messages

Lokesh Gidra 24

Inter-node messages must be minimized

  • Observation: app threads naturally create clusters of new allocated objs
  • 99% of recently allocated objects are clustered

Approach: let objects allocated by a thread stay

  • n its node
slide-7
SLIDE 7

Problem2: Limited parallelism

◼ Due to serialized traversal of object clusters across nodes

Node 0 Node 1

Node 1 idles while node 0 collects its memory

Lokesh Gidra 25

Problem2: Limited parallelism

◼ Due to serialized traversal of object clusters across nodes ◼ Solution: adaptive algorithm

Trade-off between locality and parallelism

  • 1. Prevent remote access by using messages when not idling
  • 2. Steal and access remote objects otherwise

Node 0 Node 1

Node 1 idles while node 0 collects its memory

Lokesh Gidra 26

Outline

◼ Why GC doesn’t scale? ◼ Our Solution: NumaGiC ◼ Evaluation

Lokesh Gidra 27

Evaluation

◼ Comparison of NumaGiC with –

  • 1. ParallelScavenge (PS): baseline stop-the-world GC of Hotspot
  • 2. Improved PS: PS with lock-free data structures and interleaved

heap space

  • 3. NAPS: Improved PS + slightly better locality, but no messages

◼ Metrics

  • GC throughput –

– amount of live data collected per second (GB/s) – Higher is better

  • Application performance –

– Relative to improved PS – Higher is better

Lokesh Gidra 28

slide-8
SLIDE 8

Name Description Heap Size Amd48 Intel80 Spark In-memory data analytics (page rank computation) 110 to 160GB 250 to 350GB Neo4j Object graph database (Single Source Shortest Path) 110 to 160GB 250 to 350GB SPECjbb2013 Business-logic server 24 to 40GB 24 to 40GB SPECjbb2005 Business-logic server 4 to 8GB 8 to 12GB

Experiments

1 billion edge Friendster dataset The 1.8 billion edge Friendster dataset

Lokesh Gidra 29

Hardware settings –

1. AMD Magny Cours with 8 nodes, 48 threads, 256 GB of RAM 2. Xeon E7-2860 with 4 nodes, 80 threads, 512 GB of RAM

GC Throughput (GB collected per second)

GC Throughput Heap Sizes Spark Neo4j SpecJBB13 SpecJBB05

  • n

Amd48

Lokesh Gidra 30

Improved PS NAPS NumaGiC

GC Throughput (GB collected per second)

GC Throughput Heap Sizes Spark Neo4j SpecJBB13 SpecJBB05

  • n

Amd48

NumaGiC multiplies GC performance up to 5.4X

Lokesh Gidra 31

Improved PS NAPS NumaGiC

5.4X 2.9X

GC Throughput (GB collected per second)

Heap Sizes Spark Neo4j SpecJBB13 SpecJBB05

  • n

Amd48

  • n

Intel80

Lokesh Gidra 32

GC Throughput

3.6X

slide-9
SLIDE 9

GC Throughput Scalability

Spark on Amd48 with a smaller dataset of 40GB GC Throughput # of nodes Improved PS Baseline PS NumaGiC Ideal Scalability

Lokesh Gidra 33

NAPS

Application speedup

Speedup relative to Improved PS

Lokesh Gidra 34

Spark Neo4j SpecJBB13 SpecJBB05 NAPS NumaGiC

94% 82% 36% 64% 55% 61% 27% 42%

Application speedup

Speedup relative to Improved PS

Lokesh Gidra 35

Spark Neo4j SpecJBB13 SpecJBB05 NAPS NumaGiC

12% 21% 37% 35% 26% 37% 33% 37%

Conclusion

◼ Performance of data-intensive apps relies on GC performance ◼ Memory access locality has huge effect on GC performance ◼ Enforcing locality can be detrimental for parallelism in GCs ◼ Future work: NUMA-aware concurrent GCs

Lokesh Gidra 36

slide-10
SLIDE 10

Conclusion

◼ Performance of data-intensive apps relies on GC performance ◼ Memory access locality has huge effect on GC performance ◼ Enforcing locality can be detrimental for parallelism in GCs ◼ Future work: NUMA-aware concurrent GCs

Lokesh Gidra 37

Thank You J

Large multicores provide this power

But scalability is hard to achieve because software stack was not designed for

Data analytic Cores Memory Banks I/O controllers Operating system Application Language runtime Middleware Hypervisor Hadoop, Spark, Neo4j, Cassandra… JVM, CLI, Python, R… Linux, Windows… Xen, VMWare…

Lokesh Gidra 38

Large multicores provide this power

But scalability is hard to achieve because software stack was not designed for

Data analytic Cores Memory Banks I/O controllers Operating system Application Language runtime Middleware Hypervisor Hadoop, Spark, Neo4j, Cassandra… JVM, CLI, Python, R… Linux, Windows…

Do not consider hypervisors in this talk: Software stack is already complex and hard to analyze!

Lokesh Gidra 39