An Experimental Evaluation of Garbage Collectors on Big Data - - PowerPoint PPT Presentation
The 45th International Conference on Very Large Data Bases (VLDB 2019) An Experimental Evaluation of Garbage Collectors on Big Data Applications Lijie Xu 1 , Tian Guo 2 , Wensheng Dou 1 , Wei Wang 1 , Jun Wei 1 1 Institute of Software, Chinese
1 Institute of Software, Chinese Academy of Sciences 2 Worcester Polytechnic Institute 1
The 45th International Conference on Very Large Data Bases (VLDB 2019)
2
GC root GC root GC root
Time memory usage memory size frequent and long GC pauses Memory usage
[1] https://stackoverflow.com/questions/38965787/spark-executor-gc-taking-long
4
3
Example: Spark Join Application
mapper 1 a 2 b 3 c 1 d 1 A 2 B 3 C 2 D map() map() map() map() 1, a 2, b 2 b B D 3, c 1, d 1, A 2, B 3, C 2, D 1 c C reduce() reduce() Shuffle write in-memory partition Shuffle read in-memory aggregattion 1, (a, A) 1, (d, A) 3, (c, C) 2, (b, B) 2, (b, D) a d A if cached
User code Large intermediate computing results Shuffle write/read Large intermediate data Cached data Large cached data
GC root
In-memory objects
GC root GC root
5
Objects managed by JVM
Memory space managed by the framework
3
Example: Spark Join Application
mapper 1 a 2 b 3 c 1 d 1 A 2 B 3 C 2 D map() map() map() map() 1, a 2, b 2 b B D 3, c 1, d 1, A 2, B 3, C 2, D 1 c C reduce() reduce() Shuffle write in-memory partition Shuffle read in-memory aggregattion 1, (a, A) 1, (d, A) 3, (c, C) 2, (b, B) 2, (b, D) a d A if cached
for user code
for shuffle write/read
for cached data
GC root
In-memory objects
GC root GC root
6
The framework only manages the intermediate and cached data in a logical space ⇒ Rely on garbage collectors to manage the data objects Objects managed by JVM
Memory space managed by the framework
3
Example: Spark Join Application
mapper 1 a 2 b 3 c 1 d 1 A 2 B 3 C 2 D map() map() map() map() 1, a 2, b 2 b B D 3, c 1, d 1, A 2, B 3, C 2, D 1 c C reduce() reduce() Shuffle write in-memory partition Shuffle read in-memory aggregattion 1, (a, A) 1, (d, A) 3, (c, C) 2, (b, B) 2, (b, D) a d A if cached
for user code
for shuffle write/read
for cached data
GC root
In-memory objects
GC root GC root
7
Objects managed by JVM
Parallel/CMS JVM Young Generation Old Generation G1 JVM (equal-sized regions) O E E O O S H H H H E S O O S O E O E Eden Survivor Survivor
JVM heap layout
O S Survivor H Humongous Non-allocated E Eden Old E
Parallel/CMS collector Contiguous generations G1 collector Non-contiguous generations (equal-size regions)
9
GC root live objects unused objects GC root GC root GC root GC root GC root GC root GC root GC root
Before GC Marking unused objects Sweeping unused objects Parallel GC
Stop-the-world GC
CMS/G1 GC
Concurrent GC
App threads
Stop the world
GC threads for marking for sweeping
Stop-the-world
App threads
Stop the world
Concurrent marking
Stop the world
App threads Concurrent sweeping
Stop the world
App threads
10
11
SQL Graph Machine Learning
Time memory usage memory size frequent and long GC pauses Memory usage
12
1.1 a 2.3 b 3.5 c 1.2 d 1.6 A 2.8 B 3.7 C 4.9 D 1, a 2, b 3, c 1, d 1, A 2, B 3, C 4, D 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D] 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D]
Map Stage Reduce Stage
Memory usage pattern: Long-lived accumulated records 2, [b, B] map() merge() spill() groupByKey()
1 a 2 b 3 c 1 d 1 A 2 B 3 C 2 D 1, a 2, b 3, c 1, d 1, A 2, B 3, C 2, D 1, [(a,d), A] 3, [c, C] 2, [b, (B,D)] 1, (a, A) 1, (d, A) 3, (c, C) 2, (b, B) 2, (b, D) map()
Map Stage Reduce Stage
Long-lived accumulated records join() Massive temporary records Cartesian product
gradient vector
x1 y1 x2 y2
x3 y3 x4 y4 x5 y5 x6 y6 x7 y7 x8 y8
Input matrix
Map Stage Reduce Stage
Features
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
Humongous data objects
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
n
X
i=1 , n
X
i=1
× × ×
wT xi xi yi yi
=+ = ,
n
X
i=1
grad,
n
X
i=1
loss + =
Driver broadcast the new hyperplane vector wT n Long-lived cached records
wT
Label loss value
compute(wnew)
Long-lived accumulated records 1 2 2 1 3 5 3 6 1, 2 2, 1 3, 5 3, 6 1, [2] 6, [3, 7] 2, [1] 4 1 6 3 6 7 7 4 4, 1 6, 3 6, 7 7, 4 3, [5, 6] 7, [4] 4, [1] 3, 1.0 7, 1.0 4, 1.0 1, [(2), 1.0] 6, [(3, 7), 1.0] 2, 1.0 3, 0.5 7, 0.5 1, 1.0 5, 0.5 6, 0.5 4, 1.0 1, 1.0 join() 1, 2.0 6, 0.5 2, 1.0 3, 0.5 7, 0.5 4, 1.0 2, 1.0 reduceByKey() 1, 1.0 6, 1.0
Map Stage 1st Iterative Stage
2, [(1), 1.0] 3, [(5, 6),1.0] 7, [(4), 1.0] 4, [(1), 1.0] Long-lived cached records reduceByKey()
3rd Iterative Stage 2nd Iterative Stage Cached data
13
Young Gen
1.1 a 2.3 b 3.5 c 1.2 d 1.6 A 2.8 B 3.7 C 4.9 D 1, a 2, b 3, c 1, d 1, A 2, B 3, C 4, D 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D] 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D]
Map Stage Reduce Stage
Memory usage pattern: Long-lived accumulated records 2, [b, B] map() merge() spill() groupByKey() 1 a 2 b 3 c 1 d 1 A 2 B 3 C 2 D 1, a 2, b 3, c 1, d 1, A 2, B 3, C 2, D 1, [(a,d), A] 3, [c, C] 2, [b, (B,D)] 1, (a, A) 1, (d, A) 3, (c, C) 2, (b, B) 2, (b, D) map()
Map Stage Reduce Stage
Long-lived accumulated records join() Massive temporary records Cartesian product
Old Gen P1: Long-lived accumulated records Shuffled records are accumulated in memory ⇒ Long-lived objects ⇒ Stored in Old Gen
14
Young Gen
1.1 a 2.3 b 3.5 c 1.2 d 1.6 A 2.8 B 3.7 C 4.9 D 1, a 2, b 3, c 1, d 1, A 2, B 3, C 4, D 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D] 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D]
Map Stage Reduce Stage
Memory usage pattern: Long-lived accumulated records 2, [b, B] map() merge() spill() groupByKey() 1 a 2 b 3 c 1 d 1 A 2 B 3 C 2 D 1, a 2, b 3, c 1, d 1, A 2, B 3, C 2, D 1, [(a,d), A] 3, [c, C] 2, [b, (B,D)] 1, (a, A) 1, (d, A) 3, (c, C) 2, (b, B) 2, (b, D) map()
Map Stage Reduce Stage
Long-lived accumulated records join() Massive temporary records Cartesian product
Temporary results are generated in user code (e.g., cartesian()) ⇒ short-lived objects ⇒ Stored in Young Gen Old Gen P1: Long-lived accumulated records P2: Massive temporary records Shuffled records are accumulated in memory ⇒ Long-lived objects ⇒ Stored in Old Gen
Young Gen
15
gradient vector
x1 y1 x2 y2
x3 y3 x4 y4 x5 y5 x6 y6 x7 y7 x8 y8
Input matrix
Map Stage Reduce Stage
Features
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
Humongous data objects
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
n
X
i=1 , n
X
i=1
× × ×
wT xi xi yi yi
=
+ = ,
n
X
i=1
grad,
n
X
i=1
loss + =
Driver broadcast the new hyperplane vector wT n Long-lived cached records
wT
Label loss value
compute(wnew)
Old Gen:
SVM stores training data in memory ⇒ Long-lived cached records ⇒ Sorted in Old Gen P1: Long-lived cached records
Young Gen
16
gradient vector
x1 y1 x2 y2
x3 y3 x4 y4 x5 y5 x6 y6 x7 y7 x8 y8
Input matrix
Map Stage Reduce Stage
Features
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
Humongous data objects
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
n
X
i=1 , n
X
i=1
× × ×
wT xi xi yi yi
=
+ = ,
n
X
i=1
grad,
n
X
i=1
loss + =
Driver broadcast the new hyperplane vector wT n Long-lived cached records
wT
Label loss value
compute(wnew)
Old Gen:
SVM generates large vectors (large arrays) ⇒ A vector achieves 345MB ⇒ Humongous data object ⇒ Stored in Old Gen SVM stores training data in memory ⇒ Long-lived cached records ⇒ Sorted in Old Gen P1: Long-lived cached records P2: Humongous data objects
Young Gen Old Gen
17
P1: Iterative long-lived accumulated records
Long-lived accumulated records 1 2 2 1 3 5 3 6 1, 2 2, 1 3, 5 3, 6 1, [2] 6, [3, 7] 2, [1] 4 1 6 3 6 7 7 4 4, 1 6, 3 6, 7 7, 4 3, [5, 6] 7, [4] 4, [1] 3, 1.0 7, 1.0 4, 1.0 1, [(2), 1.0] 6, [(3, 7), 1.0] 2, 1.0 3, 0.5 7, 0.5 1, 1.0 5, 0.5 6, 0.5 4, 1.0 1, 1.0 join() 1, 2.0 6, 0.5 2, 1.0 3, 0.5 7, 0.5 4, 1.0 2, 1.0 reduceByKey() 1, 1.0 6, 1.0
Map Stage 1st Iterative Stage
2, [(1), 1.0] 3, [(5, 6),1.0] 7, [(4), 1.0] 4, [(1), 1.0] Long-lived cached records reduceByKey()
3rd Iterative Stage 2nd Iterative Stage Cached data
PageRank generates shuffled records in each iteration ⇒ Iterative long-lived accumulated records ⇒ Similar memory usage pattern in each iteration
Young Gen Old Gen
18
P2: Long-lived cached records P1: Iterative long-lived accumulated records
Long-lived accumulated records 1 2 2 1 3 5 3 6 1, 2 2, 1 3, 5 3, 6 1, [2] 6, [3, 7] 2, [1] 4 1 6 3 6 7 7 4 4, 1 6, 3 6, 7 7, 4 3, [5, 6] 7, [4] 4, [1] 3, 1.0 7, 1.0 4, 1.0 1, [(2), 1.0] 6, [(3, 7), 1.0] 2, 1.0 3, 0.5 7, 0.5 1, 1.0 5, 0.5 6, 0.5 4, 1.0 1, 1.0 join() 1, 2.0 6, 0.5 2, 1.0 3, 0.5 7, 0.5 4, 1.0 2, 1.0 reduceByKey() 1, 1.0 6, 1.0
Map Stage 1st Iterative Stage
2, [(1), 1.0] 3, [(5, 6),1.0] 7, [(4), 1.0] 4, [(1), 1.0] Long-lived cached records reduceByKey()
3rd Iterative Stage 2nd Iterative Stage Cached data
PageRank generates shuffled records in each iteration ⇒ Iterative long-lived accumulated records ⇒ Similar memory usage pattern in each iteration
Young Gen Old Gen
19
P2: Long-lived cached records P1: Iterative long-lived accumulated records
Long-lived accumulated records 1 2 2 1 3 5 3 6 1, 2 2, 1 3, 5 3, 6 1, [2] 6, [3, 7] 2, [1] 4 1 6 3 6 7 7 4 4, 1 6, 3 6, 7 7, 4 3, [5, 6] 7, [4] 4, [1] 3, 1.0 7, 1.0 4, 1.0 1, [(2), 1.0] 6, [(3, 7), 1.0] 2, 1.0 3, 0.5 7, 0.5 1, 1.0 5, 0.5 6, 0.5 4, 1.0 1, 1.0 join() 1, 2.0 6, 0.5 2, 1.0 3, 0.5 7, 0.5 4, 1.0 2, 1.0 reduceByKey() 1, 1.0 6, 1.0
Map Stage 1st Iterative Stage
2, [(1), 1.0] 3, [(5, 6),1.0] 7, [(4), 1.0] 4, [(1), 1.0] Long-lived cached records reduceByKey()
3rd Iterative Stage 2nd Iterative Stage Cached data
Similar memory usage pattern in each iteration
[2] KDD Cup 2012 dataset. https://www.csie.ntu.edu.tw/cjlin/libsvmtools/datasets/binary.html. [3] Twitter social graph. http://an.kaist.ac.kr/traces/WWW2010.html.
20
Application Data-1.0 (100%) Data-0.5 (50%) GroupBy 200GB Uservisits (1.2B rows) [1] 50% rows (100GB) Join 200GB Uservisits (1.2B rows), 40GB Rankings (600M rows) [1] 50% rows (100GB, 20GB) SVM 21GB KDD2012 matrix [2] (149M rows, 54M features) 50% columns (11.2GB, 27M features) PageRank 25GB Twitter graph [3] (476M edges, 17M nodes) 50% edges (12.2GB, 238M edges)
Master
driver
slave4 slave5 slave6 slave7 slave8 slave1 slave3 slave2 Alibaba Cloud 9 nodes, Java 8, Spark 2.1.2 Each node runs 4 JVMs (tasks) Each JVM runs one task (core = 1, memory = 6.5GB) ⇒ Observe the memory usage and GC activities of each task
3
Tasks of a Spark application
1 a 2 b 3 c 1 d 1 A 2 B 3 C 2 D map() map() map() map() 1, a 2, b 2 b B D 3, c 1, d 1, A 2, B 3, C 2, D 1 c C reduce() reduce() 1, (a, A) 1, (d, A) 3, (c, C) 2, (b, B) 2, (b, D) a d A
22
6.2 6 6 0.4 0.3 0.1
Parallel CMS G1
26.1 19.5 38.3 11.3 3.5 3.3
Parallel CMS G1
45.4 36.3 39.4 19 0.9 1.2 Parallel CMS G1
App Execution Time (min) GC Time (min) 78.7 54.7 57.1 41 0.7 2.6
Parallel CMS G1
App Execution Time (min) GC Time (min)
23
Massive long-lived accumulated records
1.1 a 2.3 b 3.5 c 1.2 d 1.6 A 2.8 B 3.7 C 4.9 D 1, a 2, b 3, c 1, d 1, A 2, B 3, C 4, D 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D] 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D]
Map Stage Reduce Stage
Long-lived accumulated records map() groupByKey()
3 1 c a 7 5 f e 9 l c a c a a b A F S n C d G D m
E d Q l E H p J
24
1.1 a 2.3 b 3.5 c 1.2 d 1.6 A 2.8 B 3.7 C 4.9 D 1, a 2, b 3, c 1, d 1, A 2, B 3, C 4, D 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D] 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D]
Map Stage Reduce Stage
Long-lived accumulated records map() groupByKey()
Massive long-lived accumulated records
3 1 c a 7 5 f e 9 l c a c a a b A F S n C d G D m
E d Q l E H p J
3 1 c a 7 5 f e 9 l c a c a a b A F S n C d G D m
E d Q l E H p J
25
1.1 a 2.3 b 3.5 c 1.2 d 1.6 A 2.8 B 3.7 C 4.9 D 1, a 2, b 3, c 1, d 1, A 2, B 3, C 4, D 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D] 1, [a, d, A] 3, [c, C] 2, [b, B] 4, [D]
Map Stage Reduce Stage
Long-lived accumulated records map() groupByKey()
Massive long-lived accumulated records
3 1 c a 7 5 f e 9 l c a c a a b A F S n C d G D m
E d Q l E H p J
Time memory usage memory size frequent and long GC pauses Memory usage
26
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 200 400 600 800 1000 5 10 GC time (s) YGC Pause FGC Pause
(b) GroupBy-1.0-Slowest-CMS-Task
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 200 400 600 800 1000 1200 2 4 GC time (s) YGC Pause FGC Pause
(c) GroupBy-1.0-Slowest-G1-Task
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 250 500 750 1000 1250 1500 10 20 GC time (s) YGC Pause FGC Pause
(a) GroupBy-1.0-Slowest-Parallel-Task
Time (s) Time (s)
shuffle spill
Time (s)
shuffle spill shuffle spill A long (11 s) full GC pause due to concurrent mode failure
Parallel GC Allocate the smallest old gen & shrink with memory usage ⇒ Suffer from frequent GC G1 GC Allocate large old gen & shrink with memory usage ⇒ Less GCs than Parallel CMS GC Allocate the largest old gen & does not shrink ⇒ The fewest GC pauses
27
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 200 400 600 800 1000 5 10 GC time (s) YGC Pause FGC Pause
(b) GroupBy-1.0-Slowest-CMS-Task
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 200 400 600 800 1000 1200 2 4 GC time (s) YGC Pause FGC Pause
(c) GroupBy-1.0-Slowest-G1-Task
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 250 500 750 1000 1250 1500 10 20 GC time (s) YGC Pause FGC Pause
(a) GroupBy-1.0-Slowest-Parallel-Task
Time (s) Time (s)
shuffle spill
Time (s)
shuffle spill shuffle spill A long (11 s) full GC pause due to concurrent mode failure
Parallel GC G1 GC CMS GC
28
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 200 400 600 800 1000 5 10 GC time (s) YGC Pause FGC Pause
(b) GroupBy-1.0-Slowest-CMS-Task
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 200 400 600 800 1000 1200 2 4 GC time (s) YGC Pause FGC Pause
(c) GroupBy-1.0-Slowest-G1-Task
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 250 500 750 1000 1250 1500 10 20 GC time (s) YGC Pause FGC Pause
(a) GroupBy-1.0-Slowest-Parallel-Task
Time (s) Time (s)
shuffle spill
Time (s)
shuffle spill shuffle spill A long (11 s) full GC pause due to concurrent mode failure
Implication: Design more intelligent heap sizing polices for accommodating big data objects (especially for long-lived accumulated records)
45.4 36.3 39.4 19 0.9 1.2
Parallel CMS G1
App Execution Time (min) GC Time (min) 78.7 54.7 57.1 41 0.7 2.6 Parallel CMS G1
App Execution Time (min) GC Time (min) 6.2 6 6 0.4 0.3 0.1 Parallel CMS G1
26.1 19.5 38.3 11.3 3.5 3.3
Parallel CMS G1
30
GC root live objects unused objects GC root GC root GC root GC root GC root
31
GC root live objects unused objects GC root GC root GC root GC root GC root
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 250 500 750 1000 1250 1500 10 20 GC time (s) YGC Pause FGC Pause
(a) GroupBy-1.0-Slowest-Parallel-Task
32
Parallel GC
Stop-the-world GC
CMS/G1 GC
Concurrent marking/sweeping
App threads
Stop the world
Stop the app threads for GC (10-20s pause)
Stop-the-world
App threads
Stop the world Stop the world
App threads Concurrent marking & sweeping (less than 1s pause for remark)
Stop the world
App threads
33
Parallel GC
Stop-the-world GC
CMS/G1 GC
Concurrent marking/sweeping
App threads
Stop the world
Stop the app threads for GC (10-20s pause)
Stop-the-world
App threads
Stop the world Stop the world
App threads Concurrent marking & sweeping (less than 1s pause for remark)
Stop the world
App threads
Question:
34
App threads
Stop the world Stop the world Stop the world
App threads
CMS/G1 GC
Concurrent marking/sweeping
35
App threads
Stop the world Stop the world Stop the world
App threads
CMS/G1 GC
Concurrent marking/sweeping
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 0.00 0.05 0.10 GC time (s) FGC Pause Concurrent mark phase 500 1000 1500 2000 100 200 300 CPU utilization (%) CPU Usage Memory Usage 0.0 2.5 5.0 7.5 Memory (GB) 5 10 Concurrent GC (s)
(b) Join-1.0-Slowest-CMS-Task
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 0.0 0.5 1.0 GC time (s) FGC Pause Concurrent mark phase 500 1000 1500 2000 100 200 300 CPU utilization (%) CPU Usage Memory Usage 0.0 2.5 5.0 7.5 Memory (GB) 10 20 30 Concurrent GC (s)
(c) Join-1.0-Slowest-G1-Task
Time (s) Time (s) shuffle phase
phase
shuffle spill
shuffle phase
shuffle spill
36
App threads
Stop the world Stop the world Stop the world
App threads
CMS/G1 GC
Concurrent marking/sweeping
Implication: Design more efficient marking & sweeping algorithms for big data objects to reduce individual GC pause time
15.2 14.5 1.2 1.1
Parallel CMS G1
App Execution Time (m) GC Time (m)
OutOfMemory
38
Humongous data object (~500 MB vector array)
gradient vector
x1 y1 x2 y2
x3 y3 x4 y4 x5 y5 x6 y6 x7 y7 x8 y8 Map Stage Reduce Stage
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
Humongous data objects
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
n
X
i=1
grad(w, xi),
n
X
i=1
loss(w, xi)
n
X
i=1 , n
X
i=1
× × ×
wT xi xi yi yi
=
+ = ,
n
X
i=1
grad,
n
X
i=1
loss + =
Driver broadcast the new hyperplane vector wT loss value
wT
39
Humongous data object (~500 MB vector array)
(equal-sized regions) O E E O O S H H H H E S O O S O E O E O S Survivor H Humongous Non-allocated
G1 JVM
E Eden Old E O E E O
OOM
Root cause: Not enough contiguous space for keeping this humongous object
40
Humongous data object (~500 MB vector array)
(equal-sized regions) O E E O O S H H H H E S O O S O E O E O S Survivor H Humongous Non-allocated
G1 JVM
E Eden Old E O E E O
Implication Redesign the object allocation algorithm to balance the trade-
and reliability
(see our paper for details)
cause is that Parallel collector has the smallest available heap size that leads to the lowest spill threshold of ParallelGC tasks.
GC pauses towards the long- lived accumulated records. Due to different full GC triggering thresholds, ParallelGC suffers from 1.7x more full GC pauses than G1, and G1 suffers from 7x more full GC pauses than CMS.
records in each iteration, CMS collector’s concurrent sweeping algorithm achieves 16x shorter full GC time than G1’s incremental sweeping algorithm.
due to 1.7-12x more full GC pauses and 10x longer individual full GC pause.
CMS tasks.
resizing timing mechanism leads to 38% more full GC pauses.
41
2 4 6 8 Old Gen (GB) BeforeGC AfterGC Allocated 2 4 GC time (s) YGC Pause FGC Pause 20 40 60 80 100 120 200 400 600 800 CPU utilization (%) CPU Usage Memory Usage 2 4 6 8 Memory usage (GB)
(c) GroupBy-0.5-Slowest-G1-task
3 1 c a 7 5 f e 9 l c a c a a b A F S n C d G D m
E d Q l E H p J
GC root GC root GC root
45