Carol V. Alexandru , Sebastiano Panichella, Sebastian Proksch, Harald - - PowerPoint PPT Presentation

Carol V. Alexandru, Sebastiano Panichella, Sebastian Proksch, Harald C. Gall

Software Evolution and Architecture Lab University of Zurich, Switzerland {alexandru,panichella,proksch,gall}@ifi.uzh.ch 26.03.2018 59th CREST Open Workshop Centre for Research on Evolution, Search and Testing University College London, London, United Kingdom

The Problem Domain

• Static analysis (e.g. #Attr., McCabe, coupling...)

1

The Problem Domain

v0.7.0 v1.0.0 v1.3.0 v2.0.0 v3.0.0 v3.3.0 v3.5.0 2

• Static analysis (e.g. #Attr., McCabe, coupling...)

The Problem Domain

v0.7.0 v1.0.0 v1.3.0 v2.0.0 v3.0.0 v3.3.0 v3.5.0 2

• Static analysis (e.g. #Attr., McCabe, coupling...)
• Many revisions, fine-grained historical data

A Typical Analysis Process

www

clone

select project

3

A Typical Analysis Process

www

clone checkout

select project select revision

3

A Typical Analysis Process

www

clone checkout Purpose-built, language specific tool

Res

select project select revision

apply tool store analysis results

3

A Typical Analysis Process

www

clone checkout Purpose-built, language specific tool

Res

select project select revision

apply tool store analysis results more revisions?

3

A Typical Analysis Process

www

clone checkout Purpose-built, language specific tool

Res

select project select revision

apply tool store analysis results more revisions? more projects?

3

Redundancies all over...

4

Redundancies in historical code analysis Impact on Code Study Tools

Redundancies all over...

Redundancies in historical code analysis Few files change Only small parts of a file change Across Revisions Impact on Code Study Tools

4

Redundancies all over...

Redundancies in historical code analysis Few files change Only small parts of a file change Across Revisions Impact on Code Study Tools Repeated analysis

• f "known" code

4

Redundancies all over...

Redundancies in historical code analysis Few files change Only small parts of a file change Changes may not even affect results Across Revisions Impact on Code Study Tools Repeated analysis

• f "known" code

Storing redundant results

4

Redundancies all over...

4

Redundancies in historical code analysis Few files change Across Languages Only small parts of a file change Changes may not even affect results Each language has their own toolchain Yet they share many metrics Across Revisions Impact on Code Study Tools Repeated analysis

• f "known" code

Storing redundant results

Redundancies all over...

Redundancies in historical code analysis Few files change Across Languages Only small parts of a file change Changes may not even affect results Each language has their own toolchain Yet they share many metrics Across Revisions Impact on Code Study Tools Repeated analysis

• f "known" code

Storing redundant results Re-implementing identical analyses Generalizability is expensive

4

Redundancies all over...

Redundancies in historical code analysis Few files change Across Languages Only small parts of a file change Changes may not even affect results Each language has their own toolchain Yet they share many metrics Across Revisions Impact on Code Study Tools Repeated analysis

• f "known" code

Storing redundant results Re-implementing identical analyses Generalizability is expensive

5

#1: Avoid Checkouts

Avoid checkouts

7

clone

Avoid checkouts

7

clone checkout

read write

Avoid checkouts

7

clone checkout

read read write

analyze

Avoid checkouts

7

clone checkout

read

For every file: 2 read ops + 1 write op Checkout includes irrelevant files Need 1 CWD for every revision to be analyzed in parallel

read write

analyze

Avoid checkouts

8

clone analyze

read

Avoid checkouts

8

clone analyze Only read relevant files in a single read op No write ops No overhead for parallization

read

Avoid checkouts

8

clone analyze Only read relevant files in a single read op No write ops No overhead for parallization

Git Analysis Tool File Abstraction Layer

read

Avoid checkouts

8

clone analyze Only read relevant files in a single read op No write ops No overhead for parallization

Git Analysis Tool File Abstraction Layer

E.g. for the JDK Compiler:

class JavaSourceFromCharrArray(name: String, val code: CharBuffer) extends SimpleJavaFileObject(URI.create("string:///" + name), Kind.SOURCE) {

• verride def getCharContent(): CharSequence = code

} read

Avoid checkouts

clone analyze Only read relevant files in a single read op No write ops No overhead for parallization

Git Analysis Tool File Abstraction Layer

E.g. for the JDK Compiler:

class JavaSourceFromCharrArray(name: String, val code: CharBuffer) extends SimpleJavaFileObject(URI.create("string:///" + name), Kind.SOURCE) {

• verride def getCharContent(): CharSequence = code

} read 9

#2: Use a multi-revision representation

• f your sources

Merge ASTs

• rev. 1
• rev. 2
• rev. 3
• rev. 4

10

• rev. 1
• rev. 2
• rev. 3
• rev. 4

11

• rev. 1

Merge ASTs

• rev. 1
• rev. 2
• rev. 3
• rev. 4

12

• rev. 2

Merge ASTs

Merge ASTs

• rev. 1
• rev. 2
• rev. 3
• rev. 4

13

• rev. 3

Merge ASTs

• rev. 1
• rev. 2
• rev. 3
• rev. 4

14

• rev. 4

Merge ASTs

• rev. 1
• rev. 2
• rev. 3
• rev. 4

15

Merge ASTs

• rev. 1
• rev. 2
• rev. 3
• rev. 4

16

• rev. range [1-4]
• rev. range [1-2]

Merge ASTs

• rev. 1
• rev. 2
• rev. 3
• rev. 4

AspectJ (~440k LOC): 1 commit: 2.2M nodes All >7000 commits: 6.5M nodes

17

Merge ASTs

• rev. 1
• rev. 2
• rev. 3
• rev. 4

AspectJ (~440k LOC): 1 commit: 2.2M nodes All >7000 commits: 6.5M nodes

18

Merge ASTs

• rev. 1
• rev. 2
• rev. 3
• rev. 4

AspectJ (~440k LOC): 1 commit: 2.2M nodes All >7000 commits: 6.5M nodes

19

#3: Store AST nodes only if they're needed for analysis

public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) }} } }

What's the complexity (1+#forks) and name for each method and class?

20

140 AST nodes (using ANTLR)

parse

What's the complexity (1+#forks) and name for each method and class?

20 public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) }} } }

140 AST nodes (using ANTLR)

parse

CompilationUnit TypeDeclaration Modifiers public Members Method Modifiers public Name run Name Demo Parameters ReturnType PrimitiveType VOID Body Statements

... ... What's the complexity (1+#forks) and name for each method and class?

20 public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) }} } }

What's the complexity (1+#forks) and name for each method and class? 140 AST nodes (using ANTLR)

parse filtered parse

TypeDeclaration Method Name run Name Demo ForStatement IfStatement ConditionalExpression

7 AST nodes (using ANTLR)

21 public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) }} } }

public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) } } }

What's the complexity (1+#forks) and name for eachmethod and class? 140 AST nodes (using ANTLR)

parse filtered parse

TypeDeclaration Method Name run Name Demo ForStatement IfStatement ConditionalExpression

7 AST nodes (using ANTLR)

22

public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) } } }

What's the complexity (1+#forks) and name for eachmethod and class? 140 AST nodes (using ANTLR)

parse filtered parse

TypeDeclaration Method Name run Name Demo ForStatement IfStatement ConditionalExpression

7 AST nodes (using ANTLR)

23

#4: Use non-duplicative data structures to store your results

• rev. 1
• rev. 2
• rev. 3
• rev. 4

24

• rev. 1
• rev. 2
• rev. 3
• rev. 4

24

• rev. 1
• rev. 2
• rev. 3
• rev. 4

24

[1-1] label #attr mcc [4-4] label #attr mcc InnerClass 4 1 2 4 [2-3] label #attr mcc

• rev. 1
• rev. 2
• rev. 3
• rev. 4

[1-1] label #attr mcc [4-4] label #attr mcc InnerClass 4 1 2 4 [2-3] label #attr mcc

25

LISA also does: #5: Parallel Parsing #6: Asynchronous graph computation #7: Generic graph computations applying to ASTs from compatible languages

26

A light-weight view on multi-language analysis

Typical solutions

• Toolchains / Frameworks
• Integrate language-specific tooling
• Lots of engineering required
• Meta-models
• Translate language code to some common

representation

• Significant overhead / rigid models

52

Structure matters most

• Complexity?
• # of Functions / Attributes etc.
• Coupling between Classes
• Call graphs

53

if (true) { if (true) { } } # CYCLO: 3 if (true) { } if (true) { } # CYCLO: 4

Relative structure is similar

54

public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) }}}} class Demo: def run(): for i in range(1, 100): if i % 3 == 0 or i % 5 == 0: print(i)

Relative structure is similar

55

public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) }}}} class Demo: def run(): for i in range(1, 100): if i % 3 == 0 or i % 5 == 0: print(i)

Relative structure is similar

56

public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) }}}} class Demo: def run(): for i in range(1, 100): if i % 3 == 0 or i % 5 == 0: print(i)

class function for if print

Entities are different

57

public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) }}}} class Demo: def run(): for i in range(1, 100): if i % 3 == 0 or i % 5 == 0: print(i)

class function for if print CLASS_DECL METHOD_DECL FOR_STAT IF_STAT METHOD _INVOK classdef funcdef forstat ifelsestat atomexpr

Can filter irrelevant nodes

58

public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) }}}} class Demo: def run(): for i in range(1, 100): if i % 3 == 0 or i % 5 == 0: print(i)

class function for if print

Can filter irrelevant nodes

59

public class Demo { public void run() { for (int i = 1; i< 100; i++) { if (i % 3 == 0 || i % 5 == 0) { System.out.println(i) }}}} class Demo: def run(): for i in range(1, 100): if i % 3 == 0 or i % 5 == 0: print(i)

class function for if print

How LISA handles Multiple languages

Analysis formulation

• Signal/Collect (like “Google Pregel” for Scala)
• Graph vertices send information packets

(signals) and do something when receiving (collecting) signals.

61

Analysis formulation

• Signal/Collect (like “Google Pregel” for Scala)
• Graph vertices send information packets

(signals) and do something when receiving (collecting) signals.

62

class function

Analysis formulation

• Signal/Collect (like “Google Pregel” for Scala)
• Graph vertices send information packets

(signals) and do something when receiving (collecting) signals.

63

class function func-count signal 1 1 1 signal

Analysis formulation

• Signal/Collect (like “Google Pregel” for Scala)
• Graph vertices send information packets

(signals) and do something when receiving (collecting) signals.

64

class function collect func-count signal fc = fc+1+1 = 2

Analysis formulation

• Signal/Collect (like “Google Pregel” for Scala)
• Graph vertices send information packets

(signals) and do something when receiving (collecting) signals.

65

class function func-count signal 1 signal

Analysis formulation

• Signal/Collect (like “Google Pregel” for Scala)
• Graph vertices send information packets

(signals) and do something when receiving (collecting) signals.

66

class function collect func-count signal fc = fc+1 = 3

Analysis formulation

• Signal/Collect (like “Google Pregel” for Scala)
• Graph vertices send information packets

(signals) and do something when receiving (collecting) signals.

• Analyses use generic lables for entities

67

68

69

Light-weight entity mappings

70

Light-weight entity mappings

71

Lables used by Analyses

Light-weight entity mappings

72

Lables used by the parser or grammar

Light-weight entity mappings

73

Parsing

• AST is kept as the parser supplies it
• ANTLRv4 integration
• Filtering can be enabled
• Only AST nodes that correspond to a label

used in an analysis are kept

• Reduces graph size by a factor of 10 or more

74

Adding new languages

• 1. Integrate a parser (or generate one)
• Graph interface allows adding nodes/edges
• 2. Write a node mapping
• 3. Re-use existing analyses on new ASTs

75

76

77

To Summarize...

The LISA Analysis Process

27

www

clone

select project

The LISA Analysis Process

27

www

clone parallel parse into merged graph

select project Language Mappings (Grammar to Analysis)

determines which AST nodes are loaded

ANTLRv4 Grammar

used by Generates Parser

The LISA Analysis Process

27

www

clone parallel parse into merged graph

Res

select project

store analysis results

Analysis formulated as Graph Computation Language Mappings (Grammar to Analysis)

used by

• Async. compute

determines which AST nodes are loaded determines which data is persisted

ANTLRv4 Grammar

used by Generates Parser runs

• n graph

The LISA Analysis Process

27

www

clone parallel parse into merged graph

Res

select project

store analysis results more projects?

Analysis formulated as Graph Computation Language Mappings (Grammar to Analysis)

used by

• Async. compute

determines which AST nodes are loaded determines which data is persisted

ANTLRv4 Grammar

used by Generates Parser runs

• n graph

How well does it work, then?

Marginal cost for +1 revision

41.670 4.633 0.525 0.109 0.082 0.071 0.052 0.041 0.032 0.033 5 10 15 20 25 30 35 40 45 50 1 10 100 1000 2000 3000 4000 5000 6000 7000

Average Parsing+Computation time per Revision when analyzing n revisions of AspectJ (10 common metrics)

Seconds # of revisions

28

Overall Performance Stats

29

Language Java C# JavScript Python #Projects 1000 1000 1000 1000 #Revisions 2 Million 1.4 Million 1.5 Million 2.3 Million #Files (parsed!) 10 Billion 3 Billion 380 Thousand 1.2 Billion #Lines (parsed!) 1.6 Trillion 0.6 Trillion 43 Billion 0.3 Trillion Total Runtime (RT)¹ 2d 5h 3d 23h 2d 4h Median RT¹ 8.35s 40.5s 14.4s 24.5s

• Tot. Avg. RT per

Rev.² 97ms 183ms 57ms 83ms

• Med. Avg. RT per

Rev.² 41ms 88ms 25ms 48ms

¹ Including cloning and persisting results ² Excluding cloning and persisting results

What's the catch?

(There are a few...)

The (not so) minor stuff

• Must implement analyses from scratch
• No help from a compiler
• Non-file-local analyses need some effort

30

The (not so) minor stuff

• Must implement analyses from scratch
• No help from a compiler
• Non-file-local analyses need some effort
• Moved files/methods etc. add overhead
• Uniquely identifying files/entities is hard
• (No impact on results, though)

30

Parser performance matters

31

Javascript C# Java Python

LISA is EXTREME

32

complex feature-rich heavyweight simple generic lightweight

Thank you for your attention

59th CREST Open Workshop, London, 26.03.2018

Read the paper: http://t.uzh.ch/Fj

(upcoming EMSE publication is much more detailed)

Try the tool: http://t.uzh.ch/Fk Get the slides: http://t.uzh.ch/NR Contact me: alexandru@ifi.uzh.ch