PublishedApi Handling published Java API automatically How to - - PowerPoint PPT Presentation

V1.0 | 2017-06-27

How to provide a stable but changeable API to external developers

PublishedApi – Handling published Java API automatically

 Andreas Turban  Andreas.Turban@vector.com

> Work E-Mail

 Andreas.Turban@net-baustelle.de

> Private E-Mail

 Employed at Vector Informatik GmbH since 2006 as  Product Architect  Senior Software Development Engineer

> www.vector.com

About me

Introduction

2/40

 For the presentation:  An application programming interface (API) is a well defined set of Java classes

including their fields and methods

 The API is used by code which is NOT internal code, also called client

> By clients which are not part of the same project > Clients use only the API to build their own code not the whole project

 The client code is written by other people using the API  So the API is published and can not be changed easily

Application Programming Interface (API)

Introduction

3/40

Introduction  Why stable API matters PublishedApi PublishedApi PostProcessing API Change Documentation References

Agenda

4/40

 Suppose we write an API, which is used by different people  We do not control or have the source code of the users

> So we can‘t change all usage of the API, we have to deal with existing code

 And we want to change our API  What is part of our API and what is internal?  What can we change without breaking the existing clients?  How do we communicate the change?  These are all well known problems  The easiest solution is to give the API a version number

API stability

Why stable API matters

5/40

 Versions are used to describe compatibility of an API  Most used scheme

 „Semantic Versioning“

> http://semver.org/

 Given a version number MAJOR.MINOR.PATCH, increment the:

 MAJOR version when you make incompatible API changes,  MINOR version when you add functionality in a backwards-compatible manner, and  PATCH version when you make backwards-compatible bug fixes.

 But how do we as developers know, if a change is a major, minor or patch

change? Versioning - Quick reminder

Why stable API matters

6/40

 We as developer have to group every change in the categories:

 Major  Minor  Patch

 But this can be a tedious and error-prone process

Change Categories

Why stable API matters

7/40

 Example of a compatible looking but incompatible change  We add a new method to an interface, which is implemented by a client  Original:  New method isExecutable():

Change Example

Why stable API matters

8/40

 This change breaks the compatibility of the API  Because every client now has to implement the new method isExecutable()

Change Example

Why stable API matters

9/40

 But which changes are compatible or incompatible?  There are rules for it:  The Eclipse wiki lists 148 rules to achieve API Binary Compatibility in Java

> https://wiki.eclipse.org/Evolving_Java-based_APIs_2

 But we as developers are not good in applying over 100 rules to each tiny code

change

 The process is too tedious and error-prone  So lets write a program which does the work!

Change Rules

Why stable API matters

10/40

Introduction Why stable API matters  PublishedApi PublishedApi PostProcessing API Change Documentation References

Agenda

11/40

 We need a program analysing our code changes and check the compatibility rules

Idea

PublishedApi

Product Developer «resource» source code Java compiler API supervision «resource» class files API PostProcessing «resource» published class files «resource» API Jar «resource» Product Jars E.g. an Eclipse RCP «feedback» [on error] stops compilaton [on API class] calls writes code 12/40

 The usage of the API by external people whould be:

Idea

PublishedApi

External Developer «resource» source code «resource» API Jar «resource» Product Jars Java compiler «resource» External Jar An OSGi bundle or some other extension jar Product execution Normal OSGI bundle which start an Eclipse RCP The running Eclipse RCP done by external people Only used for compilation! Not for execution. writes code

13/40

 At Vector we call our solution PublishedApi  It compares the current java source with a reference in a database  CompatibilityDatabase  It issues errors if changes were made without properly changing the versions

Vector Solution

PublishedApi

14/40

 All API classes are annotated with  @PublishedApi  So all classes with @PublishedApi are part of a certain API and will be supervised

automatically

 These classes must be kept stable or otherwise there will be a compile error

How does it work?

PublishedApi

15/40

 Example:

How does it work?

PublishedApi

16/40

 If we make an incompatible change:  Error:

 The current Version information in your workspace and in the CompatibilityDatabase have the same

major version, but there is a breaking change in this class.

 New method “public boolean isExecutable()”.  You have to increment the major version to make this change.

How does it work?

PublishedApi

17/40

 If we make a compatible change:  There will be a warning:

 PublishedApi: The method "public void newMethod()" in the class "DefRef" is new.

How does it work?

PublishedApi

18/40

 One part of the solution is a Plug-In in the Java compiler  AnnotationProcessor

> https://docs.oracle.com/javase/7/docs/technotes/guides/apt/GettingStarted.html

 The annotation processor is automatically called by the Java compiler, if an

@PublishedApi class is compiled

 The annotation processor will stop the compilation, if there is an error  So nobody can unintentionally break the API

How does it work?

PublishedApi

19/40

Introduction Why stable API matters PublishedApi  PublishedApi PostProcessing API Change Documentation References

Agenda

20/40

 The other part of the solution is a PostProcessor  This step can change the compiled Java classes after the compiler run  Only the compile API jar is changed not the jars used for runtime execution  This is used in multiple ways  Hide certain methods  Divide the API in multiple @PublishedApiDomains

> E.g. different API for different users

 Document deprecation  Etc.

General

PublishedApi PostProcessing

21/40

 The post processor uses the ASM library to read and rewrite the class files:  http://asm.ow2.org/

> „ASM is an all purpose Java bytecode manipulation and analysis framework.

It can be used to modify existing classes or dynamically generate classes, directly in binary form.” http://asm.ow2.org/ [2017-06-29]

 The following steps are all implemented as ASM visitors on the API class files

How does it work

PublishedApi PostProcessing

22/40

 Suppose we have a class  We want to hide a method, which uses a non published type  Example:  But now we have to publish the MyInternalType  Otherwise clients will not compile

Hide Methods from PublishedApi

PublishedApi PostProcessing

23/40

 The post processor supports to strip methods from the API  Example:

Hide Methods from PublishedApi

PublishedApi PostProcessing

24/40

 The annotation @KeepSecretFromPublishedApi will remove the method from the

class file

 But only for the client during compilation  So we do not have to publish MyInternalType  And no client will see the method in code completion etc.  Note: Reflection by the client will still work

Hide Methods from PublishedApi

PublishedApi PostProcessing

25/40

 The post processor removes all implementation byte code an reference of API

classes

 This makes it possible only to provide the API classes for client not other

dependencies

 Example  A class uses an internal type in its implementation

> Field: private final InternalType myfield; > Now the Java compiler needs the internal type to analyze the class

 When we strip the whole internal implementation the Java compiler can use the

class without the internal type Delete Implementation Bytecode

PublishedApi PostProcessing

26/40

 There is the Java annotation @Deprecated  But this will deprecate the usage for clients and internal code  Sometime we just want to Deprecate the usage for clients

Deprecate API

PublishedApi PostProcessing

27/40

 The @PublishedDeprecated is transformed into the @Deprecated java annotation

during post processing

 So the client will see a Java warning, if the method is used  But internal code can use the method without warnings

Deprecate API

PublishedApi PostProcessing

28/40

 There are some cases in the Java compiler, when internal types are needed for

compilation, but are not part of the API

 Examples: > private inner classes, needs to be present due to reflection information > Subtypes of enums need to be published as well  So we need to add these classes, but no client shall use them directly  The post processor will automatically mark the internal types with  @Deprecated  @UsageNotAllowed(“Reason why it shall not be used”)

Usage Not Allowed

PublishedApi PostProcessing

29/40

 We can also split the API in multiple parts  We annotate the classes with the allowed usages, for each domain  The enum DomainType defines the possible domains  The @PublishedApiDomain defines for each class the allowed usage

PublishedApiDomain

PublishedApi PostProcessing

30/40

 The annotation processor and post processor also support the new Java 8 keyword

default at interfaces

 An interface with ChangePolicy.CHANGES_FORBIDDEN can be changed by adding

methods with default implementation Java 8 default methods

PublishedApi PostProcessing

31/40

Introduction Why stable API matters PublishedApi PublishedApi PostProcessing  API Change Documentation References

Agenda

32/40

 The PostProcessor will also calculate which changes were made since the last

release

 This can be used to:  Verify changes => Review  Create API change documentation  Analytics

Calculate Changes from Last Release

API Change Documentation

33/40

 We want to evolve our API so we need ways to document  What is currently unfinished?  What needs to be changed in the future?

Planning Changes

API Change Documentation

34/40

 We want to mark API as unfinished  @PublishedBeta

> Changes the version semantic by shifting MAJOR to MINOR etc.

 Allows to add API which could still be changed later on

Planning Changes

API Change Documentation

35/40

 @ReworkThisAtNextPublishedApiChange(“Description”)

 Marks a certain part of the API, that it has to be changed in the next MINOR

version Planning Changes

API Change Documentation

36/40

 @ReworkThisAtNextBreakingChange(“Description”)  Marks a certain part of the API, that it has to be change in the next MAJOR

version

 This is especially useful, if the next major release is way ahead

 We can search for all usages of the annotation when a new major release

is done

Planning Changes

API Change Documentation

37/40

Introduction Why stable API matters PublishedApi PublishedApi PostProcessing API Change Documentation  References

Agenda

38/40

 http://vector.com/  http://semver.org/  https://wiki.eclipse.org/Evolving_Java-based_APIs_2  http://asm.ow2.org/

Sources and References

References

39/40

 Your questions are welcome!

 Andreas Turban  Andreas.Turban@vector.com

> Work E-Mail

 Andreas.Turban@net-baustelle.de

> Private E-Mail

Questions

40/40