So why the big fuss about Design by Contract? Bertrand Meyer - - PowerPoint PPT Presentation
So why the big fuss about Design by Contract? Bertrand Meyer - - PowerPoint PPT Presentation
So why the big fuss about Design by Contract? Bertrand Meyer Software Architecture, March 2008 Verification: the vision Guardian angel approach: environment checks your software as you write it Squiggly red underscore of
2
Verification: the vision
“Guardian angel” approach: environment checks your software as you write it “Squiggly red underscore” of Microsoft Word Combination of techniques:
- Automatic testing through AutoTest
- Manual tests
- Regression tests
- CDD
- Static analysis
- Proofs
All of this integrated into EiffelStudio
3
Design by Contract
A discipline of analysis, design, implementation, management
4
Design by Contract
Together with the implementation (“how”) of each software element, describe “what” it is supposed to do: its contract Three basic questions about every software element:
- What does it assume?
- What does it guarantee?
- What does it maintain?
Precondition Postcondition Invariant
5
With and without contracts
.Net collections library EiffelBase Karine Arnout (IEEE Computer)
6
Applications of Design by Contract
- Getting the software right
- Analysis
- Design
- Implementation
- Debugging
- Testing
- Management
- Maintenance
- Documentation
7
The metaphor: contracts in business
A contract:
- Binds two parties (or more): supplier, client
- Is explicit (written)
- Specifies mutual obligations and benefits
- Usually maps obligation for one of the parties into
benefit for the other, and conversely
- Has no hidden clauses: obligations are those specified
- Often relies, implicitly or explicitly, on general rules
applicable to all contracts: laws, regulations, standard practices
8
A human contract
Client Supplier
(Satisfy precondition:) Bring package before 4 p.m.; pay fee. (Satisfy postcondition:) Deliver package by 10 a.m. next day.
OBLIGATIONS
(From postcondition:) Get package delivered by 10 a.m. next day. (From precondition:) Not required to do anything if package delivered after 4 p.m.,
- r fee not paid.
BENEFITS
deliver
9
A view of software construction
Constructing systems as structured collections of cooperating software elements — suppliers and clients — cooperating on the basis of clear definitions of obligations and benefits These definitions are the contracts
10
Correctness in software
Correctness is a relative notion: consistency of implementation vis-à-vis specification. Basic notation: (P, Q : assertions, i.e. properties of the state of the computation. A : instructions). {P } A {Q } “Hoare triple” What this means (total correctness):
- Any execution of A started in a state satisfying P will
terminate in a state satisfying Q.
11
Hoare triples: a simple example
{n > 5} n := n + 9 {n > 13} Most interesting properties:
- Strongest postcondition (from given precondition).
- Weakest precondition (from given postcondition).
“P is stronger than or equal to Q ” means: P implies Q QUIZ: What is the strongest possible assertion? The weakest?
12
A class without contracts
class ACCOUNT feature -- Access balance : INTEGER
- - Balance
Minimum_balance: INTEGER = 1000
- - Minimum balance
feature {NONE } -- Deposit and withdrawal add (sum : INTEGER)
- - Add sum to the balance.
do balance := balance + sum end Secret features
13
A class without contracts
feature -- Deposit and withdrawal operations deposit (sum : INTEGER)
- - Deposit sum into the account.
do add (sum) end withdraw (sum : INTEGER)
- - Withdraw sum from the account.
do add (– sum) end may_withdraw (sum : INTEGER): BOOLEAN
- - Is it permitted to withdraw sum from the account?
do Result := (balance - sum >= Minimum_balance) end end
14
Introducing contracts
class ACCOUNT create make feature {NONE } -- Initialization make (initial_amount: INTEGER)
- - Set up account with initial_amount
require do balance := initial_amount ensure end
15
Introducing contracts
feature -- Access balance: INTEGER
- - Balance
Minimum_balance : INTEGER = 1000
- - Lowest permitted balance
feature {NONE} -- Implementation of deposit and withdrawal add (sum : INTEGER)
- - Add sum to the balance.
do balance := balance + sum ensure increased: balance = old balance + sum end
16
Introducing contracts
feature -- Deposit and withdrawal operations deposit (sum : INTEGER)
- - Deposit sum into the account.
require not_too_small: sum >= 0 do add (sum) ensure increased: balance = old balance + sum end Precondition Postcondition
17
Introducing contracts
withdraw (sum : INTEGER)
- - Withdraw sum from the account.
require not_too_small: sum >= 0 not_too_big: sum <= balance – Minimum_balance do add (–sum)
- - i.e. balance := balance – sum
ensure decreased: balance = old balance - sum end Value of balance, captured
- n entry to routine
18
The contract
Client Supplier
(Satisfy precondition:) Make sure sum is neither too small nor too big (Satisfy postcondition:) Update account for withdrawal of sum
OBLIGATIONS
(From postcondition:) Get account updated with sum withdrawn (From precondition:) Simpler processing: may assume sum is within allowable bounds
BENEFITS
withdraw
19
The imperative and the applicative
do balance := balance - sum ensure balance = old balance - sum PRESCRIPTIVE DESCRIPTIVE How? Operational Implementation Command Dynamic What? Denotational Specification Query Static
20
Introducing contracts
may_withdraw (sum : INTEGER ): BOOLEAN
- - Is it permitted to withdraw sum from account?
do Result := (balance - sum >= Minimum_balance) end invariant not_under_minimum: balance >= Minimum_balance end
21
The class invariant
Consistency constraint applicable to all instances of a class. Must be satisfied:
- After creation
- After execution of any feature by any client
Qualified calls only: x.f (...)
22
The correctness of a class
For every creation procedure cp :
{Precp } docp {INV and Postcp }
For every exported routine r :
{INV and Prer } dor {INV and Postr }
x.f (…) x.g (…) x.h (…) create x.make (…) S1
S2 S3 S4
23
What are contracts good for?
Writing correct software (analysis, design, implementation, maintenance, reengineering) Documentation (the “contract” form of a class) Effective reuse Controlling inheritance Preserving the work of the best developers Quality assurance, testing, debugging (especially in connection with the use of libraries) Exception handling
24
Object-oriented (requirements) analysis
Same benefits as O-O programming, in particular extendibility and reusability Direct modeling of the problem domain Seamlessness and reversibility with the continuation of the project (design, implementation, maintenance)
25
Contracts for analysis
Specify semantics, but abstractly! Basic dilemma of requirements:
- Committing too early to an implementation
Overspecification
- Missing parts of the problem
Underspecification
26
Television station example
Source: OOSC
27
Schedules
note description : “ 24-hour TV schedules ” deferred class SCHEDULE feature segments: LIST [SEGMENT ]
- - Successive segments
deferred end air_time : DATE is
- - 24-hour period
- - for this schedule
deferred end set_air_time (t : DATE)
- - Assign schedule to
- - be broadcast at time t.
require t.in_future deferred ensure air_time = t end print
- - Produce paper version.
deferred end end
28
Segment
note description : “Fragments of a schedule " deferred class SEGMENT feature schedule : SCHEDULE deferred end
- - Schedule to which segment belongs
index : INTEGER deferred end
- - Position of segment in
- - its schedule
starting_time, ending_time : INTEGER deferred end
- - Beginning and end of
- - scheduled air time
next: SEGMENT deferred end
- - Segment to be played
- - next, if any
sponsor: COMPANY deferred end
- - Segment’s principal sponsor
rating : INTEGER deferred end
- - Segment’s rating (for
- - children’s viewing etc.)
… Commands such as change_next, set_sponsor, set_rating omitted … Minimum_duration : INTEGER = 30
- - Minimum length of segments,
- - in seconds
Max_interval: INTEGER = 2
- - Maximum time between two
- - successive segments, in seconds
29
Segment (continued)
invariant in_list: (1 <= index) and (index <= schedule.segments.count) in_schedule: schedule.segments.item (index) = Current next_in_list: (next /= Void ) implies (schedule.segments.item (index + 1) = next) no_next_iff_last: (next = Void) = (index = schedule.segments.count) non_negative_rating: rating >= 0 positive_times: (starting_time > 0 ) and (ending_time > 0) sufficient_duration: ending_time – starting_time >= Minimum_duration decent_interval : (next.starting_time) - ending_time <= Max_interval end
30
Commercial
note description: "Advertising segment " deferred class COMMERCIAL inherit SEGMENT rename sponsor as advertiser end feature primary: PROGRAM deferred
- - Program to which this
- - commercial is attached
primary_index: INTEGER deferred
- - Index of primary
set_primary (p : PROGRAM)
- - Attach commercial to p.
require program_exists: p /= Void same_schedule: p,schedule = schedule before: p.starting_time <= starting_time deferred ensure index_updated: primary_index = p.index primary_updated: primary = p end invariant meaningful_primary_index: primary_index = primary.index primary_before: primary.starting_time <= starting_time acceptable_sponsor: advertizer.compatible (primary.sponsor) acceptable_rating: rating <= primary.rating end
31
Diagrams: UML, BON
Text-Graphics Equivalence
32
Contracts and inheritance
Issues: what happens, under inheritance, to
- Class invariants?
- Routine preconditions and postconditions?
33
Invariants
Invariant Inheritance rule:
- The invariant of a class automatically includes the
invariant clauses from all its parents, “and”-ed. Accumulated result visible in flat and interface forms.
34
Contracts and inheritance
r require
γ
ensure
δ
r require
α
ensure
β
a1: A a1.r (…) …
Correct call in C: if a1.α then a1.r (...)
- - Here a1.β holds
end
r ++
C A D B
Client Inheritance
++ Redefinition
35
Assertion redeclaration rule
When redeclaring a routine, we may only:
- Keep or weaken the precondition
- Keep or strengthen the postcondition
36
A simple language rule does the trick! Redefined version may have nothing (assertions kept by default),
- r
require else new_pre ensure then new_post Resulting assertions are:
- original_precondition or new_pre
- original_postcondition and new_post
Assertion redeclaration rule in Eiffel
37
A contract violation is not a special case
For special cases (e.g. “if the sum is negative, report an error...”) use standard control structures, such as if ... then ... else... A run-time assertion violation is something else: the manifestation of A DEFECT (“BUG”)
38
Contracts and quality assurance
Precondition violation: Bug in the client. Postcondition violation: Bug in the supplier. Invariant violation: Bug in the supplier. {P } A {Q }
39
Contracts: run-time effect
Compilation options (per class, in Eiffel):
- No assertion checking
- Preconditions only
- Preconditions and postconditions
- Preconditions, postconditions, class invariants
- All assertions
40
Contracts for testing and debugging
Contracts express implicit assumptions behind code
- A bug is a discrepancy between intent and code
- Contracts state the intent!
In EiffelStudio: select compilation option for run-time contract monitoring at level of:
- Class
- Cluster
- System
May disable monitoring when releasing software A revolutionary form of quality assurance
41
Contract monitoring
A contract violation always signals a bug:
- Precondition violation: bug in client
- Postcondition violation: bug in routine
42
Contracts and bug types
Preconditions are particularly useful to find bugs in client code:
YOUR APPLICATION COMPONENT LIBRARY
your_list.insert (y, a + b + 1) i <= count + 1 insert (x : G ; i : INTEGER) require i >= 0 class LIST [G ] feature
43
Contract monitoring
- Enabled or disabled by compile-time options.
- Default: preconditions only.
- In development: use “all assertions” whenever possible.
- During operation: normally, should disable monitoring.
But have an assertion-monitoring version ready for shipping.
- Result of an assertion violation: exception.
Ideally: static checking (proofs) rather than dynamic monitoring.
44
The documentation issue
Who will do the program documentation (technical writers, developers) ? How to ensure that it doesn’t diverge from the code (the French driver’s license / reverse Dorian Gray syndrome) ?
The Single Product principle The product is the software
45
Contract form: Definition
Simplified form of class text, retaining interface elements
- nly:
- Remove any non-exported (private) feature
For the exported (public) features:
- Remove body (do clause)
- Keep header comment if present
- Keep contracts: preconditions, postconditions, invariant
- Remove any contract clause that refers to a secret
feature (This raises a problem; can you see it?)
46
Contract form of ACCOUNT class
class interface ACCOUNT create make feature balance: INTEGER
- - Balance
Minimum_balance : INTEGER = 1000
- - Minimum balance
deposit (sum: INTEGER)
- - Deposit sum into the account.
require not_too_small: sum >= 0 ensure increased: balance = old balance + sum
47
Contract form (continued)
withdraw (sum: INTEGER)
- - Withdraw sum from the account.
require not_too_small: sum >= 0 not_too_big: sum <= balance – Minimum_balance ensure decreased: balance = old balance – sum
- ne_more: withdrawals.count = old withdrawals.count + 1
may_withdraw (sum: INTEGER): BOOLEAN
- - Is it permitted to withdraw sum from the account?
invariant not_under_minimum: balance >= Minimum_balance consistent: balance = deposits.total – withdrawals.total end
48
Flat, interface
Flat form of a class: reconstructed class with all the features at the same level (immediate and inherited). Takes renaming, redefinition etc. into account. The flat form is an inheritance-free client-equivalent form
- f the class.
Interface form: the contract form of the flat form. Full interface documentation.
49
Uses of the contract and interface forms
Documentation, manuals Design Communication between developers Communication between developers and managers
50
50
A class with contracts
class BANK_ACCOUNT create make feature make (n : STRING)
- - Set up with name n
require n /= Void do
name := n balance := 0
ensure name = n end
name : STRING balance : INTEGER deposit ( v : INTEGER)
- - Add amount v
do balance := balance + v ensure balance = old balance + v end invariant name /= Void balance >= 0 end ensure name = n require n /= Void ensure balance = old balance + v invariant name /= Void balance >= 0
51
51
What we do with contracts today
Write better software Analyze Design Reuse Implement Use inheritance properly Avoid bugs Document software automatically Help project managers do their job Perform systematic testing Guide the debugging process (with run-time monitoring)
52
Anecdotal & non-anecdotal evidence
HP 1: invariant r = 2 ^ i HP 2: Eiffel messes up our great system! Axa Rosenberg: postcondition fails in deep_clone of TWO_WAY_LIST ! Patrice Chalin study (Concordia): Eiffel programmers do use contracts day in, day out.
53
53
Contracts for testing and debugging
Contracts provide the right basis:
- A bug is a discrepancy between intent and reality
- Contracts describe intent
A contract violation always signals a bug:
- Precondition violation: bug in client
- Postcondition violation: bug in routine
In EiffelStudio: select compilation option for contract monitoring at level of class, cluster or system.
54
54
Should we test software?
- Proofs don’t always succeed
- Some properties are not formally specified
Tests as complement of proofs?
55
From a survey of 240 companies in N. America and Europe:
- 8% release software to beta sites without testing
- 83% of their developers don't like to test code.
- 53% don't like to test their own code because they
find it tedious.
- 30% don't like to test because they find testing
tools inadequate.
Testing is tedious
56
56
“Automated testing”
What can be automated:
- Test execution
- Robust execution
- Regression testing
- Test case generation (test suites)
- Test result verification (test oracles)
- Test case minimization
57
57
AutoTest: an automatic test framework
- Input is set of classes & time
- AutoTest generates instances and
calls features with automatically selected arguments
- Oracles are contracts:
- Precondition violations: skip
- Postcondition/invariant violation: bingo!
- Manual tests can be added explicitly
- Any test (manual or automated) that fails becomes part
- f the test suite
Andreas Leitner Ilinca Ciupa
58
58
Dynamic slicing
auto_test system.ace –t 120 BANK_ACCOUNT
create {STRING} v1 v1.wipe_out v1.append_character (’c’) v1.append_double (2.45) create {STRING} v2 v1.append_string (v2) v2.fill (’g’, 254343) ... create {BANK_ACCOUNT} v3.make (v2) v3.deposit (15) v3.deposit (100) v3.deposit (-8901) ...
class BANK_ACCOUNT create make feature make (n : STRING) require n /= Void do name := n balance := 0 ensure name = n end name : STRING balance : INTEGER deposit (v : INTEGER) do balance := balance + v ensure balance =
- ld balance + v
end invariant name /= Void balance >= 0 end
59
The testbed: EiffelBase
- Version of September 2005
- 20-year history
- Showcase of Eiffel technology
- About 1800classes, 20,000 SLOC
- Extensive (but not complete) contracts
- Widely used in production applications
- Significant bugs remain
60
60
Some AutoTest results (random strategy)
Library
Total Failed Total Failed EiffelBase (Sep 2005) 40,000 3% 2000 6% Gobo Math 1500 1% 140 6% TESTS ROUTINES
61
61
AutoTest strategies
- Object pool
- For each type, initialize through creation procedures
(constructors)
- This requires proper ordering of classes
- Diversify through procedures
- Routine arguments
- Basic values: heuristics for each type
- Objects: get from pool
- Test all routines, including inherited ones (“Fragile base
class” issue)
62
Testing results and strategy
“Smart” ideas not always better Don’t believe your intuition Don’t believe standard criteria Measure and assess objectively
# bugs Class STRING
Define good assessment criteria:
- Very bad: number of tests to first bug
- Bad: time to first bug
- Better:
- Number of bugs found
- Time to find all bugs
Timeout (minutes)
Experimental law: n = a / (1 + b * t )
63
63
Conjecture: Random testing may find bugs faster if inputs evenly spread So far: basic types m n
Adaptive Random Testing (Chen et al.)
Our contribution: extend this to objects Need to define notion of distance between objects
64
64
Object distance
p ↔ q combination ( type_distance (p.type, q.type), field_distance (p, q), recursive_distance ( {[p.r ↔ q.r] | r ∈ Reference_attributes } )
=
Δ
Ilinca Ciupa
65
ART vs pure random
Results so far:
- Does not find more bugs
- Does not find bugs faster
- Finds other bugs!
66
Boolean query conjecture
The argument-less boolean queries of a well-written class yield a partition of the object state space that helps the testing process
67
“Zurich" Cursor index count 1 after before is_first is_last
68
Boolean query coverage
A set of tests for a class satisfies boolean query coverage if and only if the execution of these tests can cover all the reachable abstract object states for that class. Strategy:
- Through constraint solving, generate states that
satisfy boolean queries in invariant (e.g. after implies off)
- Through theorem proving, remove states that do
not satisfy rest of invariant (e.g. count < capacity)
69
Experimental results
Class Random Boolean Queries Routine Coverage Bugs Found Routine Coverage Bugs Found INT_STACK 33% 1 100% 2 LINKED_LIST 85% 1 99% 7 BINARY_TREE 88% 5 100% 11 ARRAYED_SET 84% 1 100% 6 FIXED_LIST 93% 12 99% 12
70
AutoTest developments
- Large-scale extensive tests (cluster computing)
- Comparison with manual efforts
- Scaling up (memory, time) (mostly done)
- Rigorous assessment of testing criteria & strategies
- Complete integration with EiffelStudio environment
- Background, unobtrusive, continuous testing
- Automatic regression testing
- Distributed cooperative testing (“Testi@home”)
71
“Contract-Driven Development”
CDD = TDD — WTC*
*Writing Test Cases Andreas Leitner
72
CDD principles
Contracts (specifications) are better than tests
- One can derive tests from a spec
- Not the other way around!
Record every failed execution Make it reproducible by retaining objects Turn it into a regression test, to be kept forever
73
Current status
AutoTest: separate tool, available for download, operational CDD: separate tool, experimental, available for download Both techniques are being integrated as a standard part of EiffelStudio Unobtrusive approach: “red squiggly underscore” Tests, proofs, model checking all complementary
74
Verification: the vision
“Guardian angel” approach: environment checks your software as you write it “Squiggly red underscore” of Microsoft Word Combination of techniques:
- Automatic testing through AutoTest
- Manual tests
- Regression tests
- CDD
- Static analysis
- Proofs
All of this integrated into EiffelStudio
75
75
The next step: proving classes correct?
Some issues:
- Providing full contracts
- Devising theories for target worlds
- Devising proof rules (language semantics)
- Specifying and proving what programs do not do
(the frame issue)
- Taking advantage of inheritance on the proof side
76
Eiffel Model Library (also known as MML)
Classes correspond to mathematical concepts: SET [G], FUNCTION [G, H ], TOTAL_FUNCTION [G, H ], RELATION [G, H ], SEQUENCE [G ], … Completely applicative: no attributes (fields), no implemented routines (all completely deferred) Specified with contracts (unproven) reflecting mathematical properties Expressed entirely in Eiffel Bernd Schoeller Tobias Widmer
77
Example MML class
class SEQUENCE [G] feature count : NATURAL
- - Number of items
last : G
- - Last item
extended (x) : SEQUENCE [G]
- - Identical sequence except x added at end
ensure Result.count = count + 1 Result.last = x Result.sub (1, count ) ~ Current mirrored : SEQUENCE [G]
- - Same items in reverse order
ensure Result.count = count … … end
78
Specifying lists
class LINKED_LIST [G] feature … remove_front
- - Remove first item
require not empty do first := first.right ensure end … end
first right right right
count = old count – 1 first = old item (2) model = old model.tail
79
Verification: the vision
“Guardian angel” approach: environment checks your software as you write it “Squiggly red underscore” of Microsoft Word Combination of techniques:
- Automatic testing through AutoTest
- Manual tests
- Regression tests
- CDD
- Static analysis
- Proofs
All of this integrated into EiffelStudio
80
The three questions
- What does it expect?
- What does it promise?
- What does it maintain?