CS 451 Software Engineering Winter 2009 Yuanfang Cai Room 104, - - PowerPoint PPT Presentation

Drexel University

CS 451 Software Engineering Winter 2009

1

Yuanfang Cai Room 104, University Crossings 215.895.0298 yfcai@cs.drexel.edu

Drexel University

Validation and Verification

 Validation: a set of activities that ensures that

the software that has been built is traceable to customer requirements.

 Are we building the right product?

 Verification: the set of activities the ensures

that software correctly implements a specific function.

 Are we building the product right?

Drexel University

Testing

 Testing only reveals the presence of defects  Does not identify nature and location of defects  Identifying & removing the defect => role of

debugging and rework

 Preparing test cases, performing testing,

defects identification & removal all consume effort

 Overall testing becomes very expensive : 30-

50% development cost

Drexel University

Levels of Testing

 The code contains requirement defects, design

defects, and coding defects

 Nature of defects is different for different

injection stages

 One type of testing will be unable to detect the

different types of defects

 Different levels of testing are used to uncover

these defects

Drexel University

User needs Acceptance testing Requirement specification System testing Design code Integration testing Unit testing

Levels of Testing

Drexel University

Unit Testing

Drexel University

Unit Testing

 Different modules tested separately  Focus: defects injected during coding  Essentially a code verification technique,

covered in previous chapter

 UT is closely associated with coding  Frequently the programmer does UT; coding

phase sometimes called “coding and unit testing”

Drexel University

Testing Strategies

 Unit Testing – focuses verification effort on the smallest

unit of software design. (The component or module)

 Considerations:



The module interface is tested to ensure that information properly flows into and out of the program unit under test.



All independent paths through the control structure are exercised to ensure that all statements in a module have been executed at least once.



Boundary conditions are tested to ensure that the module operates properly.



Error handling paths are tested.

8

Drexel University

Testing Strategies

 Unit Testing

Procedure

 Unit testing is

normally considered as an adjunct to the coding step.

 Design of tests

can be done before coding begins (Agile programming)

9

Drexel University

Testing Strategies

 Unit Testing Procedure

 Since each component is not a stand alone program, a

bit of work must be done to perform the test.

 Create a driver, usually a main program, that accepts test

case data and passes it to the component to be tested. It also may print relevant results.

 Stubs replace modules that may not be coded and are

subordinate to the component to be tested. A stub may do minimum data manipulation , provides verification of entry, and returns control to the module undergoing testing.

 Unit testing is simpler when modules are highly

cohesive.

10

Drexel University

Integration Testing

Drexel University

Integration Testing

 Focuses on interaction of modules in a

subsystem

 Unit tested modules combined to form

subsystems

 Test cases to “exercise” the interaction of

modules in different ways

 May be skipped if the system is not too large

Drexel University

Integration Testing



Why is unit testing not enough?



Data can be lost across an interface.



One module can have an inadvertent effect on another.



Global data structures can present problems



Individually acceptable imprecision may become magnified.



Integration testing is a systematic technique for constructing the software architecture while at the same time conducting tests to uncover errors associated with interfacing.



The objective is to take unit tested components and build a program structure that has been dictated by design.

Drexel University

Integration Testing



There is a tendency to attempt nonincremental integration – construct the whole program via a “big- bang” approach.



Thus to combine all components at once and test it as an entire program.



Top Down Integration is an incremental approach to construction of the software architecture. Modules are integrated by moving downward through the control hierarchy, beginning with the main control module.

Drexel University

Top Down Integration



Modules subordinate to the main control module are incorporated into the structure in either a depth first or breath first manner.

Drexel University

Integration Testing



The integration process is preformed in five steps:

1.

The main control module is used as a test driver, and stubs are substituted for all components directly subordinate to the main control module.

2.

Depending on the integration approach selected (i.e. depth or breath first), subordinate stubs are replaced

• ne at a time with actual components.

3.

Tests are conducted as each component is integrated.

4.

On completion of each set of tests, another stub is replaced with the real component.

5.

Regression testing may be conducted to ensure that new errors have not been introduced.

Drexel University

Bottom Up Integration



Begins construction and testing with atomic modules. Since all subordinates are always available, there is no need for stubs.



Bottom up integration follows the following steps:

1.

Low-level components are combined into clusters that perform a specific software subfunction.

2.

A driver (control program for testing) is written to coordinate test cases input and output.

3.

The cluster is tested.

4.

Drivers are removed and clusters are combined moving upward in the program structure.



One major disadvantage of bottom up integration is that the program as an entity does not exist until the last module is added.

Drexel University

Bottom Up Integration

Drexel University

System Testing

 Entire software system is tested  Focus: does the software implement the

requirements?

 Validation exercise for the system with

respect to the requirements

 Generally the final testing stage before the

software is delivered

 May be done by independent people  Defects removed by developers  Most time consuming test phase

Drexel University

Acceptance Testing

 Focus: Does the software satisfy user needs?  Generally done by end users/customer in

customer environment, with real data

 Only after successful AT software is deployed  Any defects found,are removed by developers  Acceptance test plan is based on the

acceptance test criteria in the SRS

Drexel University

Other forms of testing

 Performance testing

 tools needed to “measure” performance

 Stress testing

 load the system to peak, load generation tools

needed

 Regression testing

 test that previous functionality works alright  important when changes are made  Previous test records are needed for comparisons  Prioritization of testcases needed when complete test

suite cannot be executed for a change

Drexel University

Regression Testing

Drexel University

Regression Testing



Each time a new module is added, the software changes.



These changes may cause problems with functions that previously worked.



Therefore, we must re-execute some subset of tests that were already conducted to ensure that changes have not propagated unintended side effects.



Any time you test, if successful you will find errors. When those errors are corrected, software changes. Therefore, regression testing must be performed in order to ensure that those changes did not propagate unintended side effects.



Regression tests may be conducted manually – I do not recommend this.

Drexel University

System Testing



Recovery Testing – most computer-based systems must recover from faults and resume processing within a prespecified time.



Recovery testing is a system test that forces the software to fail in a variety of ways and verifies that recovery is properly preformed.

Drexel University

Security Testing



Any computer system that manages sensitive information or causes actions that can improperly harm or benefit individuals is a target for improper or illegal penetration.



Lot’s of security issues.



Security testing verifies that protection mechanisms built into the system will, in fact, protect it from improper penetration.

Drexel University

Security Testing



During security testing, the tester plays the role(s) of individual who desires to penetrate the system.



Usually it’s not the software, but people issues.



People often give out secure information. No system can truly protect against this 100%.

Drexel University

Stress Testing



So far all testing assumed normal program functions and performance.



Stress tests are designed to confront programs with abnormal quantity, frequency, or volume.



Multiple interrupts per second



Increased data rate



Test max memory



Excessive disk hunting

Drexel University

Performance Testing



Designed to test the run-time performance of software within the context of an integrated system.



Should be performed at all steps of the testing process.



Not until system elements are fully integrated that performance of a system can really be ascertained.



Often requires hardware and software instrumentation to measure resource utilization.

Drexel University

Defect logging and tracking

 A large software may have thousands of defects,

found by many different people

 Often person who fixes (usually the coder) is

different from who finds

 Due to large scope, reporting and fixing of

defects cannot be done informally

 Defects found are usually logged in a defect

tracking system and then tracked to closure

 Defect logging and tracking is one of the best

practices in industry

Drexel University

Defect logging…

 A defect in a software project has a life cycle of

its own, like

 Found by someone, sometime and logged along with

info about it (submitted)

 Job of fixing is assigned; person debugs and then

fixes (fixed)

 The manager or the submitter verifies that the defect

is indeed fixed (closed)

 More elaborate life cycles possible

Drexel University

Organizing Testing

 Software testing may be viewed as a spiral process:

31

Drexel University

Organizing Testing

 Software Testing Steps

32

Drexel University

Software Testing Strategies

 How do you know when you are done testing?

 There is no definitive answer to this question.

 Many possible answers:

 You are never done testing.  You are done testing when you run out of money or

time.

 Depends upon the certainty of failure you desire.

33

Drexel University

Summary

 Validation and Verification  Different Types of Testing

 Context  purposes