A Test Adequacy Infrastructure with Database Interaction Awareness - - PowerPoint PPT Presentation

▶
a test adequacy infrastructure with database interaction
SMART_READER_LITE
LIVE PREVIEW

A Test Adequacy Infrastructure with Database Interaction Awareness - - PowerPoint PPT Presentation

Oct 20, 2022 411 likes •825 views

A Test Adequacy Infrastructure with Database Interaction Awareness Gregory M. Kapfhammer Department of Computer Science Allegheny College (in collaboration with Mary Lou Soffa) A Test Adequacy Infrastrcture with Database Interaction Awareness,

slide-1
SLIDE 1

A Test Adequacy Infrastructure with Database Interaction Awareness

Gregory M. Kapfhammer Department of Computer Science Allegheny College (in collaboration with Mary Lou Soffa)

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 1/35

slide-2
SLIDE 2

Motivation

The Risks Digest, Volume 22, Issue 64, 2003 Jeppesen reports airspace boundary problems

About 350 airspace boundaries contained in Jeppesen NavData are incorrect, the FAA has warned. The error occurred at Jeppe- sen after a software upgrade when information was pulled from a database containing 20,000 airspace boundaries worldwide for the March NavData update, which takes effect March 20.

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 2/35

slide-3
SLIDE 3

Looking Ahead

Test adequacy infrastructure that can find faults and establish confidence in the correctness of a database-centric application Model of database interaction faults Unifed application representation Family of test adequacy criteria Experiments with real applications that measure the number of test requirements and the time and space overheads incurred by enumeration Foundation for a comprehensive methodology for testing database-centric applications

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 3/35

slide-4
SLIDE 4

Testing Challenges

Must consider the environment in which software applications actually execute Should test a program and its interaction with a database Database-centric application’s state space is well-structured, but essentially infinite (Chays et al.) Need to show program does not violate database integrity, where integrity = consistency + validity (Motro) Must locate program and database coupling points that vary in granularity

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 4/35

slide-5
SLIDE 5

Database-Centric Applications

P m i m j Dl Dk

R R2

1

E F G H A B C D I

R3

J K L

Program P interacts with two relational databases Dk and Dl

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 5/35

slide-6
SLIDE 6

Database Interactions

1

D

n

D P m

update select insert delete

Program P can view and/or modify the state of the database

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 6/35

slide-7
SLIDE 7

Database Interaction Faults: (1-v)

P

m

actual

before after

expected

insert update

P uses update or

insert to incorrectly

modify items within database Commission fault that violates database validity Structural adequacy criteria can support fault isolation

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 7/35

slide-8
SLIDE 8

Database Interaction Faults: (1-c)

P

m

actual

before after

expected

delete

P uses delete to remove incorrect items from database Commission fault that violates database completeness Structural adequacy criteria can support fault isolation

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 8/35

slide-9
SLIDE 9

Database Interaction Faults: (2-v)

P

m

actual

before after

expected

delete

P does not submit

delete to remove items

from database Commission or

  • mission fault that

violates database validity Structural adequacy criteria cannot easily support omission fault isolation

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 9/35

slide-10
SLIDE 10

Database Interaction Faults: (2-c)

P

m

actual

before after

expected

insert update

P does not submit

update or insert to

database Commission or

  • mission fault that

violates database completeness Structural adequacy criteria cannot easily support omission fault isolation

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 10/35

slide-11
SLIDE 11

Database Interaction Levels

Database Level

D1

P

Dn

A program can interact with a relational database at different levels of granularity

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 11/35

slide-12
SLIDE 12

Database Interaction Levels

UserInfo

4 32142 41601 45322 56471

Relation Level

P

D1 Dn

Brian Zorman Robert Roos Marcus Bittman Geoffrey Arnold

3

card_number pin_number user_name acct_lock

2 1

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 11/35

slide-13
SLIDE 13

Database Interaction Levels

UserInfo

4 3 32142 41601 45322 56471

Record Level

P

n

D1 D

user_name acct_lock

Brian Zorman

Robert Roos Marcus Bittman Geoffrey Arnold

1 2

card_number pin_number

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 11/35

slide-14
SLIDE 14

Database Interaction Levels

UserInfo

4 1 2 3 32142 41601 45322 56471

Attribute Level

P

D1 Dn

user_name acct_lock

Brian Zorman Robert Roos Geoffrey Arnold

card_number pin_number

Marcus Bittman

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 11/35

slide-15
SLIDE 15

Database Interaction Levels

UserInfo

4 1 2 3 32142 41601 45322 56471

Attribute Value Level

P

D1

n

D

user_name acct_lock

Brian Zorman Robert Roos Geoffrey Arnold Marcus Bittman

card_number pin_number

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 11/35

slide-16
SLIDE 16

Database Interaction Points: DML

select A1, A2, . . . , Aq from r1, r2, . . . , rm where Q delete from r where Q insert into r(A1, A2, . . . , Aq) values(v1, v2, . . . , vq) update r set Al = F(Al) where Q

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 12/35

slide-17
SLIDE 17

Analyzing Database Interaction Points

Database interaction point Ir ∈ I corresponds to the execution of a SQL DML statement Consider the relevant portions of SQL that are parsed by HSQLDB (http : //hsqldb.sf.net) Interaction points are normally encoded within Java programs as dynamically constructed Strings

select uses Dk, delete defines Dk, insert defines Dk, update defines and/or uses Dk

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 13/35

slide-18
SLIDE 18

Refined Database-Centric Application

P m i m j

R R2

1

E F G H A B C D

l

D

k

D

where set J = 500 update L < 1000 R3 select

1

* from R R2 from ) select where A < ( avg(G)

I

R3

J K L

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 14/35

slide-19
SLIDE 19

Test Adequacy Concepts

P violates a database Dk’s validity when it:

(1-v) inserts entities into Dk that do not reflect real

world P violates a database Dk’s completeness when it:

(1-c) deletes entities from Dk that still reflect real

world In order to verify (1-v) and (1-c), T must cause P to define and then use entities within D1, . . . , Dn!

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 15/35

slide-20
SLIDE 20

Data Flow Information

Interaction point: “UPDATE UserInfo SET acct_lock = 1 WHERE card_number =” + card_number + “;”;

Database Level: define(BankDB) Attribute Level: define(acct_lock) and use(card_number)

Data fl

  • w information varies with respect to

the granularity of the database interaction

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 16/35

slide-21
SLIDE 21

Database Entities

UserInfo

4 1 Brian Zorman 2 Robert Roos 3 Marcus Bittman Geoffrey Arnold 41601 45322 56471 32142

user_name acct_lock pin_number card_number

v r

A (I ) = {

32142

}

1 Geoffrey Arnold

, , . . . , ,

Enumerate database entities at the attribute value level

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 17/35

slide-22
SLIDE 22

Application Types

Interaction Approach Program Location Database−Centric Applications Embedded Inside DBMS Interface Outside DBMS Testing methodology relevant to all types of applications Current tool support focuses on Interface-Outside applications

Example: Java application that submits SQL Strings to

HSQLDB relational database using JDBC drivers

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 18/35

slide-23
SLIDE 23

The DICFG: A Unified Representation

entry lockAccount

temp1 = parameter0:c_n temp2 = LocalDatabaseEntity0:Bank temp3 = LocalDatabaseEntity1:acct_lock temp4 = LocalDatabaseEntity2:card_number

“Database-enhanced” CFG for lockAccount Automatically constructed with tool support Define temporaries to represent the program’s interaction at the levels of database and attribute

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 19/35

slide-24
SLIDE 24

The DICFG: A Unified Representation

exit G G G G

r r2 r 2 r 1

1

entry entry exit lockAccount update_lock = m_connect.createStatement() if( result_lock == 1) completed = true exit

D

qu_lck = "UPDATE UserInfo ..." + temp1 + ";" use(temp4) result_lock = update_lock.executeUpdate(qu_lck) define(temp2)

A Ir

define(temp3)

Database interaction graphs (DIGs) are placed before interaction point Ir Multiple DIGs can be integrated into a single CFG String at Ir is determined in a control-fl

  • w sensitive

fashion using enhanced BRICS JSA

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 19/35

slide-25
SLIDE 25

DICFG Construction Algorithms

1 5

CreateDIG

4

CreateDICFG CreateRepresentation CreateDIGNode GetDatabaseEntities

2 3

Iteratively construct a database aware CFG to support data flow analysis and enumerate test requirements

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 20/35

slide-26
SLIDE 26

Test Adequacy Criteria

all−attribute−value−DUs all−record−DUs all−attribute−DUs all−relation−DUs all−database−DUs

Database interaction association (DIA) involves the def and use of a database entity DIAs can be located in the DICFG with data flow analysis all-database-DUs requires tests to exercise all DIAs for all

  • f the accessed databases

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 21/35

slide-27
SLIDE 27

Generating Test Requirements

Database Seeder

Database

(P) (C)

Relational Schema Requirements Test

System Under Test Test Adequacy Criterion

Test Case Specification

Measured time and space overhead when computing family of test adequacy criteria Modifi ed ATM and mp3cd to contain appropriate database interaction points Soot 1.2.5 to calculate intraprocedural associations GNU/Linux workstation with kernel 2.4.18-smp and dual 1 GHz Pentium III Xeon processors

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 22/35

slide-28
SLIDE 28

Experiment Goals and Design

Reseach Question One: Does the incorporation of

database interactions yield more test requirements?

Reseach Question Two: Can test requirement

enumeration be performed efficiently if database interactions are included?

Experiment Metrics: Number of test requirements (T R),

time overhead (T ), and space overhead (S) Applications: ATM (1732 NCSS and 136 methods) and mp3cd (2913 NCSS and 452 methods)

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 23/35

slide-29
SLIDE 29

Number of Test Requirements: ATM

D R Rc A Av P Database Granularity 500 1000 1500 2000 Number of Associations D R Rc A Av P 39 41 81 107 203 1910

80.7% increase in number of test requirements from D to Av

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 24/35

slide-30
SLIDE 30

Number of Test Requirements: mp3cd

D R Rc A Av P Database Granularity 2000 4000 6000 8000 10000 Number of Associations D R Rc A Av P 132 259 510 940 1768 8718

92.5% increase in number of test requirements from D to Av

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 25/35

slide-31
SLIDE 31

Time Overhead: ATM

P PD PR PRc PA PAv Database Granularity 5 10 15 20 25 Time sec P PD PR PRc PA PAv 20.50 20.74 20.77 20.78 20.94 21.06

2.7% increase in time overhead from P to P + Av

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 26/35

slide-32
SLIDE 32

Time Overhead: mp3cd

P PD PR PRc PA PAv Database Granularity 10 20 30 40 Time sec P PD PR PRc PA PAv 32.72 34.66 34.82 35.13 36.37 38.23

14.4% increase in time overhead from P to P + Av

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 27/35

slide-33
SLIDE 33

Space Overhead: ATM

5 10 15 20 Node and Edge Count PAv PA PRc PR PD P Database Granularity PAv PA PRc PR PD P 17.5 16.5 16.2 15.8 15.8 15.5 17.2 16.2 15.9 15.5 15.4 15.1

Edges Nodes

Average number of {DI}CFG nodes and edges is stable

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 28/35

slide-34
SLIDE 34

Space Overhead: mp3cd

5 10 15 20 25 30 35 Node and Edge Count PAv PA PRc PR PD P Database Granularity PAv PA PRc PR PD P 29.8 26.5 24.8 23.7 23.2 22.7 24.5 22.0 20.7 19.9 19.5 19.2 Edges Nodes

mp3cd has more database interactions and larger database

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 29/35

slide-35
SLIDE 35

Average Increase in CFG Nodes

SN %

I (R, D) SN % I (Rc, R) SN % I (A, R)

ATM .6 2.5 4.3 mp3cd 2.0 3.8 9.5 SN %

I (Av, Rc) SN % I (Av, A) SN % I (Av, D) SN % I (Av, P)

ATM 7.5 5.8 10.4 12.2 mp3cd 15.5 10.2 20.4 21.6

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 30/35

slide-36
SLIDE 36

Average Increase in CFG Edges

SE%

I (R, D) SE% I (Rc, R) SE% I (A, R)

ATM 0.0 2.4 4.2 mp3cd 2.1 4.4 10.5 SE%

I (Av, Rc) SE% I (Av, A) SE% I (Av, D) SE% I (Av, P)

ATM 7.4 5.7 9.7 11.4 mp3cd 16.7 11.0 22.1 23.8

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 31/35

slide-37
SLIDE 37

Related Work

Jin and Offutt and Whittaker and Voas have suggested that the environment of a software system is important Chan and Cheung transform SQL statements into C code segments Chays et al. and Chays and Deng have created the category-partition inspired AGENDA tool suite Neufeld et al. and Zhang et al. have proposed techniques for database state generation Dauo et al. focused on the regression testing of database-driven applications

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 32/35

slide-38
SLIDE 38

Ongoing Research

Test suite execution that minimizes number of costly database restarts and initializations Test coverage monitoring through a database interaction calling context tree (DICCT) Regression test suite reduction and prioritization that incorporates database aware adequacy and test case cost Detailed empirical studies with ten case study applications

  • f varying code and database size

Comprehensive tool support to assist the testing of database-centric applications

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 33/35

slide-39
SLIDE 39

Conclusions

Must test the program’s interaction with the database Test adequacy infrastructure provides : (i) database interaction fault model, (ii) unifi ed application representation, (iii) family of test adequacy criteria Unique family of test adequacy criteria to detect all type (1) and some type (2) violations of database validity and completeness Intraprocedural database interactions can be computed from a DICFG with minimal time and space overhead Foundation for a complete testing methodology

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 34/35

slide-40
SLIDE 40

Further Resources

Gregory M. Kapfhammer and Mary Lou Soffa. A Family of Test Adequacy Criteria for Database-Driven Applications. In ESEC/FSE 2003. Gregory M. Kapfhammer. Software Testing. CRC Press Computer Science Handbook. June, 2004. http : //cs.allegheny.edu/˜gkapfham/research/diatoms/

A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 35/35