A Research Framework for Empirical Software Engineering - - PowerPoint PPT Presentation

Software Engineering Center

Information-technology Promotion Agency, Japan

A Research Framework for Empirical Software Engineering Collaboration and Its Application in a Software Development Project

Yoshiki Mitani*, Nahomi Kikuchi**, Tomoko Matsumura***, Satoshi Iwamura****, Mike Barker***, Ken-ichi Matsumoto***

*IPA/SEC,NAIST(EASE),**IPA/SEC,***NAIST(EASE),****NTT Software(EASE)

Demonstrate and verify empirical software engineering methods A new research framework In response to government policy Field data sharing between Industry & Academia Concept of a “Macro-measurement tools” Target project: A Governmental Multi-Vendor Consortium type Software Project

IWFST2005 1

IPA: Information Technology Promotion Agency, Japan SEC: Software Engineering Center EASE: Empirical Approach to Software Engineering project NAIST: Nara Institute of Science and Technology

In-process Project Measurement and Feedback Structure

IWFST2005 2

Bug Tracking System

Mailing-list Management System

Review Report Sheet Mail Bug Report XML Standard Data Format RDB EPM Basic Analyzer EPM Extended Analyzer Code Clone Analyzer Collaborating Filtering tool Feedback to in-process Software Development Project Development Environment Program Development Benchmark Data Report SEC Benchmark Database (over 1,000projects)

Configuration Management System Source Code

40 Management Items Operation Rule, Input Web Template

Operation Rule

20 Management Items Operation Rule, Input Template

(CVS) (GNATS)

Operation Rile

400 Items

Q&A Check-list Interview Check-list Analyzer

400 Management Items

1 2 3 4 5

統合 スコ ープ タ イ ム コ スト 品質 人的資源 コ ミ ュ ニケーショ ン リ スク 調達 顧客 技術 組織 基本動作 モチベーショ ン 課題管理

1 2 3 4 5

統合 スコ ープ タ イ ム コ スト 品質 人的資源 コ ミ ュ ニケーショ ン リ スク 調達 顧客 技術 組織 基本動作 モチベーショ ン 課題管理

Bug Management e-Mail Review Q&A Interview to PM

Table1 Measurement Data Items

Data item Review record, Bug report(40 items), Specification Change report, Inter company mail with basic document, Basic Design Document, CVS repository with source code Tool CVS, GNATS, Mailman, EPM Data input form & template: Review report form, GNATS web template form Operation rule CVS, GNATS, Mail, EPM Collected data are kept safely in SEC secret data room. Data collecting Method Individual IT vender development process: Weekly collect data by media from CVS and GNATS. Integration test process: Use single GNATS system and collect data from it. CVS data are same as individual development process. Data item SEC defined 400 bench mark data items, IT skill data based on IPA/ITSS Data collecting Method SEC defined bench mark data input form( EXCEL template) Data collection timing; end of basic design and end of development

Project Context

Monitoring project Observe whole development process, Join to project meeting.

Ended Project Data/ Bench mark Data

Proceeding Project Data

EPM: Empirical Project Monitor ITSS: Information Technology Skill Standard

IWFST2005 3

Table 2 Data Analysis Example

Basic Analysis

Source line of code transition, Check-in opportunity, Check-out

• pportunity & frequency,

Number & opportunity of mail, Number & opportunity of bur report, Pending bug transition, Residual bug transition,

Extended Analysis

Review quality analysis from IEEE 982.1(1988):Fault Density, Defect Density, Measurement from PSP (Personal Software Process) ODC (Orthogonal Defect Classification): Review quality File renewal analysis

Bench mark Analysis

Source Code Clone Analysis Retrieve similar project from 1,000 past project benchmark data .

Skill data Analysis

Analyze human factor from IT skill data of development persons

Code Clone Analysis IWFST2005 4

ASDP: Advanced Software Development Project

CVS, GNATS, Mailman EPM,400itemBenchmarkData,ITSS EASE project, SEC MEXT, METI Prime Minister Leadership

Measurement Target Project Software Tools Measurement Platform Industry & Academia Collaboration Framework Ministry Office Policy Government Policy

Macro Measurement Tools

Trigger Budget and Policy

Development Project Management & Operation Software Tool Management & Operation Measurement Platform Management & Operation A&I Collaboration Management & Operation

National Policy

Fig.1 “Macro Measurement Tools” and its Management & Operation for Software Project Measurement

IWFST2005 5 EPM: Empirical Project Monitor ITSS: Information Technology Skill Standard EASE: Empirical Approach to Software Engineering SEC: Software Engineering Center

MEXT: Ministry of Education, Culture, Sports, Science and Technology METI: Ministry of Economy, Trade and Industry

The major factors to realize measurement and data-sharing

• The fundamental I&A collaboration

research framework

• Concept and equipment of measurement

platform

• Thoughtful measurement plan
• Total framework coordination

IWFST2005 6 Acknowledgment This work is supported by IPA/SEC, METI and MEXT of Japan as part of the Comprehensive Development of e-Society Foundation Software program. We thank the researchers in SEC and the EASE project who kindly support

• ur project.

Software Engineering Center

Information-technology Promotion Agency, Japan