LAB OF SOFTWARE ARCHITECTURES
AND INFORMATION SYSTEMS
FACULTY OF INFORMATICS
MASARYK UNIVERSITY, BRNO
PV260 - SOFTWARE QUALITY
LECT 12. Software Quality Management Process
Bruno Rossi
brossi@mail.muni.cz
2-72
●
Software Quality Management (SQM) introduction
●
The impact of Quality on Users' choices
●
SQM categories
– Software Quality Planning (SQP)
– Software Quality Assurance (SQA)
– Software Quality Control (SQC)
– Software Process Improvement (SPI)
●
Software Process Maturity Levels
– Process Levels and attributes
– ISO/IEC 15504 (SPICE)
– Mapping CMMI to ISO/IEC 15504
●
Models for Software Quality
– Personal Software Process (PSP)
– Agile Process Maturity
– Open Source Maturity Model (OMM)
– Six Sigma
●
Do better processes lead to a better software product?
Outline
"Essentially, all models are wrong,"Essentially, all models are wrong,
but some are useful"but some are useful"
George BoxGeorge Box
4-72
●
Kitchenham gave five perspectives of software quality:
1. Transcendental view
→ can be recognized but difficult to define exactly
2. User view
→ fitness for purpose
3. Manufacturing view
→ conformance to specification
4. Product view
→ from inherent product characteristics
5. Value-based view
→ depends on customer’s willingness to pay
Introduction
B. Kitchenham and S. L. Pfleeger, “Software quality: the elusive target [special issues section],”
IEEE Software, vol. 13, no. 1, pp. 12–21, Jan. 1996.
5-72
© 2012 Steve Easterbrook creative commons license.
Introduction
We deal with process
quality in this lecture
6-72
Introduction
●
Quality is a concept that starts with the development
process, goes on with the software product and finally to
the user with the effects of software usage
7-72
●
“Software quality management (SQM) is the collection of all processes
that ensure that software products, services, and life cycle process
implementations meet organizational software quality objectives and
achieve stakeholder satisfaction” (SWEBOK 3.0)
●
SQM defines processes, process owners, requirements for the processes,
measurements of the processes and their outputs, and feedback
channels throughout the whole software life cycle.
What is Software Quality Management (SQM)
8-72
The Impact of Quality
9-72
How much Quality is needed? Should we have 100% quality in our software
products?
Think about some of the systems that are/have been market leaders, how
was the quality of the proposed solution?
The impact of Quality
Babich has shown that even by starting with the same
level of customer satisfaction what really matters is
the competition with the other companies
Babich, P. (1992) “Customer Satisfaction: How Good is Good Enough?” Quality Progress
A
B C
dissatisfied C
dissatisfied B
satisfied A
satisfied B satisfied C
This implies that you need to do better than your
competitors in terms of quality
10-72
●
What makes software different
– Really big economies of scale
→ additional physical inputs for production result in a non-proportional
increase in output with a decrease in average costs
– Increasing returns
→ “tendency for that which is ahead to get further ahead, for that
which loses advantage to lose further advantage” (W.B. Arthur)
– Network Externalities
→ the value of goods to users increases as more people adopt them
– High initial costs
→ software is complex to design and to deliver to the market
– Switching costs
→ switching to other software might be costly (e.g. training to
re-do, change of infrastructure)
The impact of Quality
11-72
Network Externalities: for some types of goods, the value of a
good increases with the number other people adopting the same
good
●
“These effects arise both because the ability to communicate and share data with
others will be greater, and because it is more likely that complementary hardware,
software, and wetware (i.e., brain cells) will be available, when there is a large
base of users of the software”, Katz & Shapiro
Network Externalities
Network Externalities
Path Dependence
Lock-in
12-72
●
Path Dependence: in common knowledge means 'history matters'
●
But there is more.. If we consider historical paths, not only the path
depends on previous events but also produces a self-reinforcing
mechanism that leads to the reinforcement of the path selected. In this
way, switching to another path will become at every step more costly
●
In general, due to increasing returns, a phenomenon can assume
contagious effects
Path Dependence
13-72
●
Given a discrete set of values of time T={t1,t2,..,tn}, Xt represents the outcome at
time t
●
ht, the history at time t, that is the combination of all the outcomes up to time t-1
●
a dynamic process is a stochastic process equipped with a outcome function G that
maps current history to next outcome
●
A process is path-dependent if the outcome in any period depends on history
A Formal definition of Path Dependence
• Weak Path Dependent: if the outcome probabilities depend upon the set of outcomes; the
sequence of events {A,B,B,A} is considered the same as {A,B,A,B} for the evolution of the process
at later stages;
• Strong Path Dependent: if the outcome probabilities depend upon the sequence of outcomes;
events {A,B,B} has a set of outcomes completely different from {B,B,A}
This essentially means that nr.
of users of a sw will depend
on the past history – for some
platforms quality will be less
important
This is specific to software → if
I have a car I do not really care
about the types of cars that
other users will have
14-72
●
So are we destined to keep the current status in the future
(e.g. using Facebook, Google, etc..)?
●
One nice answer comes from Christensen (2013) that
identified two types of technologies:
– Sustaining – an innovation that improves a product in either
expected or unexpected way, but does not lead to a paradigm
shift
– Disruptive – an innovation that can potentially create a new
market (e.g. facebook creating the whole social networking
idea, or touch screens for the whole mobile phones industry)
●
Disruptive innovations are a way to change from the status quo
→ can you name some companies that were market leaders before a
disruptive technology appeared?
Sustaining and Disruptive Innovations (1/3)
Christensen, C. (2013). The innovator's dilemma: when new technologies cause great firms to fail.
Harvard Business Review Press.
15-72
●
Disruptive technologies: initially the quality might be lower
than current technologies, but will catch-up quickly
Sustaining and Disruptive Innovations (2/3)
Christensen, C. (2013). The innovator's dilemma: when new technologies cause great firms to fail.
Harvard Business Review Press.
16-72
●
Gartner's Hype Cycle (GHC) for Emerging Technologies →
maturity of technologies in a domain
Sustaining and Disruptive Innovations (3/3)
← 2013 GHC
17-72
Q&A
18-72
a) Can you name one innovation either in Software or in the
IT world that impressed you particularly in the last few years
(that is a disruptive innovation)?
b) Thinking in terms of 5-10 years from now, what do you
think will be a disruptive technology in the software/IT
world?
General Questions
19-72
SQM Categories
20-72
SQM comprises four subcategories
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
21-72
●
A quality plan defines how an organization will reach the quality
objectives
●
Usually covers
– Quality objectives and goals
– Quality management scope
– Organisation & responsibilities
– Resource requirements
– Cost benefit analysis
– Activities and deliverables
– Schedule
– Risk analysis
A. Software Quality Planning
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
22-72
●
Quality at the organization and project Levels
A. Software Quality Planning
Source: http://www.chambers.com.au/glossary/quality_planning.php
Project /
Product
development
Project /
Product
development
Quality
management
Group
Process Group
SQMS
Development
Process
Improvement
Plan
Project Manager /
Quality
Representative
Project /
Product
development
Project
Quality Plan
Quality Management System
standards
procedures
process
description
QA objectives
& metrics
Goals
Resources
SQMS tailoring
SQMS = Software Quality Management System
23-72
●
IEEE Std 730-2014 – Outline for software quality planning:
A. Software Quality Planning
Heimann, D. I. (2014). An Introduction to the New IEEE 730 Standard on Software Quality Assurance, SQP Vol 16, N.3
1. Purpose & Scope
2. Definitions & acronyms
3. Reference documents
4. SQA Plan Overview:
4.1 Organization & independence
4.2 Software Product Risk
4.3 Tools
4.4 Standards, practices and conventions
4.5 effort, resource, schedules
5. Activities, Outcomes and tasks:
5.1 Product Assurance:
5.1.1 Evaluate plans for conformance
5.1.2 Evaluate product for conformance
5.1.3 Evaluate product for acceptability
5.1.4 Evaluate product life cycle support for
conformance
5.1.5 Measure products
5.2 Process assurance:
5.2.1 Evaluate life cycle support for conformance
5.2.2 Evaluate environments for conformance
5.2.3 Evaluate subcontractor processes for
conformance
5.2.4 Measure processes
5.2.5 Assess staff skills & knowledge
6. Additional Considerations
6.1 Contract review
6.2 Quality Measurement
6.3 Waiver and deviations
6.4 Task repetition
6.5 Risks in performing SQA
6.6 Communication strategy
6.7 Conformance process
4. SQA Records:
7.1 Analyze, identify, collect, file, maintain,
dispose
7.2 Availability of records
24-72
IEEE Std 730-2014 – how is Agile considered?
●
Agile → the product backlog is the contract, the standards help in
defining the role of the backlog as contract
●
The SQA product part in IEEE730 can be used to defined the “done”
criteria
●
Non-conformances to standards are inserted in the backlog and
addressed in sprints in which are scheduled
●
Acceptance is a continuous process in Agile
●
IEEE730 contains an appendix with details about Agile adoption of the
standard
A. Software Quality Planning
Heimann, D. I. (2014). An Introduction to the New IEEE 730 Standard on Software Quality Assurance, SQP Vol 16, N.3
25-72
●
SQA means monitoring constantly the software engineering
process to ensure that the approaches/methods/processes applied
lead to quality within the project
B. Software Quality Assurance
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
26-72
●
Relevant Standard: ISO/IEC/IEEE 12207:2008: Systems and software engineering Software
life cycle processes
→ can you spot SQA in this schema?
B. Software Quality Assurance
27-72
●
IEEE Std 730 format and content of a→ software quality assurance plan
●
IEEE Std 1061 describes a methodology—spanning the life cycle—for→
establishing quality requirements and for identifying, implementing, and
validating the corresponding measures.
●
IEEE Std 1465 (withdrawn standard) describes quality requirements→
specifically suitable for software "packages". It is expected to be replaced
by an IEEE adoption of ISO/IEC 25051
B. Summary of IEEE Stds related to SWQA
28-72
●
SQC means to constantly monitor the software engineering
process to check for conformance to applied standards (e.g.
CMMI) or produced artifacts
●
Some examples of methods
→ The Goal Question Metrics Approach (seen on the measurement lecture)
→ The Plan-Do-Check-Act method
→ Total Software Quality Control
C. Software Quality Control
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
29-72
●
Improve process effectiveness, efficiency and other characteristics
with the aim to improve software quality
●
Very often software process improvement practices are
embedded within the process (e.g. capability models)
●
Some methods:
→ Capability Maturity Model (CMM) and Capability Maturity
Model Integration (CMMI)
→ ISO/IEC 15504 (SPICE)
→ ISO 9001 Specification (seen during PA017 SEII)
→ PSP and TSP
D. Software Process Improvement
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
30-72
Software Process Maturity Levels
31-72
●
According to ISO/IEC 12207/15288:2007
What is a process?
Processes require a purpose and outcome. All processes
have at least one activity
Activities are constructs for grouping together related tasks
A task is a detailed arrangement for the implementation of
a process. It can be a requirement (“shall”), a
recommendation (“should”) or a permission (“may”)
Notes are used to explain better the intent or mechanism
of a process
32-72
●
As defined in ISO/IEC 15504-2 (SPICE)
Process Maturity Levels
→ The process is not implemented or fails to achieve
the purpose
→ The process is achieving the purpose
→ The process is now running in a managed way
(planned, monitored, adjusted) – work products are
established, controlled and maintained
→ The managed process is now implemented using a
defined process capable of achieving process
outcomes
→ The established process now operates within
defined limits to achieve its process outcomes
→ The predictable process is continuously improved
to meet relevant current and projected business goals
Capability
Level
Process Capability
0 Incomplete Process
1 Performed Process
2 Managed Process
3 Established Process
4 Predictable Process
5 Optimizing Process
33-72
●
As defined in ISO/IEC 15504-2 (SPICE)
Process Maturity Levels & Attributes
34-72
SPICE - Overall
ENG.1
Engineering
Customer-
Relationship
Support
ManagementManagement Organisation Support
ENG.1
ENG.1
CUS.1
ENG.1
MAN.1
ENG.1
ORG.1
ENG.1
SUP.1
ENG.1
ENG.1.BP1
ENG.1
CUS.1.BP1
ENG.1
MAN.1.BP1
ENG.1
ORG.1.BP1
ENG.1
SUP.1.BP1
ENG.1
ENG.1.WP1
ENG.1
CUS.1.WP1
ENG.1
MAN.1.WP1
ENG.1
ORG.1.WP1
ENG.1
SUP.1.WP1
5 Categories
24 Processes
201 Base
Practices
109 Work
Products
35-72
Example Process Definition (1/2)
5 Process
Categories
(SUP=Support)
36-72
Example Process Definition (2/2)
Overall 24 processes
are specified
37-72
●
SPICE is a two-dimensional level model
– Processes and categories on one side (Process Dimension)
includes Base practices, work products, characteristics→
→ Does the process reach its goals?
– Capability of processes on the other side (Capability
Dimension) includes levels, process attributes, process→
attributes, management practices → How well is a
specific goal met?
Process Maturity Levels & Attributes
38-72
Process Maturity Levels & Attributes
Assessment Model
Process Dimension Capability Dimension
Process categories (5)
Processes (24)
Capability Levels (6)
Process Attributes (9)
Base Practices (201)
Work Products (109)
Management Practices (33)
Resources & Infrastructure
characteristics
Indicators of process
performance
Indicators of process
capability
39-72
Process Maturity Levels & Attributes
Assessment Model
Process Dimension Capability Dimension
Process categories (5)
Processes (24)
Capability Levels (6)
Process Attributes (9)
Base Practices (201)
Work Products (109)
Management Practices (33)
Resources & Infrastructure
characteristics
Indicators of process
performance
Indicators of process
capability
Assessment Scale
●
up to 15% – N (Not performed/achieved)
●
> 15% to 50% – P (Partial)
●
> 50 to 85% - L (Large)
●
> 85% - F (Full performance / achievement)
PA1.1 PA2.1 PA2.2 PA3.1 PA3.2 PA4.1 PA4.2 PA5.1 PA5.2
ENG.1
ENG.2
MAN.2
ORG.1
N
P
L
F → who performs the assessment?
40-72
Capability Maturity Model Integrated
Upgrade from CMM appearing around year 2000
CMMI
41-72
Staged assessment of the maturity of the→ entire software process
Continuous assessment of the→ capabilities of different Process
Areas (PA)
Sample Pas:
– Requirements Development (RD)
– Requirements Management (REQM)
– Project Monitoring and Control (PMC)
– Project Planning (PP)
– Process and Product Quality Assurance (PPQA)
– Quantitative Project Management (QPM)
– Risk Management (RSKM)
– Supplier Agreement Management (SAM)
– Technical Solution (TS)
– Validation (VAL)
– ...
CMMI
42-72
Ragaisis, S., Peldzius, S., & Simenas, J. (2010). Mapping CMMI-DEV Maturity Levels toISO/IEC 15504 Capability Profiles. Assessment, 23, 24.
Mapping CMMI to ISO/IEC 15504
43-72
Ragaisis, S., Peldzius, S., & Simenas, J. (2010). Mapping CMMI-DEV Maturity Levels toISO/IEC 15504 Capability Profiles. Assessment, 23, 24.
Mapping CMMI to ISO/IEC 15504
…........................................................................................................
This was the
sample process
we saw some
slides ago
Legenda: up to 15 % – N (Not performed/achieved), > 15 % to 50 % – P (Partial), > 50 to 85
% – L (Large), and F (Full performance / achievement) > 85 % - “ML2”- “ML5” maturity
levels in CMM-i - “CL1”-“CL5” are capability levels in ISO/IEC 15504
44-72
Models for Software Quality
45-72
●
What is the name of the simplest quality process
management practice in your opinion?
→ Actually, it involves no process
Simplest Quality Management Form
46-72
●
Cowboy Coders write code according to their rules
●
Some sentences you might have heard:
“If possible, the customer should only see the final versions of the
product. It is important to minimize the contact with the customer so
time is not wasted”
“The code is mine and none is allowed to touch it!”
“I do not need any analysis, design nor documentation”
“Even if it is broken, do not touch it! Try to hide it!”
“People who need comments in order to
understand my code are too dumb to be
working with me “
Cowboy Coding
Image: https://www.cs.utexas.edu/blog/cowboy-rides-away-now
→ See http://c2.com/cgi/wiki?CowboyCoder
47-72
●
The Personal Software Process (PSP) is a disciplined software development process
that works at the individual level
Personal Software Process (PSP)
PSP0
Defining &
using
processes
PSP1
Planning &
Tracking
PSP2
Quality
Management
TSP
Team
Development
Estabilish a measured
performance baseline
Practice size and effort
estimation
Practice defect
management and improve
design practices
48-72
PSP0 – First Level
Process
scripts
PSP0 Process
Planning
Development
Postmortem
Design
Code
Compile
Test
Requirements
Time & Defect
Logs
Plan Summary
Finished Product
Scripts provide
the steps for
all processes
Templates
helpful when
following the
process
49-72
●
Emphasizes making accurate and precise size measurements
●
Incorporates measuring the size of the programs produced
●
Accounts for various types of LOC in the programs produced
●
Begins to look at process improvement
●
The following elements are added
– Estimating and reporting software size
– Use of a coding standard
– Recording process problems and improvement ideas
PSP0.1 – Improvement
50-72
●
All is based on forms, scripts and logs
PSP0.1 – Improvement
W. S. Humphrey, “A Discipline for Software Engineering”, 1995
Table C3 PSP0.1 Project Plan Summary Example
Student Student 11 Date 2/1/94
Program Object LOC Counter Program # 3A
Instructor Humphrey Language C
Program Size (LOC) Plan Actual To Date
Base(B) 87
(Measured)
Deleted (D) 0
(Counted)
Modified (M) 6
(Counted)
Added (A) 113
(T-B+D-R)
Reused (R) 0 0
(Counted)
Total New & Changed (N) 90 119 315
(A+M)
Total LOC (T) 200 396
(Measured)
Total New Reuse 0 0
Time in Phase (min.) Plan Actual To Date To Date %
Planning 10 11 36 6.4
Design 25 21 63 11.2
Code 75 97 249 44.2
Compile 20 4 35 6.2
Test 45 39 105 18.7
Postmortem 20 33 75 13.3
Total 195 205 563 100.0
Defects Injected Actual To Date To Date %
Planning 0 0 0
Design 1 3 11.5
Code 8 23 88.5
Compile 0 0 0
Test 0 0 0
Total Development 9 26 100.0
Defects Removed Actual To Date To Date
%
Planning 0 0 0
Design 0 0 0
Code 0 0 0
Compile 2 13 50.0
Test 7 13 50.0
Total Development 9 26 100.0
After Development 0 0
51-72
●
PSP1 introduces the concept of
software effort estimation and
the usage of historical data
●
Using the PROBE (PROxy-Based
Estimating) size estimating
method
●
Using linear regression, PROBE
size estimation, regression
analysis is based on historical
estimated object LOC (the x
data) and actual new and
changed LOC (the y data)
PSP1 – Second Level
Estimate Objects (proxies)
Nr. of
methods
Start with the conceptual design
Object
type
Relative
size
Reuse
categories
Estimate new and changed LOCs
Estimate development time
Calculate prediction interval
Planning estimates
W. S. Humphrey, “A Discipline for Software Engineering”, 1995
52-72
●
PSP2 introduces design and code reviews methods for
evaluating and improving the quality of your reviews
●
There are three new process elements
– PSP2 project plan summary
– PSP2 design review checklist
– PSP2 code review checklist
PSP2 – Third Level
53-72
●
The idea behind PSP is that it should lead to more team-aware
processes once developers have tried a self-disciplined approach
Personal Software Process (PSP)
PSP Skill-building
Personal Plans
Planning Methods
Earned Value
Process data
Quality Measures
Defined Processes
TSP Team-building TSP Team-working
Engineering
Disciplines
Team
Disciplines
Management
Disciplines
Committment
Aggressive plans
Quality ownership
Project goals
Plan ownership
Plan detail
Team roles
Team resources
Quality priority
Cost of quality
Follow the process
Review status
Review quality
Communication
Change management
Integrated
Product
Teams
→ see hackystat
54-72
There are around 40 Agile Maturity models that sometimes adapt SPICE/CMMI
levels/processes – each one uses different naming for levels:
Agile Process Maturity
T. Schweigert, D. Vohwinkel, M. Korsaa, R. Nevalainen, and M. Biro, “Agile Maturity Model: A Synopsis as a First Step to Synthesis,” in Systems, Software and Services
Process Improvement, F. McCaffery, R. V. O’Connor, and R. Messnarz, Eds. Springer Berlin Heidelberg, 2013, pp. 214–227.
Level1 Level2 Level3 Level4 Level5
● Rhetorical stage
● Team level
maturity
● Neutral or Chaotic
● Emergent
● Engineering Best
Practices
● Introductory
● Collaborative
● Dormant
● No Agile
● Waterfall
● Non-Agile
● Core Agile
Development
● Adherence to Agile
Principles
● Getting Started
● Improvising
● Certified stage
● Department Level
Maturity
● Collaborative
● Continuous
Practices at
Component Level
● Learn
● Novice
● Evolutionary
● Speed
● Early Adoption
● Forming
● Minimum
● Discipline Agile
Delivery
● Repeteable
Process across the
organization
● Scrum at project
level
● Practicing
● Plausible stage
● Business Level
Maturity
● Operating
(consistent
exhibition of
competence)
● Cross component
continuous
integration
● Leverage
● Intermediate
● Effective
● Reactive
● Self Service
● Agile
● Consolidated
● Agility at scale
● Scalability –
SCRUM of
SCRUMS
● Respectable stage
● Project
Management Level
Maturity
● Adaptive
● Cross Journey
Continuous
integration
● Advanced
● Adaptive
● Responsive
● The Lake effect
● Performing
● Items on the right
● SCRUM at
Enterprise Level
● Governed
● Measured stage
● Management Level
Maturity
● Innovating
● On Demand Just in
Time Release
● Optimise
● Insane
● Ambient
● Scaling
● Coexistence with
non-agile
● Enterprise
transformation
● Matured
55-72
Some approaches suggest even to reuse SPICE Process definitions, example:
Agile Process Maturity
T. Schweigert, M. Ekssir-Monfared, and M. Ofner, “An Agile Management Process Group for TestSPICE®,” in Systems, Software and Services Process Improvement, F.
McCaffery, R. V. O’Connor, and R. Messnarz, Eds. Springer Berlin Heidelberg, 2013, pp. 228–236.
56-72
Source: http://www.agigante.it/different-levels-of-agile-in-a-company/
Agile Maturity Matrix
Level1 – ad hoc
Agile
Level2 – doing
agile
Level3 – being
agile
Level4 – thinking
agile
Level5 – culturally
agile
Agile is not yet used
or agile practices are
used sporadically
Teams start to exhibit
some agile habits
Lean portfolio
management
Communities of
practice support agile
habits
Lean and agile are
part of organizational
culture
Variable quality Consistency across
teams is still variable
Mature embodiment
of essential
characteristics and
behaviour of agile
Successful use of
agile at scale
Perfecting waste
reduction, smooth
flow of delivery
Predominantly manual
testing
Some knowledge
sharing activities
under way
Disciplined Agile
delivery processes
and practices with
continual
improvement and
repeatable results
Success even with
teams in multiple
geographies
Sustainable pace of
innovation
Very little crossproject
knowledge and
collaboration
Use of agile tools and
practices become
commonplace
Respect for people
and continuous
improvement
Measurement
systems in place keep
track of business
value delivered
Continuous
organizational
learning and
optimization of the
work process and the
work products
Success achieved
primarily through
heroic individual
efforts
Solution quality
improves
Appropriate agile
governance
Autonomation:
automation with a
human touch
Standard work is
defined
57-72
●
One of the existing models is a simplified version of CMMI taking into
account the distributed nature of Open Source development
●
Trustworthy elements (TWE) are micro-characteristics that allow to define
maturity levels
Open Source Maturity Model (OMM)
Source: http://qualipso.icmc.usp.br/OMM/
PDOC – Product Documentation
STD – Use of Established and Widespread Standards
QTP – Quality of Test Plan
LCS – Licenses
ENV – Technical Environment
DFCT – Number of Commits and Bug Reports
MST – Maintainability and Stability
CM – Configuration Management
PP1 – Project Planning Part 1
REQM – Requirements Management
RDMP1 – Availability and Use of a (product) roadmap
RDMP2 – Availability and Use of a (product) roadmap
STK – Relationship between Stakeholders
PP2 – Project Planning Part 2
PMC – Project Monitoring and Control
TST1 – Test Part 1
DSN1 – Design Part 1
PPQA – Process and Product Quality Assurance
PI – Product Integration
RSKM – Risk Management
TST2 – Test Part 2
DSN2 – Design 2
RASM – Results of third party assessment
REP – Reputation
CONT – Contribution to FLOSS Product from SW Companies
58-72
●
The Goal Question Metric approach is used to define the elements of the current
OMM based on the TWEs
●
All levels of the GQM are aggregated according to following rating calculation:
Open Source Maturity Model (OMM)
R(Pi)=
∑ Mi
count (M )
R(Gi)=
∑ Pi
count (P)
R(TWEi)=
∑ Gi
count (G)
No weighting of metrics, Maturity Level calculated
by using practices (not TWEs)
R(ML)=
∑ Pi
max∑ Pi
P=practice, G=goal, ML=Maturity Level, R=rating
59-72
●
A measure of Software Quality developed at Motorola
●
The focus of Six Sigma is on eliminating defects, that is
everything that is outside from the customers specifications
●
No more than +/- six times the standard deviation from the
process mean → 3,4 Defects Per Million datapoints
●
Data is a key to understand the underlying processes and take
decisions
Six Sigma
Sigma Level Defects x 1M
2 308.537
3 66.807
4 6.210
5 233
6 3,4
60-72
Six Sigma - Process
Define Measure Analyze Improve Control
Who are the
customers and the
needs
How is the process
defined and how
are the defects
measured
What are the most
important causes of
the defects
How can the
causes of the
defects be
eliminated
What actions are
needed to sustain
improvement
In Six Sigma, a defect is defined as “Any product, service, or process
variation which prevents meeting the needs of the customer and/or
which adds cost, whether or not it is detected”
61-72
Six Sigma - Process
Define Measure Analyze Improve Control
●Define Project
scope
●Estailish formal
project
●Identify needed
data
●Obtain data set
●Evaluate data
quality
●Summarize &
baseline data
●Explore data
●Characterize
process & problem
●Update
improvement
project scope &
scale
●Identify possible
solutions
●Select solution
●Implement (pilot as
needed)
●Evaluate
●Define control
method
●Implement
●Document
62-72
Six Sigma - Tools
Define Measure Analyze Improve Control
● Benchmark
● Contract/charter
● Kano Model
● Voice of the
customer
● Voice of the
Business
● Quality Function
Deployment
(QFD)
● GQ(I)M and
indicator
templates
● Data collection
methods
● Measurement
System
Evaluation
● Cause & Effect
Diagram/Matrix
● Failure models
& effects
analysis
● Statistical
inference
● Reliability
Analysis
● Root Cause
Analysis
● Hypothesis Test
● Design of
experiments
● Modeling
● ANOVA
● Tolerancing
● Robust Design
● System
Thinking
● Decision & Risk
Analysis
● Performance
Analysis Model
● Statistical
Controls:
● Control charts
● Time series
methods
● Non-Statistical
controls:
● Procedural
adherence
● Performance
Management
● Preventive
measures
Basic Tools (Histogram, scatter plots, run charts, pareto charts, cause & effect diagram, Control chart,
descriptive statistics), baseline process flow map, project management, management by fact, sampling
techniques, survey methods, defect metrics
63-72
Six Sigma – SQFD
Diagram Source: Becker and Associates, 2000
SQFD = Software Quality Function
Deployment
64-72
●
In the lecture about measurement, we have seen the Measurement
Information Model of ISO/IEC 15939:2002. To complete the view,
the following is the measurement process
ISO/IEC 15939
65-72
●
Defining the measurement construct
ISO/IEC 15939 - Example
66-72
Customization of processes is usually quite heavy in the development world, it is difficult to find 1-1
correspondence with the original model – all derives from the project's quality objectives
The following is an adaptation of SCRUM and introduction of some maturity phases to bridge the gap
between research prototypes and industrial products releases
Final Considerations (1/2)
D. Winkler, R. Mordinyi, and S. Biffl, “Research Prototypes versus Products: Lessons Learned from Software Development Processes in Research Projects,” in Systems,
Software and Services Process Improvement, F. McCaffery, R. V. O’Connor, and R. Messnarz, Eds. Springer Berlin Heidelberg, 2013, pp. 48–59.
67-72
The overall interest of researchers in Software Process Improvement area for the
established techniques (that is not considering mixture and ad-hoc created methods)
Final Considerations (2/2)
M. Unterkalmsteiner, T. Gorschek, A. K. M. M. Islam, C. K. Cheng, R. B. Permadi, and R. Feldt, “Evaluation and Measurement of Software Process Improvement - A
Systematic Literature Review,” IEEE Transactions on Software Engineering, vol. 38, no. 2, pp. 398–424, Mar. 2012.
68-72
Do better processes lead to a better
software product?
69-72
●
The problem was studied in * by means of a systematic
literature review
●
The starting point is that models such as CMMI were born
under the assumption that “the quality of a system or
product is highly influenced by the quality of the process
used to develop and maintain it”
Relation of Process Quality to Product Quality
* Lavallée, M., & Robillard, P. N. (2011, September). Do software process improvements lead to ISO 9126 architectural
quality factor improvement. In Proceedings of the 8th international workshop on Software quality (pp. 11-17). ACM.
70-72
●
Projects that use some SPI initiatives show higher levels of
software maintainability
●
It takes significant time to see the benefits of an SPI program
●
Most of the SPI initiatives focus on defect reduction, not on
other quality concerns
●
The major conclusion is that Software Process Improvement
activities always increase process quality, but not always product
quality
Relation of Process Quality to Product Quality
* Lavallée, M., & Robillard, P. N. (2011, September). Do software process improvements lead to ISO 9126 architectural
quality factor improvement. In Proceedings of the 8th international workshop on Software quality (pp. 11-17). ACM.
71-72
For the part about increasing returns, path dependency, if you are interested :)
[1] Arthur, W. Brian (1989). Competing Technologies, Increasing Returns, and Lock-In by Historical Events, 97 Economic
Journal 642-65.
[2] Farrell, Joseph and Garth Saloner (1985). Standardization, Compatibility, And Innovation, 16 Rand Journal 70-83.
[3] Katz, M. L., & Shapiro,C. (1985). Network Externalities, Competition, and Compatibility. The American Economic
Review, 75(3), 424-440.
[4] Katz, M. L., & Shapiro, C. (1986). Technology Adoption in the Presence of Network Externalities. Journal of Political
Economy 822-841.
[5] Liebowitz, S. J. and Stephen E. Margolis (1990). The Fable of the Keys, Journal of Law and Economics, 33:1, 1-26.
[6] Liebowitz, S. J. and Stephen E. Margolis (1994). Path Dependency, Lock-In, and History, working paper, 1994b.
[7] Liebowitz, S. J. and Stephen E. Margolis (1994). Network Externality: An Uncommon Tragedy, 8 Journal of Economic
Perspectives 133-50.
[8] Economides, N. (1996). The Economics of Networks, International Journal of Industrial Organization, vol. 14, no. 6, pp.
673-699
[9] Paul A. David (2000), Path dependence, its critics and the quest for ‘historical economics, in P. Garrouste and S.
Ioannides (eds), Evolution and Path Dependence in Economic Ideas: Past and Present, Edward Elgar Publishing, Cheltenham,
England.
[10] Shapiro, C. e Varian, H.R. (1999). Information Rules: A Strategic Guide to the Network Economy, Harvard Business
School Press.
[11] Windrum, P., (2003). Unlocking a lock-in: towards a model of technological succession, in Applied Evolutionary
Economics: New Empirical Methods and Simulation Techniques, P.P. Saviotti (ed.), Cheltenham: Edward Elgar.
References
72-72
References mentioned in the slides, plus:
●
Bourque, P., & Fairley, R. E. (2014). Guide to the Software Engineering
Body of Knowledge (SWEBOK (R)): Version 3.0. IEEE Computer Society
Press.
References