or=black height=1> Yahoo! is not affiliated with the authors of this page or responsible for its content.
Quality Attribute Workshops, 2nd Edition
Quality Attribute
Workshops,
2nd Edition
Mario R. Barbacci
Robert Ellison
Anthony J. Lattanze
Judith A. Stafford
Charles B. Weinstock
William G. Wood
June 2002
TECHNICAL REPORT
CMU/SEI-2002-TR-019
ESC-TR-2002-019 . Pittsburgh, PA 15213-3890
Quality Attribute
Workshops,
2nd Edition
CMU/SEI-2002-TR-019
ESC-TR-2002-019
Mario R. Barbacci
Robert Ellison
Anthony J. Lattanze
Judith A. Stafford
Charles B. Weinstock
William G. Wood
Unlimited distribution subject to the copyright.
June 2002
Architecture Tradeoff Analysis Initiative This report was prepared for the
SEI Joint Program Office
HQ ESC/AXS
5 Eglin Street
Hanscom AFB, MA 01731-2116
The ideas and findings in this report should not be construed as an official DoD position. It is published
in the interest of scientific and technical information exchange.
FOR THE COMMANDER
Norton L. Compton, Lt Col, USAF
SEI Joint Program Office
This work is sponsored by the U.S. Department of Defense. The Software Engineering Institute is a
federally funded research and development center sponsored by the U.S. Department of Defense.
Copyright
©
2002 by Carnegie Mellon University.
Requests for permission to reproduce this document or to prepare derivative works of this document
should be addressed to the SEI Licensing Agent.
NO WARRANTY
THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE
MATERIAL IS FURNISHED ON AN AS-IS BASIS. CARNEGIE MELLON UNIVERSITY
MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY
MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE
OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE
MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF
ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT
INFRINGEMENT.
This work was created in the performance of Federal Government Contract Number F19628-00-C-
0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a
federally funded research and development center. The Government of the United States has a royalty-
free government-purpose license to use, duplicate, or disclose the work, in whole or in part and in any
manner, and to have or permit others to do so, for government purposes pursuant to the copyright
license under the clause at 252.227-7013.
Use of any trademarks in this report is not intended in any way to infringe on the rights of the
trademark holder.
For information about purchasing paper copies of SEI reports, please visit the publications portion of
our Web site (http://www.sei.cmu.edu/publications/pubweb.html). Table of Contents
CMU/SEI-2002-TR-019
i
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2
Process Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1
Scenario Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2
Test Case Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3
Test Case Architecture Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4
Results Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3
Variants in QAWs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.1
Application Before Acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2
Application During Solicitation and Proposal Evaluation Phases . . . . . 11
3.3
Application During a Competitive Fly-off . . . . . . . . . . . . . . . . . . . . . . . . 13
4
Experience and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.1
Lessons Learned from Scenario Generation and Refinement . . . . . . . . 15
4.2
Lessons Learned from Developing the Test Cases . . . . . . . . . . . . . . . . 16
4.3
Lessons Learned from Analyzing the Architecture Using Test Cases . . 16
4.4
Lessons Learned from the Results Presentation . . . . . . . . . . . . . . . . . . 17
Appendix A
Example Scenario Refinement . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Appendix B
Example Test Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
B.1
Test Case Context and Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
B.2
Quality Attribute Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
B.3
Utility Tree. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Appendix C
Example Results of the Test Case Analysis . . . . . . . . . . . . . . . . 23
C.1
Satellite Locations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
C.2
Health of the Satellites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 ii
CMU/SEI-2002-TR-019 List of Figures
CMU/SEI-2002-TR-019
iii
Figure 1:
The QAW Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Figure 2:
Common Acquisition Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Figure 3:
Acquisition Strategy Using Competitive Fly-Off. . . . . . . . . . . . . . . . . . . 13 iv
CMU/SEI-2002-TR-019 List of Tables
CMU/SEI-2002-TR-019
v
Table 1:
Agenda for Scenario Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Table 2:
Template for Scenario Refinement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Table 3:
Template for Utility Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Table 4:
Agenda for Presentation of Results. . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Table 5:
Example Scenario Refinement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Table 6:
Example Utility Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 vi
CMU/SEI-2002-TR-019 CMU/SEI-2002-TR-019
vii
Abstract
Quality attribute workshops (QAWs) provide a method for analyzing a systems architecture
against a number of critical quality attributes, such as availability, performance, security,
interoperability, and modifiability, that are derived from mission or business goals. The QAW
does not assume the existence of a software architecture. It was developed to complement the
Architecture Tradeoff Analysis Method
SM
(ATAM
SM
) in response to customer requests for a
method to identify important quality attributes and clarify system requirements before there is
a software architecture to which the ATAM could be applied. The analysis is based on apply-
ing a set of test cases to a system architecture. These test cases include questions and concerns
elicited from stakeholders associated with the system. The process of building the test cases
allows stakeholders to communicate among themselves, thereby exposing assumptions that
may not have surfaced during requirements elicitation. Our experience to date includes multi-
ple QAWs that were held with four different U.S. government acquisition programs.
This is the second edition of a technical report describing QAWs. This report clarifies the con-
text in which a QAW is applicable, provides a rationale for developing the process and
describes it in detail, and concludes with a list of lessons learned and a discussion of how these
lessons have helped evolve the process to its current state. viii
CMU/SEI-2002-TR-019 CMU/SEI-2002-TR-019
1
1
Introduction
Quality attribute workshops (QAWs) provide a method for analyzing a systems architecture
against a number of critical quality attributes, such as availability, performance, security,
interoperability, and modifiability, that are derived from mission or business goals. The analy-
sis also provides insights as to how these attributes interact, forming a basis for making
tradeoffs between these attributes [Barbacci 95, Barbacci 96, Barbacci 97].
The QAW does not assume the existence of a software architecture. It was developed to com-
plement the Architecture Tradeoff Analysis Method
SM
(ATAM
SM
)
1
[Kazman 00] in response
to customer requests for a method to identify important quality attributes and clarify system
requirements before there is a software architecture to which the ATAM could be applied. In
an ATAM evaluation, an external team facilitates short meetings between stakeholders during
which scenarios representing the quality attributes of the system are developed, prioritized,
and analyzed against the software architectural approaches cho