1	Introduction to the SEI Capability Maturity Model^â (SEI/CMM^â ) Jeff Jacobs Jeffrey Jacobs & Associates Belmont, CA phone: 650.571.7092 email: jeff@jeffreyjacobs.com http://www.jeffreyjacobs.com
2	Survey Does your organization have a well defined software development methodology/process? Does your organization use Oracle Method/CDM? Does your organization use RUP? Does your organization have a software engineering process group?
3	Agenda What is SEI/CMM^â? History Overview and characteristics of the 5 levels Structure Examples focus on Level 2 Level 2 is the hardest to attain, as it involves fundamental cultural changes The spreadsheet/questionnaire Caveats and clarifications
4	What is the SEI/CMM^â? Well defined process and framework for assessing or evaluating the maturity level of an organization Organizations may receive a formal assessment from SEI licensed assessors DoD and other organizations may require a formal assessment rating for contractors and partners Organizations may use the SEI/CMM^â materials to perform internal evaluations and as a basis for improvement Describes an evolutionary improvement path to migrate an organization from an ad-hoc, immature organization to mature, disciplined organization
5	Prescriptive, not Descriptive Describes what processes and practices should be in place to achieve a maturity level, but not how they should be performed “Software Configuration Management” should be performed, but no guidelines on how to do it
6	History The SEI (Software Engineering Institute) is a Federally Funded Research and Development Center (FFRDC) established in 1984 at Carnegie Mellon University by US Department of Defense To solve the problem of why software projects were always late, over budget and full of bugs http://www.sei.cmu.edu November 1986, SEI, in conjunction with MITRE Corporation, began developing a process maturity framework that would help organizations improve their software process
7	More History CMM conceived by Watts Humphrey SEI maturity questionnaire — 1987 Humphrey, W.S. (1989), Managing the Software Process, Addison-Wesley, Reading, MA. Paulk, M.C., Weber, C., Chrissis, M.B., Garcia, S., & Bush, M. (1992), Key Practices of the Capability Maturity Model: Version 1.1, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA. http://www.sei.cmu.edu
8	Why Should You Care? Does this Sound Familiar? Success is dependant on heroic efforts, long hours and death marches People complain about workload and stress Processes are improvised and reinvented for every project Constant fire fighting People are rewarded for solving problems, not preventing them There is no time to do it right, but there is always time to do it over
9	Higher Level Organizations are More Effective Fewer defects/higher quality Reduced schedule misses Fewer/lower cost overruns Lower personnel turnover Better predictability Numerous study results can be found at http://www.sei.cmu.edu
10	Improved Defect Detection Results
11	Characteristics of a Mature Organization Possesses an organization-wide ability to manage software development and maintenance processes The processes are documented and communicated to existing staff and new employees Work activities are carried out according to documented procedures and process The defined processes are fit for use and consistent with the way work actually gets done The defined processes are updated when necessary; improvements are developed through controlled pilot test or cost/benefit analysis
12	More Maturity... Roles and responsibilities are clear throughout projects and across the entire organization Managers monitor the quality of the software products and customer satisfaction There is an objective, quantitative basis for judging product quality and for analyzing problems with the product or process Schedules and budgets are based on historical performance and are realistic The expected results for cost, schedule, functionality and quality of the product are usually achieved
13	More Maturity... A disciplined process is followed because everyone understands the benefits of doing so and the infrastructure is in place to support it
14	Overview of the 5 Levels Evolves from one level to next Each level forms the foundation of the next
15	Characteristics of Level 1 - Initial Ad hoc, chaotic work environment Success depends on heroic effort, overtime Few processes defined Even fewer processes and procedures are documented Processes often ignored under schedule pressure Missed deadlines, poor quality Most organizations are Level 1
16	Characteristics of Level 2 - Repeatable Ability to repeat earlier success based on prior experience Basic project management processes are established Project planning Project tracking Subcontractor management Basic software development processes are in place Requirements management Software quality assurance Software configuration management Processes are project oriented
17	Characteristics of Level 3 - Defined Management and engineering processes are standardized and documented across the organization Projects use approved, tailored versions of the standard process definitions A training program exists Level 3 focuses on organization wide issues A formal focus for process improvement exists Processes are documented and maintained
18	Characteristics of Level 4 - Managed The software process is under statistical control Quantitative quality goals for products are established and met Strong knowledge and use of metrics and statistical techniques throughout the organization Schedule and quality performance is predictable
19	Characteristics of Level 5 - Optimizing “Continuous process improvement is enabled by quantitative feedback and from piloting innovative ideas and new technologies”
20	CMM Maturity Distribution
21	Structure
22	Key Process Areas (KPA) Each Level (except 1) consists of several Key Process Areas Requirements Management (L2) “Key” implies other process may be needed, but are not part of CMM, e.g. “requirements elicitation” is not a is not a CMM KPA (just manage ‘em after you find them :-)
23	Key Process Areas
24	Goals Each KPA has associated goals The degree of achievement of each Level’s KPA’s Goals measure the level of maturity Goals are achieved by satisfactorily implementing the activities
25	Level 2 KPA’s Goals (1) Requirements Management Goal 1 - System requirements allocated to software are controlled to establish a b02%2ine for software engineering and management use. Goal 2 - Software plans, products, and activities are kept consistent with the system requirements allocated to software. Software Project Planning Goal 1 - Software estimates are documented for use in planning and tracking the software project. Goal 2 - Software project activities and commitments are planned and documented. Goal 3 - Affected groups and individuals agree to their commitments related to the software project.
26	Level 2 KPA’s Goals (2) Software Project Tracking and Oversight Goal 1 - Actual results and performances are tracked against the software plans. Goal 2 - Corrective actions are taken and managed to closure when actual results and performance deviate significantly from the software plans. Goal 3 - Changes to software commitments are agreed to by the affected groups and individuals.
27	Level 2 KPA’s Goals (3) Software Quality Assurance Goal 1 - Software quality assurance activities are planned. Goal 2 - Adherence of software products and activities to the applicable standards, procedures, and requirements is verified objectively. Goal 3 - Affected groups and individuals are informed of software quality assurance activities and results. Goal 4 - Noncompliance issues that cannot be resolved within the software project are addressed by senior management.
28	Level 2 KPA’s Goals (4) Software Configuration Management Goal 1 - Software configuration management activities are planned. Goal 2 - Selected software work products are identified, controlled, and available. Goal 3 - Changes to identified software work products are controlled. Goal 4 - Affected groups and individuals are informed of the status and content of software baselines.
29	Level 2 KPA’s Goals (5) Software Subcontract Management Goal 1 - The prime contractor selects qualified software subcontractors. (CMM/SEI certified :-) Goal 2 - The prime contractor and the software subcontractor agree to their commitments to each other. Goal 3 - The prime contractor and the software subcontractor maintain ongoing communications. Goal 4 - The prime contractor tracks the software subcontractor's actual results and performance against its commitments.
30	Common Features Common Features are attributes that indicate whether the implementation and institutionalization of a KPA is effective, repeatable, and lasting There are 5 features common to all KPAs Commitment to Perform Ability to Perform Activities to Perform Measurement and Analysis Verifying Implementation
31	Common Feature #1 - Commitment to Perform Commitment to Perform describes actions the organization must take to ensure that the process is established and will endure. This involves establishing organizational policies and senior management sponsorship. Policy Statements Refers to a written organizational policy for the practices of that key process area May refer to the project or to the organization (level 3) Leadership Assign a leadership role (e.g., project manager) or describes sponsorship activities necessary for the key process area to be successfully institutionalized
32	Common Feature #2 - Ability to Perform Ability to Perform describes preconditions that must exist in the project or organization to implement the software process competently Resources and funding for the activities covered by the key process area Training, formal and informal, for practioners Orientation Indicates less depth of skill or knowledge than would be expected via training, e.g. overview or introduction to a topic for those overseeing or interfacing with the practitioners
33	Ability to Perform (2) Prerequisite Items Either outputs from another KPA or from outside the project Software development plan is a prerequisite for Project Tracking and Oversight KPA CMM sites only those that are critical to implementing the KPA; there are many other issues that may be necessary for success
34	Common Feature #3 - Activities Performed Activities Performed describes the roles and procedures necessary to implement a key process area. This typically involves: establishing plans and procedures performing the work tracking it taking corrective actions as necessary.
35	Common Feature #4 - Measurement and Analysis Measurement and Analysis describes the need to measure the process and analyze the measurements. This typically includes examples of the measurements that could be taken to determine the status and effectiveness of the Activities Performed.
36	Common Feature #5 - Verifying Implementation (1) Verifying Implementation describes the steps to ensure that the activities are performed in compliance with the process that has been established. This typically encompasses reviews and audits by management and software quality assurance. Project management oversight on both a periodic and event-driven basis Periodic - maintain an ongoing awareness of the status of the effort and be informed when significant events (e.g., design review) on the project occur
37	Verifying Implementation (2) Senior management oversight on a periodic basis Provide awareness of and insight into software process activities Adequate mechanisms for exception reporting Covers different topics or similar topics at a higher level of abstraction than project management oversight reviews Software Quality Assurance activities Particular activities considered appropriate for review and/or audit by the SQA group are described as a key practice
38	Activities Each KPA in each level has a list of activities
39	Level 2, Requirements Management Activities System requirements allocated to software are controlled to establish a baseline for software engineering and management use. Software plans, products, and activities are kept consistent with the system requirements allocated to software. The project follows a written, organizational policy for managing the system requirements allocated to software.
40	More Activities For each project, responsibility is established for analyzing the system requirements and allocating them to hardware, software, and other system components. The requirements are documented. Adequate resources and funding are provided for managing the requirements. Members of the software engineering group and other software-related groups are trained to perform their requirements management activities.
41	Still More The software engineering group reviews the requirements before they are incorporated into the software project. The software engineering group uses the requirements as the basis for software plans, work products, and activities. Changes to the requirements are reviewed and incorporated into the software project. Measurements are made and used to determine the status of the activities for managing the requirements. The activities for managing the requirements are reviewed with senior management on a periodic basis.
42	Finally The activities for managing the requirements are reviewed with the project manager on both a periodic and event-driven basis. The software quality assurance group reviews and/or audits the activities and work products for managing the allocated requirements and reports the results.
43	An Assessment Spreadsheet Simple “Y” for yes, “N” or blank for no, “N/A” for not applicable answers Percent of Goal achievement is calculated to determine level of achievement of KPA
44	Caveats and Clarifications Maturity assessment is not a guarantee of quality Activities belonging to a higher level’s KPA are often needed and implemented by a lower level organization Peer Review is a Level 3 KPA, but is a highly effective technique often used by organizations that would be rated Level 2 Achievement of maturity takes time Estimate of 2 years to fully transition from Level 1 to Level 2 Benefits accrue long before next level is reached Not a silver bullet
45	Summary SEI/CMM provides a well defined framework for assessing and evaluating a software development organization’s maturity Formal assessment is not required SEI/CMM provides a well defined framework for understanding what a mature software development organization should look like SEI/CMM is descriptive, not prescriptive SEI/CMM is comprehensive, but not complete