Project management tools for agile development teams

A number of project management tools are specifically aimed at agile development. They are designed to help plan, track, analyse and integrate work. These tools play an important role in agile development, as a means of Knowledge Management.
Common features include: Version control integration, progress tracking, easy work allocation, integrated release and iteration planning, discussion forums, and reporting and tracking of software defects
Some well-known agile project management websites include: Rally Software, versionone, TargetProcess, assembla, acunote, ppts, Mingle, Gatherspace and visionproject.

Agile methods and project management

Agile methods differ to a large degree in the way they cover project management. Some methods are supplemented with guidelines on project management, but there is generally no comprehensive support.
PRINCE2 has been suggested as a suitable, complementary project management system.

Agile methods and method tailoring

In the literature, different terms refer to the notion of method adaptation, including ‘method tailoring’, ‘method fragment adaptation’ and ‘situational method engineering’. Method tailoring is defined as:
A process or capability in which human agents through responsive changes in, and dynamic interplays between contexts, intentions, and method fragments determine a system development approach for a specific project situation.
Potentially, almost all agile methods are suitable for method tailoring. Even the DSDM method is being used for this purpose and has been successfully tailored in a CMM context.
Situation-appropriateness can be considered as a distinguishing characteristic between agile methods and traditional software development methods, with the latter being relatively much more rigid and prescriptive. The practical implication is that agile methods allow project teams to adapt working practices according to the needs of individual projects. Practices are concrete activities and products that are part of a method framework. At a more extreme level, the philosophy behind the method, consisting of a number of principles, could be adapted (Aydin, 2004).
XP makes the need for method adaptation explicit. One of the fundamental ideas of XP is that no one process fits every project, but rather that practices should be tailored to the needs of individual projects. There are no experience reports in which all the XP practices have been adopted. Instead, a partial adoption of XP practices, as suggested by Beck, has been reported on several occasions.
A distinction can be made between static method adaptation and dynamic method adaptation. The key assumption behind static method adaptation is that the project context is given at the start of a project and remains fixed during project execution. The result is a static definition of the project context. Given such a definition, route maps can be used in order to determine which structured method fragments should be used for that particular project, based on predefined sets of criteria. Dynamic method adaptation, in contrast, assumes that projects are situated in an emergent context. An emergent context implies that a project has to deal with emergent factors that affect relevant conditions but are not predictable. This also means that a project context is not fixed, but changing during project execution. In such a case prescriptive route maps are not appropriate. The practical implication of dynamic method adaptation is that project managers often have to modify structured fragments or even innovate new fragments, during the execution of a project (Aydin et al, 2005).

Agile Data

The Agile Data method describes how data professionals can be productive members of agile software development teams. Agile Data's 6 philosophies provide guidance for how data professionals can interact effectively with other team members as traditional approaches to data work don't fit well with agile approaches. More importantly the Agile Data method describes a collection of agile techniques that DBAs can adopt, including Database refactoring, agile data modeling, database regression testing, and continuous database integration.

Suitability of agile methods

There is little if any consensus on what types of software projects are best suited for agile methodologies. Many large organizations have difficulty bridging the gap between a more traditional waterfall method and an agile one.
Large scale agile software development remains an active research area.
Agile development has been widely documented (see Experience Reports, below, as well as Beck , and Boehm and Turner as working well for small (<10 developers) co-located teams.
Some things that can negatively impact the success of an agile project are:
Large scale development efforts (>20 developers), though scaling strategiesand evidence to the contrary have been described.
Distributed development efforts (non-co-located teams). Strategies have been described in Bridging the Distance and Using an Agile Software Process with Offshore DevelopmentCommand-and-control company cultures
Forcing an agile process on a development team
Several successful large scale agile projects have been documented. BT has had several hundred developers situated in the UK, Ireland and India working collaboratively on projects and using Agile methods. While questions undoubtedly still arise about the suitability of some Agile methods to certain project types, it would appear that scale or geography, by themselves, are not necessarily barriers to success.
Barry Boehm and Richard Turner suggest that risk analysis be used to choose between adaptive ("agile") and predictive ("plan-driven") methods.The authors suggest that each side of the continuum has its own home ground as follows:
Agile home ground:

- Low criticality
- Senior developers
- Requirements change very often
- Small number of developers
- Culture that thrives on chaos
- Plan-driven home ground:
- High criticality
- Junior developers
- Requirements don't change too often
- Large number of developers
- Culture that demands order

Agile Comparison with other methods

Agile methods are sometimes characterized as being at the opposite end of the spectrum from "plan-driven" or "disciplined" methods. This distinction is misleading, as it implies that agile methods are "unplanned" or "undisciplined". A more accurate distinction is that methods exist on a continuum from "adaptive" to "predictive". Agile methods lie on the "adaptive" side of this continuum.
Adaptive methods focus on adapting quickly to changing realities. When the needs of a project change, an adaptive team changes as well. An adaptive team will have difficulty describing exactly what will happen in the future. The further away a date is, the more vague an adaptive method will be about what will happen on that date. An adaptive team can report exactly what tasks are being done next week, but only which features are planned for next month. When asked about a release six months from now, an adaptive team may only be able to report the mission statement for the release, or a statement of expected value vs. cost.
Predictive methods, in contrast, focus on planning the future in detail. A predictive team can report exactly what features and tasks are planned for the entire length of the development process. Predictive teams have difficulty changing direction. The plan is typically optimized for the original destination and changing direction can cause completed work to be thrown away and done over differently. Predictive teams will often institute a change control board to ensure that only the most valuable changes are considered.
Agile methods have much in common with the "Rapid Application Development" techniques from the 1980/90s as espoused by James Martin and others.

Contrasted with other iterative development methods
Most agile methods share other iterative and incremental development methods' emphasis on building releasable software in short time periods. Agile development differs from other development models: in this model time periods are measured in weeks rather than months and work is performed in a highly collaborative manner. Most agile methods also differ by treating their time period as a strict timebox.

Contrasted with the waterfall model
Agile development has little in common with the waterfall model. As of 2008, the waterfall model is still in common use. The waterfall model is the most predictive of the methods, stepping through requirements capture, analysis, design, coding, and testing in a strict, pre-planned sequence. Progress is generally measured in terms of deliverable artifacts: requirement specifications, design documents, test plans, code reviews and the like.
The main problem with the waterfall model is the inflexible division of a project into separate stages, so that commitments are made early on, and it is difficult to react to changes in requirements. Iterations are expensive. This means that the waterfall model is likely to be unsuitable if requirements are not well understood or are likely to change in the course of the project.
Agile methods, in contrast, produce completely developed and tested features (but a very small subset of the whole) every few weeks or months. The emphasis is on obtaining the smallest workable piece of functionality to deliver business value early, and continually improving it/adding further functionality throughout the life of the project.
In this respect, agile critics incorrectly assert that these features are not placed in context of the overall project, concluding that, if the sponsors of the project are concerned about completing certain goals with a defined timeline or budget, agile may not be appropriate. Adaptations of Scrum show how agile methods are augmented to produce and continuously improve a strategic plan.
Some agile teams use the waterfall model on a small scale, repeating the entire waterfall cycle in every iteration.Other teams, most notably Extreme Programming teams, work on activities simultaneously.

Contrasted with "cowboy coding"
Cowboy coding is the absence of a defined method: team members do whatever they feel is right. Agile development's frequent re-evaluation of plans, emphasis on face-to-face communication, and relatively sparse use of documents sometimes causes people to confuse it with cowboy coding. Agile teams, however, do follow defined (and often very disciplined and rigorous) processes.
As with all development methods, the skill and experience of the users determine the degree of success and/or abuse of such activity. The more rigid controls systematically embedded within a process offer stronger levels of accountability of the users. The degradation of well-intended procedures can lead to activities often categorized as cowboy coding.

Agile

Agile software development refers to a group of software development methodologies that promotes development iterations, open collaboration, and process adaptability throughout the life-cycle of the project.

Agile methods are a family of development processes, not a single approach to software development. In 2001, 17 prominent figures[5] in the field of agile development (then called "light-weight methods") came together at the Snowbird ski resort in Utah to discuss ways of creating software in a lighter, faster, more people-centric way. They created the Agile Manifesto, widely regarded as the canonical definition of agile development and accompanying agile principles.
Some of the principles behind the Agile Manifesto[6] are:
Customer satisfaction by rapid, continuous delivery of useful software
Working software is delivered frequently (weeks rather than months)
Working software is the principal measure of progress
Even late changes in requirements are welcomed
Close, daily cooperation between business people and developers
Face-to-face conversation is the best form of communication (Co-location)
Projects are built around motivated individuals, who should be trusted
Continuous attention to technical excellence and good design
Simplicity
Self-organizing teams
Regular adaptation to changing circumstances
The manifesto spawned a movement in the software industry known as agile software development.
In 2005, Alistair Cockburn and Jim Highsmith gathered another group of people — management experts, this time — and wrote an addendum, known as the PM Declaration of Interdependence

Waterfall model

The waterfall model is a sequential software development model (a process for the creation of software) in which development is seen as flowing steadily downwards (like a waterfall) through the phases of requirements analysis, design, implementation, testing (validation), integration, and maintenance. The origin of the term "waterfall" is often cited to be an article published in 1970 by Winston W. Royce (1929–1995) although Royce did not use the term "waterfall" in this article. Ironically, Royce was presenting this model as an example of a flawed, non-working model





History
In 1970 Royce proposed what is presently referred to as the waterfall model as an initial concept, a model which he argued was flawed (Royce 1970). His paper explored how the initial model could be developed into an iterative model, with feedback from each phase influencing subsequent phases. It is only the initial model that received notice; his own criticism of this initial model has been largely ignored. The phrase "waterfall model" quickly came to refer not to Royce's final, iterative design, but rather to his purely sequentially ordered model. This article uses the popular meaning of the phrase "waterfall model". For an iterative model similar to Royce's final vision, see the spiral model.
Despite Royce's intentions for the waterfall model to be modified into an iterative model, use of the waterfall model as a purely sequential process is still popular, and, for some, the phrase "waterfall model" has since come to refer to any approach to software creation which is seen as inflexible and non-iterative. Those who use the phrase "waterfall model" pejoratively usually see the waterfall model as naive and unsuitable for an iterative process.





Model


In Royce's original waterfall model, the following phases are followed in order:
1. Requirements specification
2. Design
3. Construction (AKA implementation or coding)
4. Integration
5. Testing and debugging (AKA Validation)
6. Installation
7. Maintenance
To follow the waterfall model, one proceeds from one phase to the next in a purely sequential manner. For example, one first completes requirements specification, which are set in stone. When the requirements are fully completed, one proceeds to design. The software in question is designed and a blueprint is drawn for implementers (coders) to follow — this design should be a plan for implementing the requirements given. When the design is fully completed, an implementation of that design is made by coders. Towards the later stages of this implementation phase, disparate software components produced ined to introduce new functionality and remove bugs.
Thus the waterfall model maintains that one should move to a phase only when its preceding phase is completed and perfected. However, there are various modified waterfall models (including Royce's final model) that may include slight or major variations upon this process.





Usage
The waterfall model is widely used by such large software development houses as those employed by the U.S. Department of Defense and NASA, and for many large government projects (see "the standard waterfall model" on the Internet Archive). Those who use such methods do not always formally distinguish between the pure waterfall model and the various modified waterfall models, so it can be difficult to discern exactly which models are being used and to what extent.

Supporting arguments
Time spent early on in software production can lead to greater economy later on in the software lifecycle; that is, it has been shown many times that a bug found in the early stages of the production lifecycle (such as requirements specification or design) is cheaper, in terms of money, effort and time, to fix than the same bug found later on in the process. ([McConnell 1996], p. 72, estimates that "a requirements defect that is left undetected until construction or maintenance will cost 50 to 200 times as much to fix as it would have cost to fix at requirements time.") To take an extreme example, if a program design turns out to be impossible to implement, it is easier to fix the design at the design stage than to realize months later, when program components are being integrated, that all the work done so far has to be scrapped because of a broken design.
This is the central idea behind Big Design Up Front (BDUF) and the waterfall model - time spent early on making sure that requirements and design are absolutely correct will save you much time and effort later. Thus, the thinking of those who follow the waterfall process goes, one should make sure that each phase is 100% complete and absolutely correct before proceeding to the next phase of program creation. Program requirements should be set in stone before design is started (otherwise work put into a design based on incorrect requirements is wasted); the program's design should be perfect before people begin work on implementing the design (otherwise they are implementing the wrong design and their work is wasted), etc.
A further argument for the waterfall model is that it places emphasis on documentation (such as requirements documents and design documents) as well as source code. In less designed and documented methodologies, should team members leave, much knowledge is lost and may be difficult for a project to recover from. Should a fully working design document be present (as is the intent of Big Design Up Front and the waterfall model) new team members or even entirely new teams should be able to familiarize themselves by reading the documents.
As well as the above, some prefer the waterfall model for its simple and arguably more disciplined approach. Rather than what the waterfall adherent sees as chaos, the waterfall model provides a structured approach; the model itself progresses linearly through discrete, easily understandable and explainable phases and thus is easy to understand; it also provides easily markable milestones in the development process. It is perhaps for this reason that the waterfall model is used as a beginning example of a development model in many software engineering texts and courses.
It is argued that the waterfall model and Big Design Up Front in general can be suited to software projects which are stable (especially those projects with unchanging requirements, such as with shrink wrap software) and where it is possible and likely that designers will be able to fully predict problem areas of the system and produce a correct design before implementation is started. The waterfall model also requires that implementers follow the well made, complete design accurately, ensuring that the integration of the system proceeds smoothly.

Criticism
The waterfall model is argued by many to be a bad idea in practice, mainly because of their belief that it is impossible, for any non-trivial project, to get one phase of a software product's lifecycle perfected before moving on to the next phases and learning from them. For example, clients may not be aware of exactly what requirements they want before they see a working prototype and can comment upon it; they may change their requirements constantly, and program designers and implementers may have little control over this. If clients change their requirements after a design is finished, that design must be modified to accommodate the new requirements, invalidating quite a good deal of effort if overly large amounts of time have been invested into Big Design Up Front. Designers may not be aware of future implementation difficulties when writing a design for an unimplemented software product. That is, it may become clear in the implementation phase that a particular area of program functionality is extraordinarily difficult to implement. If this is the case, it is better to revise the design than to persist in using a design that was made based on faulty predictions and that does not account for the newly discovered problem areas.
Steve McConnell in Code Complete (a book which criticizes the widespread use of the waterfall model) refers to design as a "wicked problem" — a problem whose requirements and limitations cannot be entirely known before completion. The implication of this is that it is impossible to perfect one phase of software development, thus it is impossible if using the waterfall model to move on to the next phase.
David Parnas, in "A Rational Design Process: How and Why to Fake It", writes:
“Many of the [system's] details only become known to us as we progress in the [system's] implementation. Some of the things that we learn invalidate our design and we must backtrack.”
The idea behind the waterfall model may be "measure twice; cut once", and those opposed to the waterfall model argue that this idea tends to fall apart when the problem being measured is constantly changing due to requirement modifications and new realizations about the problem itself.
Additional criticisms of a non-iterative development approach (such as the waterfall model) include:

• 
  Unless those who specify requirements and those who design the software system in question are highly competent, it is difficult to know exactly what is needed in each phase of the software process before some time is spent in the phase following it. That is, feedback from following phases is needed to complete preceding phases satisfactorily. For example, the design phase may need feedback from the implementation phase to identify problem design areas. The counter-argument for the waterfall model is that experienced designers may have worked on similar systems before, and so may be able to accurately predict problem areas without time spent prototyping and implementing.
• 
  Constant testing from the design, implementation and verification phases is required to validate the phases preceding them. Constant prototype design work is needed to ensure that requirements are non-contradictory and possible to fulfill; constant implementation is needed to find problem areas and inform the design process; constant integration and verification of the implemented code is necessary to ensure that implementation remains on track. The counter-argument for the waterfall model here is that constant implementation and testing to validate the design and requirements is only needed if the introduction of bugs is likely to be a problem. Users of the waterfall model may argue that if designers (et cetera) follow a disciplined process and do not make mistakes that there is no need for constant work in subsequent phases to validate the preceding phases.
• 
  Frequent incremental builds (following the "release early, release often" philosophy) are often needed to build confidence for a software production team and their client.
• 
  It is difficult to estimate time and cost for each phase of the development process without doing some "recon" work in that phase, unless those estimating time and cost are highly experienced with the type of software product in question.
• 
  The waterfall model brings no formal means of exercising management control over a project and planning control and risk management are not covered within the model itself.
• 
  Very specific skill sets are required for each phase, thus there is a requirement for multiple projects to run in sequence to optimize resource use if all members stay through the course of a given project, or to suffer skill levels by using "jack of all trades" resources throughout each stage.

Modified models

In response to the perceived problems with the pure waterfall model, many modified waterfall models have been introduced. These models may address some or all of the criticisms of the pure waterfall model. Many different models are covered by Steve McConnell in the "lifecycle planning" chapter of his book Rapid Development: Taming Wild Software Schedules.
While all software development models will bear some similarity to the waterfall model, as all software development models will incorporate at least some phases similar to those used within the waterfall model, this section will deal with those closest to the waterfall model. For models which apply further differences to the waterfall model, or for radically different models seek general information on the software development process.

SQA Techniques and Tools

SQA should evaluate its needs for assurance tools versusthose available off-the-shelf for applicability to thespecific project, and must develop the others it requires.Useful tools might include audit and inspection checklistsand automatic code standards analyzers.

Software Quality Assurance During the Software

In addition to the general activities described insubsections C and D, there are phase-specific SQA activitiesthat should be conducted during the Software AcquisitionLife Cycle. At the conclusion of each phase, SQAconcurrence is a key element in the management decision toinitiate the following life cycle phase. Suggestedactivities for each phase are described below.

1. Software Concept and Initiation Phase
SQA should be involved in both writing and reviewing theManagement Plan in order to assure that the processes,procedures, and standards identified in the plan areappropriate, clear, specific, and auditable. During thisphase, SQA also provides the QA section of the ManagementPlan.

2. Software Requirements Phase
During the software requirements phase, SQA assures thatsoftware requirements are complete, testable, and properlyexpressed as functional, performance, and interfacerequirements.

3. Software Architectural (Preliminary) Design Phase
SQA activities during the architectural (preliminary) designphase include: Assuring adherence to approved design standards as designated in the Management Plan. Assuring all software requirements are allocated to software components. Assuring that a testing verification matrix exists and is kept up to date. Assuring the Interface Control Documents are in agreement with the standard in form and content. Reviewing PDR documentation and assuring that all action items are resolved. Assuring the approved design is placed under configuration management.

4. Software Detailed Design Phase
SQA activities during the detailed design phase include: Assuring that approved design standards are followed. Assuring that allocated modules are included in the detailed design. Assuring that results of design inspections are included in the design. Reviewing CDR documentation and assuring that all action items are resolved.

5. Software Implementation Phase
SQA activities during the implementation phase include theaudit of: Results of coding and design activities including the schedule contained in the Software Development Plan. Status of all deliverable items. Configuration management activities and the software development library. Nonconformance reporting and corrective action system.

6. Software Integration and Test Phase
SQA activities during the integration and test phaseinclude: Assuring readiness for testing of all deliverable items. Assuring that all tests are run according to test plans and procedures and that any nonconformances are reported and resolved. Assuring that test reports are complete and correct. Certifying that testing is complete and software and documentation are ready for delivery. Participating in the Test Readiness Review and assuring all action items are completed.

7. Software Acceptance and Delivery Phase
As a minimum, SQA activities during the software acceptanceand delivery phase include assuring the performance of afinal configuration audit to demonstrate that alldeliverable items are ready for delivery.

8. Software Sustaining Engineering and Operations Phase
During this phase, there will be mini-development cycles toenhance or correct the software. During these developmentcycles, SQA conducts the appropriate phase-specificactivities described above.

SQA Relationships to Other Assurance Activities

Some of the more important relationships of SQA to othermanagement and assurance activities are described below.

1. Configuration Management Monitoring
SQA assures that software Configuration Management (CM)activities are performed in accordance with the CM plans,standards, and procedures. SQA reviews the CM plans forcompliance with software CM policies and requirements andprovides follow-up for nonconformances. SQA audits the CMfunctions for adherence to standards and procedures andprepares reports of its findings.

The CM activities monitored and audited by SQA includebaseline control, configuration identification,configuration control, configuration status accounting, andconfiguration authentication. SQA also monitors and auditsthe software library. SQA assures that:
Baselines are established and consistently maintained for use in subsequent baseline development and control.

Software configuration identification is consistent and accurate with respect to the numbering or naming of computer programs, software modules, software units, and associated software documents.

Configuration control is maintained such that the software configuration used in critical phases of testing, acceptance, and delivery is compatible with the associated documentation.

Configuration status accounting is performed accurately including the recording and reporting of data reflecting the software's configuration identification, proposed changes to the configuration identification, and the implementation status of approved changes.

Software configuration authentication is established by a series of configuration reviews and audits that exhibit the performance required by the software requirements specification and the configuration of the software is accurately reflected in the software design documents.

Software development libraries provide for proper handling of software code, documentation, media, and related data in their various forms and versions from the time of their initial approval or acceptance until they have been incorporated into the final media.

Approved changes to baselined software are made properly and consistently in all products, and no unauthorized changes are made.

2. Verification and Validation Monitoring
SQA assures Verification and Validation (V&V) activities bymonitoring technical reviews, inspections, and walkthroughs.The SQA role in formal testing is described in the nextsection. The SQA role in reviews, inspections, andwalkthroughs is to observe, participate as needed, andverify that they were properly conducted and documented.SQA also ensures that any actions required are assigned,documented, scheduled, and updated.
Formal software reviews should be conducted at the end ofeach phase of the life cycle to identify problems anddetermine whether the interim product meets all applicablerequirements. Examples of formal reviews are thePreliminary Design Review (PDR), Critical Design Review(CDR), and Test Readiness Review (TRR). A review looks atthe overall picture of the product being developed to see ifit satisfies its requirements. Reviews are part of thedevelopment process, designed to provide a ready/not-readydecision to begin the next phase. In formal reviews, actualwork done is compared with established standards. SQA'smain objective in reviews is to assure that the Managementand Development Plans have been followed, and that theproduct is ready to proceed with the next phase ofdevelopment. Although the decision to proceed is amanagement decision, SQA is responsible for advisingmanagement and participating in the decision.
An inspection or walkthrough is a detailed examination of aproduct on a step-by-step or line-of-code by line-of-codebasis to find errors. For inspections and walkthroughs, SQAassures, at a minimum, that the process is properlycompleted and that needed follow-up is done. The inspectionprocess may be used to measure compliance to standards.

3. Formal Test Monitoring
SQA assures that formal software testing, such as acceptancetesting, is done in accordance with plans and procedures.SQA reviews testing documentation for completeness andadherence to standards. The documentation review includestest plans, test specifications, test procedures, and testreports. SQA monitors testing and provides follow-up onnonconformances. By test monitoring, SQA assures softwarecompleteness and readiness for delivery.
The objectives of SQA in monitoring formal software testingare to assure that:
The test procedures are testing the software requirements in accordance with test plans.
The test procedures are verifiable.
The correct or "advertised" version of the software is being tested (by SQA monitoring of the CM activity).
The test procedures are followed.
Nonconformances occurring during testing (that is, any incident not expected in the test procedures) are noted and recorded.
Test reports are accurate and complete.
Regression testing is conducted to assure nonconformances have been corrected.
Resolution of all nonconformances takes place prior to delivery.

Software testing verifies that the software meets itsrequirements. The quality of testing is assured byverifying that project requirements are satisfied and thatthe testing process is in accordance with the test plans andprocedures.

Software Quality Assurance Activities

Product evaluation and process monitoring are the SQAactivities that assure the software development and controlprocesses described in the project's Management Plan arecorrectly carried out and that the project's procedures andstandards are followed. Products are monitored forconformance to standards and processes are monitored forconformance to procedures. Audits are a key technique usedto perform product evaluation and process monitoring.Review of the Management Plan should ensure that appropriateSQA approval points are built into these processes.

Product evaluation is an SQA activity that assures standardsare being followed. Ideally, the first products monitoredby SQA should be the project's standards and procedures. SQAassures that clear and achievable standards exist and thenevaluates compliance of the software product to theestablished standards. Product evaluation assures that thesoftware product reflects the requirements of the applicablestandard(s) as identified in the Management Plan.

Process monitoring is an SQA activity that ensures thatappropriate steps to carry out the process are beingfollowed. SQA monitors processes by comparing the actualsteps carried out with those in the documented procedures.The Assurance section of the Management Plan specifies themethods to be used by the SQA process monitoring activity.

A fundamental SQA technique is the audit, which looks at aprocess and/or a product in depth, comparing them toestablished procedures and standards. Audits are used toreview management, technical, and assurance processes toprovide an indication of the quality and status of thesoftware product.

The purpose of an SQA audit is to assure that proper controlprocedures are being followed, that required documentationis maintained, and that the developer's status reportsaccurately reflect the status of the activity. The SQAproduct is an audit report to management consisting offindings and recommendations to bring the development intoconformance with standards and/or procedures.

SOFTWARE QUALITY ASSURANCE

Standards and Procedures

Establishing standards and procedures for softwaredevelopment is critical, since these provide the frameworkfrom which the software evolves. Standards are theestablished criteria to which the software products arecompared. Procedures are the established criteria to whichthe development and control processes are compared.Standards and procedures establish the prescribed methodsfor developing software; the SQA role is to ensure theirexistence and adequacy. Proper documentation of standardsand procedures is necessary since the SQA activities ofprocess monitoring, product evaluation, and auditing relyupon unequivocal definitions to measure project compliance.

Types of standards include:

Documentation Standards specify form and content for planning, control, and product documentation and provide consistency throughout a project. The NASA Data Item Descriptions (DIDs) are documentation standards (see Appendix B).
Design Standards specify the form and content of the design product. They provide rules and methods for translating the software requirements into the software design and for representing it in the design documentation.
Code Standards specify the language in which the code is to be written and define any restrictions on use of language features. They define legal language structures, style conventions, rules for data structures and interfaces, and internal code documentation.

Procedures are explicit steps to be followed in carrying outa process. All processes should have documented procedures.Examples of processes for which procedures are needed areconfiguration management, nonconformance reporting andcorrective action, testing, and formal inspections.
If developed according to the NASA DID, the Management Plandescribes the software development control processes, suchas configuration management, for which there have to beprocedures, and contains a list of the product standards.Standards are to be documented according to the Standardsand Guidelines DID in the Product Specification. Theplanning activities required to assure that both productsand processes comply with designated standards andprocedures are described in the QA portion of the ManagementPlan.

SOFTWARE QUALITY ASSURANCE

Concepts and Definitions

Software Quality Assurance (SQA) is defined as a planned andsystematic approach to the evaluation of the quality of andadherence to software product standards, processes, andprocedures. SQA includes the process of assuring thatstandards and procedures are established and are followedthroughout the software acquisition life cycle. Compliancewith agreed-upon standards and procedures is evaluatedthrough process monitoring, product evaluation, and audits.Software development and control processes should includequality assurance approval points, where an SQA evaluationof the product may be done in relation to the applicablestandards.

Methodology of Software Quality Assurance

Software testing is as much an art as a science. In large, complex applications, such as operating systems, it is practically impossible to iron out every single bug before releasing it both from a difficulty point of view and due to time constraints. Different software applications require different approaches when it comes to testing, but some of the most common tasks in software QA include:

PPQA audits
Process and Product Qualty Assurance is the activity of ensuring that the process and work product conform to the agreed upon process.
The following quality control activities are often confused as quality assurance activities:

Peer Reviews
Peer reviews of a project's work products are the most efficient defect removal (quality control) activity.

Validation testing
Validation testing is the act of entering data that the tester knows to be erroneous into an application. For instance, typing "Hello" into an edit box that is expecting to receive a numeric entry.

Data comparison
Comparing the output of an application with specific parameters to a previously created set of data with the same parameters that is known to be accurate.

Stress testing
A stress test is when the software is used as heavily as possible for a period of time to see whether it copes with high levels of load. Often used for server software that will have multiple users connected to it simultaneously. Also known as Destruction testing.

Usability testing
Sometimes getting users who are unfamiliar with the software to try it for a while and offer feedback to the developers about what they found difficult to do is the best way of making improvements to a user interface.

Advantages of Software Quality Assurance

An SQA plan can take a number of paths, testing for different capabilities and performing different analysis, depending on the demands of project, the users, and the software itself. But any rigorous SQA plan carried out scrupulously by seasoned QA professionals will confer certain benefits:
Improved customer satisfaction Improved customer satisfaction means longer, more profitable customer relationships, positive customer testimonials, and waves of referral business generated from positive word of mouth.
If customers are dissatisfied with a product they have purchased from a particular software vendor, they're likely never to recommend that product nor buy from that software vendor again. Bugs and defects, in addition to seriously hampering an application's functionality, look sloppy and unprofessional, and reflect poorly on a company's reputation.
What's more, without proper testing, it is virtually impossible to know how new users will respond to an application's functions, options, and usability features. Unbiased software quality assurance specialists come to a project fresh, with a clear outlook, and so serve as the first line of defense against unintuitive user interfaces and broken application functionality. A quality application is guaranteed to result in enhanced customer satisfaction.
Reduced cost of development Because the process of software quality assurance is designed to prevent software defects and inefficiencies, projects that incorporate rigorous, objective testing will find that development costs are significantly reduced since all later stages of the development life cycle become streamlined and simplified. With SQA, all further testing and development including user testing and customer deployments will go more smoothly, and of course more quickly -- which means your software development project will consistently reach completion on time and within budget, release after release.
Reduced cost of maintenance Bug-infested applications are troublesome to support. The combined cost of unnecessary recalls, returns, and patches can be frightful. And that says nothing of what will have to be spent on ongoing customer support, be it by telephone, email, or in person. All these costs and more can be dramatically reduced by releasing only rigorously quality-assured products. Software vendors that invest in quality now can avoid big losses in the future.

Software quality assurance

Software quality assurance (SQA) consists of a means of monitoring the software engineering processes and methods used to ensure quality. It does this by means of audits of the quality management system under which the software system is created. These audits are backed by one or more standards, usually ISO 9000 or CMMI.
It is distinct from software quality control which includes reviewing requirements documents, and software testing. SQA encompasses the entire software development process, which includes processes such as software design, coding, source code control, code reviews, change management, configuration management, and release management. Whereas software quality control is a control of products, software quality assurance is a control of processes.
Software quality assurance is related to the practice of quality assurance in product manufacturing. There are, however, some notable differences between software and a manufactured product. These differences stem from the fact that the manufactured product is physical and can be seen whereas the software product is not visible. Therefore its function, benefit and costs are not as easily measured. What's more, when a manufactured product rolls off the assembly line, it is essentially a complete, finished product, whereas software is never finished.[citation needed] Software lives, grows, evolves, and metamorphoses, unlike its tangible counterparts. Therefore, the processes and methods to manage, monitor, and measure its ongoing quality are as fluid and sometimes elusive as are the defects that they are meant to keep in check.

Appraisal Considerations

Choices that affect a CMMI-based appraisal include the following:
· Which CMMI model to use for the appraisal (for this constellation, the choice would be between the CMMI for Development model and the CMMI for Development +IPPD model)
· Establishing the appraisal scope, including the organizational unit to be appraised, the CMMI process areas to be investigated, and the maturity level or capability level(s) to be appraised
· Selecting the appraisal method
· Selecting the appraisal team members
· Selecting appraisal participants from the appraisal entities to be interviewed
· Establishing appraisal outputs (e.g., ratings or instantiation-specific findings)
· Establishing appraisal constraints (e.g., time spent on site)
The SCAMPI MDD allows the selection of predefined options for use in an appraisal. These appraisal options are designed to help organizations align CMMI with their business needs and objectives.
Documentation of CMMI appraisal plans and results must always include a description of the appraisal options, model scope, and organizational scope selected. This documentation confirms whether an appraisal meets the requirements for benchmarking.
For organizations that wish to appraise multiple functions or groups, CMMI’s integrated approach enables some economy of scale in model and appraisal training. One appraisal method can provide separate or combined results for multiple functions.
The appraisal principles for the CMMI Product Suite remain the same as those used in appraisals for other process improvement models. Those principles are as follows:
· Senior management sponsorship
· A focus on the organization’s business objectives
· Confidentiality for interviewees
· Use of a documented appraisal method
· Use of a process reference model (e.g., a CMMI model) as a base
· A collaborative team approach
· A focus on actions for process improvement

SCAMPI Appraisal Methods

The SCAMPI appraisal methods are the generally accepted methods used for conducting appraisals using CMMI models. The SCAMPI Method Definition Document (MDD) defines rules for ensuring the consistency of appraisal ratings. For benchmarking against other organizations, appraisals must ensure consistent ratings. The achievement of a specific maturity level or the satisfaction of a process area must mean the same thing for different appraised organizations.
The SCAMPI family of appraisals includes Class A, B, and C appraisal methods. SCAMPI A is the most rigorous method and the only method that can result in a rating. SCAMPI B provides options in model scope, but the characterization of practices is fixed to one scale and is performed on implemented practices. SCAMPI C provides a wide range of options, including characterization of planned approaches to process implementation according to a scale defined by the user.

Appraisal Requirements for CMMI

The ARC document describes the requirements for several types of appraisals. A full benchmarking class of appraisal is defined as a Class A appraisal. Less formal methods are defined as Class B or Class C methods. The ARC document was designed to help improve consistency across appraisal methods, and to help appraisal method developers, sponsors, and users understand the tradeoffs associated with various methods [SEI 2006a].
Depending on the purpose of the appraisal and the nature of the circumstances, one class may be preferred over the others. Sometimes self-assessments, initial appraisals, quick-look, or mini-appraisals, incremental appraisals, or external appraisals are appropriate, and other times a formal benchmarking appraisal is appropriate.A particular appraisal method is declared an ARC Class A, B, or C appraisal method based on the sets of ARC requirements that the method developer addressed when designing the method.

Using CMMI Appraisals

Many organizations find value in measuring their progress by conducting an appraisal and thus earning a maturity level rating or a capability level achievement profile. These appraisals are typically conducted for one or more of the following reasons:
· To determine how well the organization’s processes compare to CMMI best practices and identify areas where improvement can be made
· To inform external customers and suppliers about how well the organization’s processes compare to CMMI best practices
· To meet the contract requirements of one or more customersAppraisals of organizations using a CMMI model must conform to the requirements defined in the Appraisal Requirements for CMMI (ARC) document. These appraisals focus on identifying improvement opportunities and comparing the organization’s processes to CMMI best practices. Appraisal teams use a CMMI model and ARC-conformant appraisal method to guide their evaluation of the organization as well as how they report their conclusions. The appraisal results are then used (by a process group, for example) to plan improvements for the organization.

CMMI Models

CMMI models describe what have been determined to be best practices that organizations have found to be productive and useful to achieving their business objectives.
Regardless of your type of organization, to apply CMMI best practices, you must use professional judgment when interpreting them for your situation, needs, and business objectives. Although process areas depict the characteristics of an organization committed to process improvement, you must interpret the process areas using an in-depth knowledge of CMMI, your organization, the business environment, and the specific circumstances involved.
As you begin using a CMMI model to improve your organization’s processes, map your real-world processes to CMMI process areas. This mapping enables you to initially judge and later track your organization’s level of conformance to the CMMI model you are using and to identify opportunities for improvement.
To interpret practices, it is important to consider the overall context in which these practices are used and to determine how well the practices satisfy the goals of a process area in that context. CMMI models do not explicitly prescribe nor imply particular processes that are right for any organization or project. Instead, CMMI describes minimal criteria necessary to plan and implement processes selected by the organization for improvement based on business objectives.CMMI practices purposely use nonspecific phrases such as “relevant stakeholders,” “as appropriate,” and “as necessary” to accommodate the needs of different organizations and projects. The specific needs of a project may also differ at various points during its life.

Adopting CMMI

Research has shown that the most powerful initial step to process improvement is to build strong organizational support through strong senior management sponsorship. To gain senior management sponsorship, it is often beneficial to expose senior management to the performance results experienced by others who have used CMMI to improve their processes.
For more information about CMMI performance results, see the SEI Web site at www.sei.cmu.edu/cmmi/results.html [SEI 3].
The senior manager, once committed as the process improvement sponsor, must be actively involved in the CMMI-based process improvement effort. Activities performed by the senior management sponsor include (but are not limited to) the following:
· Influence the organization to adopt CMMI.
· Choose the best people to manage the process improvement effort.
· Monitor the process improvement effort personally.
· Be a visible advocate and spokesperson for the process improvement effort.
· Ensure that adequate resources are available to enable the process improvement effort to be successful.
Given sufficient senior management sponsorship, the next step is establishing a strong, technically competent process group that represents relevant stakeholders to guide process improvement efforts.
For an organization with a mission to develop software-intensive systems, the process group might include engineers representing the different technical disciplines across the organization and other selected members based on the business needs driving improvement. For example, a systems administrator may focus on information-technology support, whereas a marketing representative may focus on integrating customers’ needs. Both members could make powerful contributions to the process group.Once your organization has decided to adopt CMMI, planning can begin with an improvement approach such as the IDEALSM (Initiating, Diagnosing, Establishing, Acting, & Learning) model.

Support

Support process areas cover the activities that support product development and maintenance. The Support process areas address processes that are used in the context of performing other processes. In general, the Support process areas address processes that are targeted toward the project and may address processes that apply more generally to the organization. For example, Process and Product Quality Assurance can be used with all the process areas to provide an objective evaluation of the processes and work products described in all the process areas.
The Support process areas of CMMI are as follows:
· Configuration Management
· Process and Product Quality Assurance
· Measurement and Analysis
· Decision Analysis and Resolution
· Causal Analysis and Resolution

Engineering

Engineering process areas cover the development and maintenance activities that are shared across engineering disciplines. The Engineering process areas were written using general engineering terminology so that any technical discipline involved in the product development process (e.g., software engineering or mechanical engineering) can use them for process improvement.
The Engineering process areas also integrate the processes associated with different engineering disciplines into a single product development process, supporting a product-oriented process improvement strategy. Such a strategy targets essential business objectives rather than specific technical disciplines. This approach to processes effectively avoids the tendency toward an organizational “stovepipe” mentality.
The Engineering process areas apply to the development of any product or service in the development domain (e.g., software products, hardware products, services, or processes).
The technical foundation for IPPD is grounded in a robust systems engineering approach that encompasses development in the context of the phases of the product’s life. The Engineering process areas provide this technical foundation. The implementation of IPPD is further addressed through amplifications to specific practices in the Engineering process areas that emphasize concurrent development and focus on all phases of the product’s life.
The Engineering process areas of CMMI are as follows:
· Requirements Development
· Requirements Management
· Technical Solution
· Product Integration
· Verification
· Validation

Project Management

Project Management process areas cover the project management activities related to planning, monitoring, and controlling the project.
The Project Management process areas of CMMI are as follows:
· Project Planning
· Project Monitoring and Control
· Supplier Agreement Management
· Integrated Project Management +IPPD
· Risk Management
· Quantitative Project Management
Integrated Project Management (IPM) has one goal that applies only when using CMMI with the IPPD group of additions.

Process Management

Process Management process areas contain the cross-project activities related to defining, planning, deploying, implementing, monitoring, controlling, appraising, measuring, and improving processes.
The Process Management process areas of CMMI are as follows:
· Organizational Process Focus
· Organizational Process Definition +IPPD
· Organizational Training
· Organizational Process Performance
· Organizational Innovation and Deployment
Organizational Process Definition (OPD) has one goal that applies only when using CMMI with the IPPD group of additions.

Four Categories of CMMI Process Areas

Process areas can be grouped into four categories:
· Process Management
· Project Management
· Engineering
· Support
Although we are grouping process areas this way to discuss their interactions, process areas often interact and have an effect on one another regardless of their defined group. For example, the Decision Analysis and Resolution process area provides specific practices to address the formal evaluation that is used in the Technical Solution process area for selecting a technical solution from alternative solutions. Technical Solution is an Engineering process area and Decision Analysis and Resolution is a Support process area.Being aware of the interactions that exist among CMMI process areas and which process areas are Basic and Advanced will help you apply CMMI in a useful and productive way. The following sections describe the interactions of process areas within the categories and only briefly describe the interactions among process areas in other categories. Interactions among process areas that belong to different categories are described in references within the Related Process Areas section of the process areas in Part Two. Refer to Chapter 2 for more information about references.

Understanding Maturity Levels

To support those using the staged representation, all CMMI models reflect maturity levels in their design and content. A maturity level consists of related specific and generic practices for a predefined set of process areas that improve the organization’s overall performance. The maturity level of an organization provides a way to predict an organization’s performance in a given discipline or set of disciplines. Experience has shown that organizations do their best when they focus their process improvement efforts on a manageable number of process areas at a time and that those areas require increasing sophistication as the organization improves.
A maturity level is a defined evolutionary plateau for organizational process improvement. Each maturity level matures an important subset of the organization’s processes, preparing it to move to the next maturity level. The maturity levels are measured by the achievement of the specific and generic goals associated with each predefined set of process areas.
There are five maturity levels, each a layer in the foundation for ongoing process improvement, designated by the numbers 1 through 5:
1. Initial
2. Managed
3. Defined
4. Quantitatively Managed
5. Optimizing

Remember that maturity levels 2 through 5 use the same terms as capability levels 2 through 5. This was intentional because the concepts of maturity levels and capability levels are complementary. Maturity levels are used to characterize organizational improvement relative to a set of process areas, and capability levels characterize organizational improvement relative to an individual process area.

Maturity Level 1: Initial
At maturity level 1, processes are usually ad hoc and chaotic. The organization usually does not provide a stable environment to support the processes. Success in these organizations depends on the competence and heroics of the people in the organization and not on the use of proven processes. In spite of this chaos, maturity level 1 organizations often produce products and services that work; however, they frequently exceed their budgets and do not meet their schedules.
Maturity level 1 organizations are characterized by a tendency to over commit, abandonment of processes in a time of crisis, and an inability to repeat their successes.

Maturity Level 2: Managed
At maturity level 2, the projects of the organization have ensured that processes are planned and executed in accordance with policy; the projects employ skilled people who have adequate resources to produce controlled outputs; involve relevant stakeholders; are monitored, controlled, and reviewed; and are evaluated for adherence to their process descriptions. The process discipline reflected by maturity level 2 helps to ensure that existing practices are retained during times of stress. When these practices are in place, projects are performed and managed according to their documented plans.
At maturity level 2, the status of the work products and the delivery of services are visible to management at defined points (e.g., at major milestones and at the completion of major tasks). Commitments are established among relevant stakeholders and are revised as needed. Work products are appropriately controlled. The work products and services satisfy their specified process descriptions, standards, and procedures.

Maturity Level 3: Defined
At maturity level 3, processes are well characterized and understood, and are described in standards, procedures, tools, and methods. The organization’s set of standard processes, which is the basis for maturity level 3, is established and improved over time. These standard processes are used to establish consistency across the organization. Projects establish their defined processes by tailoring the organization’s set of standard processes according to tailoring guidelines. (See the glossary for a definition of “organization’s set of standard processes.”)
A critical distinction between maturity levels 2 and 3 is the scope of standards, process descriptions, and procedures. At maturity level 2, the standards, process descriptions, and procedures may be quite different in each specific instance of the process (e.g., on a particular project). At maturity level 3, the standards, process descriptions, and procedures for a project are tailored from the organization’s set of standard processes to suit a particular project or organizational unit and therefore are more consistent, except for the differences allowed by the tailoring guidelines.
Another critical distinction is that at maturity level 3, processes are typically described more rigorously than at maturity level 2. A defined process clearly states the purpose, inputs, entry criteria, activities, roles, measures, verification steps, outputs, and exit criteria. At maturity level 3, processes are managed more proactively using an understanding of the interrelationships of the process activities and detailed measures of the process, its work products, and its services.
At maturity level 3, the organization must further mature the maturity level 2 process areas. The generic practices associated with generic goal 3 that were not addressed at maturity level 2 are applied to achieve maturity level 3.

Maturity Level 4: Quantitatively Managed
At maturity level 4, the organization and projects establish quantitative objectives for quality and process performance and use them as criteria in managing processes. Quantitative objectives are based on the needs of the customer, end users, organization, and process implementers. Quality and process performance is understood in statistical terms and is managed throughout the life of the processes [SEI 2001].
For selected subprocesses, detailed measures of process performance are collected and statistically analyzed. Quality and process-performance measures are incorporated into the organization’s measurement repository to support fact-based decision making [McGarry 2000]. Special causes of process variation are identified and, where appropriate, the sources of special causes are corrected to prevent future occurrences. (See the definition of “special cause of process variation” in the glossary.)
A critical distinction between maturity levels 3 and 4 is the predictability of process performance. At maturity level 4, the performance of processes is controlled using statistical and other quantitative techniques, and is quantitatively predictable. At maturity level 3, processes are typically only qualitatively predictable.

Maturity Level 5: Optimizing
At maturity level 5, an organization continually improves its processes based on a quantitative understanding of the common causes of variation inherent in processes. (See the definition of “common cause of process variation” in the glossary.)
Maturity level 5 focuses on continually improving process performance through incremental and innovative process and technological improvements. Quantitative process improvement objectives for the organization are established, continually revised to reflect changing business objectives, and used as criteria in managing process improvement. The effects of deployed process improvements are measured and evaluated against the quantitative process improvement objectives. Both the defined processes and the organization’s set of standard processes are targets of measurable improvement activities.
A critical distinction between maturity levels 4 and 5 is the type of process variation addressed. At maturity level 4, the organization is concerned with addressing special causes of process variation and providing statistical predictability of the results. Although processes may produce predictable results, the results may be insufficient to achieve the established objectives. At maturity level 5, the organization is concerned with addressing common causes of process variation and changing the process (to shift the mean of the process performance or reduce the inherent process variation experienced) to improve process performance and to achieve the established quantitative process improvement objectives.

Advancing through Maturity Levels
Organizations can achieve progressive improvements in their organizational maturity by achieving control first at the project level and continuing to the most advanced level—organization-wide continuous process improvement—using both quantitative and qualitative data to make decisions.
Since improved organizational maturity is associated with improvement in the range of expected results that can be achieved by an organization, it is one way of predicting the general outcomes of the organization’s next project. For instance, at maturity level 2, the organization has been elevated from ad hoc to disciplined by establishing sound project management. As your organization achieves the generic and specific goals for the set of process areas in a maturity level, you are increasing your organizational maturity and reaping the benefits of process improvement. Because each maturity level forms a necessary foundation for the next level, trying to skip maturity levels is usually counterproductive.
At the same time, you must recognize that process improvement efforts should focus on the needs of the organization in the context of its business environment and that process areas at higher maturity levels may address the current needs of an organization or project. For example, organizations seeking to move from maturity level 1 to maturity level 2 are frequently encouraged to establish a process group, which is addressed by the Organizational Process Focus process area that resides at maturity level 3. Although a process group is not a necessary characteristic of a maturity level 2 organization, it can be a useful part of the organization’s approach to achieving maturity level 2.
This situation is sometimes characterized as establishing a maturity level 1 process group to bootstrap the maturity level 1 organization to maturity level 2. Maturity level 1 process improvement activities may depend primarily on the insight and competence of the process group staff until an infrastructure to support more disciplined and widespread improvement is in place.
Organizations can institute specific process improvements at any time they choose, even before they are prepared to advance to the maturity level at which the specific practice is recommended. In such situations, however, organizations should understand that the success of these improvements is at risk because the foundation for their successful institutionalization has not been completed. Processes without the proper foundation may fail at the very point they are needed most—under stress.
A defined process that is characteristic of a maturity level 3 organization can be placed at great risk if maturity level 2 management practices are deficient. For example, management may commit to a poorly planned schedule or fail to control changes to baselined requirements. Similarly, many organizations prematurely collect the detailed data characteristic of maturity level 4, only to find the data uninterpretable because of inconsistencies in processes and measurement definitions.
Another example of using processes associated with higher maturity-level process areas is in the building of products. Certainly, we would expect maturity level 1 organizations to perform requirements analysis, design, integration, and verification. These activities are not described until maturity level 3, however, where they are described as the coherent, well-integrated engineering processes that complement a maturing project management capability, put in place so that the engineering improvements are not lost by an ad hoc management process.