Monday, December 10, 2007

Software Methodologies

Software Engineering Methodologies

In software engineering and project management, a methodology is a codified set of practices (sometimes accompanied by training materials, formal educational programs, worksheets, and diagramming tools) that may be repeatably carried out to produce software.

Software engineering methodologies span many disciplines, including project management, analysis, specification, design, coding, testing, and quality assurance. All of the methodologies guiding this field are collations of all of these disciplines.

Methodology Versus Method

Disagreement exists regarding the relationship between the terms method and methodology. In common use, the methodology is frequently substituted for method; seldom does the opposite occur. Some argue this occurs because methodology sounds more scholarly or important than method. A footnote to the word methodology in the 2006 American Heritage Dictionary notes that, "the misuse of methodology obscures an important conceptual distinction between the tools of scientific investigation (properly methods) and the principles that determine how such tools are deployed and interpreted." In academia, the distinction between practice (i.e., method) and the philosophical basis for the practice (i.e., methodology) tends to be more clearly delineated.

In Software Engineering in particular, the discussion continues. One could argue that a software engineering method is a recipe, a series of steps, to build software, while a methodology is a codified set of recommended practices, sometimes accompanied by training materials, formal educational programs, worksheets, and diagramming tools. In this way, a software engineering method could be part of a methodology. Also, some authors believe that in a methodology there is an overall philosophical approach to the problem. Using these definitions, Software Engineering is rich in methods, but has fewer methodologies. There are two main stream types of methodologies: Structured Methodology (Information Engineering, SSADM and others), which encompass many methods and software processes; and Object Oriented Methodology (OOA/OOD and others).

Methodology

In general, almost all methodologies can be classified into two

  1. Planned Development.
  2. Unplanned development.

Planned development methodologies insist that there should be non code artefacts that describe the system before actual development can start. That is, we will need to produce documents for requirements, architecture and design, test plans and test cases, before we actually start development. This approach has its advantages and is invaluable in distributed development or development following an onsite-offshore model. However, the draw back of this methodology is heavy investment upfront in time and material in planning the non-development activities. Another draw back is once development starts and change requests start getting implemented, most of the analysis and design time artefacts soon get out of sync. One may argue that this need not be the case always and he/she will not be wrong. There is no need why all artefacts can not be in sync and not just a Project Plan or a Requirement Traceability matrix. My own experience over the years has only proved that cases such as the latter are extremely few and far between considering even the process heavy CMM level 5 companies! The main problem is not because these processes do not offer vale. On the contrary they do. The problem is one of implementing these processes and ensuring that these are followed. Planned processes are generally so heavy weight that they are often delegated to oblivion, only to be dusted back to life when there is a quality audit.

Unplanned development relies on informal communication between the development team and the requirements team for building an application. The specifications are narrated like story. Development team starts building very simple building blocks first and continue building and refactoring as new requirements come in or existing requirements change. Refactoring is key in this type of development and implementations mostly pattern driven to allow efficient recapturing. This is the XP or eXtreme Programming methodology. XP also advocates Test Driven Development strongly. My own take on this is that while Test Driven development is the way to go, for teams that are not co-located, XP is far from ideal.

For development that involves an onsite-offshore model, a combination of planned and unplanned methodology yields the best results. The planned phases of a Rational Unified Process are invaluable for Requirement Gathering, Analysis and Design. However these phases are not really significant for the implementation discipline where Test Driven development advocated by XP makes a compelling case.

The most important takeaway from this discussion is that we need both planned and unplanned processes and we need a painless way of ensuring that these processes are followed, a process that is as lean on documentation as possible while allowing the processses to pass quality audits