A conceptual framework for evolving software processes.
Software processes are complex entities that may last for long periods of time and are carried out through the interaction of humans and computerized tools. They need to continuously evolve in order to cope with different kinds of changes or customizations both in the organization and in the technologies used to support software production activities.In recent years, many software process support technologies have been developed, and have currently been further extended and used in trial projects. Moreover, some research prototypes have generated commercial products, that are marketed and currently used in industrial organizations. Despite these significant efforts and results, however, there is still little conceptual characterization and assessment of the properties of software processes and related support environments. It is difficult to compare and assess existing approaches. Even a common characterization of the problems to be addressed seems to be problematic and difficult to achieve. This is particularly true when we consider the process evolution problem, for which it does not seem that a common view of the issue has been established yet.This paper aims at proposing a conceptual framework to describe and assess flexible and evolving software processes. It is based on the assumption that a software process is composed of two main components: a software production process to carry out software production activities, and a software meta-process to improve and evolve the whole software process.The general requirements and properties of the process domain are first discussed, and the meta-process concept is introduced. Then, we discuss several process related concepts and, in particular, the relationship between the meta-process and the rest of the software process. Methods and technologies needed to support the meta-process are highlighted and discussed. Finally, we apply the resulting framework to an example, in order to show the potential and expected benefits of the proposed approach.
Introduction
During the last decades the problem of producing high quality software products has become
increasingly complex and difficult to manage. One reason for this is the rapid evolution of
technologies and methods to produce software, together with an increased complexity of the
applications to be developed. These two factors are strongly interrelated: advances in the
technology enable creation of new products, services and activities, or modification of old ones,
which in their turn produce new needs, feedbacks, and requirements to software technology
providers. Secondly, software processes are human-oriented [CFFS92], and the interactions
among humans and between humans and the tools that support their activities are characterized
by high variability and unpredictability. This fact further increases the complexity
of the resulting software process, and puts hard demands on management. Finally, software
processes may last for long periods of time, and are thus likely to undergo many changes during
their lifetime in order to cope with new requirements [BL79]. Examples of such changes
are the substitution of (part of) the technologies used to produce software applications, or
the modification of the development strategy and procedures. Other important reasons for
process evolution is the need to (dynamically) customize the process to accommodate requirements
or preferences of individuals that take part in the process, or to cope with various
unanticipated situations.
There is a wide consensus among researchers and practitioners that the complex nature of
software processes is among the main reasons for the software crisis we have been experiencing
during the last decades. The insight that the quality of products and services intrinsically
depends on the quality of the associated process, has steered work on software quality towards
improvement of software processes. More specifically, researchers are concentrating on the
following topics:
ffl How can we describe a software process, in order to understand it and to facilitate
communication and teaching of the its rules and procedures?
ffl How can we better manage and improve the software process?
ffl How can we have humans and computerized tools cooperating in a coordinated and
controlled way to support the software process?
Around software process management and improvement, several recent initiatives have been
launched:
Industrial initiatives to improve software processes have been adopted within software
development organizations. Cooperations between academia and industry have also been
established. A well-know initiative in this area is DARPA's Software Engineering Insti-
tute, created at Carnegie Mellon University in Pittsburgh in 1983 [SEI91].
ffl A new research area has grown up, supported by scientific events that facilitate the
exchange and discussion of results, problems and early experiences ([Der92] [IEE91a]
[IEE91b]).
ffl Outside the area of software production, similar efforts emerge. Within the area of
information systems and office automation, for example, these are carried out under
the common heading of workflow support [HL91]. So far, there has been little cross-fertilization
between the areas.
Taking into consideration this situation and the many initiatives launched in the area, we
believe that two aspects deserve further attention:
ffl The recent intensive work on software processes has led to terminology and definitions of
software processes which are rather confusing. Many different terms are used for the same
or similar concepts, and vice-versa. As a consequence, it is difficult to compare different
technologies and solutions, in order to understand their particularities and assess their
specific merits and drawbacks.
ffl Software processes are not the only production processes. We should aim at reusing
the experience and research from other disciplines, such as information systems, or CIM
systems for VLSI design and production.
It therefore seems worthwhile to establish a reference framework of (software) process concepts
to be used within the software process community as a common set of terms and concepts. The
aim is to facilitate communication and assessment of emerging software process technologies,
especially with respect to software process evolution (see below). It can also be used to exploit
commonalities with related disciplines or research areas. Similar efforts have been done in the
recent past (see [DNR91], [FH92], [CFFS92], and [Lon93]), but further work is still needed.
To properly take into account the different requirements and issues emerged in the above
discussion, we propose to base such a framework on the following assumptions:
ffl A software process is complex and long-lived and needs to evolve to cope with different
kinds of changes, as we discussed above.
ffl A software process is therefore composed of two parts: the actual software production
process and the software meta-process. The software production process includes all the
activities used to build, deliver, and maintain a software product. The software meta-process
(or simply meta-process) includes all the activities related to the evolution of the
entire software process (i.e. both the software production process and the meta-process).
Its goal is to introduce into the software process all the innovations that can enhance the
overall quality of the delivered products, of the software production process and of the
meta-process itself.
In our opinion, the meta-process is a crucial component in such a reference framework, and
represents an effective support to software process improvement. An organization (and the
process through which it operates) must be seen as an organic entity, where the production
process and the meta-process cooperate in a highly interacting way. To evaluate the performance
of a software process, and thus drive its evolution, it is necessary to observe the joint
behavior of the software production process and the meta-process, i.e., the result of their
interaction. Therefore, the meta-process must be built so that it can observe and manipulate
the whole software process (including itself). This means that a software process has to be
conceived as a reflective system.
Summing up, our work is based on the assumption that a software process is an organic entity
whose goals are 1) to produce high-quality software products, and 2) to observe and improve
its own structure according to its performances and the organization's overall strategy and
objectives. The paper elaborates and discusses this basic assumption and is structured as
follows: Section 2 presents the basic concepts used throughout the paper. To accomplish this,
we have tried to provide a synthesis of the current practice, by associating each term with a
definition that is as independent as possible from any specific technology or approach. Section
3 provides the fundamentals of the conceptual framework, by presenting a more detailed
discussion of the meta-process, and of the concepts related to the evolution of a software
process model. Section 4 analyzes the technological and methodological issues related to
the development and support of the meta-process. Section 5 provides examples to put the
presented concepts into practice. Finally, Section 6 provides some conclusions and points at
possible applications of the proposed framework.
2 Basic Concepts
In this section we will present the basic concepts of the proposed framework. As a first step
we informally introduce and motivate some basic definitions, and then we put them in a more
general and systematic context.
A software production process can be coarsely characterized as a partially ordered network of
interacting activities, aiming at producing a software product.
Activities are carried out by human agents (possibly organized in teams) supported by tools.
They work upon software components and related documents, called software items. The set
of software items constitutes what we called a software product.
The software production process needs to be modeled (i.e. explicitly represented) in order to
be effectively repeatable, supportive (automatable), and manageable. In particular, we want
to be able to control a process and to support it. The former issue is related to the ability
to ensure that the process evolves according to specific and desirable rules and procedures.
The latter is related to the ability to automate production steps, and guide and solicit human
activities.
Both the actual process and the corresponding process model need to be continuously evolved
(modified, refined or customized) in order to cope with different kinds of change requests.
Thus, a software production process is part of an encompassing process, called software pro-
cess, which also includes meta-activities, to support the evolution of the whole software pro-
cess. We denote the set of such meta-activities with the term meta-process. This is described
by a meta-process model, being a part of the software process model. The elements produced
by the software process are denoted (software process) artifacts and consist of software
items (produced by the software production process) and software process models and tools
in various forms (see further section 3.1.)
To produce a model of a software process we need software process languages offering suitable
features to describe and manipulate the process artifacts. 1
Parts of the same software process can potentially be adopted under different circumstances
or by different organizations, so software process models are partly reusable. A project defines
the context of a specific instantiation of a software process. A project context includes the
1 Note that the software process model is an artifact and as such is one of the entities that are described
using the process formalism. This aspect of self-reference is the fundamental reason for considering software
processes as reflective systems.
profile of the product to be developed, scheduling, budgetary and organizational constraints,
available resources, specific tools to be used, and any other information that characterizes the
activities related to the production of a specific software product.
Let us now provide a more systematic presentation of process-related definitions:
Process: All the "real-world" elements involved in the production and maintenance of
a product, i.e. artifacts, production technology, activities, agents and process support. It
is composed of a production process, a process support and a meta-process.
ffl Process model: A description of a process expressed in a suitable process modeling
language. A model is always an abstraction of the reality it represents, and is as such
only a partial description, i.e. there are always parts or aspects of the process that are
not captured in the model. It can e.g. consist of templates or types, and instances of
these, to describe the various process elements.
ffl Artifact: The products and "raw material" of a process. An artifact produced by a
process may later be used as raw material by the same or a different process to produce
other artifacts. Artifacts are often persistent and versioned.
ffl Production technology: Set of tools used to support the agents of a process to perform
operations on artifacts.
ffl Activity: An atomic or composite production step of a process. It incorporates and
implements procedures, rules and policies, and aims at generating or modifying a given
set of artifacts. Activities may be organized into networks with both horizontal (chaining)
and vertical (decomposition) dimensions. Activities are associated with roles, production
technology and artifacts.
ffl Agent: A human performing the activities related to a role. An agent is characterized
by skills, cost and availability.
ffl Tool: Computer program supporting or automating a part of the work related to a role.
Tools are characterized by the operations they implement, their cost of use and their
availability. Tools may be further characterized according to the kind of interaction they
support: active tools are able to start an interaction, for example to urge an individual
to accomplish a specific action, while the interaction with passive tools is always initiated
by humans.
ffl Role: A role describes a set of responsibilities and obligations to be covered by an agent.
related to a role are the skills required of its agent. Role descriptions are part of process
models.
ffl Process (modeling) language: Formal notation used to express process models, both
for production processes and meta-processes. Process models both in the analysis, design,
and implementation phase could be modeled (Section 3.3).
ffl Process support technology: Technology used to manipulate and execute process
models. In particular, we can consider here process modeling languages, process modeling
support tools and their operating environment, and process model interpreters.
ffl Process support: A process model and the appropriate process support technology to
define, modify, analyse, and execute it.
ffl Production process: Part of the process in charge of producing and maintaining the
product to be delivered.
ffl Meta-process: Part of the process which is in charge of maintaining and evolving the
whole process, i.e. the production process, the process support and the meta-process
itself.
ffl Software production process: All the real-world elements involved in the production
and maintenance of a software product.
ffl Software production process model: A description of a software production process.
ffl Software item: Artifact produced by the software production process. This includes
software documents and components, such as user requirements, design specifications,
source code, test data, documentation, or releases - and aggregates of such with mutual
dependencies.
ffl Software product: The set of required software items to be developed, delivered, and
possibly maintained.
ffl Software production technology: The production technology of the software production
process. Typical examples of its constituent tools are: compilers, CASE tools and
reverse engineering tools 2 .
ffl Software meta-process: Meta-process whose goal is to maintain and evolve the whole
software process, i.e. the software production process, the software process support and
the software meta-process itself.
ffl Software meta-process model: A model of a software meta-process.
ffl Software process: A software production process, its software meta-process and the
corresponding software process support.
ffl Software process model: A model of a software process.
ffl Software process modeling language: Formal notation used to express software
process models.
There may be sublanguages and corresponding submodels for different aspects of the
software process. Object-oriented entity-relationship modeling is often used to describe
software items and their relationships, while software activities are often expressed either
by production rules (triggers), task networks, process programs, or hybrids. Tools are
often described by activity "envelopes" [BCN90], with operation signatures to express
formal i/o parameters and other constraints.
Since we want to model both the meta-process and the software production process, the
process modeling language must offer reflective features, i.e. it must be possible to create
models that are able to modify their own structure. Non-reflective formalisms can model
part of the software process (typically the software production process), but can not be
used to describe in a single, integrated system both process and meta-process activities.
ffl Software project: Instantiation of a software process to produce a specific software
product, in a given organization, with specific temporal and other constraints and objectives

ffl Software process evolution: Effect of the operations issued within the software meta-process
to modify (partially replace, refine or customize) any of the software production
process, the software meta-process, and the software process support. Evolution thus
encompasses both the "real-world" (the software production process and the software
meta-process) and its description (the software process model), as will be further discussed
in Section 3.
Summing up, the software production process and the software meta-process are entities of
2 Note that businesses other then software production can be modeled and supported using the framework
presented in this paper, by substituting the software production technology with other kinds of technologies.
For example, information systems are processes in which humans and computer tools cooperate to achieve a
specific business goal (e.g. to provide financial, banking or similar kinds of services). The production technology
used within these processes are EDP systems and similar computerized applications.
the real-world, described through models that enable the use of computers to support and
control them. As discussed in more detail later, such models are manipulated in different ways
in order to achieve the above-mentioned goals (control and support), and to evolve according
to the evolution of the external, real-world.
All process elements can be considered a versioned and composite object, controlled by a
configuration management (CM) system. This raises the classical issues of change and consistency
control, change propagation etc. Traditionally, CM needs PM to control activities
related to updating, change propagation, auditing, re-building etc. Conversely, the process elements
themselves are (or should be) under CM control, with corresponding "meta-activities"
- see next section and [CJM
3 Software Process Evolution: the Meta-process
In the previous section, a software process has been defined as the composition of a software
production process, its software meta-process 3 and the corresponding software process sup-
port. To fully understand the nature of "software process evolution", we have to understand
how these components can evolve, both considering them in isolation, and taking into account
possible mutual dependencies.
Firstly, it is obviously possible to change a production process without modifying the meta-
process. This means that we change the way we conduct production activities, but we do not
change the way we model and evolve them. Also, since models are partial, any change in the
real-world process which does not effect the modeled parts, need not be reflected by changes
to the process support.
Secondly, we can improve or modify the meta-process, i.e., the way we model and evolve the
software process, without affecting the production process.
Thirdly, it is possible to change the process support, even if neither the production process
nor the meta-process is changed. That is, both the process model and its associated tools
may be extended (i.e. cover parts of the process that were not described in the initial model),
refined (i.e. give more details) and made more powerful (e.g. giving better user guidance) -
without needing to change the real-world process.
In summary, the software production process, the meta-process and their corresponding models
may all evolve independently of one another.
3.1 Evolution of Process Support
To characterize the evolution of the software process support, we need to better understand
the nature and structure of a software process model. Its own evolution will drive and cause
the evolution of the entire process support.
A software process model is composed of several model variations, holding refined or customized
embodiments and representations of the same model. Variations can be classified
into three different categories (or levels):
3 We will in the following only consider processes in the domain of software. The abbreviated forms "pro-
duction process" and "meta-process" will frequently be used with the meaning "software production process"
and "software meta-process" respectively.
Purpose: Purpose:
Variation Application Change
of model of model
Template none create, modify
Enactable create modify
Enacting create modify

Figure

1: Operations on variations at model application and model change.
1. Template model variation:
This is the definition of the process model as expressed in the process modeling language.
Depending on the underlying paradigm of the language, a template may consist of type
definitions, re-entrant code, or simply source code that may generate several run-time
occurrences.
2. Enactable model variation:
This is a uniquely identifiable instance obtained from a template variation. Furthermore,
it contains the necessary context-specific information to make it possible to execute 4
it. Notice thus that at a given time it is possible to have several enactable variations
originated from the same template variation.
3. Enacting model variation:
When enactment actually starts, with a binding to a process model interpreter and with
an explicit enactment state, an enacting variation is created from an enactable.
We then have an application dimension to describe primitive operations on the three
variations of process models, in order to create these from one another.
There is also a change dimension to characterize the modifications that can be applied to
an existing variation, according to given requirements. For example, a template variation of a
software process model can be changed to specify an alternative or better way to accomplish a
given goal, or can be specialized/customized to cope with more specific requirements. But it
is for example also possible to modify an enactable variation, without changing the template
variation of the software process model, before the enacting variation is created and can start
executing.
The two relevant, primitive operations on variations are create and modify. The enactment of
a software process model involves six possible operations of creation and modification of the
three model variations. Depending on the purpose of an enactment step, different operations
are invoked as shown in figure 1:
ffl When the purpose is to apply an existing model, new enactable and enacting variations
are created.
ffl When the purpose is to change an existing model, this is accomplished either by creating
or modifying a template, or by modifying enactable or enacting variations.
Operations of model variations are implemented by the tools of the software process support,
e.g. the process language editor, the process language interpreter, the debugger for enacting
In line with common practice, we use enactment as a synonym for execution in the domain of process
models
variations etc. Let us consider the three process support systems the three authors are
involved in. In SLANG [BFG93], there is one Process Engine tool, which creates the enactable
and enacting variations (in an intertwined way), starting from the template variation of the
software process model. In EPOS [C + 90], there is a Planner tool which creates the enactable
variation, and a second Execution Manager tool that creates and operates on (executes)
enacting variations (these two tools work intertwined). In Process WEAVER [FER93], there
is a process instantiation tool which copies a template, provides it with enactment parameters
and links it with an interpretor to form the enactable variation. An enacting variation is
created by a process starter tool and is then simply managed as a UNIX process. Both
the process instantiation and starter tools can be accessed from enacting models, to allow
process support for the meta-process. In addition, all three process support systems above
offer process model editors to help creating and modifying template variations.
Evolution of the software process support is more than simply creation or modification of
template variations. Different scenarios can be envisaged, by considering different sequences
of variation operations. We consider here three significant styles of evolution:
ffl The first style is what we call delayed change of the software process support. When
we modify the template variation of a software process model, we do not propagate
the corresponding modification to the existing enactable and enacting variations. The
modification is "seen" only when new enactable and enacting variations of the process
model are created.
ffl The second style is what we call busy change of the software process support. When
we modify the template variation of a software process model, we also propagate (the
consequences of) this modification to the enactable and enacting variations generated
from it.
ffl The third style is what we call local customization of an enactable or enacted variation
of a software process model. In this case, we are interested in a change that does not
affect the template variation of the software process model, and which is just effective as
long as the affected variation exists.
If the software process is described by a single integrated model (a reflective system), these
operations can be invoked from some other activity variation which is part of the meta-process.
In general, therefore, the evolution of a software process model can be carried out under the
control of some enacting meta-variations. This fact is particularly significant when we shall
consider the structure of the meta-process in the following.
An important comment concerns the granularity of the operations along the two dimensions.
For the sake of simplicity, we have implicitly assumed that a variation is associated with an
process model. Actually, it is possible to extend or redefine the variation concept e.g.
to the activity level, so that each activity can have a template variation and several enactable
and enacting variations. This extension does not substantially affect the general scheme
presented above. Indeed, it allows us to characterize real situations where a software process
model evolves through the invocation of operations applied e.g. to single activity variations
in the software process model.
Notice, also, that the borderlines between the different kinds of model variations may technically
vanish when adopting specific technologies within the process formalism and the support
tools. In particular, late/dynamic binding can facilitate incremental creation and evolution
of template, enactable and enacting variations, thus providing a high degree of flexibility and
variability.
3.2 Evolution of Processes
The previous section was concerned with the evolution of process support (and specially
models) in its various forms. Up to now very little has been said about the evolution of
the process itself. That is, the focus has been to identify a new process that meets new
requirements and to build a process model in sufficient detail to allow enactment support for
such a new process.
Outside the scope of laboratories it is however unlikely that the new process, and the corresponding
enactment of the new process model, can be installed without further preparation.
The migration from the existing process to the target process is in itself a (meta-)process: a
transient change process. Part of the problem addressed by the meta-process is therefore to
define a process model for the transient process.
There are numerous advantages of creating an explicit model of the transient process. Firstly,
such a model constitutes a proof that the target process is attainable from the existing process.
If it is impossible to define the transient process, it is also unlikely that the defined target
process can replace the existing process.
Secondly, it may guide the preparation of the environment to accept the new target process.
This preparation may be very far-reaching, and include technical aspects such as the introduction
of new tools in the environment or the transformation of data to correctly represent
new types and information structures, as well as non-technical aspects such as user training,
motivation, and the preparation of new user documentation.
Thirdly, it may be used to plan and to budget the introduction of the new process. Process
improvement is ultimately measured in terms of cost. Benefits from changing an existing
process must therefore be measurable either in the short term (e.g. reduced end-to-end time
or resource costs), in the medium term (e.g. better sharing of resources or higher degree
of reuse across projects), or in the long term (e.g. higher skills or better team motivation).
However, benefits from improved processes must be put in relation to the cost of process
change - i.e. the cost of the transient process.
Finally, the transient process model can help to measure progress in introducing a new target
process: the state of the transient process indicates how far a process improvement activity
has come to reach its goal of installing a new process.
3.3 Structure of the Meta-process: Process Lifecycle
In the previous sections we have provided concepts to understand the nature of software
processes, both in the real-world and their corresponding models, and to characterize how
they can be changed. In this section we shall consider evolution from a more general point
of view, and analyze the properties of the meta-process. As stated earlier, the meta-process
is the process concerned with creating and evolving the software process. This means that
it must include all those activities that allow the agents to apply all changes to the existing
process and its corresponding process support. We will therefore in this section describe the
meta-process in terms of its principal constituent meta-activities.
Meta-activities can be built and composed in many different ways, i.e., it is not possible to
identify one universal meta-process for all possible software processes. However, it is possible
to define some basic general meta-activities that constitute the skeleton of any meta-process.
Technology provision This meta-activity is in charge of producing or acquiring the technologies
and tools used within the production process (software production technologies)
and the software process support (software process support technology). Notice that
if this involves the actual production of technology, this leads to a "process within the
process", and thus introduces a level of recursion in the software process.
Process requirements analysis This meta-activity takes into account the software production
process, the meta-process and the existing software process support to provide
(new) requirements to the design meta-activity. The resulting requirements specify (part
of) the features and properties that the software process has to offer.
Process design When a new software process is created, or if new requirements for an
existing process are identified, this meta-activity provides the general and detailed architecture
of the software process. Design specifications might affect different parts of
the software process, i.e. the production process, the meta-process, and/or the process
support. An important input to this meta-activity is the description of the technology
that has to be integrated and used within the software process.
Process implementation This meta-activity is in charge of implementing the design specification
produced by the previous meta-activity. As described previously, this includes
two conceptually distinct aspects:
ffl Changing the process support (Section 3.1)
If the design specification requires the creation/modification of the process support,
its implementation is accomplished by creating or modifying model variations of
the software process model, possibly at fine granularity. In addition comes possible
changes to the support tools.
ffl Changing the real-world process (Section 3.2)
If the design specification requires the modification/creation of the real-world process
then this usually involves the definition of a transient change process to manage
the installation of the new process.
Process assessment This meta-activity provides quantitative and qualitative information
describing the performance of the software process. Such information is used by the
process requirements analysis meta-activity.
We want to stress once again that the proposed set of meta-activities does not impose any
specific meta-process lifecycle, but should give an indication of the fundamental steps that
have to be taken in order to manage a software process. It should be possible to support a
meta-process following an evolutionary and incremental lifecycle, if we are able to technically
build software process support that allows us to launch multiple meta-activities on different
meta-activity variations.
Notice also the evident analogy between the meta-process of the software process domain
and traditional software production lifecycles. This suggests that we have to identify and
adopt technologies and methods that might be quite similar, at least in principle, to what
is used in software production. For instance, we might use executable specifications in both
the analysis, design and implementation meta-activities in order to support a meta-process
based on a transformational lifecycle. A particular aspect of the software process domain,
however, is its evolving and reflective nature, which imposes specific requirements on the
software process support technology (see Section 4).
3.4 Key Roles in the Software Process
The following main roles can be identified in relation to the the principal activities of the
software process:
Process Owner: is responsible for the software process under consideration, provides informal
process descriptions and evaluates process performance, thus producing additional
feedback and requirements for process improvement. The process owner is the main
source of information for the process analysis meta-activity.
Process Technology Provider: is responsible for providing the software process technol-
ogy. Meta-activity: technology provision.
Process Designer: is responsible for providing the process support to the software process:
the template variation of the process model and specifications of the process support
tools. Meta-activities: process requirements analysis and process design.
Process Manager: is responsible for the implementation of the software process and in
particular the transformation of the template variation of the process model into its enactable
and enacted variations. The process manager is also in charge of collecting data
and information describing the performance of the software process. Meta-activities:
process implementation and process assessment.
Process Agent: a human who operates within the software production process. A process
agent uses the set of domain/application-specific rules, tools and technologies, all of
which are controlled, assisted or enforced by the software process support.
4 Techniques and Methods for Process Evolution
As mentioned, a process consists of a production process and a meta-process, each consisting
of "external" process elements (artifacts, agents, tools etc. - i.e., the real-world) and the
"internal" process support. The latter again consists of a process model in different variations
and of process model support tools.
A natural question is which aspects of the above process elements or support (process pa-
rameters) are inclined or allowed to change, and due to what inputs or feedbacks? Further,
how can the change be implemented and propagated, by which meta-activity (when), and by
which human roles? All this involves discussing the detailed requirements and architecture
of the meta-process, its implementation and thus the specific procedures, rules and activities
used to produce, enact and maintain the software process.
In the previous section, we discussed some aspects mainly of process models, their evolution
and associated meta-activities. In the following, we will first elaborate on the nature of the
meta-process, and categorize some available techniques to manage process evolution. Then we
sketch a framework for method support for such evolution and discuss some current method
approaches. The focus will be evolution of models, not tools or external process artifacts.
4.1 Nature of the Meta-processes
The need for and character of process support and for process changes will differ between
organizations. Sometimes a fixed process is imposed by management; other times the process
can change almost continuously by process agents, acting as process managers, in a controlled
manner. In the latter case, we want to treat meta-activities as flexible and integrated parts
of the process. They need to be activated at appropriate times and points in the process
lifecycle, because of feedbacks or incremental detailing of model definitions. They can often
technically be modeled and implemented as normal production activities. Then what is so
special about meta-activities?
From previous discussions, four observations naturally come out:
ffl Meta-activities have deep semantics and may occur frequently:
These activities may fundamentally affect process behavior, as their meta-artifacts have
a large influence "range". Frequent process changes also necessitate technologies like
dynamic schemas and rapid prototyping (see next subsection).
ffl Meta-activities must be carefully controlled:
Regardless of the underlying machinery for change support, we need methods to structure
and control the meta-process. We cannot allow undisciplined access or transformations
of meta-models and meta-tools.
ffl There is a potentially infinite recursion of meta-meta processes:
If the meta-process is a normal process to be created, controlled, and maintained, what is
the (meta-meta-)process to produce a meta-process, and so on? Like in any other reflective
framework, we must let some of the meta-tools have embedded semantics to "root"
this recursion. Note the analogy to bootstrapped systems, e.g. self-defining compilers
with two "recursion" steps per new version of the compiler.
ffl There are lessons to be learned from software engineering with respect to
methods, techniques and tools:
What are the mechanisms, principles, methods, and policies we implement within the
meta-process to produce (build and maintain) the software process? Existing software
production technologies and methods should here be carefully assessed to verify their
suitability within the software process and meta-process domain. We could even suggest
CASE tools for process support, assisting the development and maintenance of process
models and their tools. Such integrated meta-tools would combine individual meta-tools
and other technologies. They could coherently offer meta-activity support for model
definition, analysis, creation, evolution, assessment etc. A high degree of flexibility (meta-
meta-level!) is needed for such CASE tools to be effective.
4.2 Mechanisms to Assist Process Evolution
Software technology embodies an arsenal of techniques that can be used to support meta-
processes: rule-based expression of constraints, rapid prototyping, structured analysis and
design, top-down and/or bottom-up design, object-oriented design and coding, modularization
with explicit interfaces, reusable libraries, reflection, dynamic binding, verification and
validation including testing, configuration management, schema evolution, assessment and
propagation of side-effects, access control and human roles, etc.
To properly support process model evolution and transformation, we will probably need
several kinds of mechanisms. The following five technologies seem most important:
Flexible architectures to support reconfigurable, modular, and distributed systems.
This is needed in order to dynamically reconfigure the process support environment upon
changes.
ffl Reflection: i.e., a process modeling language should support late / dynamic binding and
allow executable code to be manipulated as data (like in Lisp, Smalltalk, or Prolog). This
is necessary to make it possible to dynamically reason upon, manipulate, and interpret
a model ("code"), e.g. represented by "reified" classes and meta-classes [MN88]. This
is needed if we want to describe the software process as a single, integrated model.
Therefore, most process support environments - with highly diverse process modeling
languages - rely on such techniques to achieve model flexibility during enactment.
ffl Schema evolution and -versioning:
Technically we have to manage evolving types/classes:
- versioned or modified classes,
with all possible consequences for class instances 5 (see below also).
- subclasses or generics/parameterized classes.
views.
instance-level delegation.
- ad-hoc, embedded binding mechanisms in the process support tools.
Schema evolution has been discussed extensively in the database literature [B + 87]. It
has a fundamental impact on persistent data, structured according to a given schema.
However, in process models we mostly need to consider "soft" or behavioral changes, not
"hard" changes of the structure of stored data (altering attribute layout, surgery on class
hierarchy).
ffl Impact analysis: This is needed to assess the effect of changes, e.g. to check that stated
constraints are still satisfied after a (potential) change.
Restricting access rights:
e.g. by modularization, interfaces, and scoping, or by associating roles to activities.
4.3 Methods and Principles to Guide Process Evolution
The previous subsection dealt with technical mechanisms to handle process support evolution.
We shall now briefly study how we can define a method, i.e. meta-level rules, policies etc.,
to control and guide this evolution. We will assume that the same method used to define
or provide the initial process support also can be used to evolve it later, as the definition
of the process support is evolutionary. Namely, we need an evolutionary production method
for software processes. This method must cover the entire lifecycle of a process: Technology
Provision, Analysis, Design, Implementation, and Assessment. It is our opinion that no such
total method has reached sufficient maturity. What we can consider today are method fragments
that may be applied to the individual meta-activities. Exploiting established methods
from software engineering, we might come up with the following list:
ffl Technology provision:
The various process support tools need to embody a significant number of the mechanisms
described in section 4.2. Due to their research-oriented nature, it should be expected
that technology provision will be prototype-oriented with evolutionary introduction of
new features.
ffl Process requirement analysis:
Generally, we believe that conventional methods for analysing information systems (like
SADT[Ros77], semantic modeling, object-oriented analysis) can be used for certain as-
pects. However, our current knowledge of especially the fine-grained process aspects is
5 "Software artifacts have longer life-time and are more stable than their models!"[Est92].
very low, and we will need to rely on combinations of knowledge acquisition techniques
and formal techniques.
ffl Process design:
We should expect that process modeling languages for design (and for implementation
below) to embody known support for modularity and reuse. A future trend (once a
significant number of process supported projects have been successfully completed) will
be towards process design from reusable process fragments, and the role of the process
designer will change character from being very technical (with good programming skills)
towards more focus on quality methods and techniques.
ffl Process implementation:
Process implementation is probably the most innovative area and the role of a process
manager is quite new, with responsibilities somewhere between a quality manager and a
project manager. Typical for the processes to manage is that they cross organizational
boundaries, i.e. they involve several teams or sub-teams. Especially in the domain of
process evolution will this activity entangle new methods and techniques, both technical
and managerial.
ffl Process assessment:
Here it seems that the emerging process capability method [Hum88] will have its application

We must regretfully conclude this section by saying that there is very little (new) method work
on systematic support for process evolution, both in the production process and meta-process.
A large repertoire of technologies is available, both for process and meta-process modeling and
instrumentation, but a complete method base is missing. One significant example, however,
of a method encompassing both the production process and the meta-process, is the PRISM
model of change [Mad91]. This method suggests a general scheme to apply and track changes
to different kinds of artifacts such as software items and roles, but also process models and
other meta-process related entities.
5 An Example: a Bank EDP Center
The EDP center of a bank is usually a large organization, whose goal is to provide the bank
departments with computer applications to support and conduct the bank's business, i.e. to
offer efficient and valuable financial services to its customers.
The EDP center has a software process in which analysts and programmers (human agents)
use software development tools and follow specific rules and procedures, in order to produce
the above-mentioned software products. The management of the EDP center must coordinate
and manage the different activities carried out within it, to ensure that the software products
are delivered and maintained according to the EDP center's quality plan. Moreover, it has to
maintain and evolve the process used to deliver such software products, i.e. it manages the
meta-process. Such a meta-process is in charge of acquiring new software development tools
(e.g. a new COBOL compiler or a reverse engineering tool), introducing new methods and
approaches to software development (e.g. a prototype-based development), or updating the
procedures used in the software process (e.g. modifying the CM operating procedures).
Both the production process and the meta-process can be described to better provide guidance
and support to the EDP center's employees. Let us assume that the meta-process is not
formally modeled and supported. Moreover, assume the process support includes only the
model of a part of the software production process, namely CM activities. It is worthwhile to
see how different kinds of changes then can be accomplished, and how they can be described
using the conceptual framework presented in this paper.
ffl Enriching the process with productivity measurements
Let us suppose that we decide to introduce a measurement activity in the software pro-
cess, to evaluate programmer productivity. This requirement is established by the managing
director of the bank (process owner) who wants to have a clearer assessment of
the productivity of the EDP center. In designing such a change to the process, the software
process designer realizes that it does not affect the current model of the software
production process, since, at least in the first place, this does not deal with CM activ-
ities. He decides that the software production process has to be modified, by having
programmers filling in an information form as soon as they release a source module. The
implementation of such a design specification is accomplished by sending out a memorandum
to all programmers, with the indications on how to accomplish the above-mentioned
measurement activity, and by distributing a copy of the form.
ffl Revising the process model by CM procedures for check-in
Later on, we may want to change the way a specific part of the CM activity is carried
out. In particular, we want to add a step to the check-in procedure, so that an e-mail
is automatically sent to the programmer who is releasing a module, to invite him to
complete the productivity form. We might also require that check-in operations that
have not been terminated yet, are restarted using the new activity specification.
Using the terminology introduced in the previous sections, the above specification requires
that, after we have modified the template variation of the check-in activity, we
have to modify its enactable and enacted variations accordingly. To implement such a
modification, therefore, we have to use a model editor to modify the template variation
of the check-in activity, by adding the invocation of the e-mail tool. Moreover, we need
to invoke some monitor/debugger tool to propagate the modification to ongoing check-in
activities, i.e., existing enactable and enacted variations. The model editor and the mon-
itor//debugger are both meta-tools in the process support technology used to create the
software process support.
Notice that since the meta-process is not modeled and thus not managed by the software
process support. It is up to the process manager/designer to properly invoke the editor
and the monitor, in order to accomplish the required modification of the software process
support.
ffl Modeling the implementation meta-activity: busy vs. delayed propagation of
model changes
Finally, we may want to change the process support by introducing a model of the implementation
meta-activity. We are therefore introducing a way to automatically support
changes to the software process. As in the previous case, we have to invoke the editor to
specify the new template variation describing the implementation meta-activity. Such a
variation will contain a formal description of the steps we have been "manually" conducting
so far, for example to modify the check-in activity model. In particular, it will contain
the invocations of the model editor and the monitor, and will specify how to use them to
accomplish the required evolution strategy (delayed change, busy change, . As soon
as this editing activity is terminated, and the template variation of the implementation
meta-activity is therefore available, it is possible to implement further changes to the
process model by creating enactable and enacted variations of this newly created meta-
activity. Eventually, we have "manually" introduced changes to the software process
model to "automatically" support and guide future modifications.
It seems to us that this example demonstrates that the proposed framework can be fruitfully
used to describe and analyze the evolution of a software process, i.e. the software production
process, the meta-process, and the software process support. In particular, it is possible to
describe the operations occurring in an integrated production process / meta-process system,
according to a high-level reference model and using a set of coherent and consistent concepts.
6 Conclusions
The paper has presented a reference framework of concepts for (software) processes, with
emphasis on their evolution. The main concepts are as follows: A (whole) process consists of
a production process, a meta-process, and the process support. The production process is the
set of "external" production elements (real-world activities, artifacts, tools, agents, roles, and
embedding project). The meta-process is the set of similar "external" meta-elements, that
evolve and control the whole process. The process support is the "internal" process model
and tools, that govern both the production process and the meta-process.
The meta-activities are divided into five phases in a process lifecycle: Technology provi-
sion, Process requirements analysis, Process design, Process implementation, and Process
assessment. These phases can be active in a fine-grained and incremental way, e.g. based on
feedbacks from later phases.
A process model describes the above-mentioned process elements, possibly at a fine granu-
larity, and is dynamically and incrementally (re)elaborated and transformed in the process
implementation meta-activity. It goes from a template model variation, via an enacted model
variation to an enacting model variation.
The framework is applicable for technology providers and technology users. It should not
act as a mere check-list, e.g. does system X offer formal "meta-process tools" or "enacting
template models". Rather, it should provide a systematic way to understand, assess and
compare process support systems. We can compare it with the Object-Oriented Database
System manifesto [ABD
Section 1 in the paper points at some related work. Much underlying technology can be
taken and adapted from general software engineering, including dynamic instrumentation
and binding facilities.
However, we still miss a reference method for process evolution that covers non-trial applica-
tions, and that is coupled to tools and roles for the process actors of the meta-process. We
also need better understanding of the process implementation issue, especially with respect
to managing process evolution.
The framework will be evaluated in the ESPRIT Basic Research Action #7082, PROMOTER,
where it will be applied to different scenarios and process management systems.

Acknowledgements

Thanks go to the participants of the 2nd European Software Process Workshop and to our
partners in PROMOTER. In particular, we want to acknowledge the contribution of Bob
Snowdon at ICL, who has provided us with many valuable ideas and comments. Individually,
the authors also want to express their thanks to colleagues at Politecnico di Milano, CEFRIEL,
NTH, Cap Gemini Innovation and in the Eureka Software Factory project.



--R

The Object-Oriented Database System Manifesto
The ATIS reference model.
Process Modeling in the Large with SLANG.
Semantics and implementation of schema evolution in object-oriented databases
Characteristics of large systems.
Design of the Kernel EPOS Software Engineering Environ- ment

Towards a Reference Framework for Fundamental (Software) Process Concepts.

Fundamental Software Process Concepts.
In discussing dynamic typing in the Adele versioned software engineering database.
Process WEAVER: Adding Process Support to UNIX.
Software Process Development and Enact- ment: Definitions

Workflow Management Software: the Business Opportunity.
Characterizing the Software Process: A Maturity Framework.
IEEE Computer Society.
IEEE Computer Society.
A structured conceptual and terminological framework for software process engineering.
The process cycle.

Structured Analysis for Requirements Definition.

--TR

--CTR
Ulrike Becker-Kornstaedt , Roman Reinert, A concept to support process model maintenance through systematic experience capture, Proceedings of the 14th international conference on Software engineering and knowledge engineering, July 15-19, 2002, Ischia, Italy
Gregor Joeris, Change management needs integrated process and configuration management, ACM SIGSOFT Software Engineering Notes, v.22 n.6, p.125-141, Nov. 1997
Bjrn Decker , Jrg Rech , Klaus-Dieter Althoff , Andreas Klotz , Edda Leopold , Angie Voss, eParticipative process learning: process-oriented experience management and conflict solving, Data & Knowledge Engineering, v.52 n.1, p.5-31, January 2005
software process;process evolution and improvement;process modeling;metaprocess