Distributed component architecture for scientific applications.
The ideal goal of not having the user dealing with concurrency aspects has proven hard to achieve in the context of system (compiler, run-time) supported automatic parallelization for general purpose languages and applications. More focused approaches, of automatic parallelization for numerical applications with a regular structure have been successful. Still, they cannot fully handle irregular applications (e.g the solution of Partial Differential Equations (PDEs) for general geometries).This paper describes a new approach to the parallelization of scientific codes. We make use of the object-oriented and generic programming techniques in order to make parallelism implicit (invisible for the user). Instead of generating a new solution based on a existing one, we take advantage of the application characteristics in order to capture the concurrency infrastructure and to provide part of the solution process to the user. Our goal is to achieve "transparent" concurrency by giving the user the illusion of a sequential programming environment. We isolate the user from the tedious aspects of geometrical data representation, communication patterns computation and communication generation in the process of writing a parallel solver for PDEs. The user concentrates on providing only the local numerical computations which is a straight forward mapping from the numerical algorithm.Furthermore, we describe a system that demonstrates our approach. We address the issues of efficiency for our system and we show that our approach is scalable.
Introduction
Most of the scientic computing applications are concerned
with the solution of the Partial Dierential
Equations (PDEs) which describe some physical phe-
nomena. Typical application areas are computa-
tional
uid dynamics, computational biology, and so
forth. The solution of PDEs, either by Finite Element
Method (FEM), or by Finite Dierence (FD), involves
the discretization of the physical domain and the computation
of the solution at the discretized points. The
computation is then assembled into a global system
of linear equations.
The discrete solution algorithm of a PDE is called
solver. The scientic computing eld is concerned
with employing powerful computing resources for
solving numerical analysis problems, such as PDEs.
For example, for simulating a protein folding modeled
by PDEs, 10000 particles are used and
Copyright c
2002, Australian Computer Society, Inc. This paper
appeared at the 40th International Conference on Technology of
Object-Oriented Languages and Systems (TOOLS Pacic 2002),
Sydney, Australia. Conferences in Research and Practice in Information
Technology, Vol. 10. James Noble and John Potter,
Eds. Reproduction for academic, not-for prot purposes permitted
provided this text is included.
time steps are required. This would require
days (therefore for a feasible computation time frame,
one would need steps). In this exam-
ple, he biological data consists of 1500 protein elds,
protein sequences, so 15000 protein models.
In order to predict the protein structure, all possible
sets (structures) have to be generated (optimize
by performing a selection based on a \energy func-
The only way to deal with the size of data arising
from large, numerical computations and the number
of iteration steps involved is to use parallel com-
puting. Parallel computing has been employed extensively
in the scientic computing eld in an explicit
manner. Most of the parallel scientic applications
use the Fortran language, in conjunction with
message passing paradigm (Message Passing Interface
to specify the decomposition, map-
ping, communication and synchronization. Reuse has
been explored in its incipient phase as function li-
braries. Even though at limit Fortran libraries account
for some reuse, Fortran applications are hardy
extendible. As a consequence, despite their similar
structure, most of the parallel application are re-designed
from scratch. Given that the process of writing
distributed memory applications is complex and
error-prone, this means low productivity.
Our goal is to abstract away from the user the
distributed computing aspects. That is, to give the
user the illusion of a sequential programming model.
Our thesis is that scalable parallelizing compilers for
real application codes are very far in time.
With our approach, we are taking advantage of the
application specic features to \automate" the parallelization
process. We separate the user data (numer-
ical application specic data) from the parallelization
algorithm. Therefore, we capture the concurrency infrastructure
for the class of application at hand (PDE
solvers) and dynamically use it during the user's solution
process. We use generic programming techniques
in order to couple user data and the workload partitions
for the transparent distributed solution process.
In the reminder of the paper we will refer to the
FEM solution process, since we treat the FD case
as a particularization of the general solution process.
Thus, the key features of FEM solvers are:
The applications are data parallel and loosely
synchronous. Domain decomposition is a technique
used to break down the physical domain
into smaller sub-domains, that can be treated
separately. With this method, data at the border
between sub-domains is logically connected with
data from other sub-domains. In a distributed
memory setting this means that data residing on
remote processors are needed locally. The computation
steps consist of independent local com-
putations, followed by communication.
The physical domain is described by a geomet-
rical, discretized structure. This usually translates
into nodes, elements, or faces (edges, i.e.
the connection between the elements). Any user
application specic abstractions (matrices, vec-
tors, etc.) or attributes (e.g. pressure, temper-
ature) are indexed according to the nodes, ele-
ments, or faces. The numerical computation consists
mainly on iterations over the entities (nodes,
elements, edges).
The applications are inherently dynamic: experimentation
with dierent geometrical data struc-
ture(degree of freedom, element shapes) or different
numerical algorithms (time discretization
schemes, iteration schemes, etc.) is at the core
of the physical phenomena simulation (numerical
applications).
We make use of the application domain features
(PDE solvers) and object-oriented techniques in solving
the problem of transparent concurrency for numerical
applications.
1.1 Contributions
This paper makes the following contributions:
A distributed component model for scien-
tic computation. We present a component
model suitable for concurrent scientic applications
that enables us to achieve the \illusion" of a
sequential programming model. We distinguish
between active and passive components and their
visibility at dierent levels: system and user. We
take advantage of the loosely synchronous feature
of the class of applications we refer. Therefore,
our component model allows for optimal commu-
nication. Moreover, we use generic programming
techniques in order to be able to couple user data
and algorithms with our concurrency infrastruc-
ture. In our distributed component model there
is no notion of global address or name space, or
remote invocations.
Automatic data consistency. We introduce
the notion of dependent and independent data
items. Dependent data needs to be globally con-
sistent, while independent data does not. The
distinction allows us to guide the user in invoking
the global consistency phase during the computa-
tion. The communication is then automatically
taken care of.
An architecture for the scalable solution
of PDEs. Our solution exploits the similarities
of the applications, as well as the genericity of
the parallelization process in order to capture the
concurrency infrastructure that can be reused as-is
by any scientic application (i.e. distributed
PDE solver) programmer. The hardest aspects of
the solution process are dealt with and therefore
isolated from the user. That is, geometrical data
representation, computation of the communication
patterns and communication generation.
The reminder of the paper is organized as follows.
Section 2 overviews the existing approaches to the
problem we are trying to solve (transparent con-
currency) and motivates our approach. Section 3
presents a system for the transparent concurrency of
the parallel PDE solvers. Section 4 describes a prototype
framework implementation of our system. It
also discusses the design rationale that drove our ap-
proach. Section 5 concludes our paper and gives some
directions for future research.
Existing Approaches
Several approaches exist to support the parallelization
of scientic applications. The spectrum of the
approaches lies between manual parallelization, at
one end, to the fully automatic parallelization, at the
other end. Despite its inconveniences, manual parallelization
is still the most popular approach to writing
concurrent scientic applications, due to its e-ciency
and taylorability. On the other hand, writing concurrent
scientic applications is an error-prone, complex
task. Automatic parallelization is one way to
tackle the complexity and the reliability of concurrent
applications. Research into parallelizing compilers
for scientic applications has been successful
for Fortran applications with simple data representations
and regular access patterns (Ujaldon, Sharma,
1993). Compile-time analysis cannot handle
arbitrary data access patterns that depend on
run-time information. Run-time analysis has been
used to address the issue of compiling concurrent loop
nests in the presence of complicated array references
and irregularly distributed arrays (Wu, Das, Saltz,
Berryman & Hiranandani 1995). However, these approaches
are limited to loop level parallelism and simple
data layouts (arrays) in the context of Fortran lan-
guages. The code excerpt in gure 1 exemplies the
applicability of the compiler support for parallelization

The loop level parallelism is ne grain and results
in a lot of communication. Also, the compiler support
can only handle arbitrary access patterns in the
context of simple data layouts, such as arrays.
Object-oriented/based distributed systems that
support data parallel applications have been proposed
(Chang, Sussman & Saltz 1995), (Hassen,
They either do not support
complex data representations, with general distribu-
tion, or many of the concurrency aspects are still visible
to the user. A complete survey of models and
languages for parallel computation can be found in
(Skillicorn & Talia 1998). We will only refer to the
object-oriented models. In (Skillicorn & Talia 1998),
they are classied into external and internal models
according to whether the parallelism is orthogonal to
the object model, or integrated with the object model.
We are interested in the internal object models, because
these are closely related with data-parallelism,
since at the top level the language appears sequen-
tial. Existing approaches require communication to
be explicit, but reduce some of the burden of the synchronization
associated with it. Our model aims to
hide communication and synchronization to the user.
With our prototype implementation, we succeed in
achieving this to the extent that the synchronization
phase has to be triggered by the user. Anyway, we
will extend our system implementation to automatically
detect and trigger the synchronization phase
when necessary.
Chaos++ (Chang et al. 1995) is a library that provides
support for distributed arrays and distributed
pointer-based data structures. The library can be
used directly by the application programmers to parallelize
applications with adaptive and/or irregular
data access patterns. The object model is based on
Compilers - regular case:
Example:
1.Gather data dependence info
(i.e. f[+,3],[0,3]g)
2.Data and computation decomposition
(i.e. the iteration space).
3.Code (communication) generation
(based on data flow information
and owner-compute rule.)
Run-time support - irregular case:
Example:
1.Build the communication schedule
(i.e. a translation table lists the
home processor and the local address
for each array element)
2.Move the data based on the schedule
The transformed code:
Call DataMove(y, DS)

Figure

1: Compile-time and run-time support for parallelization.
mobile and globally addressable objects. A mobile object
is an object that knows how to pack and unpack
itself to and from a message buer. A globally addressable
object is an object that it is assigned to one
processor, but allows copies to reside on other processors
(referred to as ghost objects).
The rst problem we see with this approach is
that the user is expected to provide implementations
of the pack and unpack functions (that support deep
copy) when subclassing the mobile component. On
one hand, the packing and unpacking tasks are low
level operations that expose the user to many of the
concurrency aspects (what to pack, when to pack,
where to unpack, what to unpack). On the other
hand, more mobile objects may contain a pointer to
same-sub-objects. The user has to make sure that
only one copy of a sub-object exists at any point
during the program execution. The use of globally
addressable objects can alleviate some of these prob-
lems. The global object is intended in order to support
the global pointer concept. The main problem that
we see is that the contents of the ghost objects are
updated by explicit calls to data exchange routines.
Our approach is dierent from this one by not mixing
the concurrency aspects at the user level. We do
not let the user see the active components (i.e. associated
with a process) of our model. Also, we do
not need to naively transport the entire content of an
object for every communication. We only update the
parts of the objects that are needed for subsequent
computations and avoid the overhead associated with
communicating entire objects.
In contrast with the generative approach, such as
compiler support for parallelization we use a constructional
approach. Therefore we construct part of
the solution. We want to achieve the \illusion" of
a sequential programming environment, as opposed
to transforming sequential programs to run in par-
allel. Our approach is due to the limited compiler
support for dynamic analysis and e-ciency consider-
ations. Our approach is based on capturing the concurrency
infrastructure of a class of applications and
reusing it for every new application. This idea is similar
with the notion of skeletons, ready made building
blocks (?), or abstractions characteristic to a class of
applications. In our case, we are closer to algorithmic
skeletons, those that encapsulate structure. (Botorog
Kuchen 1995) explore the approach of algorithmic
skeletons, or common parallelization patterns for another
class of applications, i.e. adaptive multigrid
methods. Moreover, (Botorog & Kuchen 1995) list
a series of requirements for a language supporting algorithmic
skeletons, among which data access control
and polymorphism. The authors introduce the notion
of parallel abstract data type (PADT) in order to account
for the required features. Furthermore, the host
language used for experimentation is an imperative
language, i.e. the C programming language. We argue
that object-oriented models and generic programming
naturally fulll the requirements for implementing
algorithmic skeletons. Therefore we concentrate
on an e-cient data parallel object model suitable for
high performance, parallel applications.
In contrast with object-oriented distributed
middle-ware, we do not support any notion of global
name or address space. That is because e-ciency is
important for the applications we address and such
methods wouldn't fulll this requirement. Moreover,
we are not giving the user access to the distributed
computing aspects such as communication, synchro-
nization, etc. As a consequence, we achieve both,
e-ciency and transparency of the concurrency.
3 A View of a System for Transparent Con-
currency
In this section we describe a system for transparent
concurrency of the distributed solution of the PDEs
from the user's perspective. In this perspective, we
emphasize the following requirements for our system:
Applicability - the class of the applications we
address is the parallelization for the general solution
of PDEs (FEM, FD etc. PDEs are at
the core of most of engineering and the natural
sciences. With such a large class of applications,
our system is most likely to be highly relevant
for a large applied research community.
Usability - we expose the user to a small, well
tested, well documented component set, together
with the control thread (the way the components
play together), that can be easily learned and
used.
Extendibility - the user should be able to use
our system in conjunction with his/her own data
and algorithms in order to complete the solution
process.
E-ciency - last, but not least, e-ciency is an
important requirement of the scientic applica-
tions. Our architectural approach is driven by
e-ciency as well.
In succeeding to meeting our requirements, we
have accounted for the following:
To achieve large applicability, we only focus on
capturing the concurrency infrastructure, that
is the load balanced data distribution, nding
the data that needs to be communicated (i.e.
computing the communication patterns), knowing
when the communication takes place. We
only employ a high-level mathematical interface
in the form of the geometrical data description
for the discretized physical domain. Our solution
is general and it does not involve any algorithmic
or discrete mathematics interface. With
our solution, any existing computational kernels
(e.g. BLAS (Dongarra, Croz, Hammarling
Hanson 1988), Lapack (Angerson, Bai, Don-
garra, Greenbaum, McKenney, Du Croz, Ham-
marling, Demmel & Bischof 1990)) or numerical
library (e.g. Dipack (Langtangen 1999)) can be
employed by the user for the solution process.
Most of the existing numerical libraries for the
distributed solution of PDEs are still low-level
and complex. Here, by low-level, we mean that
the user is still aware of the data distribution and
communication aspects, as well as of many other
low-level aspects (renumbering, data access o-
set, etc. (Smith 1998)). By complex, we mean
that the libraries provide a rich, mixed function-
ality. Part of the functionality accounts for numerical
abstractions (linear algebra solvers, time
discretization schemes, etc. More general functionality
(sometimes duplicated by each library,
in its own \philosophy"), such as geometrical
data representation, local element to global mesh
renumbering, etc., it is mixed in also. Therefore
the libraries become large, hard to use and
learn. We separate the parallel solution process
and the application specic data and algorithm.
We achieve high usability by designing a small
set of well tested, well documented components,
with a narrowly focused functionality.
Our solution is high level and reusable through
the use of encapsulation. We hide the details of
concurrency from the user, while achieving reuse
as-is of the entire concurrency infrastructure. We
also hide the tedious details of the geometrical
data representation from the user. At the same
time, the component-based design accounts for
the reuse of any existing (numerical) software artifacts

Our solution is e-cient because we employ a
truly distributed object model. That is, in our
model, there is no notion of a global name or
address space, or remote invocations. The active
objects are loosely synchronous. Communication
only takes place at particular times during
the application life time. We optimize communication
by knowing exactly when communication
takes place and aggregating data into large messages

The main concepts/steps involved in the distributed
solution of the PDEs are:
1. Geometric data representation.
2. Data structure creation.
3. O-processor data updates.
4. Local numerical computation.
Our system accounts for the rst three phases, while
the user is responsible for providing the numerical
algorithm for a particular PDE.
3.1 Geometrical Data Representation
Geometrical data representation is one of the hard
aspects of scientic applications that use general
meshes. Even though the applications use similar
geometries, many dierent representations coexist in
practice, making existing scientic codes hard to un-
derstand, modify, maintain and extend. We isolate
the geometrical data representation in a component
with a well dened interface for accessing all the
needed geometrical attributes. At the system level,
the geometrical data representation can be easily re-
placed, without aecting the system functionality, or
requiring any modications in any other system mod-
ules. Therefore, our system takes over the task of
providing the user with a geometrical data representation
to be used for the implementation of user's
application.
The user species the structure of the input domain
as an input le, called mesh or grid, for the sys-
tem. The user also species the number of the processors
available on his/her system. The le describing
the domain has a prescribed, well documented for-
mat. Such les are usually obtained with the help of
tools called mesh generators 2 .
The system reads the data from the le into the internal
components. Dierent element shapes used for
the discretization of the input domain can be specied
in the mesh le.
The system uses a load-balanced partitioning algorithm
(as provided by METIS 3 ) for breaking down
the input mesh structure data into smaller regions.
All the details related with the geometrical data representation
are encapsulated by our system compo-
nent. The user gets access to all geometrical data aspects
through our component interface, which is the
Subdomain. At the system level, the geometrical data
representation can be easily replaced, without aect-
ing the system functionality, or requiring any modi-
cations in any other system modules.
3.2 Data Structure Creation
The system creates a number of regions from the input
data structure equal with the number of the processors
available. Then it associates each data region
with a process 4 , transparently for the user. The internal
boundary geometrical mesh data is duplicated
on each process, such as the user has access, locally,
to all the o-processor data needed during one computation

The user has access to the geometrical data local
to one processor through the Subdomain compo-
nent. The component presents the user with a uniform
view of the geometrical data structure that can
be employed in a sequential programming model for
implementing a numerical algorithm (solver). All the
distributed computing aspects that the component incorporates
are invisible to the user.
The user has to subclass the a system provided
component UserData for dening any attribute (e.g.
pressure, temperature) or data abstraction dened on
the mesh structure that it is involved in the computation
of the nal result. The user provides the concrete
An example of such tool can be found at
http://www.sfb013.uni-linz.ac.at/ joachim/netgen/
4 In our model a single process runs on a single processor.
interface for storing and retrieving a user dened data
item to/from a specic mesh location (element, node,
etc.
3.3 O-Processor Data Updates
The concurrency structure of the applications we
address (the solution of PDEs) consists of independent
local computation, followed by communication
phases. Therefore, they are loosely synchronous
(Chang et al. 1995). In order to automatically
provide for the o-processor data updates, we
need to know when and what to communicate. That
is, we need to know the communication patterns and
when to generate communication. Our system computes
the communication patterns, but it is the user
that explicitly invokes the update phase. Then, the
system performs the updates transparently.
Each region associated with a process is stored in
the Subdomain component. The \invisible" part of
this component makes use of the local data in order to
account for the distributed computing part. The component
computes the communication patterns, creates
the communication data container objects and
maintains the global data consistency transparent for
the user. The user has to call a system provided
generic function, Update, which makes sure that the
user dened data is globally consistent.
3.4 Local numerical Computation
Our system treats the user data dierently, depending
on the dependency attribute:
1. We call dependent data any dened user property
that is the nal result (i.e. the unknown
for which the equations are solved, e.g. pres-
sure, temperature, etc.), or updated by another
dependent data item (e.g. some of the coe-cient
matrices computed based on the unknown).
2. We call independent data any user data that it is
not assigned the result of an expression computed
based on dependent data.

Figure

shows how the system supports the above
tasks transparently to the user, and what is the user
contribution in completing the solution process. As
shown in gure 2, we employ a Master/Worker concurrency
model. A master process is responsible for
reading in the domain data from an input le and
distributing the subdomains to the workers. We actually
use a hybrid master/worker and SPMD (Single
Process Multiple Data) concurrency model. We use
the SPMD model due to the key observation for our
parallelization system: all the worker processes execute
a similar task on the local domain (data). That
is because the class of application we address share
the same feature, namely, they are data parallel.
In gure 2 we associate each subdomain with a
unique process. We say that a subdomain has communication
capabilities, that is, it \knows" how to
\pack/unpack itself" and send/receive. Inside the
system there are other components with similar fea-
tures. We call these components active components.
In contrast, some other components, passive, capture
only the structure data, and/or the algorithm associated
with it, without having any communication ca-
pabilities. At the user level, only the \passive" components
are employed, or visible.
The user sees only the \passive interface" of the
Subdomain. This will allow the user to manipulate the
appropriate geometrical data. We hide the geometrical
data representation from the user. The system
instantiates and \computes" the \right" subdomain
for each worker. The user acts only at a subdomain
level, in a sequential fashion. The system replicates
the user algorithm and data over all the workers. The
workers communicate transparently for the user, by
using messages.
In gure 3 we show the dierence between a code
excerpt written in a \classical sequential manner",
and the same code excerpt enriched with our system
functionality to look sequential, but execute dis-
tributed. We do not show the \classical concurrent
model" for such an application (i.e. the MPI) ver-
sion, since we assume that its complexity is evident
to the reader and the space would not allow for such
an illustration. In gure 3, we emphasize the dier-
ence between the two models by using grey for our
model required modications. We use black for the
similar parts. It is easy to see the data items B and
are candidates for the dependent data. Therefore,
in the code excerpt above, these data specialize our
component UserData. Also, the loc data variable reects
the data the user sees, i.e. a Subdomain. On
the other hand, the data items A and C are independent
and they do not require any modication to the
sequential algorithm.
An important observation here it is that the user is
the one who has to \observe" the dierence between
the dependent and independent data. With proper
guidance provided by the system documentation and
the user experience, this \task" is straight forward.
We have implemented a prototype frame-
work( (Johnson 1997), (Bassetti, Davis &
Quinlan 1998)) to demonstrate our approach, using
the C++ programming language (Stroustrup 2000).
We have used the METIS library for the load
balanced (graph) partitioning of the irregular mesh.
We use the Object Oriented Message Passing Interface
library for communication.

Figure

4 depicts a view of the prototype, using
the UML (Fowler, Scott & Booch 1999) (Unied
Modeling Language) formalism. For brevity we only
show the key components and interfaces in the UML
diagram.
Our design is based on a careful study of the application
domain (PDE solvers) and their behavior.
Our key observation is that the applications share a
similar structure, with dierences residing in the \nu-
merical problem parameters", rather than the concurrency
infrastructure. Therefore, by separating the
parallel algorithm and geometrical data from the application
specic parameters, data and algorithms, we
were able to come up with a solution to automate the
parallelization process.
In gure 4, the UserAlg component is \the hook"
for the user to \anchor" his/her computation into the
framework. The user subclasses the abstract component
UserAlg 6 and provides his/her main control
ow which complements the framework's control
ow
in completing a particular application. The user also
hooks the data representation for his/her application
here. As we have mentioned earlier( 3.4), the data
can be independent or dependent. Further more, it
can be user dened, or components imported by the
user from other libraries. In this case, we say that
the user extends the framework through composition.
Composition is the rst mechanism used to extend
6 In this particular implementation, components and classes are
the same
User supplied
application
specific part
The input file
Communication
Worker 1 Worker n
Mesh data
Master
Data
decomposition
Communication
(Metis library)
library (MPI)

Figure

2: The main building blocks of the system.
the framework. The user dependent data has to be
dened by subclassing the framework container component
UserData. Subclassing is the second mechanisms
used to extend the framework.
The user has access to the geometrical data for
a subdomain through the interface of the Subdomain
component. The user algorithm component is parameterized
with the Subdomain component. The Subdomain
is created by the framework, such as the user
has a contiguous, consistent view of his/her data. The
user writes his/her application as for a single Subdo-
main. In a SPMD fashion, the framework instantiates
a number of workers which replicate the user provided
computation for all the subdomains. A worker
process is modeled by a Worker component, which
is not visible for the user. The component receives
its work from a master process, in a Master/workers
fashion. The Master component reads in the input
data as provided by the user (the discretized physical
domain) and breaks it down into smaller partitions
that it sends to the workers. Based on their local
data provided by the Subdomain component, a worker
sets up the communication patterns in cooperations
with other workers. The generic function Update uses
the communications patterns for a Subdomain and
the container component UserData to automatically
generate communication every time the user calls the
function, after updating a user dependent data item.
The component-based design we have chosen
for our framework is due to our constructional ap-
proach, i.e. we construct part of the solution process.
The generative approach would be to analyze an existing
solution and generate a new one (e.g. compiler
driven parallelization). The compiler techniques
(data
ow analysis, dynamic analysis) are limited to
the regular applications, that use simple data layout.
Frameworks fulll best our need for the user to be
able to plug in his own data representations and al-
gorithms, i.e. to provide the remaining part of the
solution process.
The benets of our architecture choice range from
the ability to \automate" the parallelization process
for a general class of applications which has never
been achieved before (general, irregular applications),
to the data locality and communication optimization,
the data encapsulation concept naturally provides.
Intensively researched e-ciency issues such as data locality
and communication optimization come for free.
Or almost, at least. They come at the cost of whatever
makes object-oriented languages slow: abstraction
penalty, dynamic binding penalty and inheritance
penalty. At last, but not least, the (object-
oriented) framework has been the most eective route
to reuse (Parsons, Rashid, Speck & Telea 1999).
Genericity is an important aspect of our design.
Because the parallel structure of the numerical applications
we refer to can be expressed independently of
the representation, the concurrency infrastructure is
based on the concept of generic programming. We use
generic programming to be able automatically generate
communication for user data. We use containers
as place holders for later dened user data to be able
to pack/unpack and send/receive the data. This solution
enables us to free the user from any distributing
computing aspects, from data distribution, and data
coherence and consistency, to data communication.
The concurrency model we use is the hybrid
master/workers and SPMD. SPMD is the widely used
concurrency model for numerical applications, since
they are data parallel. We use a special process, a
master, to evenly divide the data and send the work
to the worker processes. This way, the workers will
have approximately the same amount of work load,
and the computation will be balanced.
The validation of the framework has two aspects.
At the usability level, our system is open source and
it will be made available to the researchers in the
application domain area to experiment with it. We
Classical sequential model:
A FEM Poisson Solver using a tetrahedral mesh
void main() f
ComputeB();
ComputePressure();
void ComputePressure() f
for
void ComputeA() f
void ComputeC() f
void ComputeB() f
for
for
Our non-classical sequential model:
A FEM Poisson Solver using a tetrahedral mesh
void main() f
Update(P, loc data);
ComputeB();
ComputePressure();
void ComputePressure() f
for
Update(P, loc data);
void ComputeA() f
void ComputeC() f
void ComputeB() f
B.Init;
Update(B, loc data);
for
(j+k+1)%loc data.GetNve(),
local data.GetNve());
B.SetAt(knode, temp);
Update(B, loc data);

Figure

3: A comparison of two sequential programming models.
UserMain:
public UserAlg
UserDefined:
public UserData User supplied
components
Subdomain
Worker CommPattern
Master
Metis
MeshStruct
UserAlg
virtual void Main()
OOMPI
IntBoundary
UserData
User defined
External packages
internally used by
the framework
Framework internals
User visible
template void Update(UserData &, Subdomain&)

-myDomain

Figure

4: The infrastructure of the framework.
are implementing a test suite to test the functionality
of our framework and for the documentation purposes
as well.
From the e-ciency point of view, we are interested
in the application speed-up. We intend to run
the framework on clusters of PCs or NOWs (Network
of Workstations) running Linux operating system.
Therefore we will measure the application speed-up
by running x sized problems on an increasing number
of processors. Then we will measure for dierent
size of problems as well.
5 Conclusion and Future Work
In this paper we have presented a new approach towards
the parallelization of scientic codes, i.e. a
constructional approach. In contrast to the generative
approach (e.g.compiler driven parallelization), we
construct part of the solution, instead of generating a
new solution based on a existing one. We use a component
based architecture in order to be able to allow
the user to build on our concurrency infrastructure.
With our approach, we get closer to the ideal goal
of not having the user to deal with the concurrency
at all, without restricting the generality of the application
class. Therefore, we are able to handle the
distributed solution of PDEs for general geometries
(meshes).
Given that e-ciency is an important constraint
of the class of the applications we address, we
show how a \truly distributed" component model
can alleviate the e-ciency problems of the object-oriented
middle-ware ( (Java RMI n.d.), (CORBA
Success Stories n.d.), (Distributed Component Object
Model (DCOM) n.d. The paper attempts
to explore the appropriateness of objects in conjunction
with concurrency (a much desired association
(Meyer 1993)) in the context of high performance
computing. High performance scientic computing
is known as a community traditionally reluctant to
object-oriented techniques because of the poor performance
implementations of object-oriented languages
and systems.
In the future, we will work more towards bringing
evidence that our approach is scalable. We intended
our system architecture for a cheap,
exible
distributed computing platform, consisting of clusters
of Linux PCs, or NOWs. With a scalable approach, a
potentially \unlimited" number of computers can be
used for gaining computational power.



--R

An overview of a compiler for scalable parallel machines
ans Sorensen
LAPACK: A portable linear algebra library for high-performance computers
Optimizations for parallel object-oriented frame- works
Algorithmic skeletons for adaptive multigrid methods


Distributed Component Object Model (DCOM) (n.

UML Distiled: A Brief Guide to the Standard Object Modeling Language

How frameworks compare to other object-oriented reuse techniques
Computational Partial Di

Systematic concurrent object-oriented programming
"framewok"

The transition of numerical soft- ware: From nuts-and-bolts to abstractions



--TR
An extended set of FORTRAN basic linear algebra subprograms
Systematic concurrent object-oriented programming
Object-oriented runtime support for complex distributed data structures
A flexible operation execution model for shared distributed objects
Models and languages for parallel computation
The C++ Programming Language
Computational Partial Differential Equations
Distributed Memory Compiler Design For Sparse Problems
An Overview of a Compiler for Scalable Parallel Machines
Algorithmic Skeletons for Adaptive Multigrid Methods
Run-Time Techniques for Parallelizing Sparse Matrix Problems
A "Framework" for Object Oriented Frameworks Design
generic programming;concurrency;scientific applications;distributed components