Asynchronous Transfer Mode and other Network Technologies for Wide-Area and High-Performance Cluster Computing.
We review fast networking technologies for both wide-area and high performance cluster computer systems. We describe our experiences in constructing asynchronous transfer mode (ATM)-based local- and wide-area clusters and the tools and technologies this experience led us to develop. We discuss our experiences using Internet Protocol on such systems as well as native ATM protocols and the problems facing wide-area integration of cluster systems. We are presently constructing Beowulf-class computer clusters using a mix of Fast Ethernet and Gigabit Ethernet technology and we anticipate how such systems will integrate into a new local-area Gigabit Ethernet network and what technologies will be used for connecting shared HPC resources across wide-areas. High latencies on wide-area cluster systems led us to develop a metacomputing problem-solving environment known as distributed information systems control world (DISCWorld). We summarize our main developments in this project as well as the key features and research directions for software to exploit computational services running on fast networked cluster systems.
Introduction
Cluster computing has become an important part of the high-end computing world.
Many of the applications traditionally run on high-end supercomputers are now successfully
run on computer clusters. In this paper we describe our experiences in building
high performance computer clusters and in linking them together as clusters of
clusters over wide areas. This approach towards an Australian "National Computer
Room" was started in 1996 and led us to two important conclusions about wide-area
computing. Firstly from a technical standpoint, the limitations of latency and of net-work
reliability are likely to be more important than bandwidth limitations in the future
and therefore successful wide-area systems need to be constructed accordingly.
ondly, the political and administrative issues behind supercomputer resource location
and ownership mean that in the future it will be attractive to loosely cluster compute
resources nationally and even internationally. Institutions have strong reasons to wish
to retain ownership and control of their own computer resources. Clusters of local re-
sources, each of which itself may be a cluster of some sort, are therefore an attractive
architecture to target for software development.
In this article we try to divide cluster computing systems issues into those relevant
to achieving high performance (typically on local area compute clusters) and those
issues pertaining to the successful sharing of resources between institutions (typically
wide-area issues).
The size and utilisation of computer clusters varies enormously. The recent IEEE
Workshop on Cluster Computing [3] demonstrated this as well as the different uses
of the term cluster computing. We will use cluster in its broader sense to describe a
collection of networked compute resources that may be heterogeneous in hardware and
in operating systems software, but which are capable of cooperating in some way on a
distributed application. This may be using any of the parallel computing software models
such as PVM[8], MPI[25] or HPF[17] or through some higher granularity object
or service based system like Globus[7], Legion[10], CORBA[27] or DISCWorld[16].
We use the term Beowulf-class cluster in the sense it now widely adopted to describe
a cluster that is dedicated in some sense, and is not just a random collection of networked
computers. The term "constellation" is now being adopted to describe a compute
cluster that uses shared memory or bus based technology. We focus on clusters
and Beowulf-class systems in describing our work in this paper.
At the time of writing the watershed sizes for clusters are limited by the number of
ports on commercially available network switches. For example, many groups report
work on modest clusters of 8 or 16 processors. We continue to operate our prototype 8
node Beowulf system for student experiments. Modern switches are typically capable
of interconnecting 16, 24 or even 48 compute nodes using Fast Ethernet at 100MBit/s
bandwidth between all nodes. The limits in size from a single switch therefore might be
used as the delimiting size for distinguishing large and small computer clusters. This
number is at present larger than the number of processors that can be economically
linked using shared memory technology. High end systems such as the ASCI systems
and Tera computer architecture do achieve considerably higher number of nodes interlinked
with shared memory, but can hardly be described as commodity economical
architectures.
Switches such as the 510 series from Intel allow switches to be stacked using high
bandwidth links between individual switches to preserve the full point to point band-width
between compute nodes. We have constructed a 120 node Beowulf class system
using this flat switch architecture and dual processor nodes. The Intel 510 series is
limited to linking 7 of these 24 port switches in this manner and this limit of 168
networked nodes represents well the limitations of current network technology. Fast
Ethernet technology is used to link individual nodes, with Gigabit Ethernet uplinking
used between the switch backplane and the file server node. Beowulf systems larger
than this are being built, but must use some sort of hierarchical network architecture
and are no longer "as commodity" in nature.
It is likely that network technology will move forward with economics and technical
improvements and that commodity switchgear will soon allow bigger systems.
systems will therefore continue to encroach on the traditional supercomputing
market area.
Scalable software to run on such systems is available for some applications prob-
lems, but our experience is that some effort is still needed to ensure the major contribution
towards latency in parallel applications running on clusters does not remain
the operating system kernel or communications software. Achieving low latency in
sending messages between nodes in a Beowulf system appears to be a more important
limitation at present than achieving high bandwidth.
We describe our experiences in constructing ATM-based clusters in section 2. We
attempt to summarise our metacomputing approach to cluster management in section 3
and describe how some of the latency and distributed computing problems, that are so
important to wide-area clusters, can be tackled. We review some of the networking
technologies actively used for cluster computing systems in section 4 and discuss their
relative technical and economic advantages.
In this section we describe some experiences from an experimental wide area system
we ran from 1996 to 1998, employing local and wide area ATM technology to connect
high performance computing clusters around Australia. The Research Data networks
(RDN) Cooperative Research Centre (CRC) was set up as a joint venture involving our
own University of Adelaide, the Australian National University in Canberra, Monash
University in Melbourne, the University of Queensland and Telecom Australia (Tel-
stra). The map shown in figure 1 shows the initial broadband network we were able
to set up and use. It connected Adelaide, Melbourne, Sydney, Canberra and Brisbane;
some of its latency and performance aspects are discussed below. We trialed links of
34Mbit/s and 155Mbit/s over various branches of the network and experimented with
various combinations of: cross mounted long distance filesystems; video conferencing;
simple shared whiteboard style collaborative software; and cross connected computer
clusters. We describe the Asynchronous Transfer Mode (ATM) technology used for
these experiments in section 4.
The main features of our experiments were to analyse the effects of having relatively
high wide area bandwidths available for cluster computing yet having perceptibly
high latencies arising from the long distances. We were able to construct well-connected
compute clusters having the peculiar bisection bandwidth and latency properties
that arise from having part of the cluster at each geographically separate site.
Although a number of wide area broadband networks have been built in the USA
[4, 26], it is unusual to have a network that is fully integrated over very long distances
rather than local area use of ATM technology. Telstra built the Experimental Broad-band
Network (EBN) [21] to provide the foundation for Australian broadband application
development. Major objectives of this were to provide a core network for service
providers and customers to collaborate in the development and trial of new broadband
applications, and to allow Telstra and developers to gain operational experience with
public ATM-based broadband services.
OCEAN
Kilometres
Network (EBN)
Broadband
Telstra's
Experimental
@KAH,Great Australian Bight
Bass Strait

Figure

1: Telstra's Experimental Broadband Network (EBN). The EBN is a 34Mbit/s
ATM network connecting research and commercial partner sites.

Figure

shows how our prototype storage and processing hardware were arranged
at both Adelaide and Canberra (separated by some 1200km) and connected by the
EBN. The Adelaide and Canberra cells consisted of a number of DEC AlphaStations,
interconnected locally by 155MBit/s ATM, and with the cells at the two sites connected
across the EBN at 34MBit/s.
We experimented extensively with cluster computing resources at Adelaide and
Canberra in assessing the capabilities of a distributed high performance computing
system operating across long distances. It is worth considering the fundamental limitations
involved in these very long distance networks.
Although we employ OC-3c (155MBit/s) multi-mode fibre for local area networking
we were restricted to an E-3 (34MBit/s) interface card to connect Adelaide to Melbourne
and hence to Canberra. In practice this was not a severe limitation as for realistic
applications were were unable to fully utilise a 155MBit/s link over wide areas
except in the most contrived circumstances. This would not be the case for a shared
link however.
The line-of-sight distances involved in parts of the network were: Adelaide/Melbourne
ASX-1000
Switch
Compute
File Servers
X-Terminals
DEC Alpha
SGI
Power
Challenge
Workstation
Multi-Mode Fibre, OC-3c (155Mbps)
Farm
Storage Works
RAID
Tape Library
DEC Alpha
Workstation
Farm
Storage Works
RAID
Canberra
Network (EBN)
Broadband
Experimental
Telstra's
Adelaide
Brisbane
Sydney
Melbourne
at Adelaide and Canberra
on the EBN
Project Resources
Gigaswitch
Fileserver

Figure

2: DHPC Project Hardware Resources at Adelaide and Canberra connected
via Telstra's EBN. While traffic can be sent between sites via the 'ordinary' Internet,
bandwidth-intensive experiments were carried out by specifically routing traffic over
the EBN.
732 km. Consequently the effective network distances between Adelaide and the other
cities are shown in table 1. The light-speed-limited latencies shown in table 1 are calculated
on the basis of the vacuo light-speed (2.997810 5 kms -1 ). It should therefore be
noted that this is a fundamental physics limitation and does not take into consideration
implementation details.
EBN City Network Distance from Adelaide (km) Light-speed-Limited Latency (ms)
Melbourne 660 2.2

Table

1: Inter-city distances (from Adelaide) and Light-speed-limited latencies for the
Experimental Broadband Network.
We made some simple network-performance measurements using the Unix ping
utility, which uses the Internet Control Message Protocol (ICMP), for various packet
sizes which are initiated at Adelaide, and bounced back from a process running at
other networked sites. These are shown in table 2. By varying the packet sizes sent it
is possible to derive crude latency and bandwidth measurements. It should be emphasized
these measurements are approximations of what is achievable and are only for
comparison with the latency limits in table 1.
The times in table 2 were averaged over 30 pings and represent a round-trip time.
Measurements are all to a precision of 1 ms except for those for Syracuse which had a
significant packet loss and variations that suggest an accuracy of 20 ms is appropriate.
The ping measured latency between Adelaide and Canberra appears to be approx-
Ping Mean Time Mean Time Mean Time Mean Time
Packet Canberra Syracuse USA local machine local machine
Size via EBN via Internet via ethernet via ATM switch
(Bytes) (ms) (ms) (ms) (ms)
1008
2008
8008 22 441

Table

2: Approximate Performance Measurements using Ping.
imately 15 ms. This is to be compared with the theoretical limit for a round-trip of
7.6 ms. The switch technology has transit delays of approximately 10 s per switch.
Depending upon the exact number of switches in the whole system this could approach
a measurable effect but is beyond the precision of ping to resolve. We believe that allowing
for non-vacuo light-speeds in the actual limitation our measured latency is close
(within better than a factor of two) to the best achievable. We believe that variations
caused by factors such as the exact route the EBN takes, the slower signal propagation
speed over terrestrial copper cables, the routers and switch overheads and small
overheads in initiating the ping, all combined, satisfactorily explain the discrepancy in
latencies. The EBN appears to provide close to the best reasonably achievable latency.
Also of interest is the bandwidth that can be achieved. The actual bandwidth
achieved by a given application will vary depending upon the protocols and buffering
layers and other traffic on the network but these ping measurements suggest an
approximate value of 2  8kB/(22 - 22.7MBit/s. This
represents approximately 84% of the 27MBit/s of bandwidth available to us, on what
was an operational network.
The Unix utility ttcp was useful for determining bandwidth performance measurements
which are outwith the resolution possible with ping. A typical achievable band-width
between local machines on the operational 155MBit/s fibre network is 110.3
MBit/s compared with a typical figure on local 10MBit/s ethernet of 6.586MBit/s Both
these figures are representative of what was a busy network with other user traffic on
them.
In 1998 and 1999 an additional link connecting Australia and Japan was made
available to us with a dedicated 1MBit/s of bandwidth available for experimentation.
We carried out some work in collaboration with the Real World Computing Partner-
ship(RWCP) [28] in Japan and found that there was an effective latency of around
200ms single trip between Adelaide and Tsukuba City in Japan over this ATM net-work

All these experiences suggest that the major limitations for wide area cluster computing
in the future are more likely to be from latency limitations rather than from
bandwidth limitations.
The Telstra Experimental Broadband network was closed at the end of 1998 and
we have had an alternative ATM network supplied by Optus Pty Ltd made available to
us in 1999. This network has similar latency characteristics but has the advantage that
we can apply to burst up to a full 155MBit/s between Adelaide and Canberra at will. In
practice, we have found that 8MBit/s is entirely adequate for our day to day wide-area
cluster computing operations.
It is interesting to reflect on our experiences in setting up and using these net-
works. Staff at Telstra's research laboratories were instrumental in setting up the initial
circuits and allocating bandwidth to our experiments. As the trials proceeded this became
a well automated process with little human intervention required except when we
needed to split bandwidth allocations between video, cluster and file system circuits
for example. One of the problems with the experimental set up was that only permanent
virtual circuits (PVCs) were available to us and that the network configuration
(on the routing hosts) needed to be changed (manually) each time a reconfiguration
was required. One of the promises of the ATM standard is that of Switched Virtual
Circuits (SVCs) which could be manipulated in software to reallocate bandwidth to
different applications. Although we were able to experiment with pseudo SVCs using
the proprietary facilities of our ATM switch gear, it was not possible to enable proper
switch virtual circuits across different vendors' switch gear at the different sites in our
collaboration. To our knowledge this is still not available on wide area ATM networks
and is, we believe a disappointing limitation of this technology in practice.
We expended considerable time and effort in carrying out these experiments, which
was particularly difficult when we had to reconfigure routers and operating systems
and driver software every time we needed to carry out a different experiment. ATM
technology has matured since the start of our experiments but we believe it will not
be the technology of choice at the cluster computing level. ATM is likely to still find
a place as a network backbone technology, certainly for long distance networks and
perhaps even still in local area networks. Driver support and administration tools for
ATM based cluster computing do not appear to be forthcoming and consequently we
believe it much more likely that Internet Protocols implemented on top of ATM or other
technologies perhaps will be much more useful for cluster computing.
3 Metacomputing Cluster Management
Our vision at the start of our wide area network trials was of a "National Computer
Room". We imagined this as a network and software management infrastructure that
would enable institutions to share access to scarce high-performance computing re-
sources. This seemed important given the scarcity of supercomputing systems in Australia
in particular. We have now revised this vision however. While there are still some
areas in which a computer sharing mechanism is important, the whole supercomputing
industry has continued to be shaken up. Global reduction in the number of supercomputer
vendors still in business can be attributed to a number of factors, but not least of
these is the greater availability of cluster computing resources. The need for institutions
to own and control their own resources appears to be a strong one, and relatively
cheap cluster computing systems enables this phenomena. We believe therefore that
although there is still a need for software to enable compute resource sharing, the resources
are more likely to be clusters themselves and this affects the characteristics of
software and applications that can integrate them.
Software such as Netsolve, Globus, Legion, Ninf and our own DISCWorld system
are all aimed at enabling the use of distributed computing resources for applications.
Our favoured approach is to encapsulate applications (including parallel ones) as services
that can run in a metacomputing environment and we have focussed our efforts
in to building Java middleware to allow this. Other approaches have taken experiences
from the parallel computing era to allow wide area systems to behave as a more tightly
coupled parallel system running multi processor applications across the wide area net-
works. The choice is a matter of granularity and of how to group parts of the application
together. The wide area parallel approach is attractive for its simplicity in that the
tools and technologies from parallel computing can be redeployed with only software
re-engineering efforts. The metacomputing approach recognizes that networks (espe-
cially wide-area ones will remain unreliable, and that long distance latencies will not
improve (short of some new physics breakthrough) and that the fundamental distributed
computing problems need to be considered when building wide-area cluster systems.
Distributed Information Systems Control World (DISCWorld) [16] is a metacomputing
model or framework with a series of prototype systems developed to date. The
basic unit of execution in the DISCWorld is that of a service. Services are pre-written
software components or applications. These are either written in Java or are legacy
codes that have been provided with a Java wrapper. Users can compose a number of
services together to form a complex processing request. Jobs can be scheduled across
the participating nodes [19]. An example DISCWorld application is a land planning
system [5], where a client application requires access to land titles information at one
site, and digital terrain map data at another, and aerial photography or satellite imagery
stored at another site. DISCWorld itself does not build on parallel computing
technology, but can embed parallel programs as services. A support module known as
JUMP provides integration with message-passing parallel programs which might run
on a conventional supercomputer as an or on a cluster[13].
An environment such as DISCWorld is ideal for use across a cluster or a distributed
network of clusters. The nodes within the cluster can be used as federated hosts connected
via a high-speed, dedicated network. Some or all of the nodes can be used as
a parallel processor farm, running those services which are implemented as parallel
programs. We employed both these approaches in our multi cluster experiments across
our ATM network. The ability to make intelligent decisions on where to schedule services
in the DISCWorld environment, in order to minimize either execution time or
total resource cost relies on the ability to characterise both the services and the nodes
in the environment.
Fast Networking Technologies
In this section we review the major networking technologies and describe their roles
for wide area and high-performance cluster computing. We discuss Asynchronous
Transfer Mode (ATM); Fast Ethernet; Gigabit Ethernet; Scalable Coherent Interconnect
(SCI); and Myrinet technologies. Each of these technologies may have differing
roles to play in wide-area and cluster computing over the next few years.

Table

3 summarises the networking technologies we consider here. The data in the
table has been combined from our own benchmarking experiments as well as reports
in the literature [6, 22, 23, 29]. It can be seen that for problems which do not require
gigabit-order bandwidth Fast Ethernet seems a logical choice. If gigabit-order band-width
is required, the decision is not so clear-cut: there is a complex trade-off between
the bandwidth and latency attainable, the approximate cost per node, and whether the
technology is considered commodity. When we use the term commodity, we refer to
the fact that the technology is available to the mass market, and is available from more
than a few specialist vendors.
Technology Theoretical Measured TCP/IP Latency Approx Cost
Bandwidth Bandwidth Per Node ($US)
Ethernet 10MBit/s 6.58MBit/s 1.2ms 80
Fast Ethernet 100MBit/s 68MBit/s 1ms 150
ATM 155MBit/s 110.3MBit/s 1ms 2500
Gigabit Ethernet 1000MBit/s 950MBit/s 12ms 1200
SCI 1600MBit/s 106MBit/s 4s 1400
Myrinet 1200MBit/s 1147MBit/s 117s 1700

Table

3: Interconnection network technologies characteristics as measured and also
reported in the open literature.
We were able to trial our ATM network of clusters using sponsorship from the
Australian Commonwealth government. Table 3 shows that ATM is still not an especially
economic choice. It is our belief that Gigabit Ethernet technology will be widely
adopted and that it will be correspondingly driven down in price.
4.1 ATM
Asynchronous Transfer Mode (ATM) [1] is a collection of communications protocols
for supporting integrated data and voice networks. ATM was developed as a standard
for wide-area broadband networking but also finds use as a scalable local area networking
technology. ATM is a best effort delivery system - sometimes known as bandwidth-
on-demand, whereby users can request and receive bandwidth dynamically rather than
at a fixed predetermined (and paid for) rate. ATM guarantees the cells transmitted in a
sequence will be received in the same order. ATM technology provides cell-switching
and multiplexing and combines the advantages of packet switching, such as flexibility
and efficiency of intermittent traffic, with those of circuit switching, such as constant
transmission delay and guaranteed capacity. ATM uses a point-to-point, full-duplex
transmission medium, and provides connection-oriented protocols.
In addition to supporting native ATM protocols such as ATM Adaption Layer
(AAL) 3/4 and 5, the use of LANE allows the ATM network to be viewed as part
of a TCP/IP network (to allow multicast and broadcast). There have been a number of
studies that consider the use of the AALs for high-speed interconnections in parallel
computing [20]. We employ a 155MBit/s local-area ATM network [14, 15] as well as
a 34MBit/s broadband network (EBN).
ATM allows guaranteed bandwidth reservation across a circuit (a link or a number
of links with defined end-points. There are three different types of bandwidth reser-
vation: constant bit rate (CBR); variable bit rate (VBR); and available bit rate (ABR).
CBR is used when the traffic between sites is constant and will rarely, if ever change in
bandwidth requirements. VBR is used to characterise traffic that has a mean bandwidth
requirement that can change slightly. Traffic along a VBR circuit can operate at the cir-
cuit's maximum bandwidth only for short amounts of time; intermediate switches may
drop cells that exceed the VBR parameters. Finally ABR exists for bursty traffic which
cannot be easily characterised. When ABR traffic is sent it uses any available link ca-
pacity; if there is not enough link capacity for the traffic the ABR traffic is queued until
the buffers fill, in which case cells are dropped.
ATM has been popular with the Telecommunications Industry [21] for broadband
networks, where CBR and VBR traffic is used for dedicated (statically- and dynamically-
allocated) customer links (for voice and constant-rate data). As typical data is characterised
as bursty, it does not make sense to reserve bandwidth in a CBR or even VBR
capacity. ABR, which uses any available bandwidth on an ATM link, must be used to
avoid wasting capacity. There have not been many sites that have adopted ATM as a
local area network. The major factor prohibiting widespread adoption is the cost of
ATM switches and of ATM interface cards for individual nodes.
4.2 Other Notable Networking Technologies
High Performance Parallel Interface (HiPPI) is a point-to-point link that uses twisted-pair
copper cables to connect hosts via crossbar switches. HiPPI gets its name because
data is transmitted in parallel; the connecting cables have 50 cores: are used to
transmit data, one bit per line. The standard allows for transmission rates of 800MBit/s
and 1600MBit/s but the maximum length of copper cables is 25 metres. A serial version
of HiPPI is available, using fibre optic media, which allows a maximum distance of
10km between ends. HiPPI has been successfully used for networking supercomputer
systems, but now seems likely to be overtaken by other cheaper technologies.
Fibre Channel (FC), defined by the Fibre Channel Standard, is a circuit-switching
and also packet-switching technology that allows transmission at multiple rates. Data is
sent in frames of 2148 bytes (2048 bytes of data). FC is successfully used in interconnecting
hosts but is also likely to be overtaken by the economics of other technologies.
4.3 Internet or Native Technology Protocols
Many networking technologies have proprietary communications protocols, such as
ATM Adaptation Layers (AAL). While these proprietary protocols provide optimized
communication, there is usually a trade-off for code portability.
Using proprietary protocols can also be fraught with pitfalls. The majority of users
do not wish to make every optimisation to their code, or because users are working with
heterogeneous mixtures of networking hardware, the custom protocols are not widely
used. For example, experiments with ATM's AAL5 uncovered a bug in the implementation
that the manufacturer's engineers said could only be fixed by purchasing a much
faster machine [20]. We experimented with implementing message passing communications
software on raw ATM Adaptation layers and conclude that since this approach
has not been widely supported by ATM vendors, this will not be a feasible approach in
the long term. We conclude from our experiments that using a well-known, standard
protocol such as IP is the easiest, and most portable approach. Most manufacturers
provide for the encapsulation of IP packets within their proprietary protocols (such as
ATM's LAN Emulation). IP promises to be around a long time with the advent of IPv6,
which features IP tunneling, allowing standard IP traffic to be encapsulated within IPv6
packets[18].
In summary, we have found that while 10MBit/s desktop connections are still very
common, most current machines are now capable of effectively utilising a 100MBit/s
connection. For clusters of workstations with medium bandwidth requirements 100MBit/s
Fast Ethernet is a viable and affordable solution. If high bandwidth is required then a
trade-off between the cost and performance is necessary: SCI provides the best latency
but the measured bandwidth is nowhere near as high as Myrinet; Myrinet has the largest
bandwidth of all the systems we consider, and the latency is still under a millisecond. If
a larger latency can be tolerated, and bandwidth is not critical, then we believe the open
standards of Gigabit Ethernet may be an appropriate choice. We reiterate that we believe
the economics of large markets will ensure Gigabit Ethernet technology becomes
widespread for cluster computing.
5 Summary and Conclusions
In this article we have reviewed a number of fast networking technologies for cluster
computing and related our experiences with ATM in particular. Preliminary experiments
and experiences now being reported leads us to believe that the combination of
Fast Ethernet and Gigabit Ethernet will be the chosen route for most general purpose
Beowulf class cluster computing systems. We believe that the technical criteria being
fairly finely balanced with only factor of two advantages for one technology over an-
other, that the economics of the mass market will dictate which technology becomes
most widespread. This of course is a feedback phenomena as cheaper switches and
network cards will be adopted even more widely.
We believe ATM may still have a role to play in wide area cluster computing systems
but it is preferable that it be transparent to cluster users and cluster system soft-
ware. Internet protocol will surely be implemented on top of ATM and users and cluster
software will continue to interface to that.
We believe there are still many interesting developments to be made in software to
manage wide area clusters as well as high-performance clusters. It seems important
to recognise that will message passing level technology can play a useful part in high
performance systems, there is some considerable research still to be done to address
the latency and reliability issues for wide area cluster systems.
6

Acknowledgements

Thanks to J.A.Mathew for assistance is carrying out some of the measurements reported
in this work. It is also a pleasure to thank all those who helped in conducting
trials of the EBN: K.J.Maciunas, D.Kirkham, S.Taylor, M.Rezny, M.Wilson and
M.Buchhorn.



--R

Available at http://www.


CASA Project.

A Comparison of High Speed LANs
The Grid: Blueprint for a New Computing In- frastructure
PVM: Parallel Virtual Machine A Users' guide and Tutorial for Networked Parallel Computing
Gigabit Ethernet Alliance.

IEEE Standards On-line
IEEE Standards On-Line

Geographic Information Systems Applications on an ATM-Based Distributed High Performance Computing System

DISCWorld: An Environment for Service-Based Metacomputing
High Performance Fortran Forum (HPFF).
Internet Engineering Task Force.
Scheduling in Metacomputing Systems.
Using ATM In Distributed Applications.
Telstra's Experimental Broadband Network.
A comparison of two Gigabit SAN/LAN technolo- gies: Scalable Coherent Interface and Myrinet
An Assessment of Gigabit Ethernet as Cluster Interconnect.
ATM Performance Characteristics on Distributed High Performance Computers.
Message Passing Interface Forum.
Applications and Enabling Technology for NYNET Upstate Corridor.
Object Management Group.

3Com Corporation.
--TR
PVM: Parallel virtual machine
The grid
DISCWorld
Myrinet
Geographic Information Systems Application on an ATM-based Distributed High Performance Computing System
Geostationary-satellite imagery applications on distributed, high-performance computing
Legion-a view from 50,000 feet
An Assessment of Gigabit Ethernet as Cluster Interconnect
DISCWorld;gigabit ethernet;ATM;fast ethernet;metacomputing;cluster computing