Performance Comparison of Three Modern DBMS Architectures.
The introduction of powerful workstations connected through local area networks (LANs) inspired new database management system (DBMS) architectures that offer high performance characteristics. The authors examine three such software architecture configurations: client-server (CS), the RAD-UNIFY type of DBMS (RU), and enhanced client-server (ECS). Their specific functional components and design rationales are discussed. Three simulation models are used to provide a performance comparison under different job workloads. Simulation results show that the RU almost always performs slightly better than the CS, especially under light workloads, and that ECS offers significant performance improvement over both CS and RU. Under reasonable update rates, the ECS over CS (or RU) performance ratio is almost proportional to the number of participating clients (for less than clients). The authors also examine the impact of certain key parameters on the performance of the three architectures and show that ECS is more scalable that the other two.
Introduction
Centralized DBMSs present performance restrictions due to their limited resources. In
the early eighties, a lot of research was geared towards the realization of database ma-
chines. Specialized but expensive hardware and software were used to built complex
systems that would provide high transaction throughput rates utilizing parallel processing
and accessing of multiple disks. In recent years though, we have observed different
trends. Research and technology in local area networks have matured, workstations became
very fast and inexpensive, while data volume requirements continue to grow rapidly
[2]. In the light of these developments, computer systems-and DBMSs in particular-
in order to overcome long latencies have adopted alternative configurations to improve
their performance.
In this paper, we present three such configurations for DBMSs that strive for high
throughput rates, namely: the standard Client-Server[23], the RAD-UNIFY type of
DBMS[19], and the Enhanced Client-Server architecture[16]. The primary goal of this
study is to examine performance related issues of these three architectures under different
workloads. To achieve that, we develop closed queuing network models for all
architectures and implement simulation packages. We experiment with different workloads
expressed in the context of job streams, analyze simulated performance ratios, and
derive conclusions about system bottlenecks. Finally, we show that under light update
rates (1%-5%) the Enhanced Client-Server offers performance almost proportional to the
number of the participating workstations in the configuration for 32 or less workstations.
On the other hand, the RAD-UNIFY performs almost always slightly better than the
pure Client-Server architecture.
In section 2, we survey related work. Section 3 discusses the three DBMS architectures
and identifies their specific functional components. In section 4, we propose
three closed queuing network models for the three configurations and talk briefly about
the implemented simulation packages. Section 5 presents the different workloads, the
simulation experiments and discusses the derived performance charts. Conclusions are
found in section 5.
Related Work
There is a number of studies trying to deal with similar issues like those we investigate
here. Roussopoulos and Kang [18] propose the coupling of a number of workstations
with a mainframe. Both workstations and mainframe run the same DBMS and the
workstations are free to selectively download data from the mainframe. The paper
describes the protocol for caching and maintenance of cached data.
Hagman and Ferrari [11] are among the first who tried to split the functionality
of a database system and off-load parts of it to dedicated back-end machines. They
instrumented the INGRES DBMS, assigned different layers of the DBMS to two different
machines and performed several experiments comparing the utilization rates of the CPUs,
disks and network. Among other results, they found that generally there is a 60%
overhead in disk I/O and a suspected overhead of similar size for CPU cycles. They
attribute these findings to the mismatch between the operating and the DBMS system.
The cooperation between a server and a number of workstations in an engineering design
environment is examined in [13]. The DBMS prototype that supports a multi-level
communication between workstations and server which tries to reduce redundant work
at both ends is described.
DeWitt et al. [8] examine the performance of three workstation-server architectures
from the Object-Oriented DBMS point of view. Three approaches in building a server
are proposed: object, server and file server. A detailed simulation study is presented with
different loads but no concurrency control. They report that the page and file server
gain the most from object clustering, page and object servers are heavily dependent
on the size of the workstation buffers and finally that file and page servers perform
better in experiments with a few write type transactions (response time measurements).
Wilkinson and Niemat in [26] propose two algorithms for maintaining consistency of
workstation cached data. These algorithms are based on cache, and notify locks and
new lock compatibility matrices are proposed. The novel point of this work is that server
concurrency control and cache consistency are treated in a unified approach. Simulation
results show that cache locks always give a better performance than two-phase locking
and that notify locks perform better than cache locks whenever jobs are not CPU bound.
Alonso et al. in [2] support the idea that caching improves performance in information
retrieval systems considerably and introduce the concept of quasi-caching. Different
caching algorithms- allowing various degree of cache consistency- are discussed and
studied using analytical queuing models. Delis and Roussopoulos in [6] through a simulation
approach examine the performance of server based information systems under light
updates and they show that this architecture offers significant transaction processing
rates even under considerable updates of the server resident data. In [17], we describe
modern Client-Server DBMS architectures and report some preliminary results on their
performance, while in [7], we examine the scalability of three such Client-Server DBMS
configurations.
Carey et al. in [5] examine the performance of five algorithms that maintain consistency
of cached data in client-server DBMS architecture. The important assumption
of this work is that client data are maintained in cache memory and they are not disk
resident. Wang and Rowe in [25] a similar study examine the performance of five more
cache consistency algorithms in a client-server configuration. Their simulation experiments
indicate that either a two phase locking or a certification consistency algorithm
offer the best performance in almost all cases. Some work, indirectly related to the issues
examined in this paper, are the Goda distributed filing system project [20] and the cache
coherence algorithms described in [3].
The ECS model-presented here-is a slightly modified abstraction of the system
design described in [18] and is discussed in [16]. In this paper, we extend the work in
three ways: first, we give relative performance measures with the other two existing
DBMS configurations, analyze the role of some key system parameter values, and finally,
provide insights about the scalability of the architectures.
3 Modern DBMS Architectures
Here, we briefly review three alternatives for modern database system architectures and
highlight their differences. There is a number of reasons that made these configurations
a reality:
1. The introduction of inexpensive but extremely fast processors on workstations with
large amount of main memory and medium size disk.
2. The ever-growing volume of operational databases.
3. The need for data staging: that is, extracting for a user class just a portion of the
database that defines the user's operational region.
Although the architectures are general, they are described here for the relational model
only.
3.1 Client-Server Architecture (CS)
The Client-Server architecture(CS) is an extension of the traditional centralized database
system model. It originated in engineering applications where data are mostly processed
in powerful clients, while centralized repositories with check in and out protocols are
predominantly used for maintaining data consistency.
In a CS database [23], each client runs an application on a workstation (client) but
does database access from the server. This is depicted in Figure 1(a). Only one client
is shown for the sake of presentation. The communication between server and clients is
done through remote calls over a local area network (LAN) [22]. Applications processing
is carried out at the client sites, leaving the server free to carry out database work only.
The same model is also applicable for a single machine in which one process runs the
server and others run the clients. This configuration avoids the transmission of data
over the network but obviously puts the burden of the system load on the server. In this
paper, we assume that server and client processes run on different hardware.
3.2 RAD-UNIFY Type of DBMS Architecture(RU)
The broad availability of high speed networks and fast processing diskless workstations
were the principal reasons for the introduction of the RAD-UNIFY type of DBMS ar-
chitecture(RU) presented by Rubinstein et al. in [19]. The main objective of this configuration
is to improve the response time by utilizing both the client processing capability
and its memory. This architecture is depicted in Figure 1(b). The role of the server is
to execute low level DBMS operations such as locking and page reads/writes. As Figure
1(b) suggests, the server maintains the lock and the data manager. The client performs
the query processing and determines a plan to be executed. As datapages are being
retrieved, they are sent to the diskless client memory for carrying out the rest of the
processing.
In the above study, some experimental results are presented where mostly look up
operations perform much better than in traditional Client-Server configurations. It is
also acknowledged that the architecture may perform well only under the assumption of
light update loads. More specifically in the system's prototype, it is required that only
one server database writer is permitted at a time. The novel point of the architecture
is the introduction of client cache memory used for database processing. Therefore, the
clients can use their own CPU to process the pages resident in their caches and the server
degenerates to a file server and a lock manager. The paper [19] suggests that the transfer
of pages to the appropriate client memory gives improved response times at least in the
case of small to medium size retrieval operations. This is naturally dependent on the
size of each client cache memory.
3.3 Enhanced Client-Server Architecture (ECS)
The RU architecture relieves most of the CPU load on the server but does little to the
biggest bottleneck, the I/O data manager. The Enhanced Client-Server Architecture
reduces both the CPU and the I/O load by caching query results and by incorporating
in the client a full-fledged DBMS for incremental management of cached data
[15, 16].
Shared Database
Server DBMS
Commun. Software
CS Server
Commun. Software
Application Soft.
Client
Commun. Software
Application Soft.
Server DBMS
Commun. Software
Shared Database
Shared Database
Server DBMS Locking
and Data Managers
Commun. Software
Commun. Software
Application Soft.
Cached
Data
LAN LAN LAN
RAD-UNIFY Server ECS Server
Client Client
Client
Client DBMS Client DBMS

Figure

1: CS, RU, and ECS architectures
Initially, the clients start off with an empty database. Caching query results over
time permits a user to create a local database subset which is pertinent to the user's
application. Essentially, a client database is a partial replica of the server database.
Furthermore, a user can integrate into her/his local database private data not accessible
to others. Caching in general presents advantages and disadvantages. The two
major advantages are that it eliminates requests for the same data from the server and
boosts performance with client CPUs working on local data copies. In the presence of
updates though, the system needs to ensure proper propagation of new item values to
the appropriate clients. Figure 1(c) depicts this new architecture.
Updates are directed for execution to the server which is the primary site. Pages
to be modified are read in main memory, updated and flushed back to the server disk.
Every server relation is associated with an update propagation log which consists of
timestamped inserted tuples and timestamped qualifying conditions for deleted tuples.
Only updated (committed) tuples are recorded in these logs. The amount of bytes
written in the log per update is generally much smaller than the size of the pages read
in main memory.
Queries involving server relations are transmitted to and processed initially by the
server. When the result of a query is cached into a local relation for the first time, this
local relation is bound to the server relations used in extracting the result. Every
such binding is recorded in the server DBMS catalog by stating three items of interest:
the participating server relation(s), the applicable condition(s) on the relation(s), and a
timestamp. The condition is essentially the filtering mechanism that decides what are
the qualifying tuples for a particular client. The timestamp indicates what is the last
time that a client has seen the updates that may affect the cached data. There are two
possible scenarios for implementing this binding catalog information. The first approach
is that the server undertakes the whole task. In this case, the server maintains all the
information about who caches what and up to what time. The second alternative is that
each client individually keeps track of its binding information. This releases the server's
DBMS from the responsibility to keep track of a large number of cached data subsets
and the different update statuses which multiplies quickly with the number of clients.
Query processing against bound data is preceded by a request for an incremental
update of the cached data. The server is required to look up the portion of the log
that maintains timestamps greater than the one seen by the submitting client. This is
possible to be done once the binding information for the demanding client is available.
If the first implementation alternative is used this can be done readily. However, if
the second solution is followed then the client request should be accompanied along
with the proper binding template. This will enable the server to perform the correct
tuple filtering. Only relevant fractions (increments) of the modifications are propagated
to the client's site. The set of algorithms that carry out these tasks are based on the
Incremental Access Methods for relational operators described in [15] and involve looking
at the update logs of the server and transmitting differential files [21]. This significantly
reduces data transmission over the network as it only transmits the increments affecting
the bound object, compared with traditional CS architectures in which query results are
continuously transmitted in their entirety.
It is important to point out some of the characteristics of the concurrency control
mechanism assumed in the ECS architecture. First, since updates are done on the
server, a 2-OE locking protocol is assumed to be running by the server DBMS (this is
also suggested by a recent study [25]). For the time being, and until commercial DBMSs
reveal a 2-OE commit protocol, we assume that updates are single server transactions.
Second, we assume that the update logs on the servers are not locked and, therefore, the
server can process multiple concurrent requests for incremental updates.
4 Models for DBMS Architectures
In this section, we develop three closed queuing network models one for each of the three
architectures. The implementation of the simulation packages is based on them. We
first discuss the common components of all models and then the specific elements of
each closed network.
4.1 Common Model Components and DBMS Locking
All models maintain a W orkLoad Generator that is part of either the client or the
workstation. The W orkLoad Generator is responsible for the creation of the jobs (of
either read or write type). Every job consists of some database operation such as selec-
tion, projection, join, update. These operations are represented in an SQL-like language
developed as part of the simulation interface. This interface specifies the type of the operation
as well as simple and join page selectivities. The role of the W orkLoad Generator
is to randomly generate client job from an initial menu of jobs. It is also responsible for
maintaining the mixing of read and write types of operations. When a job finishes suc-
cessfully, the W orkLoad Generator submits the next job. This process continues until
an entire sequence of queries/updates is processed. To accurately measure throughput,
we assume that the sites' query/update jobs are continuous (no think-time between
jobs).
All three models have a Network Manager that performs two tasks :
1. It routes messages and acknowledgments between the server and the clients.
2. It transfers results/datapages/increments to the corresponding client site depending
on the CS/RU/ECS architecture respectively. Results are answers to the
queries of the requesting jobs. Datapages are pages requested by RU during the
query processing at the workstation site. Increments are new records and tuple
NetworkParameters Meaning V alue
time overhead for a 10 msec
remote procedure call
net rate network speed 10 Mbits/sec
mesg length average size of 300 bytes
requesting messages

Table

1: Network Parameters
identifiers of deleted records used in the ECS model incremental maintenance of
cached data.
The related parameters to the network manager appear in Table 1. The time
overhead for every remote call is represented by init time [14] while net rate is the
network's transfer rate(Mbits/sec). The average size of each job request message is
mesg length.
Locking is at the page level. There are two types of locks in all models: shared
and exclusive. We use the lock compatibility matrix described in [10] and the standard
two-phase locking protocol.
4.2 Client-Server Model

Figure

2 outlines the closed queueing network for the CS model. It is an extension of the
model proposed in [1]. It consists of three parts: the server, the network manager and
the client. Only one client is depicted in the figure. The client model runs application
programs and directs all DBMS inquiries and updates through the network manager to
the CS server. The network manager routes requests to the server and transfers the
results of the queries to the appropriate client nodes.
A job submitted at the client's site, passes through the Send Queue, the network,
the server's Input Queue, and finally arrives at the Ready Queue of the system where
it awaits service. A maximum multiprogramming degree is assumed by the model. This
limits the maximum number of jobs concurrently active and competing for system resources
in the server. Pending jobs remain in the Ready Queue until the number of
active jobs becomes less than the multiprogramming level (MPL). When a job becomes
active, it is queued at the concurrency control manager and is ready to compete for system
resources such as CPU, access to disk pages and lock tables (which are considered to
be memory resident structures). In the presence of multiple jobs, CPU is allocated in a
WorkLoad
Generator
Server
Client
Commit Queue
CMT
Output
Queue
Receive Queue
Send Queue
MPL
Concurrency
Control Queue
CCM
Update Queue
UPD
Blocked Queue
Read Queue
RD
Abort Queue
ABRT
Update
Blocked Operation
Read
Abort Operation
Ready Queue
Input
Queue
Processing
Queue
Network
Manager

Figure

2: Queuing Model for CS Architecture
round-robin fashion. The physical limitations of the server main memory (and the client
main memory as we see later on) allocated to database buffering may contribute to extra
data page accesses from the disk. The number of these page accesses are determined by
formulas given in [24] and they depend on the type of the database operation.
Jobs are serviced at all queues with a FIFO discipline. The concurrency control
manager (CCM) attempts to satisfy all the locks requested by a job. There are several
possibilities depending on what happens after a set of locks is requested. If the
requested locks are acquired, then the corresponding page requests are queued in the
ReadQueue for processing at the I/O RD module. Pages read by RD are buffered in
the P rocessingQueue. CPU works (PRC) on jobs queued in P rocessingQueue. If
only some of the pages are locked successfully, then part of the request can be processed
normally (RD;PRC) but its remaining part has to reenter the concurrency control manager
queue. Obviously, this is due to a lock conflict and is routed to CCM through the
Blocked Queue. This lock will be acquired when the conflict ceases to exist.
For each update request, the corresponding pages are exclusively locked, and subsequently
buffered and updated (UPD). If the job is unsuccessful in obtaining a lock, it
is queued in the Blocked Queue. If the same lock request passes unsuccessfully a predefined
number of times through the Blocked Queue, then a deadlock detection mechanism
ServerParameters Meaning V alue
server cpu mips processing power of server 21MIPS
disk tr average disk transfer time 12msec
main memory size of server main memory 2000 pages
instr per lock instructions executed per lock 4000
instr selection instructions executed per selected page 10000
instr projection instructions per projected page 11000
instr join instructions per joined page 29000
instr update instructions per updated page 12500
instr RD page instructions to read a page from disk 6500
instr WR page instructions to write a page to disk 8000
ddlock search deadlock search time
kill time time required to kill a job 0.2 sec
mpl multiprogramming level 10

Table

2: Server Parameters
is triggered. The detection mechanism simply looks for cycles in a wait-for graph that
is maintained throughout the processing of the jobs. If such a cycle is found, then the
job with the least amount of processing done so far is aborted and restarted. Aborted
jobs release their locks and rejoin the system's Ready Queue. Finally, if a job commits,
all changes are reflected on the disk, locks are released, and the job departs.
Server related parameters appear in Table 2 (these parameters are also applicable to
the other two models). Most of them are self-describing and give processing overheads,
time penalties, ratios and ranges of system variables. Two issues should be mentioned:
first, the queuing model assumes an average value for disk access, thereby avoiding issues
related to data placement on the disk(s). Second, whenever a deadlock has been detected,
the system timing is charged with an amount equal to ddlock search   active jobs
kill time. Therefore, the deadlock overhead is proportional to the number of active
jobs.
The database parameters (applicable to all three models) are described by the set
of parameters shown in Table 3. Each relation of the database consists of the following
information: a unique name (rel name), the number of its tuples (card), and the size
of each of those tuples (rel size). The stream mix indicates the composition of the
streams submitted by the W orkLoad Generator(s) in terms of read and write jobs.
Note also that the size of the server main memory was defined to hold just a portion
of the disk resident database which is 25% in the case of multiprogramming equal to
one(we have applied this fraction concept later on with the sizes of the client main mem-
DataParameters Meaning V alue
name of db name of the database TestDataBase
dp size size of the data page 2048 bytes
num rels number of relations 8
fill factor page filling factor 98%
rel name name of a relation R1, R2,., R8
card cardinality of every relation 20k
rel size size of a relation tuple (bytes) 100
stream mix query/update ratio 10

Table

3: Data Parameters
ories). For the above selected parameter values every segment of the multiprogrammed
main memory is equal to 200 pages (main memory=mpl).
4.3 RAD-UNIFY Model
Abort Queue
ABRT
Ready Queue
Concurrency
Control Queue
CCM
UPD
Update Queue
Read Queue
RD
Input
Queue
Send Queue
WorkLoad
Generator
Rec. Queue
Network
Manager
Commit Queue
CMT
Output
Queue
RAD-UNIFY Server
Diskless Client
Blocked Queue
MPL
Processing
Queue
Aborted or
Committed Job
more

Figure

3: Queuing Model for RAD-UNIFY Architecture

Figure

3 depicts a closed network model for the RAD-UNIFY type of architecture. There
are a few differences from the CS model:
ffl Every client uses a cache memory for buffering datapages during query processing.
ffl Only one writer at a time is allowed to update the server database.
Every client is working off its finite capacity cache and any time it wants to request
an additional page forwards this request over to the server.
ffl The handling of the aborts is done in a slightly different manner.
The W orkLoad Generator creates the jobs which, through the proper SendQueue,
the network and the server's InputQueue are directed to the server's ReadyQueue. The
functionality of the MPL processor is modified to account for the single writer require-
ment. One writer may coexist with more than one readers at a time. The functionality of
the UPD and RD processors, their corresponding queues as well as the queue of blocked
jobs is similar to that of the previous model.
The most prominent difference from the previous model is that the query job pages
are only read (ReadQueue and RD service module) and then through the network are
directed to the workstation buffers awaiting processing. The architecture capitalizes on
the workstations processing capability. The server becomes the device responsible for
the locking and page retrieval which make up the "low" level database system opera-
tions. Write type of jobs are executed solely on the server and are serviced with the
UpdateQueue and the UPD service module.
As soon as pages have been buffered in the client site (P rocessingQueue), the
local CPU may commence processing whenever is available. The internal loop of the
RU client model corresponds to the subsequent requests of the same page that may
reside in the client cache. Since the size of the cache is finite, this may force request of
the same page many times-depending on the type of the job [24]. The replacement is
performed in either a least recently used (LRU) discipline or the way specific database
operations call for (i.e. case of the sort-merge join operation). The wait-for graph of
the processes executing is maintained at the server which is the responsible component
for deadlock detection. Once a deadlock is found, a job is selected to be killed. This job
is queued in the AbortQueue and the ABRT processing element releases all the locks
of this process and sends an abort signal to the appropriate client. The client abandons
any further processing on the current job, discards the datapages fetched so far from the
server and instructs the W orkLoad Generator to resubmit the same job. Processes that
commit notify with a proper signal the diskless client component, so that the correct
job scheduling takes place. In the presence of an incomplete job (still either active or
pending in the server) the W orkLoad Generator takes no action awaiting for either a
commit or an abort signal.
4.4 Enhanced Client-Server Model
The closed queuing model for the ECS architecture is shown in Figure 4. There is a
major difference from the previous models:
ffl The server is extended to facilitate incremental update propagation of cached data
and/or caching of new data. Every time a client sends a request, appropriate
sections of the server logs need to be shipped from the server back over to the
client. This action decides whether there have been more recent changes than the
latest seen by the requesting client. In addition, the client may demand new data
(not previously cached on the local disk) for the processing of its application.
Ready Queues
MPL
Concurrency
Control Queue
CCM
Update Queue
UPD
Blocked Queue
Abort Queue
ABRT
Update
Blocked Operation
Abort Operation
Receive
Messages
Network
Manager
Read Type Xaction
Send Queue
WorkLoad
Generator
Commit Queue
CMT
Rec. Queue
Update
Processing
Upd
Input
Queue
Output
Queue
Messages
ECS Server
ISM
Data Access
and Processing
Client
RDM
LOGRD Queue

Figure

4: Queuing Model for ECS Architecture
Initially, client-disk cached data from the server relations can be defined using the
parameter ff Rel i
that corresponds to the percentage of server relation
cached in each participating client. Jobs initiated at the
client sites are always dispatched to the server for processing through a message output
queue (Send Queue). The server receives these requests in its Input Queue via the
network and forwards them to the Ready Queue. When a job is finally scheduled by the
concurrency control manager (CCM ), its type can be determined. If the job is of write
type, it is serviced by the Update, Blocked, Abort Queues and the UPD and ABRT
processing elements. At commit time, updated pages are not only flushed to the disk but
a fraction (write log fract) of these pages is appended in the log of the modified relation
(assuming that only fraction of page tuples is modified per page at a time). If it is just
a read only job it is routed to the LOGRD Queue. This queue is accommodated by the
log and read manager (LOGRDM) which decides what pertinent changes need to be
transmitted before the job is evaluated at the client or the new portion of the data to be
downloaded to the client. The increments are decided after all the not examined pieces
of the logs are read and filtered through the client applicable conditions. The factor that
decides the amount of new data cached per job is defined for every client individually
by the parameter cont caching perc. This last factor determines the percentage of new
pages to be cached every time (this parameter is set to zero for the initial set of our
experiment). The log and read manager sends -through a queue- either the requested
data increments along with the newly read data or an acknowledgment.
The model for the clients requires some modification due to client disk existence.
Transactions commence at their respective WorkLoad Generators and are sent through
the network to the server. Once the server's answer has been received, two possibilities
exist. The first is that no data has been transferred from the server, just an acknowledgment
that no increments exist. In this case, the client's DBMS goes on with the
execution of the query. On the other hand, incremental changes received from the server
are accommodated by the increment service module (ISM) which is responsible for reflecting
them on the local disk. The service for the query evaluation is provided through
a loop of repeating data page accesses and processing. Clearly, there is some overhead
involved whenever updates on the server affect a client's cached data or new data are
being cached.

Table

4 shows some additional parameters for all the models. Average client disk
access time, client CPU processing power, and client main memory size are described by
client dist tr, client cpu mips and client main memory respectively. The num clients
is the number of clients participating in the system configuration in every experiment
and instr log page is the number of instructions executed by the ECS server in order to
process a log page.
ClientParameters Meaning V alue Applicable in Model
client disk tr average disk transfer time 15 msec ECS
client cpu mips processing power of client 20 MIPS RU, ECS
client main memory size of client main memory 500 Pages RU, ECS
initial cached fraction 0.30 or 0.40 ECS
of server relation Rel i
instr log page instructions to process 5000 ECS
a log page
log fract fraction of updated pages 0.10 ECS
to be recorded to the log
cont caching perc continuous caching factor 0 ECS
clients number of clients 4,8,16,24.56 CS, RU, ECS

Table

4: Additional CS, RU and ECS parameters
4.5 Simulation Packages
The simulation packages were written in C and their sizes range from 4.6k to 5.4k lines
of source code. Two-phase locking protocol is used and we have implemented time-out
mechanisms for detecting deadlocks. Aborted jobs are not scheduled immediately but
they are delayed for a number of rounds before restart. The execution time of all three
simulators for a complete workload is about one day on a DECstation 5000/200. We
also ensure-through the method of batch means[9]-that our simulations reach stability
(confidence of more than 96%).
5 Simulation Results
In this section, we discuss performance metrics and describe some of the experiments
conducted using the three simulators. System and data related parameter values appear
in tables 1 to 4.
5.1 Performance Metrics, Query-Update Streams, and Work
Load Settings
The main performance criteria for our evaluation are the average throughput (jobs/min),
and throughput speedup defined as the ratio of two throughput values. Speedup is
related to the throughput gap [12] and measures the relative performance of two system
configurations for several job streams. We also measure server disk access reduction
which is the ratio of server disk accesses performed by two system configurations, cache
memory hits and various system resource utilizations.
The database for our experiments consists of eight base relations. Every relation
has cardinality of 20000 tuples and requires 1000 disk pages. The main memory of the
server can retain 2000 pages for all the multiprogrammed jobs while each client can
retain the most 500 pages for database processing in its own main memory for the case
of RU and ECS configurations. Initially, the database is resident on the server's disk but
as time progresses parts of it are cached either in the cache memory of the RU clients or
the disk and the main memory of the ECS clients.
In order to evaluate the three DBMS architectures, the W orkLoad Generator
creates several workloads using a mix of queries and modifications. A Query-Update
Stream(QUS) is a sequence of query and update jobs mixed in a predefined ratio. In
the first two experiments of this paper, the mix ratio is 10, that is each QUS includes
one update per ten queries. QUS jobs are randomly selected from a predetermined set
of operations that describe the workload. Every client submits for execution a QUS and
terminates when all its jobs have completed successfully. Exactly the same QUS are
submitted to all configurations. The lenght of the QUSs was selected to be 132 jobs
since that gave confidence of more than 96% in our results.
Our goal is to examine system performances under diverse parameter settings. The
principal resources all DBMS configurations compete for are CPU cycles and disk ac-
cesses. QUS consists of three groups of jobs. Two of them are queries and the other is
updates. Since the mix of jobs plays an important role in system performance (as Boral
and DeWitt show in [4]), we chose small and large queries. The first two experiments
included in this paper correspond to low and high workloads. The small size query set
(SQS) consists of 8 selections on the base relations with tuple selectivity of 5% (2 of
them are done on clustered attributes) and 4 2-way join operations with join selectivity
0.2. The large size query set (LQS) consists of 8 selections with tuple selectivity equal to
selections with tuple selectivity of 40%(4 of these selections are done on clustered
attributes), 3 projections, and finally 4 2-way joins with join selectivity .40. Update
jobs (U) are made up of 8 modifications with varying update rates (4 use clustered at-
tributes). The update rates give the percentage of pages modified in a relation during
an update. For our experiments, update rates are set to the following values: 0% (no
modifications), 2%, 4%, 6% and 8%.
Given the above group classification, we formulate two experiments: SQS-U, and
LQS-U. A job stream of a particular experiment, say SQS-U, and of x% update rate
consists of queries from the SQS group and updates from the U that modify x% of
Throughput CS and RU Throughput Rates (SQS-U)
Clients

Figure

5: CS and RU Throughput
the server relation pages. Another set of experiments designed around the Wisconsin
benchmark can be found in [16].
5.2 Experiments: SQS-U and LQS-U
In

Figure

5, the throughput rates for both CS and RU architectures are presented. The
number of clients varies from 4 to 56. There are clearly two groups of curves: those of
the CS (located in the lower part of the chart) and those of the RU (located in the upper
part of the chart). Overall, we could say that the throughput averages at about 11.6 jobs
per minute for the CS and 23.7 for the RU case. From 4 to 16 CS clients, throughput
increases as the CS configuration capitalize upon their resources. For more than 24 CS
clients, we observe a throughput decline for the non-zero update streams attributed to
high contention and the higher number of aborted and restarted jobs. The RU 0% curve
always remains in the range between 32 and 33 jobs/min since client cache memories
essentially provide much larger memory partitions than the corresponding ones of the
CS. The "one writer at a time" requirement of the RU configuration plays a major role
in the almost linear decrease in throughput performance for the non-zero update curves
as the number of clients increases. Note that at 56 clients, the RU performance values
obtained for QUS with 4% to 8% update rates are about the same with those of their CS
counterparts. It is evident from the above figure that since RU utilizes both the cache
Throughput ECS Throughput (SQS-U)
Clients

Figure

memories and the CPU of its clients for DBMS page processing, it performs considerably
better than its CS counterpart (except in the case of many clients submitting non-zero
update streams where RU throughput rates are comparable with those obtained in the
CS configuration). The average RU throughput improvement for this experiment was
calculated to be 2.04 times higher than that of CS.

Figure

6 shows the ECS throughput for identical streams. The 0% update curve
shows that the throughput of the system increases almost linearly with the number of
clients. This benefit is due to two reasons: 1) the clients use only their local disks
and achieve maximal parallel access to already cached data 2) the server carries out
a negligible disk operations (there are no updates therefore the logs are empty) and
handles only short messages and acknowledgments routed through the network. No
data movement across the network is observed in this case. As the update rate increases
(2%, 4%) the level of the achieved throughput rates remains high and increases almost
linearly with the number of workstations. This holds up to stations. After that, we
observe a small decline that is due to higher conflict rate caused by the increased number
of updates. Similar declines are observed by all others (but the 0% update curve). From

Figures

5 and 6, we see that the performance of ECS is significantly higher than those
of the CS and RU. For ECS the maximum processing rate is 1316.4 jobs/min (all 56
clients attached to a single server with 0% updates) while the maximum throughput
0% RU/CS
2% RU/CS
4% RU/CS
6% RU/CS
8% RU/CS
0% ECS/CS
2% ECS/CS
4% ECS/CS
6% ECS/CS
8% ECS/CS21e+015Throughput
Clients

Figure

7: ECS/CS and RU/CS Throughput Speedup
value for the CS is about 12.6 jobs/min and for the RU 32.9 jobs/min. The number of
workstations after which we observe a decline in job throughput is termed "maximum
throughput threshold" (mtt) and varies with the update rate. For instance, for the
2% curve it comes at about 35 workstations and for the 6%, 8% curves appears in the
region around 20 workstations. The mtt greatly depends on the type of submitted jobs,
the composition of the QUS as well as the server concurrency control manager. A more
sophisticated (flexible) manager (such as that in [26]) than the one used in our simulation
package would further increase the mtt values.

Figure

7 depicts the throughput speedup for RU and ECS architectures over CS (y
axis is depicted in logarithmic scale). It suggests that the average throughput for ECS is
almost proportional to the number of clients at least for the light update streams (0%,
2%, and 4%) in the range of 4 to 32 stations. For the 4% update stream, the relative
throughput for ECS is 17 times better than its CS-RU counterparts (at 56 clients). It
is worth noting that even for the worst case (8% updates), the ECS system performance
remains about 9 times higher than that of CS. The decline though starts earlier, at
clients, where it still maintains about 10 times higher job processing capability. At the
lower part of the Figure 7, the RU over CS speedup is shown. Although RU performs
generally better than CS under the STS-U workload for the reasons given earlier, its
corresponding speedup is notably much smaller than that of ECS. The principal reasons
2% ECS/CS
4% ECS/CS
6% ECS/CS
8% ECS/CS1e+015RU/CS reduction curves
Reduction
Clients
Server Disk Reduction (SQS-U)

Figure

8: ECS/CS and RU/CS Disk Reduction
for the superiority of the ECS are the off loading of the server disk operations and and
the parallel use of local disks.

Figure

8 supports these hypotheses. It presents the server disk access reduction
achieved by ECS over CS and similarly the one achieved by RU over CS. ECS server
disk accesses for 0% update streams cause insignificant disk access (assuming that all
pertinent user data have been cached already in the workstation disks). For 2% update
rate QUSs the reduction varies from 102 to 32 times over the entire range of clients. As
expected, this reduction drops further for the more update intensive streams (i.e. curves
4%, 6%, and 8%). However, even for the 8% updates the disk reduction ranges from 26
to 8 times which is a significant gain. The disk reduction rates of the RU architecture
over the CS vary between 2.4 and 2.6 times throughout the range of the clients and for
all QUS curves. This is achieved predominantly by the use of the client cache memories
which are larger than the corresponding main memory multiprogramming partitions of
the CS configuration avoiding so many page replacements.

Figure

9 reveals one of the greatest impediments in the performance of ECS that
is the log operations. The above graph shows the percentage of the log pertinent disk
performed (both log reads and writes) over the total number of server disk
accesses. In the presense of more than 40 clients the log disk operations constitute a
large fraction of server disk accesses (around 69%).
Percentage of Log Accesses over Total Server Disk Accesses
Clients
Percentage

Figure

9: Percentage of Server Page Accesses due to Log Operations

Figure

depicts the time spent on the network for all configurations and for
update rates: 0%, 4%, 8%. The ECS model causes the least amount of traffic in the
network because the increments of the results are small in size. The RU models causes
the highest network traffic because all datapages must be transferred to the workstation
for processing. On the other hand, CS transfers only qualifying records(results). For 56
clients, the RU network traffic is almost double the CS (2.15), and 7 times more than
that of ECS.

Figure

11 summarizes the results of the LTS-U experiment. Although the nature
of query mix has been changed considerably, we notice that the speedup curves have not
(compare with Figure 7). The only difference is that all the non-zero update curves have
been moved upwards closer to the 0% curve and that the mtts appear much later from
to 50 clients. Positive speedup deviations are observed for all the non-zero curves
compared with the rates seen in Figure 7. It is interesting to note that gains produced
by the RU model are lower than in the STS-U experiment. We offer the following
explanation: as the number of pages to be processed by the RU server disk increases
significantly (larger selection/join selectivities and projection operators involved in the
experiment) imposing delays at the server site (disk utilization ranges between .93 and
workstation CPUs can not contribute as much to the system average throughput.
4% ECS
8% ECS21e+0552RU related
Curves
CS related
Curves
Clients
Time Spent in the Network (SQS-U)

Figure

10: Time Spent on the Network
5.3 Effect of Model Parameters
In this section, we vary one by one some of the system parameters which have a significant
impact on the performance of the examined architectures, run the SQS-U experiment
and finally compare the obtained results with those of the Figure 7. The parameters
that vary are:
Client disk access time to 9msec: This corresponds to 40% reduction in average
access time. Figure 12 presents the throughput speedup achieved for the ECS experiments
using this new client access time over the ECS results depicted in Figure 6. Not
surprisingly, the new ECS performance values are increased an average 1.23 factor for
all curves which represents very serious gains. The no-update curve indicates a constant
53% throughput rate improvement while the rest curves indicate significant gains in the
range 4 to 40 clients. For more than 40 clients the gains are insignificant and the explanation
for that is that the large number of updates (that increases linearly with the
number of submitting clients) imposes serious delays on the server. Naturally, the RU
and CS throughput values are not affected at all in this experiment.
Server disk access time set to 7 msec: We observe that all but one(8%) non-zero
update curves approach the 0% curve and the mtts area appear much later at about 40
stations. By providing a faster disk access time server jobs are executed faster and cause
0% ECS/CS
2% ECS/CS
4% ECS/CS
6% ECS/CS
8% ECS/CS
0% RU/CS21e+015
Rest RU/CS curves
Throughput
Clients

Figure

11: RU/CS and ECS/CS Throughput Speedups for LQS-U
fewer restarts.
Client CPU set to 110MIPS: The results are very similar to those of Figure 6 with
the exception that we notice an average increase of 15.16 jobs/min throughput rate for
all update curves. This indicates that extremely fast processing workstations alone have
moderate impact on the performance of the ECS architecture.
Server CPU set to 90MIPS: We observe a shifting of all curves towards the top right
corner of the graph. This pushes the mtts further to the right and allows for more clients
to work simultaneously. There are two reasons for this behavior: 1) updates are mildly
consuming the CPU and therefore, by providing a faster CPU they finish much faster
and fast CPU results in lower lock contention which in turn leads to fewer deadlocks.
Server log processing is also carried out faster as well.
5.4 Other Experiments
In this section, we perform four more experiments to examine the role of "clean" up-date
workloads, specially designed QUSs so that only a small number of clients submits
updates, the effect of the continuous caching parameter cont caching perc in the ECS
model as well as the ability of all models to scale-up.
Pure Update Workload: In this experiment, the update rate (6%) remains constant
Throughput
ECS Speedup for Client Disk Access 9msec over 15msec
Clients

Figure

12: ECS Speedup for Client Disk Access Time at 9msec over Client Access Time
at 15 msec
and the QUS are made up of only update jobs (pure update workload). Figure 13
gives the throughput rates for all configurations. The CS and ECS models show similar
curves shape-wise. For the range of 4-10 clients their throughput rates increase and
beyond that point under the influence of the extensive blocking delays and the created
deadlocks their performance declines considerably. At 56 clients the ECS achieved only
half of the throughput obtained than that when 8 clients when used. Overall, CS is
doing better than ECS because the latter has to also write into logs (that is extra disk
page accesses) at commit time. It is also interesting to note the almost linear decline in
the performance of the RU configuration. Since there is strictly one writer at a time in
this set of experiments all the jobs are sequenced by the MPL processor of the model
and are executed one at a time (due to lack of readers). The concurrency assists both
CS and ECS clients in the lower range (4 to 8) to achieve better throughput rates than
their RU counterparts, but later the advantages of job concurrency (many writers at the
same time - CS,ECS cases) diminish significantly.
Limited Number of Update Clients: In this experiment, a number of clients is
dedicated to updates only and the rest clients submit read only requests. This simulates
environments in which there is a (large) number of read-only users and a constant number
of updaters. This class of databases includes systems such as those used in the stock
CS
RU
Throuhgput
Clients
Pure Update Workload

Figure

13: QUSs consist of Update Jobs Only
markets. Figure 14 presents the throughput speedup results of the SQS-U experiment.
Note that three clients at each experiment are designated as writers and the remaining
query the database. The RU/CS curves remain in the same performance value levels as
those observed in Figure 7 with the exception that as the number of clients increases the
effect of the writers is amortized by the readers. Thus, all the non-zero update curves
converge to the 0% RU/CS curve for more than clients. The ECS/CS curves suggest
spectacular gains for the ECS configuration. The performance of the system increases
almost linearly with the number of participating clients for all the update curves. Note
as well that the mtts have been disappeared from the graph. Naturally, the mtts appear
much later when more than 56 clients are used in the configuration (not shown in the

Figure

Changing data locales for the ECS clients: So far, we have not considered the
case in which clients continuously demand new data from the server to be cached in
their disks. The goal of this experiment was to address this issue and try to qualify
the degradation in the ECS performance. To carry out this experiment, we use the
parameter cont caching perc (or ccp) that represents the percentage of new pages from
the relations involved in a query to be cached in the client disk. This parameter enabled
us to simulate the constantly changing client working set of data. Figure 15 shows the
0%, 2% and 6% curves of Figure 6 (where ccp is 0%) superimposed with the curves for
Throughput
Limited Number of Update Job Strems
Clients
0% RU/CS
6% ECS/CS
8% ECS/CS
4% ECS/CS
2% ECS/CS
0% ECS/CS
8% RU/CS

Figure

14: Throughput Speedup Rates for Three Writer Only Clients per Experiment
the same experiment (SQS-U) but with ccp equal to 1% and 2%. Taken into account
that every query requires a new part of the server data, this percentage contributes
to large numbers of additional disk accesses and augmented processing time on the
part of the server. This very fact makes the almost linear format of the original 0%
curve to disappear. More specifically, while 56 clients in the original experiment attain
1316.3 jobs/min under ccp=1% they achieve 416.1 jobs/min and under
accomplish only 206.6 jobs/min. The performance degradation for the heavy updating
QUS (i.e. 6%) is less noteworthy since there is already serious server blocking.
Ability to scale-up: We are also interested in the scale-up behavior of all architectures
under the presence of a large number of workstations. For this purpose, we ran an
experiment where the number of clients ranges from 4 to 120 per server. Figure 16
depicts the resulting curves for the STS-U experiment. The graph indicates that beyond
the mtts region the speedup of the ECS over CS is gradually decreasing (after 40 clients).
For more update intesive QUSs the speedup decline is less sharp than that of the 2%
curve. This detarioration is due to the saturation of the server's resources that the great
client number creates. Note however, even at the 120 clients, the ECS architecture can
process between 6.1 to 19.9 times more jobs than the CS architecture (non-zero update
curves). The 0% update curve shows no saturation and continues to increase almost
linearly. The major reasons for this are that the performance of the CS configuration for
0% ECS
2% ECS
0% ECS
2% ECS
6% ECS
Throughput SQS-U Experiment with Three cont_caching_perc (ccp) Values
Clients
6% ECS
6% ECS
2% ECS
Figure

15: Experiments with Three Values for the ccp parameter
more than 56 clients remains stable in the range between 9.00 and 12.5 job/min (due to
the high utilization of its resources) and that the network shows no signs of extremely
heavy utilization. The gains for the RU configuration remain in the same levels with
those reported in experiment STS-U.
6 Conclusions
We have presented, modeled and compared three contemporary DBMS architectures all
aiming for high throughput rates. The simulation results show the following characteristics

ffl The RAD-UNIFY architecture gives an average 2.1 times performance improvement
over the CS by utilizing the main memories and the CPUs of the clients.
ffl Although the ECS performance declines when the QUS queries to updates ratio
decreases, it still offers serious speedup rates over the other two architectures.
Under pure update workloads, the ECS gives the worst performance for more than
clients.
ffl Under light update rates (1%-5%), the performance speedup of ECS over both CS
and RU is almost proportional to the number of participating workstations in the
10.0030.0050.00Model Scalability Experiment
Throughput
Clients
0% ECS/CS
2% ECS/CS
4% ECS/CS
6% ECS/CS
8% ECS/CS
0% RU/CS
Rest of RU/CS curves

Figure

Scalability Experiment
range of 4-32 for our experiments. For streams without updates, ECS/CS speedup
increases almost linearly with the number of workstations.
ffl Under either heavier update rates and many concurrently imposed modifications,
the ECS Server becomes the system bottleneck. We have introduced the "maxi-
mum throughput thresholds" (mtts) to identify these bottleneck points with respect
to each update rate.
ffl Faster workstation disk access time improves the performance of ECS.
ffl ECS performance is diminished significantly whenever clients constantly demand
new data elements from the server.
ffl Database environments with few exclusive writers and a number of readers offer
very good performance results for the ECS configuration.
ffl Under all, but pure update workloads, the ECS architecture is more scalable than
the other two.
Future work includes experimentation using local databases to the workstations,
exploring the behavior of the configurations under different server concurrency control
protocols and developing periodic update propagation strategies for bringing transaction
throughput close to the 0% update curves.
Aknowledgements: The authors are grateful to the anonymous referees for their valuable
comments and suggestions as well as, Jennifer Carle and George Panagopoulos for
commenting on earlier versions of this paper.



--R

Models for Studing Concurrency Control Performance: Alternatives and Implications.
Data Caching Issues in an Information Retrieval System.
Cache Coherence Protocols: Evaluation Using a Multiprocessor Simulation Model.
A Methodology for Database System Performance Evaluation.
Data Caching Tradeoffs in Client-Server DBMS Architectures
Server Based Information Retrieval Systems Under Light Update Loads.
Performance and Scalability of Client-Server Database Architec- tures
A Study of Three Alternative Workstation- Server Architectures for Object-Oriented Database Systems
Computer Systems Performance Evaluation.
Granularity of Locks and Degrees of Consistency in a Shared Database
Performance Analysis of Several Back-End Database Architectures
Performance Considerations for an Operating System Transaction Manager.
Cooperative Object Buffer Management in the Advanced Information Management Prototype.
An Environment for Developing Fault-Tolerant Software
The Incremental Access Method of View Cache: Concept
Evaluation of an Enhanced Workstation-Server DBMS Architecture
Modern Client-Server DBMS Architectures
Principles and Techniques in the Design of ADMS
Benchmarking simple database operations.
A Highly Available File System for a Distributed Workstation Environment.
Differential Files: Their Application to the Maintenance of Large Databases.
Unix Networking Programming.
Architectures of Future Data Base Systems.
Database and Knowledge-Base Systems: Volume <Volume>II-The</Volume> New Technologies
Cache Consistency and Concurrency Control in a Client/Server DBMS Architecture.
Maintaining Consistency of Client-Cached Data
--TR
Performance analysis of several back-end database architectures
Cache coherence protocols: evaluation using a multiprocessor simulation model
Principles and techniques in the design of ADMS:.F:6WWp
Concurrency control performance modeling: alternatives and implications
Benchmarking simple database operations
Performance Considerations for an Operating System Transaction Manager
Coda
Data caching issues in an information retrieval system
A study of three alternative workstation server architectures for object-oriented database systems
Maintaining consistency of client-cached data
Architecture of future data base systems
UNIX network programming
An Environment for Developing Fault-Tolerant Software
An incremental access method for ViewCache
Data caching tradeoffs in client-server DBMS architectures
Cache consistency and concurrency control in a client/server DBMS architecture
Evaluation of an enhanced workstation-server DBMS architecture
Differential files
Principles of Database and Knowledge-Base Systems
A methodology for database system performance evaluation
Cooperative Object Buffer Management in the Advanced Information Management Prototype
Performance and Scalability of Client-Server Database Architectures

--CTR
Svend Frlund , Pankaj Garg, Design-time simulation of a large-scale, distributed object system, ACM Transactions on Modeling and Computer Simulation (TOMACS), v.8 n.4, p.374-400, Oct. 1998
Vinay Kanitkar , Alex Delis, Time Constrained Push Strategies in Client-Server Databases, Distributed and Parallel Databases, v.9 n.1, p.5-38, January 1, 2001
Vinay Kanitkar , Alex Delis, Real-Time Processing in Client-Server Databases, IEEE Transactions on Computers, v.51 n.3, p.269-288, March 2002
Alexander Thomasian, Distributed Optimistic Concurrency Control Methods for High-Performance Transaction Processing, IEEE Transactions on Knowledge and Data Engineering, v.10 n.1, p.173-189, January 1998
Alex Delis , Nick Roussopoulos, Performance and Scalability of Client-Server Database Architectures, Proceedings of the 18th International Conference on Very Large Data Bases, p.610-623, August 23-27, 1992
Je-Ho Park , Vinay Kanitkar , Alex Delis, Logically Clustered Architectures for Networked Databases, Distributed and Parallel Databases, v.10 n.2, p.161-198, September 2001
Alex Delis , Nick Roussopoulos, Techniques for Update Handling in the Enhanced Client-Server DBMS, IEEE Transactions on Knowledge and Data Engineering, v.10 n.3, p.458-476, May 1998
Vinay Kanitkar , Alex Delis, Efficient processing of client transactions in real-time, Distributed and Parallel Databases, v.17 n.1, p.39-74, January 2005
Alfredo Goi , Arantza Illarramendi , Eduardo Mena , Jos Miguel Blanco, An Optimal Cache for a Federated Database System, Journal of Intelligent Information Systems, v.9 n.2, p.125-155, Sept./Oct. 1997
DBMS architectures;software architecture configurations;RAD-UNIFY type;design rationales;database management systems;simulation models;performance evaluation;client-server;functional components;workstations;simulation results;local area networks;software engineering