A decision-theoretic approach to database selection in networked IR.
In networked IR, a client submits a query to a broker, which is in contact with a large number of databases. In order to yield a maximum number of documents at minimum cost, the broker has to make estimates about the retrieval cost of each database, and then decide for each database whether or not to use it for the current query, and if, how many documents to retrieve from it. For this purpose, we develop a general decision-theoretic model and discuss different cost structures. Besides cost for retrieving relevant versus nonrelevant documents, we consider the following parameters for each database: expected retrieval quality, expected number of relevant documents in the database and cost factors for query processing and document delivery. For computing the overall optimum, a  divide-and-conquer algorithm is given. If there are several brokers knowing different databases, a preselection of brokers can only be performed heuristically, but the computation of the optimum can be done similarily to the single-broker case. In addition, we derive a formula which estimates the number of relevant documents in a database based on dictionary information.
Introduction
Networked information retrieval (NIR) is a new research area challenged by the rapid growth of
the Internet and the widespread use of IR servers like e.g. WAIS or systems supporting the Z39.50
protocol (see e.g. [Kahle et al. 93]). Besides classical applications like bibliographies, more recently
digital libraries have become accessible through the internet. The ultimate goal of NIR is to develop
systems letting a user access all resources available on the network, but in a way that gives him the
impression of a single large IR database.
Roughly speaking, there are two major tasks that have to be performed in NIR, namely resource
discovery and collection fusion. Given a user query, the former deals with the problem of selecting
those databases which should be used for answering the query, whereas the latter deals with the
database-specific transformations of the original query and the merging of the results.
In this paper, we address the resource discovery issue, which consists of two subtasks, namely
database detection and database selection. Database detection can be performed relatively easily,
either by exploiting the name conventions used in the domain name service of the internet (e.g.
names of ftp servers should start with 'ftp.', names of Web servers with `www.') or by establishing
central registries (e.g. the directory-of-servers for WAIS systems).
For database selection, there is the problem of precisely formulating the goal that should be
reached by performing this task. Some approaches (see section only consider the retrieval quality
of the overall NIR task (i.e. resource discovery plus collection fusion). This way, however, the
underlying assumptions do not become clear, and it is also difficult to figure out which parts of such
a system are doing well and where it fails. For resource discovery alone, no appropriate evaluation
measures have been described so far.
Author's address: University of Dortmund, Informatik VI, 44221 Dortmund, Germany. Email:
fuhr@ls6.informatik.uni-dortmund.de. voice: +49 231 755 2045. fax: +49 231 755 2045.
In this paper, we develop a model which defines a decision-theoretic criterion for optimum
database selection. This model considers relevance as well as other important factors present in
networked IR (e.g. costs for query processing and document delivery). We start from the Probability
Ranking Principle (PRP, see [Robertson 77]), where it can be shown that optimum retrieval
performance is achieved when documents are ranked according to decreasing probability of relevance.
Here performance can be measured either in terms of precision and recall (which, in turn, refer to
relevance), or by means of a decision-theoretic model which attributes different costs to the retrieval
of relevant and nonrelevant documents.
In the following section, we describe the basic model for optimumdatabase selection, thus deriving
an optimum selection rule. Then we discuss the consequences of this model for different application
situations. In section 4, we present an algorithm for computing the optimum selection for a specific
query and a number of databases given. Then we briefly discuss the situation where we have a
hierarchic network strucure. Section 6 discusses related work, and the final section presents the
conclusions and gives an outlook on further work.
Optimum database selection
Our model extends the two assumptions underlying the PRP by an additional one for considering
the costs of retrieval in different databases:
1. Relevance judgements are based on a binary relevance scale.
2. The relevance judgement for a document is independent of that for any other document.
3. The costs for retrieving a set of documents from a database are independent of those for other
queries or other databases.
The first assumption also can be generalized to multi-valued scales, (see [Bookstein 83] for the
model and [Fuhr 89] for an application of this model), but here we want to avoid burdening our
presentation with features that are not essential. The second assumption not only excludes effects
due to similarity or other kinds of dependence between documents, we also ignore the effect of
duplicates (i.e. retrieval of the same document from different databases). The third assumption
(which we have added to those from the PRP) restricts the nature of the cost factors such that we
can regard costs for specific databases and queries in isolation; thus, our model is not applicable if
e.g. a database provider offers discount rates for retrieval from multiple databases from the same
provider or for a set of queries
In the following, we assume a basic setting as follows: A broker has access to a set of IR databases
to which it may send a query. In response, each database produces a ranked list of documents,
and the broker may request any number of documents from this list. Each database has its own
performance curve in terms of recall and precision, and there are database-specific costs for the
retrieval of documents. Given a specific query, we now want to retrieve a maximum number of
relevant documents at minimum cost, i.e. one of the two parameters is specified by the user, and the
broker aims at optimizing the other one.
More specifically, in order to select the databases to be used for processing a query, for each
database D i the broker estimates a function C R
giving the specific costs for retrieving n relevant
documents from this database. Based on this information, a global function C R (n) can be derived
which specifies the minimum costs for retrieving n relevant documents from all databases combined.
As usual in decision-theoretic models, costs may stand for money as well as for computing time,
response time or the time a user spends for doing her job.
On a coarse-grained level, we may assume that there is a cost function C s
retrieving k
documents from database D i . However, in most cases the cost can be split up in fixed costs C 0
processing a query and a factor C d
for each document delivered from the query result. With sign(:)
denoting the signum function, we have
In order to consider relevance, we assume that we know the following parameters for each database
ffl the expected recall-precision curve P i (R) (for the specific query) and
ffl the expected number of relevant documents R i in the database.
Then we estimate the expected number of documents s i to be selected in order to retrieve r i relevant
documents from database D i via the relationship r i =s
As in traditional IR, we also assume user costs (or benefits) C R and C N for a user viewing a
relevant document or a nonrelevant document, respectively.
Combining equations 1 and 2, we can estimate the cost for retrieving r i relevant documents from
database
Given the database-specific cost functions, we can estimate the overall minimum costs C R (n) for
retrieving n relevant documents. For that, let us assume that we have l databases and the numbers
r i of relevant documents to be retrieved from the different databases are represented as a vector
Now we can formulate the optimum selection rule: For a given number n of relevant documents
to be retrieved, determine ~r such that the expected overall cost C R (n) is minimum, i.e.
~r
l
This selection rule is independent of the type of the database-specific cost function. Thus, it
holds also for alternatives to the function (3) - as long as the cost function C R
does not depend
on other queries or other databases.
3 Interpretation of results
Now we want to discuss the consequences of the cost functions 3 and 4. For that, we will ignore
the fact that C R (n) is a discrete function and assume it to be continuous. Then we can make some
observations that hold for the optimum solution.
First, we can make a general statement about those databases D i that contribute to the query
result, i.e. r i ? 0. By using Lagrange multipliers, we find out that
is equal for all these databases. Roughly speaking, this means that the costs for the last relevant
document retrieved from each of the databases involved are equal. If we asssume the typical situation
with a cost function like eqn (3) and a monotonously decreasing recall-precision curve, then we can
conclude that the cost differential is monotonously increasing.
In order to make further observations, we have to distinguish certain cases depending on the
structure of the cost factors:
meaning
database
documents retrieved from D i
documents retrieved (selected) from D i
relevant documents in D i
expected recall-precision function for D i
costs for selecting n documents from D i
fixed costs for query processing in D i
costs for retrieving a document from D i
expected costs for retrieving n relevant documents from D i
user costs for viewing a relevant document
user costs for viewing a nonrelevant document
expected global costs for n relevant documents
R recall
precision
query
d document
indexing weight of term t j in document dm
of indexing weights for term t j in database
search term weight of term t j
of database D i (# documents)

Table

1: Notations used throughout this paper
1. C 0
Assuming that we only have to pay per document delivered, but not
for processing the query, we get C R
databases. Sample functions are depicted in
figure 1. Since a specific number of total relevant documents implies an equal slope @C R
for all curves, all databases for which there is a point with this slope on the curve will contribute
to the optimum solution. In figure 1, the points corresponding to two solutions a and b are
marked, showing that for the first solution, only two of the databases are involved. The set
of databases involved grows as the total number of relevant documents increases; a database
contributing to a small number always will stay involved for larger numbers, too. This feature
is important for incremental retrieval where a user specifies neither the cost nor the number
of relevant documents in advance.
If, in addition, we have equal costs per document C d
i for all databases, then we can also make
statements about recall and precision. In this case, the databases involved operate at the same
precision level. Figure 2 shows the points for four different solutions e.g. for
b, only databases 1 and 4 reach this precision level.
2. C 0
l]. If there are databases with nonzero query processing costs, then
the set of databases that actually contribute to the solution will depend on the total number
n of relevant documents. Here databases involved for small values of n may not contribute to
the optimum solution as n grows (see the example in table 2). With regard to incremental
retrieval, we have a conflict here: Given that the user first requested n 1 documents and then
another documents, the minimum costs for this stepwise procedure may be higher than for
retrieving relevant documents at once.
a
a

Figure

1: Sample cost functions for C 0
solutions a, b
Towards application
With the derivation of the overall cost function C R
i (n) in eqn (4), we have defined a rule for optimum
database selection. So each method for database selection should aim at approximating this
optimum. In most applications, it will be difficult to estimate the parameters occurring in C R
This situation is similar to (or even worse than) the difficulties with the PRP, where the estimation
of the probability of relevance of a document also poses problems. However, with our model in
mind, we can start with crude approximations of the parameters, apply it and then check how far
our estimates deviate from the real parameters - thus telling us where we should spend effort for
improving our methods.
In the following, we describe a general procedure for applying our model, and we also describe an
algorithm for computing the optimum C R
i (n). In principle, the following steps have to be performed:
1. For each database D i , estimate the number of relevant documents R i .
2. For each database D i , determine (or assume) a recall-precision function P i (R).
3. Compute the database-specific cost functions C R
4. Derive the global cost function C R (n) as combination of databases such that, for each value
of n, the costs are minimum.
For the first step, the appendix describes a method for estimating R i in the case of probabilistic
systems and linear retrieval functions; it follows that the estimation procedure treats databases
like metadocuments, where the formula for estimating R i has the same structure as the retrival
function for ordinary documents. So (at least in this special case) this step can be performed rather
efficiently, even for a large number of databases.
The second step - estimating the recall-precision (RP) curves of the databases - poses more
difficulties: little is known about the behaviour of query-specific RP curves. Results from the TREC
conference [Harman 95] indicate that there is a great variation in these curves, so further research will
be required for achieving good estimates of this parameter. For the time being, heuristic methods
will have to be applied instead. For example, a simple assumption would be a linear function, with
thus leading to the approximation has to be
chosen. In the absence of any query-specific knowledge, one might assume that the RP function is
rR R R R
Pa
c
d

Figure

2: Sample recall-precision curves with optimum solutions
equal for all queries. However, in some cases additional information may be available. In practical
applications with a set of heterogeneous databases, very often a query contains a condition which
cannot be evaluated by the IR system running a specific database; then already P (0) will be very
low (see [Fuhr 96]). It also may be feasible to assume functions that are typical for certain kinds of
systems, e.g. Boolean vs. probabilistic systems.
In the third step, we use R i and P i (R) from the previous steps plus the cost factors C 0
for deriving the database-specific cost functions C R
Given these parameters, we now describe a divide-and-conquer algorithm for computing the
overall optimum cost function C R (n). Our algorithm is based on the following assumptions:
1. For each database, the costs per additional relevant document are monotonously increasing
(i.e. for all i and all k ? 0, C R
2. We do not consider fractions of (expected) relevant documents for individual databases.
The first assumption is essential for reducing the computational complexity of the algorithm. Without
this assumption, it is shown in [Goevert 97] that our optimizing problem is equivalent to the
knapsack problem, and thus it is NP-complete. The second assumption keeps the algorithm simple;
below, we also discuss alternatives.
The algorithm is shown in figure 3. Here the procedure costrel gives us the database-specific costs
retrieving n relevant documents from D i . The overall optimum cost function is computed
in procedure ocf. Given a list S of databases and a maximum number n of relevant documents
to be retrieved, this procedure returns a data structure giving the vectors r and the costs C R
n. The crucial point in this procedure is the merge step of the divide-and-conquer
strategy: when we merge two cost functions C1 and C2, for each number of relevant documents, we
test whether a single function or the combination of both C1 and C2 is cheapest. Here we exploit
the fact that the cost differentials are monotonously increasing and that in case both arguments
contribute to the optimum solution, the cost differentials must be equal. Since we have discrete cost
functions, we consider the discrete cost differences d1 and d2 instead of the differentials and assume
that at each point, the differentials lie between the corresponding two subsequent differences. An
example for the application of this algorithm is shown in table 2.
For retrieving n relevant documents from l databases, this algorithm takes O(n \Delta l \Delta log l) time.
TYPE listofdb: LIST OF database;
fcost function as array with #relevant docs as indexg
fcost function contains costs and list of document requestsg
numdocs: integer
fdocument request contains database id and # docsg
fgives costs and number of docs to be retrieved for getting r relevant docs from database db,
yields infinity cost if r exceeds total number of relevant docsg
fcomputes optimum cost function for list of databases Sg
BEGIN
C[0].cost := 0; C[0].gdl := empty();
fcompute elements of cost functiong
costrel(db,i,C[i].cost, gd.numdocs);
C[i].gdl := insert(empty(),gd);
OD
divide and conquerg
divide: split(S,S1,S2);
conquer: C1 := ocf(S1,n); C2 := ocf(S2,n);
merge: fmerge cost functionsg
point of equal cost differenceg
C2[j2+1].cost-C2[j2].cost FI;
C1[i].cost AND c12 ! C2[i].cost
THEN C[i].cost := c12; C[i].gdl := concat(C1[j1].gdl,C2[j2].gdl)
C1[i].cost ! C2[i].cost THEN C[i] := C1[i]
END.

Figure

3: Algorithm for computing optimum cost function
As mentioned above, we have not considered that a database also may yield fractions of (expected)
relevant documents, which also could contribute to the requested sum of relevant documents (e.g.
our algorithm only yields an approximation of the optimum
solution. An algorithm for computing the optimum should not restrict to integer values for the
number of relevant documents r i from each database. The correct restriction would be to assume
integer values for the number of documents selected, from which the corresponding fractions of
relevant documents are derived. However, this approach affects the complexity of the merge step
(which is now n), where all possible intermediate points between i and would have to be
computed. In the worst case where we would have to retrieve (almost) all documents of a database
D i in order to find the last relevant one, there would be an intermediate point for each document
in the database. Since different databases may yield different intermediate points, we would have to
compute up to
points. Thus, the the corresponding algorithm would take O(N \Delta l \Delta log l)
in the worst case. The corresponding algorithm is described in [Goevert 97], along with a modified
version which restricts the number of intermediate points between i and to a given constant
c by dropping additional points. Thus, the algorithm presented here corresponds to
the correct optimum is computed with
cost gdl cost gdl cost gdl
7 22 h(14; 1)i 26 h(43; 2)i 21 h(6; 1); (13; 2)i

Table

2: Example for computing optimum costs of two databases
5 Multiple brokers
So far, we have assumed that there is a single broker which knows all databases in question. For an
information system that aims at covering all information sources that can be reached via a network,
this assumption is not realistic. For this reason, we now consider the case of multiple brokers, where
each broker knows some databases and/or some other brokers.
A reasonable extension of our decision-theoretic approach would be the development of a procedure
allowing a broker to decide whether or not to contact another broker for processing a specific
query.
There are two possibilities for representing metaknowledge about another broker B in broker A:
1. Treat B like a database. Thus, a typical broker would represent a large database. In the
general case, this poses obvious difficulties for cost estimation, since a broker stands for a
number of databases with different costs, modelling broker costs can only be an approximation
of the real costs. Especially when several databases have to be used for processing a query,
the approximation may be totally wrong. However, even if we have no query processing costs
or equal costs per document for all databases of broker B, then we still cannot make a proper
estimate of the costs that would be caused by sending the query to broker B. The reason is
that in applying equation (3) for estimating these costs, the sum
(where r i denotes the number of relevant documents retrieved from database D i ) cannot be
expressed as function of the sums
2. Use more detailed metaknowledge about B. If B keeps the sum of probabilistic weights (i.e. the
weights v j from appendix A) for each query condition and each database, then the knowledge
about all databases of B can be modelled as a distribution over the v j s. Assuming a normal
distribution, this distribution can be represented as a pair of mean and variance. Let A possess
this knowledge about B.
Given this information for all conditions of a (linear) query, the distribution of the numbers
of relevant documents w.r.t. the whole query again is a normal distribution. Thus, A could
estimate the probability that a specific database known by B has at least r relevant documents.
Knowing the total number of databases managed by B, A can also estimate the probability
that there is at least one such database with more than R documents.
However, what we actually need is the expected cost E(C R (n)) for retrieving n relevant documents
via B. Comparing these estimates with the corresponding figures of A's databases
would yield the optimum solution. Even if we had the same simplified cost assumptions as
above, we would have to sum up the probabilities for all possible cases, which is not practical
in most cases.
From these considerations, it follows that our approach cannot be used for deciding whether
or not to contact another broker for processing the current query. So we would have to use other
(heuristic) criteria for this decision. For this purpose, we could use the number of relevant documents
that can be accessed via broker B. This figure can be estimated by both methods mentioned above,
where the latter also gives us the variance; thus, we could decide to access B only in case its number
of relevant documents exceeds a specific threshold with a certain probability.
Now assume that we have such a decision procedure that determines which brokers to access and
which not. How can we achieve an optimum selection of databases in this case?
The clue to the solution of this problem lies in the divide-and-conquer algorithm described in
section 4. Let us assume that different brokers access disjoint sets of databases. Then we can perform
the divide step according to the broker structure. Thus, if broker A contacts brokers B and C, then
both B and C first determine their local optimum cost functions C R (n) and send them back to
A. For A, these two functions (we could also add additional overhead costs for invoking another
broker) are treated in the same way as those of its local databases in order to arrive at a global
optimum. Thus, for a specific number of relevant documents, A determines how many documents
to be requested from B and C, which, in turn, split up these requests for their local databases. Of
course, the whole procedure can be invoked recursively in case B or C also contact other brokers,
and so on.
In order to apply this procedure for the case where databases may be "served" by more than
one broker, we have to devise a communication structure that considers each database at most once.
This is most important for determining the optimum cost function, but also plays a role in estimating
the number of relevant documents accessible via a certain broker.
First, let us assume that two brokers which know each other also synchronize their metaknowledge
about databases. That is, if they have "overlapping" metaknowledge referring to the same databases
or other brokers, then each of them passes only information about the nonoverlapping sources to the
other. However, this local strategy is not able to avoid any overlapping of a pair of brokers, since
both may be linked indirectly to the same knowledge source.
For the task of determining the optimum cost function, this problem can be solved dynamically
in the following way: When a broker B sends its local optimum function back to the requesting
broker A, it appends the identifications of the databases involved in the optimum solution. Thus, A
will be able to detect duplicate databases, and request a revised cost function without these specific
databases. In addition, B also could store this information in order to be considered for future
requests from A (assuming that A always calls the same set of brokers, and that brokers never drop
a database).
Finally, metaknowledge propagation also raises the issue of maximum propagation distance. If
for any pair of related brokers (A; B), A has metaknowledge about everything B knows, then any
broker will have metaknowledge about all sources that can be reached directly or indirectly. On the
other hand, this means that knowledge about a new database is propagated to all brokers in the
world! One possible strategy for overcoming this situation might be that a broker does not pass on
metaknowledge that it has about other brokers.
6 Related work
The Gloss system described in [Gravano et al. 94] and [Gravano & Garcia-Molina 95] is similar
to our approach in that it also focuses on the task of database selection. However, Gloss is based
on the vector space model and thus does not refer to the concept of relevance, which is explicitly
justified in [Gravano & Garcia-Molina 95]: "To see a problem with this rank, consider a database
db that contains, say, three relevant documents for some query q. Unfortunately, it turns out that
the search engine at db does not include any of these documents in the answer to q. So the user will
not benefit from these three relevant documents." Obviously, the authors of Gloss are unaware of
probabilistic IR models. Thus, they come up with a heuristic solution: They define several measures
of goodness, one of which is the sum of RSVs of all documents in a database w.r.t. the current query.
Actually, this parameter is equivalent to the estimated number of relevant documents as derived in
the appendix of this paper. The experiments performed with Gloss confirm the general applicability
of this strategy; however, the evaluation was restricted in that only the distribution of RSVs was
considered, and no attempt was made to consider relevance in terms of retrieval quality. Since Gloss
ranks databases only according to values of this goodness measure, this corresponds to the most
simple case in our model, where the cost parameters as well as the recall-precision curves are the
same for all databases.
The approach presented in [Callan et al. 95] covers both problems of database selection and
collection fusion. With respect to the first task, a collection ranking formula is developed; this
formula is similar to document retrieval based on tf \Delta idf weighting, where collections are treated like
documents. In order to evaluate the quality of this ranking, it is compared with the ideal ranking
(based on the actual number of relevant documents) by summing up the mean squared error between
the ideal rank number and the actual rank number (assigned by the algorithm) of each collection.
However, considering the ranks only ignores the differences in the number of relevant documents;
e.g. two subsequent ranks may differ by a single relevant document or by a magnitude in these
numbers. In contrast, using the cost measures described in this paper, a more objective and realistic
evaluation of database selection is possible.
Both approaches mentioned before as well as the estimation formula given in the appendix of
this paper are based on a term-wise weighting of collections. A query-based strategy is described in
[Voorhees et al. 95]. Here relevance feedback information from a set of training queries is used in
order to rank collections for a new query.
The Harvest system ([Bowman et al. 94]) is a distributed search system for data on the internet.
A 'gatherer' collects information about the local data and passes it to a broker. Thus, the Harvest
broker contains indexes of documents, whereas the brokers considered here only store indexes of
databases (i.e. the dictionaries), which is much less data. Harvest brokers may be organized in
hierarchies. However, Harvest leaves the broker specification open, in order to allow for different
designs.
In the system design of the University of Michigan Digital Libary project ([Atkins et al. 96]),
there is a 'mediator' which, among other tasks, also has to perform database selection. The metadata
describing a source contains information about content and prices as well as about the funtionality
and the interface of the search engine managing the source. So far, no description of the selection
algorithm used has been published.
There are also some approaches (e.g. [Danzig et al. 92]) which tackle the problem of database
selection from the opposite side: By assuming that the overall system may decide in which database
a new document should be placed, the goal is to maximize the differences in content between the
databases involved. Thus, the task of database selection becomes easier. However, these approaches
are more in the spirit of distributed database systems where data placement is related to efficiency
issues.
7 Conclusions and outlook
Database selection is a major problem in networked information retrieval. In contrast to other,
more heuristic approaches, we have developed a theoretical model which yields an optimum solution
in a decision-theoretic sense. In order to apply this model, the cost factors for query processing
and document delivery, the expected recall-precision curve and the expected number of relevant
documents must be known for each database. The estimation of the latter two parameters is crucial
for a successful application of our model. We have started experimental investigations of these issues.
As the major benefit of the work presented here, we view our model as a framework for all
approaches dealing with database selection. Since our model comprises all relevant parameters, it
helps in making the task of database selection more transparent. This is important in analyzing
experimental results, e.g. determining whether the estimation of the number of relevant documents or
of the recall-precision curve failed. As we have shown, most work done so far works with simplified
assumptions with respect to the parameters involved in our model. Thus, we may distinguish
different classes of tasks, ranging from rather simplified approaches to the most general task where
all parameters have to be considered.
The approach described here is currently being implemented as part of the information brokering
system developed within the German digital library project MeDoc (see [Boles et al. 96]). MeDoc
aims at the development of a virtual digital library (for computer science) involving fulltext as well
as bibliographic databases. The brokering system has to provide access to a range of heterogeneous
systems, e.g. WAIS, NCSTRL and Z39.50-based systems. A first prototype of the system has
been released recently.
The grand challenge that we are facing currently is the availability of numerous databases already
established on the Internet, where we have no possibility to change the placement of documents,
neither the indexing nor the retrieval method used in a system, and the task is to perform efficient
and effective retrieval while giving the user the impression of a single large, virtual database.



--R

Toward Inquiry-Based Education Through Interacting Software Agents
MeDoc Information Broker - Harnessing the Information in Literature and Full Text Databases

http://SunSite.
Outline of a General Probabilistic Retrieval Model.
Harvest: A Scalable

Searching Distributed Collections with Interference Networks.
Distributed Indexing of Autonomous Internet Services.

Optimum Polynomial Retrieval Functions Based on the Probability Ranking Principle.
Extending Probabilistic Datalog.
Database Selection in Networked Information Retrieval Systems.
Generalizing GIOSS to Vector-Space Databases and Broker Hierarchies
The Effectiveness of GlOSS for the Text Database Discovery Problem.
Overview of the Second Text Retrieval Conference (TREC-2)
Interfaces for Distributed Systems of Information Servers.

The Probability Ranking Principle in IR.
Evaluation of an Inference Network-Based Retrieval Model
Learning Collection Fusion Strategies.

On Modeling Information Retrieval with Probabilistic Inference.



The expected number of relevant documents in D can be approximated by E(reljq

--TR
Optimum polynomial retrieval functions based on the probability ranking principle
Evaluation of an inference network-based retrieval model
Interfaces for distributed systems of information servers
The effectiveness of GIOSS for the text database discovery problem
On modeling information retrieval with probabilistic inference
Overview of the second text retrieval conference (TREC-2)
Searching distributed collections with inference networks
Learning collection fusion strategies
The Harvest information discovery and access system
Object-oriented and database concepts for the design of networked information retrieval systems
A probabilistic model for distributed information retrieval
Students access books and journals through MeDoc
Effective retrieval with distributed collections
Evaluating database selection techniques
Toward Inquiry-Based Education Through Interacting Software Agents
Generalizing GlOSS to Vector-Space Databases and Broker Hierarchies
Provider Selection - Design and Implementation of the Medoc Broker

--CTR
Jamie Callan , Fabio Crestani , Henrik Nottelmann , Pietro Pala , Xiao Mang Shou, Resource selection and data fusion in multimedia distributed digital libraries, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada
Leif Azzopardi , Mark Baillie , Fabio Crestani, Adaptive query-based sampling for distributed IR, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA
Luo Si , Jamie Callan, Using sampled data and regression to merge search engine results, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland
James C. French , Allison L. Powell, Metrics for evaluating database selection techniques, World Wide Web, v.3 n.3, p.153-163, 2000
Henrik Nottelmann , Norbert Fuhr, Evaluating different methods of estimating retrieval quality for resource selection, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada
Kartik Hosanagar, A utility theoretic approach to determining optimal wait times in distributed information retrieval, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil
Shengli Wu , Fabio Crestani, Distributed information retrieval: a multi-objective resource selection approach, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, v.11 n.Supplement, p.83-99, September
Unified utility maximization framework for resource selection, Proceedings of the thirteenth ACM international conference on Information and knowledge management, November 08-13, 2004, Washington, D.C., USA
Fabio Crestani , Shengli Wu, Testing the cluster hypothesis in distributed information retrieval, Information Processing and Management: an International Journal, v.42 n.5, p.1137-1150, September 2006
Jared Cope , Nick Craswell , David Hawking, Automated discovery of search interfaces on the web, Proceedings of the fourteenth Australasian database conference, p.181-189, February 01, 2003, Adelaide, Australia
Sally McClean, Result merging methods in distributed information retrieval with overlapping databases, Information Retrieval, v.10 n.3, p.297-319, June      2007
Christoph Baumgarten, Retrieving Information from a Distributed Heterogeneous Document Collection, Information Retrieval, v.3 n.3, p.253-271, October 2000
Modeling search engine effectiveness for federated search, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil
Rinat Khoussainov , Nicholas Kushmerick, Specialisation dynamics in federated web search, Proceedings of the 6th annual ACM international workshop on Web information and data management, November 12-13, 2004, Washington DC, USA
Panagiotis G. Ipeirotis , Luis Gravano, When one sample is not improving text database selection using shrinkage, Proceedings of the 2004 ACM SIGMOD international conference on Management of data, June 13-18, 2004, Paris, France
semisupervised learning method to merge search engine results, ACM Transactions on Information Systems (TOIS), v.21 n.4, p.457-491, October
Nick Craswell , Peter Bailey , David Hawking, Server selection on the World Wide Web, Proceedings of the fifth ACM conference on Digital libraries, p.37-46, June 02-07, 2000, San Antonio, Texas, United States
Allison L. Powell , James C. French , Jamie Callan , Margaret Connell , Charles L. Viles, The impact of database selection on distributed searching, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.232-239, July 24-28, 2000, Athens, Greece
James Caverlee , Ling Liu , Joonsoo Bae, Distributed query sampling: a quality-conscious approach, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA
Clement Yu , King-Lup Liu , Weiyi Meng , Zonghuan Wu , Naphtali Rishe, A Methodology to Retrieve Text Documents from Multiple Databases, IEEE Transactions on Knowledge and Data Engineering, v.14 n.6, p.1347-1361, November 2002
James C. French , A. C. Chapin , Worthy N. Martin, Multiple viewpoints as an approach to digital library interfaces, Journal of the American Society for Information Science and Technology, v.55 n.10, p.911-922, August 2004
Matthias Bender , Sebastian Michel , Peter Triantafillou , Gerhard Weikum , Christian Zimmer, Improving collection selection with overlap awareness in P2P search engines, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil
James C. French , Allison L. Powell , Jamie Callan , Charles L. Viles , Travis Emmitt , Kevin J. Prey , Yun Mou, Comparing the performance of database selection algorithms, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.238-245, August 15-19, 1999, Berkeley, California, United States
Panagiotis G. Ipeirotis , Luis Gravano, Distributed search over the hidden web: hierarchical database sampling and selection, Proceedings of the 28th international conference on Very Large Data Bases, p.394-405, August 20-23, 2002, Hong Kong, China
Josiane Xavier Parreira , Sebastian Michel , Matthias Bender, Size doesn't always matter: exploiting pageRank for query routing in distributed IR, Proceedings of the international workshop on Information retrieval in peer-to-peer networks, November 11-11, 2006, Arlington, Virginia, USA
Fang Liu , Clement Yu , Weiyi Meng, Personalized Web Search For Improving Retrieval Effectiveness, IEEE Transactions on Knowledge and Data Engineering, v.16 n.1, p.28-40, January 2004
James C. French , Allison L. Powell , Fredric Gey , Natalia Perelman, Exploiting a controlled vocabulary to improve collection selection and retrieval effectiveness, Proceedings of the tenth international conference on Information and knowledge management, October 05-10, 2001, Atlanta, Georgia, USA
Jamie Callan , Margaret Connell, Query-based sampling of text databases, ACM Transactions on Information Systems (TOIS), v.19 n.2, p.97-130, April 2001
Henrik Nottelmann , Gudrun Fischer, Search and browse services for heterogeneous collections with the peer-to-peer network Pepper, Information Processing and Management: an International Journal, v.43 n.3, p.624-642, May, 2007
Jack G. Conrad , Xi S. Guo , Peter Jackson , Monem Meziou, Database selection using actual physical and acquired logical collection resources in a massive domain-specific operational environment, Proceedings of the 28th international conference on Very Large Data Bases, p.71-82, August 20-23, 2002, Hong Kong, China
James C. French , Allison L. Powell , Fredric Gey , Natalia Perelman, Exploiting Manual Indexing to Improve Collection Selection and Retrieval Effectiveness, Information Retrieval, v.5 n.4, p.323-351, October 2002
M. Elena Renda , Umberto Straccia, Automatic structured query transformation over distributed digital libraries, Proceedings of the 2006 ACM symposium on Applied computing, April 23-27, 2006, Dijon, France
Henri Avancini , Leonardo Candela , Umberto Straccia, Recommenders in a personalized, collaborative digital library environment, Journal of Intelligent Information Systems, v.28 n.3, p.253-283, June      2007
Milad Shokouhi , Justin Zobel , Saied Tahaghoghi , Falk Scholer, Using query logs to establish vocabularies in distributed information retrieval, Information Processing and Management: an International Journal, v.43 n.1, p.169-180, January 2007
Allison L. Powell , James C. French, Comparing the performance of collection selection algorithms, ACM Transactions on Information Systems (TOIS), v.21 n.4, p.412-456, October
Sebastian Michel , Matthias Bender , Nikos Ntarmos , Peter Triantafillou , Gerhard Weikum , Christian Zimmer, Discovering and exploiting keyword and attribute-value co-occurrences to improve P2P routing indices, Proceedings of the 15th ACM international conference on Information and knowledge management, November 06-11, 2006, Arlington, Virginia, USA
Josiane Xavier Parreira , Sebastian Michel , Gerhard Weikum, p2pDating: Real life inspired semantic overlay networks for Web search, Information Processing and Management: an International Journal, v.43 n.3, p.643-664, May, 2007
Yannis Tzitzikas , Nicolas Spyratos , Panos Constantopoulos, Mediators over taxonomy-based information sources, The VLDB Journal  The International Journal on Very Large Data Bases, v.14 n.1, p.112-136, March 2005
Thomas Eiter , Michael Fink , Hans Tompits, A knowledge-based approach for selecting information sources, Theory and Practice of Logic Programming, v.7 n.3, p.249-300, May 2007
Robert M. Losee , Lewis Church Jr., Information Retrieval with Distributed Databases: Analytic Models of Performance, IEEE Transactions on Parallel and Distributed Systems, v.15 n.1, p.18-27, January 2004
resource discovery;probabilistic retrieval;probability ranking principle;networked retrieval