OPTIMAL DESIGNS FOR THE GENERALIZED PARTIAL CREDIT

arXiv:1803.06517v1 [] 17 Mar 2018

MODEL

Paul-Christian Bürkner1 , Rainer Schwabe2 , Heinz Holling1
1
2

institute of psychology, university of münster, germany

institute of mathematical stochastics, university of magdeburg, germany

March 20, 2018

Correspondence should be sent to paul.buerkner@gmail.com

2

OPTIMAL DESIGNS FOR THE GENERALIZED PARTIAL CREDIT MODEL
Abstract
Analyzing ordinal data becomes increasingly important in psychology, especially in
the context of item response theory. The generalized partial credit model (GPCM) is
probably the most widely used ordinal model and finds application in many large scale
educational assessment studies such as PISA. In the present paper, optimal test designs
are investigated for estimating persons’ abilities with the GPCM for calibrated tests
when item parameters are known from previous studies. We will derive that local
optimality may be achieved by assigning non-zero probability only to the first and last
category independently of a person’s ability. That is, when using such a design, the
GPCM reduces to the dichotomous 2PL model. Since locally optimal designs require
the true ability to be known, we consider alternative Bayesian design criteria using
weight distributions over the ability parameter space. For symmetric weight
distributions, we derive necessary conditions for the optimal one-point design of two
response categories to be Bayes optimal. Furthermore, we discuss examples of common
symmetric weight distributions and investigate, in which cases the necessary conditions
are also sufficient. Since the 2PL model is a special case of the GPCM, all of these
results hold for the 2PL model as well.
Key words: optimal design; Bayesian design; partial credit model; 2PL model; Rasch
model; item response theory.

3
1. Introduction
Item response theory (IRT) provides a flexible and powerful approach for designing and
analyzing data of psychological tests. While IRT very often deals with dichotomous responses
(e.g., categorized as either 1 ’right’ or 0 ’wrong’), there are many situations where a more
advanced scoring is to be advised. Consider, for instance, a complicated task in which multiple
intermediate results have to be obtained to correctly solve the whole task. Scoring such a task in
a dichotomous way comes with substantial loss of information as someone solving all intermediate
questions failing only in the last step would receive the same score as someone who did not even
manage to take the first step. A natural solution is to give persons partial credit for the
intermediate results. The obtained response is no longer dichotomous but ordinal ranging from 0
(nothing correct), over 1 (first step correct) to J (all J steps correct). An IRT model dealing with
such ordinal responses is the Partial Credit Model (PCM; Andersen, 1973; Andrich, 1978) or its
extention, the generalized partial credit model (GPCM; Muraki, 1992, 1993). It is a
mathematically convenient generalization of the dichotomous 2PL model (which is in turn a
generalization of the Rasch model) for more than two possible response categories. As such, it
combines the ability of an examinee with the item difficulty and discrimination and maps them to
the probability of each response category.
The GPCM is probably the most frequently used ordinal model in psychological research.
Among others, it has been applied in many large scale educational assessment studies such as
PISA, TIMSS, PIRLS, or NAEP (OECD, 2017; Martin, Gregory, & Stemler, 2000; Allen,
Donoghue, & Schoeps, 2001). In addition, there is a great body of literature discussing
application of the GPCM in large scale assessment studies from a more technical perspective (e.g.,
see Mazzeo & von Davier, 2014; von Davier & Sinharay, 2010; Von Davier & Sinharay, 2013).
These studies were mostly concerned with comparability of large scale results as well as model
fitting itself, while to our knowledge optimizing the statistical properties of the resulting
parameter estimates has not yet been systematically investigated yet. We believe that given the
GPCM’s broad application in practice, it is of great relevance to investigate under which
conditions the GPCM performs best.

4
Applying principles of optimal design (Berger & Wong, 2009; Atkinson, Donev, & Tobias,
2007; Holling & Schwabe, 2013) to IRT models can lead to considerable efficiency gains and
reduce costs of test administration by reducing the number of items and / or subjects required for
achieving the same level of precision (Holling & Schwabe, 2016). In IRT, one typically
distinguishes between two types of optimal design problems. The first is about selecting items
with specific properties for the efficient estimation of person parameters, while the second is
about selecting persons with specific abilities for the efficient estimation of item parameters.
These problems are referred to as optimal test design and optimal sampling design, respectively
(Holling & Schwabe, 2016), with the former being the more common one (van der Linden, 2006).
Optimal test designs are relevant in so called calibrated tests, in which item parameters were
estimated in prior studies and can thus be considered as known with reasonable precision. The
design question is to select the items from the calibrated item pool most appropriate for the
participating persons. Optimal test designs are also applicable in tests making use of automatic
item generation (e.g., see Geerlings, Glas, & van der Linden, 2011). In this case, the item
parameters can be inferred from the set of rules, which were used to create the items, provided
that the influence of the rules on the item parameters were investigated before.
Most IRT models, including the GPCM, are non-linear in the parameters so that the related
optimal designs are dependent on the parameters to be estimated. Such optimal designs that are
optimal only for certain parameter values – but not for others – are called locally optimal designs.
Among others, locally optimal designs for the GPCM will be investigated in the present paper.
Despite being relevant for theoretical reasons and often being a prerequisite more advanced
optimal designs, locally optimal designs are of limited practical use themselves as one has to guess
the true value of the parameter before constructing the design. If the guess is far away from the
truth, such locally optimal design might become relatively inefficient.
There are different possibilities to overcome the problem of locally optimal designs. One
solution considered in the present paper are so called Bayesian optimal designs. Instead of trying
to find an optimal design for a specific parameter value, one specifies a weight distribution over
the set of possible parameters to express the uncertainty in the subject or item parameters. For
instance, when generating a test design, one might assume that a subject’s ability will be

5
somewhere between −2 and 2 with all abilities in between being equally likely. This would
translate into a uniform distribution over the interval [−2, 2]. A design that is optimal given the
specified uncertainty in the parameter values is then called Bayes optimal.
The paper is structured as follows. In Section 2 and 3, the GPCM is introduced in detail and
locally optimal designs are investigated. In Section 4, we introduce and discuss Bayesian optimal
designs in relation to the GPCM. Examples for common weight distributions are given in Section
5. We end with a discussion of the findings in Section 6. All proofs are provided in the Appendix.
2. The Model
In a test situation with ordinal items, the response of a person p on item i is symbolized by
Ypi , which can take on one of J + 1 values ranging from 0 to J. In the GPCM, the probability
πpij that person p achieves category j ∈ {0, ..., J} on item i is given by (cf. Muraki, 1992, 1993):
exp
πpij := πj (θp , τi , αi ) := P (Ypi = j; θp , τi , αi ) := P
J

P

k=0 exp

with

0
X

j
s=1 αis (θp


− τis )

P

k
s=1 αis (θp


− τis )

αis (θp − τis ) := 0

(1)

(2)

s=1

for notational convenience. The model results from the assumption that, given only two
adjacent categories, the probability for the higher of the two is given by the dichotomous 2PL
model:
P (Y = j; θp , τi , αi |Ypi ∈ {j, j − 1}) =

exp(αij (θp − τij ))
.
1 + exp(αij (θp − τij ))

(3)

The parameter θp denotes the ability of the pth person. The higher θp , the higher the
probability for reaching higher categories (Agresti, 2010). The parameter vector τi = (τi1 , ..., τiJ )
denotes the so called thresholds of the ith item. Thresholds in the GPCM can be interpreted as
follows. If a person has an ability equal to τij , the probabilities for this person of achieving
category j and j − 1 are equal, that is πpij = πpi(j−1) (i.e. τij are the intersection points of the
respective category response curves). The higher the thresholds, the lower the probability for
achieving higher categories. This does not imply, however, that thresholds must be ordered in the

6
sense that τ1 ≤ τ2 ≤ ... ≤ τJ (Adams, Wu, & Wilson, 2012). Rather, all thresholds may take on
any real value. The vector αi = (αi1 , ..., αiJ ) > 0 denotes the positive discrimination parameters
of the ith item. The higher αij , the steeper the curve favoring the jth category over the (j − 1)th
category with increasing θp . Since the focus of the present paper is on optimal test designs, we
consider the item parameters τi and αi to be known (i.e. chosen by the design) and the person
parameters θp as the quantities to be estimated. This situation arises in calibrated tests where
item parameters were estimated in prior studies and the design question is to select the most
appropriate items from the item pool. Assuming conditional independence of the items for given
θp , the joint density of all items is simply the product of the single-item densities (see van der
Linden & Glas, 2010 for a detailed discussion about conditional independence in IRT models). For
notational convenience, we will suppress the indices p and i in the following where appropriate.
The original PCM was first derived by Rasch (1961) and subsequently by Andersen (1973),
Andrich (1978), Masters (1982), and Fischer (1995) each with a different but equivalent
formulation (considering the special case of discrimination parameters fixed to one; c.f. Fischer,
1995; Adams et al., 2012). Andersen (1973) and Fischer (1995) derived the PCM in an effort to
find a model that allows the independent estimation of person and item parameters – a highly
desirable property – for ordinal variables. Andrich (1978, 2005) provided another derivation:
When two dichotomous processes are independent, four results can occur: (0, 0), (1, 0), (0, 1), (1, 1).
Using the Rasch model for each of the two processes, the probability of the combined outcomes is
given by the Polytomous Rasch Model (Andersen, 1973; c.f. Wilson, 1992; Wilson & Adams,
1993). When thinking of these processes as steps between ordered categories, (0, 0) corresponds to
Y = 0, (1, 0) corresponds to Y = 1, and (1, 1) corresponds to Y = 2. The event (0, 1), however, is
assumed to be impossible because the second step cannot be successful when the first step was
not. For an arbitrary number of ordered categories, Andrich (1978) proved that the Polytoumous
Rasch Model becomes the PCM (with discrimination parameters fixed to one) when considering
the set of possible events only. The GPCM, which generalizes the 2PL model to more than two
ordinal response categories, was later proposed by Muraki (1992).

7
3. Locally Optimal Designs
In the context of optimal test designs for the GPCM, an experimental design is the set
ξ := {(τ1 , α1 ), (τ2 , α2 ), ..., (τN , αN )} of known parameters of the N administrated items. It can be
chosen by the experimenter in order to optimize the information of the experiment. Note that
each τi and each αi (i ∈ {1, ..., N }) is a vector of length J. The set of all designs is denoted as Ξ.
When only considering a single item, we simply write (τ, α) instead of ξ and drop the index i.
There are several optimal design criteria discussed in the literature (see Atkinson et al., 2007
for an overview), but arguably the most common criterion is D-optimality. A design ξ is called
D-optimal if it minimizes the determinant of the estimator’s covariance matrix. In the present
context, the parameter θ to be estimated is the unidimensional person parameter so that the
determinant of the covariance matrix reduces to the unidimensional variance. Thus, we can define
a D-optimal design as
ξ ∗ := arg min Var(θ̂; ξ),

(4)

ξ∈Ξ

where θ̂ is an estimator of the parameter θ. According to the Cramér-Rao bound (Cramér, 2016;
Rao, 1992), the variance of an unbiased estimator cannot be smaller than the inverse of the
Fisher-Information M :
Var(θ̂, ξ) ≥ M (θ, ξ)−1

(5)

Thus, for (asymptotically) efficient estimators, we can equivalently maximize the
Fisher-Information instead of minimizing the estimator’s variance. Under certain regularity
conditions (cf. Lehmann & Casella, 2006), M can be written as


 2

d
d
M (θ, ξ) := Var
log f (Y ; θ, ξ) = − E
log f (Y ; θ, ξ) ,
dθ
d θ2

(6)

where f (Y ; θ, ξ) is the product density of all items given the ability parameter θ and the design ξ
assuming conditional independence of the items. Due to additivity of the Fisher-Information in
this case, we could equivalently write (6) as the sum of the single item Fisher-Informations. Most
often in optimal design theory, the aim is to minimize the variance of the estimator. In case of
asymptotic efficient estimators (such as maximum-likelihood estimators), which by definition

8
meet the Cramér-Rao bound, this is equivalent to maximizing the Fisher-Information and we will
use this approach in the derivation of optimal designs in the present paper.
Proposition 1. Defining Aj :=

Pj

s=1 αs ,

the Fisher information M (θ, τ, α) of a single item

following the GPCM is given by
M (θ, τ, α) =

J
X


A2j πj − 

j=1

J
X

2
Aj π j  .

(7)

j=1

As an immediate result of Proposition 1 we obtain:
Theorem 1. The Fisher information M of a single item has a unique global maximum in
P
∗
= 0.
{π ∈ RJ+1
| Jj=0 πj = 1}, which is given by π0∗ = πJ∗ = 1/2 and π1∗ = π2∗ = ... = πJ−1
+
Theorem 1 implies that an item following the GPCM would be optimal for a person with
ability θ0 , when the threshold vectors τ and α are chosen so that only the first and last category
have non-zero probability and are equally likely. In other words, such an item is optimal if it is
from the dichotomous 2PL model (with discrimination α = AJ ). Due to additivity of the Fisher
information, all items of a locally optimal design ξ ∗ for the PCM have to satisfy the above
condition.
We want to briefly illustrate the efficiency gain achieved by optimal items with a simple
example. Suppose we administer a four category item with threshold vector τ = (−1, 0, 1) and
discrimination vector α = (1, 1, 1). Then, participants with low (θ = −1), medium (θ = 0), and
high (θ = 1) ability achieve the categories 0 to 3 with probabilities πl ≈ (0.41, 0.41, 0.15, 0.02),
πm ≈ (0.13, 0.37, 0.37, 0.13), and πh ≈ (0.02, 0.15, 0.41, 0.41), respectively. Such values are not too
uncommon in practice. When compared to an optimal item, the ratio of the Fisher-information is
M ∗ /Mm ≈ 2.86 for participants with medium ability, and M ∗ /Ml = M ∗ /Mh ≈ 3.75 for
participants with low or high ability. Hence, the variance of an efficient estimator is about three
to four times higher when applying this item rather than optimal items tailored to the
participants’ abilities.
Unfortunately, there exists no item with a threshold vector τ ∗ that leads to the optimal
probabilities of Theorem 1, but it may be approximated:

9
Proposition 2. Define τ̃α (c) := (αJ c, 0, 0, ..., −α1 c) ∈ RJ with c > 0 and


J
J
Cα := x ∈ R | ∀k ∈ {1, ..., J} : αk xk = −αJ−k+1 xJ−k+1

(8)

for some discrimination vector α ∈ RJ+ .
(a) It is τ̃α (c) ∈ CαJ .
(b) For each θ ∈ R there is some ρ = ρ(θ, α) with 0 < ρ < 1 such that πJ (θ, τ̃α (c), α) → ρ,
π0 (θ, τ̃α (c), α) → 1 − ρ and πj (θ, τ̃α (c), α) → 0 for 0 < j < J as c → ∞.
(c) It is ρ(0, α) = 1/2.
(d) For τ ∈ CαJ with symmetric α satisfying αk = αJ−k+1 it holds that
πj (θ, τ, α) = πJ−j (−θ, τ, α) for all j ∈ {0, ..., J}.
According to Proposition 2 (b) and (c), we can formally define τ ∗ as τ ∗ := limc→∞ τ̃α (c)
when assuming θ0 = 0, which can be achieved by a location shift without loss of generality.
Proposition 2 may be generalized to arbitrary θ0 , but because of the location shift argument, this
is not required for the present paper. Of course, it is impossible to create items with a threshold
vector equal to τ ∗ . That is for a fixed number of more than two response categories, a locally
optimal design does not exist. However, one may choose reasonably large values of c so that the
GPCM closely approximates the dichotomous 2PL model with corresponding locally optimal
design. The latter is well known as a one-point design, in which all items have difficulty equal to
the persons ability and as high as possible discrimination. This may also be inferred directly from
Proposition 1 when considering the special case of only two response categories.
4. Bayesian Optimal Designs
From a Bayesian perspective, a locally optimal design is the result of setting all prior mass to
one point θ0 . This seems quite infeasible and is only justified if the true ability is not too far away
from θ0 or otherwise the locally optimal design in θ0 might perform poorly. One solution is to
account for the a-priori uncertainty in the ability parameter by imposing a non-degenerate weight
distribution Π over the parameter space Θ. An optimal design ξ ∗ ∈ Ξ taking this uncertainty into

10
account is called a Bayesian optimal design. It is obtained via some sort of averaging over the
locally optimal designs’ Fisher information, where the Fisher information of each locally optimal
design is weighted according to the weight distribution (Atkinson et al., 2007). In the present
paper, we will consider two common Bayesian design criteria that each perform a different kind of
averaging (Firth & Hinde, 1997):
ψ0 (ξ) := E [log(M (θ, ξ))] ,


ψ−1 (ξ) := − E M (θ, ξ)−1 ,

(9)
(10)

where
Z
E [X] =

X Π(θ) d θ

(11)

is the expected value of a random variable X under the distribution Π of θ. The criterion ψ0
considers the average logarithm of the Fisher information, while the criterion ψ−1 aims at
minimizing the average asymptotic variance. Of those two, ψ0 is mathematically more convenient
and has a natural Bayesian interpretation (Chaloner & Larntz, 1989). Both criteria are concave
(Firth & Hinde, 1997). Note that, in general, these design criteria are not fully Bayesian in the
sense that they do not require computing a posterior distribution. Instead, they borrow the
concept of prior distributions to obtain robust designs for frequentist analysis via maximum
likelihood methods. Hence, the here discussed designs are sometimes referred to as
’pseudo-Bayesian’ designs (Firth & Hinde, 1997). For a review of fully Bayesian design criteria see
Chaloner and Verdinelli (1995).
In theory, the weight distribution Π of θ may have any form. In practice, however, one
typically uses symmetric weights, both for mathematical convenience and because there is rarely
any prior information available that favors an asymmetric weight distribution for an unbounded
parameter such as the ability θ. Accordingly, we will focus on symmetric weight distributions in
the following. Within this class, there are – among others – the continuous uniform, normal, and
logistic distribution, each forming a so called location scale family of distributions.
For the Rasch model, Bayesian optimal designs have already been investigated in Graßhoff,
Holling, and Schwabe (2012) and the present section aims at generalizing these results to the

11
GPCM. The following Lemma underlines the relevance of locally optimal designs in the context of
Bayesian optimal design criteria.
Lemma 1. If the weight distribution Π is symmetric around some ability θ0 , α is fixed to any
value, and τ is considered variable, the optimal one-point design with respect to ψ0 and ψ−1 is
the locally optimal design for θ0 that is τ = τ ∗ .
According to Lemma 1, the same problem that occurs for the locally optimal design occurs
for the optimal one-point design under the Bayesian criteria as well. That is, it does not exist for
a fixed number of more than two response categories and when considering the number of
categories variable, the GPCM reduces to the dichotomous 2PL model. Accordingly, we need to
investigate the optimal discrimination parameter α in case of only two response categories:
Lemma 2. If the weight distribution Π is symmetric around some ability θ0 , τ is fixed to τ ∗ ,
and α ∈ R+ is considered variable, it holds that:
(a) ψ0 is maximized by the unique solution α0∗ of
Z
α

θ π1 (θ, τ ∗ , α) Π(θ) d θ = 1.

(12)

∗ of
(b) ψ−1 is maximized by the unique solution α−1

Z
α

θ
Π(θ) d θ =
π0 (θ, τ ∗ , α)

Z
π1

1
Π(θ) d θ.
π0 (θ, τ ∗ , α)

(θ, τ ∗ , α)

(13)

Lemma 2 is in line with our intuition that the optimal α decreases with increasing scale of Π.
Lemmas 1 and 2 do not state under which conditions the optimal one-point design is Bayes
optimal. To answer this question, we have to make use of approximate design theory (cf. Kiefer,
1974) and introduce so called sensitivity functions measuring the quality of a design (cf. Graßhoff
et al., 2012). If the sensitivity function is uniformly bounded at a certain value for a given design
ξ ∗ , no improvement is possible and ξ ∗ will be optimal. For non-linear models, sensitivity functions
of D-optimal designs are given in Firth and Hinde (1997). In case of the GPCM, these sensitivity

12
functions can be written as


E M (ξ, θ)v M (ξ, θ)−1 M (τ, α, θ)
φv (τ, α, ξ) =
− 1,
E [M (ξ, θ)v ]

(14)

where v ∈ {−1, 0} corresponds to the two Bayesian design criteria discussed above. Since, for the
GPCM, the design criteria are concave, Bayes optimality of a design ξ ∗ is equivalent to
sup φv (τ, α, ξ ∗ ) = 0,

(15)

(τ,α)∈X

where X denotes the design region of a single item. Moreover, when ξ ∗ is optimal, φv (τ, α, ξ ∗ ) = 0
holds if and only if (τ, α) is a design point of ξ ∗ . It may now be asked, under which conditions the
optimal one-point design is also Bayes optimal.
Theorem 2. Necessary conditions for the threshold τ ∗ to be Bayes optimal under a
symmetric weight distribution Π and fixed discrimination α are
Z
1
ψ0 :
πJ (θ, τ ∗ , α) π0 (θ, τ ∗ , α) Π(θ) d θ ≥
6
Z
ψ−1 :
(πJ (θ, τ ∗ , α) π0 (θ, τ ∗ , α))−1 Π(θ) d θ ≤ 6

(16)
(17)

For a fixed discrimination of α = 1, Theorem 2 provides the same conditions for the GPCM
as can be found for the Rasch model (see Graßhoff et al., 2012). By Jensen’s inequality, the
necessary condition for ψ−1 implies the condition for ψ0 , that is the former condition is more
restrictive. For scale families of symmetric weight distributions, Theorem 2 may be further
extended.
Theorem 3. Let (Πs )s>0 be a scale family of symmetric distributions around some ability θ0 .
(a) There exist unique values s−1 and s0 with 0 < s−1 ≤ s0 ≤ ∞, such that the necessary
conditions of Theorem 2 on ψ−1 and ψ0 are satisfied for all s ≤ s−1 and s ≤ s0 , respectively,
and violated otherwise.
(b) There exists a unique threshold s̃0 , 0 ≤ s̃0 < s0 , such that the optimal one-point design τ ∗ is
Bayes optimal with respect to ψ0 for all weight distributions Π with s ≤ s̃0 and fails to be
optimal for s > s̃0 .

13
According to Theorem 3, the optimal one-point design for a symmetric scale distribution
remains Bayes optimal as long as the scale parameter does not exceed a certain value that
depends on the design criterion, the discrimination α, and on the location scale family. As
R
πJ π0 Π(θ) d θ decreases monotonically with increasing α, the values s̃0 , s0 , and s−1 decrease
monotonically with increasing α as well. For some common distribution families, examples are
discussed in more detail in the upcoming section.
Lastly, we want to turn our attention to Bayes optimal designs for the dichotomous 2PL
model, when both τ and α are allowed to vary. It is of particular interest, in which cases the
optimal one-point designs derived in Lemma 2 are also Bayes optimal. In next section, we show
numerically that for certain important weight distributions under φ0 or φ−1 , the optimal
one-point design is never Bayes optimal. This implies that the Bayes optimal designs under φ0 or
φ−1 (if existent) have to have at least two distinct design points. Whether this holds in general
for all symmetric weight distributions requires further investigation.
5. Examples
Among others, Graßhoff et al. (2012) discuss Bayes optimal designs for the Rasch model in
case of the continuous uniform, normal, and logistic families. They also derive the family specific
values of s−1 and s0 (named τ1 and τ2 , respectively, in their paper) for the Rasch model (i.e. for
α = 1). We have already established that, when using the optimal one-point design, the GPCM is
equivalent to the 2PL model. We can obtain s0 and s−1 for the GPCM by dividing the respective
values for Rasch model by the sum AJ of all discrimination parameters. Theoretically, this allows
to numerically investigate, whether the necessary conditions on the optimality of the one-point
design are also sufficient, by computing the sensitivity function across the design space.
Practically, this becomes increasingly complicated for items with more response categories, since
the design space is of dimension J (or 2J when also varying α). Accordingly, we will focus on
items with up to three response categories, which may also be visualized conveniently. For each
example, we will consider two cases: (a) three response categories with varying τ and
discrimination parameters fixed to one (hence Aj = j for all j) as well as (b) two response
categories with varying τ and α.

14
Example 1 (Continuous uniform distribution): For the continuous uniform distribution with
support in [θ1 , θ2 ] = [θ0 − s, θ0 + s] and constant density f (θ) =

1
2s ,

we have s−1 ≈ 2.1773/J and

s0 ≈ 2.5757/J. The scalars 2.1773 and 2.5757 are numeric solutions of the equations
exp(s) exp(−s) = 4s and (3 − s) exp(s) = 3 + s, respectively (Graßhoff et al., 2012). For J = 2, the
sensitivity function is displayed in Figure 1 and 2, with s−1 and s0 being displayed in the center.
In the Figures, yellowish regions are values of the sensitivity function that are greater than zero
hence indicating non-optimality of the one-point design. At s−1 and s0 , or smaller values, the
sensitivity functions do not exceed zero, and are unimodal with maximal value zero at τ ∗ . This
demonstrates, at least numerically and for J = 2, that the necessary conditions of Theorem 2 are
also sufficient for the continuous uniform weight distribution. When s is further increased, the
one-point design becomes visibly non-optimal and a two-point design appears to be favored.
[Figure 1 about here.]
[Figure 2 about here.]
∗ ≈ 3.1560 for
For the dichotomous 2PL model with varying τ and α, we have α−1
+
≈ 2,
s = s−1 ≈ 2.1773. However, the necessary condition (17) for φ−1 is only satisfied for α ≤ α−1
∗ ) is not Bayes optimal (see also Figure 3
implying that the optimal one-point design (τ ∗ , α−1

left-hand side). The same result may be obtained for other values of the scale s in the sense that
∗ exceeds α+ independently of s. Similarly, we have α∗ ≈ 3.6186 for s = s ≈ 2.5757, but the
α−1
0
0
−1

necessary condition (16) for φ0 is only satisfied for α ≤ α0+ ≈ 2, (see also Figure 3 right-hand
side). Again, α0∗ > α0+ can be shown to hold for other values of s as well.
[Figure 3 about here.]
Example 2 (Normal distribution): Similar results are obtained for the normal weight
distribution. Here, the scale s is simply the standard deviation parameter of the normal
distribution. The critical values are approximately s−1 ≈ 1.177/J and s0 ≈ 1.683/J. Again,
numerical computation shows that the sensitivity functions do not exceed zero for s ≤ s−1 or
s ≤ s0 (see Figure 5 and 4). Thus, the necessary conditions appear to be sufficient for the normal

15
weight distribution when J = 2. Accordingly, the optimal one-point design remains Bayes optimal
for items with three response categories as long as the standard deviation does not exceed the
critical values.
[Figure 4 about here.]
[Figure 5 about here.]
∗ ≈ 1.3586 for
For the dichotomous 2PL model with varying τ and α, we have α−1
+
s = s−1 ≈ 1.177, but the necessary condition (17) for φ−1 is only satisfied for α ≤ α−1
≈ 1.

Similarly, we have α0∗ ≈ 1.7350 for s = s0 ≈ 1.683, but the necessary condition (16) for φ0 is only
satisfied for α ≤ α0+ ≈ 1.002. This may be shown to hold for other values of s as well.
Example 3 (Logistic distribution): For the logistic distribution with scale parameter s, the
critical values are given by s−1 ≈ 0.603/J and s0 = 1/J. According to Figure 7, the necessary
condition for ψ0 is also sufficient if J = 2. However, the sensitivity function of ψ−1 behaves
somewhat differently for the logistic weight distribution in the sense that the necessary condition
is apparently not sufficient (see Figure 6 center). However, for scale values only slightly below
s−1 , the optimal one-point design appears to be Bayes optimal (see Figure 6 left hand side),
indicating the the best possible sufficient condition, it very close to the necessary condition
derived in the present paper.
∗ ≈ 2.6518 for
For the dichotomous 2PL model with varying τ and α, we have α−1
+
≈ 1.953.
s = s−1 ≈ 0.603, but the necessary condition (17) for φ−1 is only satisfied for α ≤ α−1

Similarly, we have α0∗ ≈ 2.9217 for s = s0 ≈ 1, but the necessary condition (16) for φ0 is only
satisfied for α ≤ α0+ ≈ 1.6868. This may be shown to hold for other values of s as well.
[Figure 6 about here.]
[Figure 7 about here.]
6. Discussion
In the present paper, we investigated locally optimal designs as well as Bayes optimal designs
for the GPCM. We demonstrated that for a fixed number of more than two response categories,

16
the locally optimal design does not exist. However, when allowing the number of categories to
vary, the locally optimal design exists and assigns non-zero probability only to the first and last
response category independently of a person’s ability. That is, when using such a design, the
GPCM reduces to the ordinary 2PL model, which may seem paradoxically at first glance.
However, it is plausible given that the extremes will, from a mathematical perspective, often
contain the most information. Also, it has some analogies to optimal designs in other contexts, for
instance, in linear regression, where the optimal design puts weight only on the minimum and
maximum predictor values (e.g., see Holling & Schwabe, 2013).
When fixing the discrimination parameters, the locally optimal one-point design proved to be
relatively robust. In particular, when allowing the number of response categories to vary, it
turned out to be Bayes optimal when considering common Bayesian design criteria and symmetric
weight distribution with not too large variation. It has to be emphasized that the maximal
variation such that it remained Bayes optimal for all three criteria can be considered as
sufficiently large for application. For instance, when using a normal weight distribution with
standard deviation of s ≈ 0.59 as well as items with three response categories and a
discrimination of one, subjects in the lowest 2.5% ability-region (according to the weight
distribution) completely solve the task only with probability 9% or less, while subjects in the
highest 2.5% ability-region completely solve the task with probability 91% or more.
Despite being mathematically reasonable, the results pose some practical problems when
designing items with optimal properties: An item with more than two response categories, on
which subjects score on the first or last categories (nearly) always – independent of their ability –
would look rather strange and be practically impossible to construct. One could only aim at
reducing the probability of intermediate responses to approximate the optimal design. The
obvious solution to completely remove partial credit from the items and score only as true or
false, might be tempting, but will definitely lead to information loss – and thus less efficient
designs – for complicated items consisting of multiple tasks. Giving this paradox, it might be
asked whether the (G)PCM is a reasonable model for ordinal responses. It may be mathematical
convenient and elegant, but – as shown in the present paper – lacks generally applicable optimal
designs. It is thus of great interest to study optimal designs of alternative ordinal models such as

17
the graded response model (Walker & Duncan, 1967; Van Der Ark, 2001) or the sequential model
(Tutz, 1990, 2000), which potentially favor items with properties much closer to real items. If so,
one would have to think of switching to those models also in large scale educational assessment
studies such as PISA, which had previously applied the GPCM.
When focusing on dichotomous responses and thus on the 2PL model, we provided equations
whose unique solutions are the optimal discrimination parameters in the set of one-point designs
when applying symmetric weight distributions. However, for some important examples, we
established numerically that these discrimination parameters do not lead to Bayes optimal
designs. That is, any Bayes optimal design for the 2PL model with varying threshold and
discrimination (if existent) has to have at minimum two distinct design points at least for the
examples discussed in the present paper.
For future research, some open questions remain. It is still unclear how the optimal designs
derived here perform for asymmetric weight distributions due to the complexity of the involved
derivatives and integrals. Furthermore, it has to be noted that optimal test designs and optimal
sampling designs are not equivalent as there are multiple threshold as well as discrimination
parameters per item, but only one ability parameter per person. Accordingly, derivation of
optimal sampling designs for the GPCM still remains an open topic. Finally, as discussed above,
there are other reasonable ordinal models, whose optimal designs are potentially of great
relevance in practice. While locally optimal designs for the graded response model have been
derived in Schmidt and Schwabe (2015), Bayes optimal designs for this model as well as optimal
designs in general for the sequential model remain to be investigated.

18
7. Appendix
The indexes p and i for persons and items are suppressed for notational convenience.
Proof. (Proposition 1) We abbreviate the normalizing function in the denominator of (1) as
z (= 1/π0 ) for convenience. For a single item, the response Y ∈ {0, ..., J} is categorically
distributed so that the log-likelihood l equals:
l(Y ; θ, τ, α) := log (f (Y ; θ, τ, α)) =

J
X

1j (Y ) log(πj ) + q

j=0

=

J
X

1j (Y ) log(πj ) + (1 −

j=1

J
X

1j (Y )) log(π0 ) + q

j=1

 
J 
X
πj
=
1j (Y ) log
+ log(π0 ) + q
π0
j=1
!
j
J
X
X
1j (Y )
αs (θ − τs ) − log(z) + q,
=

(18)

s=1

j=1

where f is the density of Y defined in (1), 1j is the indicator function for Y = j and q is a
constant independent of θ. The first derivative of l with respect to θ equals
P

PJ
j
J
A
exp
α
(θ
−
τ
)
X
j
s
s
s=1
j=1
d l(θ, τ, α)
=
Aj 1j (Y ) −
dθ
z
j=1

=

J
X

Aj (1j (Y ) − πj )

(19)

j=1

Thus, M is equal to the variance of the random variable
X :=

J
X

AJ 1j (Y ).

(20)

j=1

Since (1j (Y ))2 = 1j (Y ) and 1j (Y )1k (Y ) = 0 for j 6= k, as well as E(1j (Y )) = πj , we have
Var(X) = E(X 2 ) − E(X)2 =

J
X
j=1

which completes the proof.


A2j πj − 

J
X

2
Aj π j  ,

j=1



19
Proof. (Theorem 1) The random variable X defined in (20), is bounded between 0 and AJ .
It is minimal if Y = 0 and maximal if Y = J. According to Popoviciu’s inequality for variances
(Popoviciu, 1965; Peajcariaac & Tong, 1992), Var(X) (and hence M ) is maximal if and only if
π0 = πJ = 1/2 and πj = 0 for 0 < j < J.



Proof. (Proposition 2)
(a) This can easily be checked.
(b) We have
J
X

exp

k=0

k
X

!
αs (θ − τ̃αs (c))

=1+

s=1

J−1
X

!
exp (Ak θ − α1 αJ c))

+ exp(AJ θ)

(21)

k=1

Hence, for c → ∞ it holds that π0 = 1/(1 + exp(AJ θ)), πJ = exp(AJ θ)/(1 + exp(AJ θ)) and πj = 0
for all 1 ≤ j < J.
(c) This follows directly from the proof of (b).
(d) For τ ∈ CαJ with α satisfying αk = αJ−k+1 (k ∈ {1, ..., J}) we have
!
!
J−j
J−j
X
X
exp
αs (−θ − τs ) = exp −AJ−j θ −
αs τs
s=1

s=1

= exp −(AJ − Aj )θ −

j
X

!
αs τs

s=1

= exp(−AJ θ) exp

j
X

!
αs (θ − τs )

(22)

s=1

since AJ = Aj + AJ−j and

PJ−j
s=1

αs τs =

Pj

s=1 αs τs

for all j ∈ {0, ..., J}. Again for τ ∈ CαJ the

denominator of πJ−j (−θ, τ, α) can be written as
!
!
J
k
J
k
X
X
X
X
exp
αs (−θ − τs ) = exp(−AJ θ)
exp (AJ − Ak )θ −
αs τs
k=0

s=1

= exp(−AJ θ)

= exp(−AJ θ)

k=0
J
X
k=0
J
X
k=0

s=1

exp AJ−k θ −

J−k
X

!
αs τs

s=1

exp

k
X
s=1

!
αs (θ − τs ) .

(23)

20
Hence, we find
πJ−j (−θ, τ, α) =

exp(−AJ θ)
πj (θ, τ, α) = πj (θ, τ, α)
exp(−AJ θ)

(24)


In the following proofs, we will suppress the dependency of M , πj , and Π on θ for notational
convenience. Whenever we apply the degenerate locally optimal design τ ∗ , we implicitly reduce
the number of categories to two, as in this case, only two response categories have positive
probability. This way, the derived results remain valid without writing down any limits of the
form limτ →τ ∗ thus simplifying the notation.
Proof. (Lemma 1) For j ≥ i the derivatives of πj with respect to τi and αi equal
d πj
= αi
d τi

j
X

exp

J
X

!
αs (θ − τs )

s=1
J
X

= αi πj

k
X

exp

πk − 1

= αi

πj

k=i

αs (θ − τs )

!!
−z

/z 2

s=1

k=i

!

!

J
X

!
πk

!
− πj

.

(25)

k=i

For j < i the derivatives equal
d πj
= αi
d τi

exp

j
X

!
αs (θ − τs )

s=1

= αi πj

J
X

J
X

exp

k
X

!!!
αs (θ − τs )

/z 2

s=1

k=i

!
πk

.

(26)

k=i

Thus, we find
J
X
d πj
j=1

d τi




!
J
J
J
X
X
X
= αi 
πj 
πk −
πj 
j=1

k=i

j=i

(27)

21
and hence



J
J
J
X
X
X
d
π
d
π
dM
j
j
=
− 2
A2j
Aj πj  
Aj
d τi
d τi
d τi
j=1
j=1
j=1


!
J
J
J
X
X
X
2 

= αi
Aj π j
πk −
A2j πj
j=1


− 2

j=i

k=i
J
X

 
Aj πj  

j=1

J
X


Aj πj 

j=1

J
X

!
πk

k=i

−

J
X

!
Aj πj  .

(28)

j=i

Using Proposition 2 (b), (28) evaluated in point τ ∗ is equal to
dM ∗
(τ , α) = αi A2J πJ π0 (πJ − π0 )
d τi

(29)

Under suitable regularity conditions on Π, differentiation with respect to τi and integration with
respect to θ may be interchanged so that we arrive at
Z
d ψ0 ∗
(τ , α) =
d τi
Z
d ψ−1 ∗
(τ , α) =
d τi

Z
dM ∗
(τ , α)M (τ ∗ , α)−1 Π d θ = αi (πJ − π0 ) Π d θ
d τi
Z
dM ∗
αi
(πJ − π0 )
∗
−2
(τ , α)M (τ , α) Π d θ = 2
Π dθ
d τi
πJ π0
AJ

(30)
(31)

as
M (τ ∗ , α) = A2k πJ π0

(32)

By symmetry of Π and πj (. , τ ∗ ) (in the sense of Proposition 2 (d)) we have
Z

and hence

dψ0 ∗
dτi (τ , α)

=

dψ−1 ∗
dτi (τ , α)

πJ2 π0 Π

Z
dθ =

πJ π02 Π d θ

(33)

= 0 for all i ≤ J due to symmetry. Since ψv (v ∈ {−1, 0}) is

concave (Firth & Hinde, 1997), τ ∗ is a global maximum of the two design criteria within the set of
one-point designs independent of the choice of α.



Proof. (Lemma 2) Although, we only need J = 1 for the proof, we compute the derivatives of

22
M with respect to α for arbitrary J. For j ≥ i the derivatives of πj after αi equal
! J
!
!!
j
k
X
X
X
d πj
αs (θ − τs )
= −(θ − τi ) exp
exp
αs (θ − τs ) − z
/z 2
d αi
s=1
s=1
k=i
!
!
!
J
J
X
X
= −(θ − τi )πj
πk − 1 = −(θ − τi ) πj
πk − πj
k=i

k=i

.

(34)

For j < i the derivatives equal
! J
!!!
j
k
X
X
X
d πj
= −(θ − τi ) exp
αs (θ − τs )
/z 2
exp
αs (θ − τs )
d αi
s=1
s=1
k=i
!
J
X
= −(θ − τi )πj
πk .

(35)

k=i

Thus, we find
J
X
d πj
j=1

d αi




!
J
J
J
X
X
X
= −(θ − τi ) 
πj 
πk −
πj 
j=1

(36)

j=i

k=i

and hence





J 
J
J 
X
X
X
d πj
d πj 
d Aj
d Aj
dM
=
πj + A2j
− 2
πj + Aj
2Aj
Aj π j  
d αi
d αi
d αi
d αi
d αi
j=1
j=1
j=1


!
J
J
J
X
X
X
2


= −(θ − τi )
Aj π j
πk −
A2j πj
j=1

k=i

j=i


 

!
!
J
J
J
J
X
X
X
X
− 2
Aj π j   
Aj π j 
πk −
Aj π j 
j=1

j=1

k=i

j=i





J
J
J
X
X
X
+
2Aj πj − 2 
Aj π j  
πj  .
j=i

j=1

(37)

j=i

Using Proposition 2 (b), (37) evaluated in point τ ∗ is equal to
dM ∗
(τ , α) = AJ πJ π0 (2 − AJ (θ − τi∗ )(πJ − π0 )) ,
d αi

(38)

23
which for J = 1 (dichotomous 2PL model; AJ = α), can be written as
dM
(0, α) = απ1 π0 (2 − αθ(π1 − π0 ))
dα
Due to symmetry of π in the sense of Proposition 2 (d) and since Π is symmetric, we have
Z
Z
x
− θπ1 (π1 π0 ) Π d θ = θπ0 (π1 π0 )x Π d θ
for any x ∈ Z. For symmetric Π, the integral

R

(39)

(40)

π1 π0 Π d θ is monotonically decreasing in α,

whereas
Z
I0 (α) :=

θπ1 Π d θ

(41)

is monotonically increasing, which both is immediately evident from the graph of π1 and π0
changing with α. For ψ0 we have
Z
Z
dM
2
d ψ0
−1
(0, α) =
(0, α)M (0, α) Π d θ =
(1 − αθπ1 ) Π d θ
dα
dα
α

(42)

Since I0 (α) is monotonically increasing in α, αI0 (α) is monotonically increasing and unbounded.
Furthermore, limα→0 αI0 (α) = 0 and hence there exists a unique solution α0∗ of
Z
α θπ1 Π d θ = 1,
which (together with τ ∗ ) constitutes the optimal one-point design for ψ0 . For ψ−1 we have
Z
Z
dM
2
(1 − αθπ1 )
d ψ−1
−2
(0, α) =
(0, α)M (0, α) Π d θ = 3
Π dθ
dα
dα
α
π1 π0

(43)

(44)
(45)

The integral
Z
I−1 (α) :=

(1 − αθπ1 )
Π dθ
π1 π0

(46)

is monotonically decreasing in α for symmetric Π, which again can be inferred from the graph of
π1 and π0 changing with α. Since limα→0 I−1 (α) = 4 and limα→∞ I−1 (α) = −∞, there exists a
∗ of
unique solution α−1

Z
α

θ
Π dθ =
π0

Z

1
Π d θ,
π1 π0

which (together with τ ∗ ) constitutes the optimal one-point design for ψ−1 .

(47)


24
Proof. (Theorem 2) For a symmetric weight distribution, the first derivatives of φ0 and φ−1
are zero in point τ ∗ , which can be seen directly from the first derivatives of ψ0 and ψ−1 in τ ∗ . For
i ≥ n we compute the elements of the Hessian matrix of M as



!  J
!
J
J
J
2
X
X
X
X
d M
2
2
= αi αn 
Aj π j 
πk − 
Aj πj 
πk
d τi τn
j=1
j=n
k=n
k=i


!
!
!!
J
J
J
J
X
X
X
X
+
A2j πj 
πk
πv −
πk
j=1


−

J
X

J
X

A2j πj 

j=i


− 2 

− 2

!
πk

J
X


Aj πj 

j=i

J
X

J
X

!
πk


 


!  J
J
J
J
X
X
X
X
−
Aj πj  
Aj π j 
πk − 
Aj πj 
j=n

k=n



k=i



J
X
−
A2j πj 

k=n

j=1



v=n

k=i



j=1

k=i

j=i




!  J
!
J
J
J
X
X
X
X

Aj πj 
Aj πj 
πk − 
πk

Aj π j 

j=1

j=1


+

J
X

Aj πj 

j=1


−

J
X

j=n

k=n



J
X

!
πk

Aj πj 

j=i

J
X
k=n

!
πk

J
X

πv

−

v=n

k=i



J
X

k=i

!

!!
πk

k=i


 !!
J
X
−
Aj πj 
.

(48)

j=i

This holds for i < n as well due to symmetry of the Hessian matrix. Equation (48) can be slightly
simplified, but remains too complicated to determine analytically under which conditions the
Hessian matrix of φv (v ∈ {0, −1}) is positive definite for some Bayes optimal one-point design τΠ
for a weight distribution Π of unspecified form. If, however, Π is symmetric we know from Lemma
1 that τ ∗ is the Bayes optimal one-point design and (48) remarkably simplifies to
d2 M ∗
(τ , α) = αi αn πJ π0 A2J (1 − 6πJ π0 ) = αi αn M (τ ∗ , α)(1 − 6πJ π0 )
d τi τn

(49)

and hence
d2 φ0 ∗
(τ , α) =
d τi τn

Z

M (τ ∗ , α)−1

d2 M ∗
(τ , α) Π d θ = αi αn
d τi τn

Z
(1 − 6πJ π0 ) Π d θ.

(50)

Within the design space of the threshold vector τ , this result is of limited use since the τ ∗ is at
the border of the design region and the Hessian matrix is constant. However, considering that in

25
τ ∗ the GPCM reduces to the dichotomous 2PL model, the necessary condition for τ ∗ to be Bayes
optimal for v = 0 can be directly inferred from (50) as
Z
(1 − 6πJ π0 ) Π d θ ≤ 0

(51)

or equivalently
Z

1
πJ π0 Π d θ ≥ .
6

(52)

R
Similarly, when denoting C := ( M (τ ∗ )−1 Π d θ)−1 , we obtain
d2 φ−1 ∗
(τ , α) = C
d τi τn

d2 M ∗
M (τ ∗ , α)−2
(τ , α) Π d θ
dτi τn
Z
= αi αn C (πJ π0 J 2 )−1 (1 − 6πJ π0 ) Π d θ

 Z
Z
6
∗ −1
Π dθ
= αi αn C M (τ ) Π d θ − C 2
J


6
= αi αn 1 − C 2 .
J
Z

Thus, the necessary condition for τ ∗ to be Bayes optimal for v = −1 becomes
Z
6≥

(πJ π0 )−1 Π d θ.

(53)


Proof. (Theorem 3) This can be proved in the same way as Theorem 2 in Graßhoff et al.
(2012), so we do not spell out the details here.



26
References
Adams, R. J., Wu, M. L., & Wilson, M. (2012). The Rasch rating model and the disordered
threshold controversy. Educational and Psychological Measurement, 72 (4), 547–573. doi:
10.1177/0013164411432166
Agresti, A. (2010). Analysis of ordinal categorical data. Chichester: John Wiley & Sons. doi:
10.1002/9780470594001
Allen, N., Donoghue, J., & Schoeps, T. (2001). The NAEP 1998 technical report (NCES
2001509). Washington, DC: National Center for Education Statistics.
Andersen, E. B. (1973). Conditional inference for multiple-choice questionnaires. British Journal
of Mathematical and Statistical Psychology, 26 (1), 31–44. doi:
10.1111/j.2044-8317.1973.tb00504.x
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43 (4),
561–573. doi: 10.1007/BF02293814
Andrich, D. (2005). The Rasch model explained. In Applied Rasch measurement: A book of
exemplars (pp. 27–59). Springer. doi: 10.1007/1-4020-3076-2 3
Atkinson, A. C., Donev, A. N., & Tobias, R. D. (2007). Optimum experimental designs, with SAS
(Vol. 34). Oxfort: Oxford University Press. doi: 10.1007/978-3-642-04898-2 434
Berger, M. P., & Wong, W.-K. (2009). An introduction to optimal designs for social and
biomedical research (Vol. 83). John Wiley & Sons. doi: 10.1002/9780470746912
Chaloner, K., & Larntz, K. (1989). Optimal bayesian design applied to logistic regression
experiments. Journal of Statistical Planning and Inference, 21 (2), 191–208. doi:
10.1016/0378-3758(89)90004-9
Chaloner, K., & Verdinelli, I. (1995). Bayesian experimental design: A review. Statistical Science,
273–304. doi: 10.1214/ss/1177009939
Cramér, H. (2016). Mathematical methods of statistics (Vol. 9). Princeton: Princeton University
Press.
Firth, D., & Hinde, J. (1997). On Bayesian D-optimum design criteria and the equivalence
theorem in non-linear models. Journal of the Royal Statistical Society: Series B (Statistical

27
Methodology), 59 (4), 793–797. doi: 10.1111/1467-9868.00096
Fischer, G. H. (1995). The derivation of polytomous Rasch models. In Rasch models (pp.
293–305). Springer. doi: 10.1007/978-1-4612-4230-7 16
Geerlings, H., Glas, C. A., & van der Linden, W. J. (2011). Modeling rule-based item generation.
Psychometrika, 76 (2), 337–359. doi: 10.1007/s11336-011-9204-x
Graßhoff, U., Holling, H., & Schwabe, R. (2012). Optimal designs for the Rasch model.
Psychometrika, 77 (4), 710–723. doi: 10.1007/s11336-012-9276-2
Holling, H., & Schwabe, R. (2013). An introduction to optimal design: Some basic issues using
examples from dyscalculia research. Zeitschrift für Psychologie, 221 (3), 124–144. doi:
10.1027/2151-2604/a000142
Holling, H., & Schwabe, R. (2016). Statistical optimal design theory. In Handbook of item
response theory, volume two: Statistical tools (pp. 313–340). Chapman and Hall/CRC.
Kiefer, J. (1974). General equivalence theory for optimum designs (approximate theory). The
Annals of Statistics, 849–879.
Lehmann, E. L., & Casella, G. (2006). Theory of point estimation. Springer Science & Business
Media, Berlin. doi: 10.1007/978-1-4615-8071-3
Martin, M. O., Gregory, K. D., & Stemler, S. E. (2000). TIMSS 1999 technical report.
International Study Center Chestnut Hill, MA.
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47 (2), 149–174.
doi: 10.1007/BF02296272
Mazzeo, J., & von Davier, M. (2014). Linking scales in international large-scale assessments.
Handbook of international large-scale assessment: Background, technical issues, and
methods of data analysis, 228–258.
Muraki, E. (1992). A generalized partial credit model: Application of an em algorithm. ETS
Research Report Series, 1992 (1). doi: 10.1002/j.2333-8504.1992.tb01436.x
Muraki, E. (1993). Information functions of the generalized partial credit model. ETS Research
Report Series, 1993 (1). doi: 10.1002/j.2333-8504.1993.tb01538.x
OECD. (2017). PISA 2015: Technical report. Retrieved from
http://www.oecd.org/pisa/data/2015-technical-report/

28
Peajcariaac, J. E., & Tong, Y. (1992). Convex functions, partial orderings, and statistical
applications. Academic Press.
Popoviciu, T. (1965). Sur les équations algébriques ayant toutes leurs racines réelles.
Mathematica, 9 , 129–145.
Rao, C. R. (1992). Information and the accuracy attainable in the estimation of statistical
parameters. In Breakthroughs in statistics (pp. 235–247). Springer. doi:
10.1007/978-1-4612-0919-5 16
Rasch, G. (1961). On general laws and the meaning of measurement in psychology. In
Proceedings of the fourth berkeley symposium on mathematical statistics and probability
(Vol. 4, pp. 321–333).
Schmidt, M., & Schwabe, R. (2015). Optimal cutpoints for random observations. Statistics,
49 (6), 1366–1381. doi: 10.1080/02331888.2015.1006225
Tutz, G. (1990). Sequential item response models with an ordered response. British Journal of
Mathematical and Statistical Psychology, 43 (1), 39–55. doi:
10.1111/j.2044-8317.1990.tb00925.x
Tutz, G. (2000). Die Analyse Kategorialer Daten: Anwendungsorientierte Einführung in
Logit-Modellierung und Kategoriale Regression. Oldenbourg: Oldenbourg Verlag.
Van Der Ark, L. A. (2001). Relationships and properties of polytomous item response theory
models. Applied Psychological Measurement, 25 (3), 273–282. doi:
10.1177/01466210122032073
van der Linden, W. J. (2006). Linear models for optimal test design. Springer Science & Business
Media. doi: 10.1007/0-387-29054-0
van der Linden, W. J., & Glas, C. A. (2010). Statistical tests of conditional independence
between responses and/or response times on test items. Psychometrika, 75 (1), 120–139.
doi: 10.1007/s11336-009-9129-9
von Davier, M., & Sinharay, S. (2010). Stochastic approximation methods for latent regression
item response models. Journal of Educational and Behavioral Statistics, 35 (2), 174–193.
doi: 10.3102/1076998609346970
Von Davier, M., & Sinharay, S. (2013). Analytics in international large-scale assessments: Item

29
response theory and population models. Handbook of international large-scale assessment:
Background, technical issues, and methods of data analysis, 155–174.
Walker, S. H., & Duncan, D. B. (1967). Estimation of the probability of an event as a function of
several independent variables. Biometrika, 54 (1-2), 167–179. doi: 10.2307/2333860
Wilson, M. (1992). The ordered artition model: An extension of the partial credit model. Applied
Psychological Measurement, 16 (4), 309–325. doi: 10.1177/014662169201600401
Wilson, M., & Adams, R. J. (1993). Marginal maximum likelihood estimation for the ordered
partition model. Journal of Educational and Behavioral Statistics, 18 (1), 69–90. doi:
10.2307/1165183

FIGURES

30

s = 0.8

s = 1.0886

s = 1.5

τ2

5.0
2.5

φ−1(τ, ζ*)

0.0

0.0

−2.5

−0.5

−5.0
−5.0 −2.5 0.0

−1.0
2.5

5.0 −5.0 −2.5 0.0

τ1

2.5

5.0 −5.0 −2.5 0.0

2.5

5.0

Figure 1.
Contour of the sensitivity function for ψ−1 in case of three response categories for a uniform weight distribution with
varying values of the scale parameter s.

FIGURES

31

s=1

s = 1.2878

s = 1.8

5.0

φ0(τ, ζ*)

τ2

2.5
0.0

0.0

−2.5

−0.5

−5.0
−5.0 −2.5 0.0

−1.0
2.5

5.0 −5.0 −2.5 0.0

τ1

2.5

5.0 −5.0 −2.5 0.0

2.5

5.0

Figure 2.
Contour of the sensitivity function for ψ0 in case of three response categories for a uniform weight distribution with
varying values of the scale parameter s.

FIGURES

32

0.3

0.3

0.0

0.0

−0.3

−0.3

−0.6

−0.6

1

2

α

3

4

1

2

α

3

4

Figure 3.
Illustration of the optimal one-point discrimination parameter of the 2PL model in relation to the necessary condition
for Bayes optimality. Left: criterion φ−1 with a uniform weight distribution with scale s = 2.1773. Right: criterion
∗
φ0 with a uniform weight distribution with scale s = 2.5757. The intersection of the blue and black line indicate α−1
+
and α0∗ , respectively, while the intersection of the red and black line indicate α−1
and α0+ , respectively. Note that
+
∗
α−1
> α−1
and α0∗ > α0+ .

.

FIGURES

33

s = 0.3

s = 0.5885

s = 0.7

τ2

5.0
2.5

φ−1(τ, ζ*)

0.0

0.0

−2.5

−0.5

−5.0
−5.0 −2.5 0.0

−1.0
2.5

5.0 −5.0 −2.5 0.0

τ1

2.5

5.0 −5.0 −2.5 0.0

2.5

5.0

Figure 4.
Contour of the sensitivity function for ψ−1 in case of three response categories for a normal weight distribution with
varying values of the scale parameter s.

FIGURES

34

s = 0.5

s = 0.839

s=1

5.0

φ0(τ, ζ*)

τ2

2.5
0.0

0.0

−2.5

−0.5

−5.0
−5.0 −2.5 0.0

−1.0
2.5

5.0 −5.0 −2.5 0.0

τ1

2.5

5.0 −5.0 −2.5 0.0

2.5

5.0

Figure 5.
Contour of the sensitivity function for ψ0 in case of three response categories for a normal weight distribution with
varying values of the scale parameter s.

FIGURES

35

s = 0.29

s = 0.3015

s = 0.32

τ2

5.0
2.5

φ−1(τ, ζ*)

0.0

0.0

−2.5

−0.5

−5.0
−5.0 −2.5 0.0

−1.0
2.5

5.0 −5.0 −2.5 0.0

τ1

2.5

5.0 −5.0 −2.5 0.0

2.5

5.0

Figure 6.
Contour of the sensitivity function for ψ−1 in case of three response categories for a logistic weight distribution with
varying values of the scale parameter s.

FIGURES

36

s = 0.25

s = 0.5

s = 0.6

5.0

φ0(τ, ζ*)

τ2

2.5
0.0

0.0

−2.5

−0.5

−5.0
−5.0 −2.5 0.0

−1.0
2.5

5.0 −5.0 −2.5 0.0

τ1

2.5

5.0 −5.0 −2.5 0.0

2.5

5.0

Figure 7.
Contour of the sensitivity function for ψ0 in case of three response categories for a logistic weight distribution with
varying values of the scale parameter s.

