Morphological Irregularity Correlates with Frequency
Shijie Wu Ryan Cotterell Timothy J. O'Donnell

Charge-to-heat transducers exploiting the Neganov-Trofimov-Luke effect for light detection in rare-event searches
    [
    April 17, 2019
=================================================================================================================


We present a study of morphological irregularity. Following recent
  work, we define an information-theoretic measure of irregularity
  based on the predictability of forms in a language. Using a neural
  transduction model, we estimate this quantity for the forms in 28
  languages. We first present several validatory and exploratory
  analyses of irregularity. We then show that our analyses provide
  evidence for a correlation between irregularity and frequency:
  higher frequency items are more likely to be irregular and irregular
  items are more likely be highly frequent. To our knowledge, this
  result is the first of its breadth and confirms longstanding proposals from the linguistics
  literature. The correlation is more robust when aggregated at the
  level of whole paradigms—providing support for models of
  linguistic structure in which inflected forms are unified by
  abstract underlying stems or lexemes.
  Code is available at <https://github.com/shijie-wu/neural-transducer>.





§ INTRODUCTION

Irregularity is a pervasive phenomenon in the inflectional morphology
of the world's languages and raises a number of questions about
language design, learnability, and change. Nevertheless, irregularity
remains an understudied phenomenon and many basic questions remain
unanswered <cit.>. Do all languages
exhibit irregularity? What is the relationship between irregularity
and frequency? Is irregularity best thought of as a property of
individual forms, or a property of more abstract objects like
morphological paradigms?  In this paper, we examine these questions,
focusing in particular on the relationship between irregularity and
frequency.

One of the fundamental challenges in studying irregularity is defining
the phenomenon in a way that is applicable across languages. We begin the
paper by addressing this question. First, we formalize the problem of
inflectional morphology and present a novel, information-theoretic
measure of the degree of irregularity of an inflected form. This
definition builds on recent work that defines (ir)regularity in terms
of the probabilistic predictability of a form given the rest of the
language <cit.>.  Making use of
a state-of-the-art model of morphological inflection, we estimate our
measure of irregularity across a large number of word forms from 28
languages drawn from the UniMorph database
<cit.>. Based on these estimates we perform
three studies. First, we validate our estimates by examining the
predictions on English past tense forms—showing that the model's
predicts accord with human judgements of irregularity. We also examine
the overall rate of accuracy of our model. Second, we examine the
degree of irregularity across languages, showing that the model
predicts wide variance in the average amount of irregularity between
the languages in our sample. Finally, we provide empirical evidence
for a correlation between irregularity and frequency across
languages. While this relationship has been observed for individual languages <cit.>, this is the first confirmation of the effect across this many languages.
This result is especially relevant given recent discussions calling the relationship into question  <cit.>. We find,
furthermore, that the correlation between irregularity and frequency
is much more robust when irregularity is considered as a property of
whole lexemes (or stems/paradigms) rather than as a
property of individual word forms. We discuss the implications of
these findings.



§ FORMALIZING INFLECTIONAL MORPHOLOGY


In this work, each word type is represented as a triple consisting of
the following components:


  


  * A lexeme[This terminology is characteristic of
    word-and-paradigm approaches to morphology. In
    item-and-arrangement approaches, this might be called the
    stem <cit.>. ] ℓ: An arbitrary
  integer or string that indexes an abstract word (e.g., go,
  which provides an index to forms of the verb go such as
  goes and went).


  * A slot σ: An arbitrary integer, string, or more
  structured object that indicates how the word is inflected (e.g.,
  [pos=v,tns=past,person=3rd,num=sg]
  for the form went).


  * A surface form w: A string over a fixed phonological or
  orthographic alphabet Σ (e.g., went).
  


A paradigm  (boldface ℓ) is a lexeme-specific map
from slots to surface forms for lexeme ℓ.[See
  <cit.> for a tour of alternative views of
  inflectional paradigms.]  Typically, slots are indexed by structured
entities—known as morpho-syntactic feature vectors or
morpho-syntactic tags—represented by a set of key-value pairs:
σ
=[k_1=v_1,…,k_n=v_n]. For example, the English verb form
runs, which has the feature vector
[tns=pres,per=3rd,
num=sing].  In what follows, the keys k_i and
the corresponding values v_i are taken from the universal inventory,
defined by the UniMorph annotation scheme and denoted ℳ
<cit.>.  We use dot notation to refer to
specific forms or sets of forms in a paradigm indexed by some slot
go.past= went.
 
Given the pieces just sketched, a complete model of inflectional
morphology will specify a joint distribution over surface forms,
lexemes, and slots, that is ℙ(w, ℓ, σ), or one of
its associated conditional distributions, such as
ℙ(ℓ, σ| w)—the distribution over lexemes and
features, given a surface form; or
ℙ(w|ℓ, σ)—the conditional probability of a
surface form given a lexeme and inflectional features. In this paper,
we will focus on the latter, defining a probabilistic model to
approximate this distribution and using that to estimate degrees of
irregularity.



§ OPERATIONALIZING IRREGULARITY

The informal distinction between regular and irregular forms is an
important one for many theories of grammar
<cit.>, language processing
<cit.>, and language acquisition
<cit.>. However,
there have been few proposals for how the notion can be characterized
precisely or measured quantitatively.

Clearly, the regularity of a form (or rule) can only be defined with
respect to the language as a whole—what makes something irregular is
that it does not behave in the way that would be expected given other
forms in the language. But what is meant by expected? Here, we
follow recent work by defining the notion of expectedness in terms of
a probabilistic model of inflection which approximates
ℙ(w|ℓ, σ)
<cit.>. However, there remains a
wrinkle.  A form like went is highly expected as the past tense
of go for an adult speaker of English, but is also
irregular. How do we capture this?

We take the correct notion of expectedness to be the expectedness of
the word form treated as if it were the first instance of that
lexeme which had been observed.  Thus, we base our measures of
regularity on the conditional probability of a word type w given the
rest of the forms in the language with the target lexeme removed.


    (w |ℓ, σ, ℒ_-ℓ)


Of course, since the target language ℒ is generally
infinite, we will need to make use of some model-based estimate of
this probability (w |ℓ, σ, ℒ_-ℓ).
In essence, our definition of irregularity is based on
wug-testing <cit.> such a probabilistic model
to see how robustly it generalizes to the target form w. In
practice, we will estimate this quantity by performing a holdout
evaluation of the target form under our model.

More irregular forms will tend to have a lower wug-test
probability (w |ℓ, σ, ℒ_-ℓ) than most
regular forms. However, the absolute value of such a probability is
not directly interpretable.  To turn these probabilities into
interpretable values which directly measure irregularity, we take the
negative log odds of the probability of the correct word form.


    ι(w) = -log[ (w |ℓ, σ, ℒ_-ℓ)/1-(w |ℓ, σ, ℒ_-ℓ)]


We refer to this quantity as the degree of irregularity of a
form. If probability of the correct form w is exactly 0.5, then
<ref> will be 0. However, if
(w |ℓ, σ, ℒ_-ℓ) > ∑_w^'≠
  w(w^'|ℓ, σ, ℒ_-ℓ), then
<ref> will be negative. Otherwise, the quantity is
positive. In other words, the metric is more strongly positive when a
form is less predictable given other forms in the language and more
strongly negative when a form is more strongly predictable.  The
midpoint at 0 occurs when there is an equal amount of probability
mass on the correct form and all other forms.

Note that this definition of ι neatly addresses several
challenges in studying the notion of (ir)regularity. First, it doesn't
require us to define a binary notion of regular versus irregular or
even to explicitly define any such notion at all—a model may treat
regularity as an implicit rather than explicit feature of a form or
paradigm. Second, and relatedly, we do not require data annotated with
the regularity of forms to train or test our model. Third, this
definition inherently captures the idea of degree of regularity, for
instance, capturing the distinction between wholly suppletive forms
such as went and semi-productive inflectional classes such as
ring/rang, sing/sang, etc. Fourth and
finally, regularity is known to be correlated with other features of
morphological structure, such as productivity. Our definition
sidesteps the tricky issue of disentangling these different properties
of inflection.

Note that our definition of ι conditions on
ℒ_-ℓ—the language without the target
lexeme—rather than on ℒ_-w—the language
without the target word. Thus, we are measuring the probability
that the model will generalize to the correct form without any
evidence of a lexeme at all. Thus, we rule out predictability that
comes from similar forms within a paradigm . For example, in our
approach a model cannot make use of the irregularity of the past tense
form ring to guess that the past participle form was more
likely to be rung. We discuss the implications of this
assumption in more detail below <ref>.



§ MODELING MORPHOLOGICAL INFLECTION

Our goal is to estimate
(w |ℓ, σ, ℒ_-ℓ) from data. We do this
by using a structured probabilistic model of string transduction which
we call . In the following sections, we describe this model,
how we handle syncretism in the model, our training (holdout and test)
scheme, and our estimates of the degree of irregularity ι.



 §.§ A Lemma-Based Model




In linguistic morphology, a major division is between
item-and-arrangement or morpheme-based models and
word-and-paradigm or word-based models
<cit.>.  Following <cit.>,
we adopt a word-based approach. To do this, we designate a unique
surface form for each paradigm  known as the lemma. The
lemma is associated with a slot which we notate σ̌:
.σ̌∈Σ^*. The lemma can be thought of as a
dictionary or citation form of a word and is traditionally chosen by
lexicographers of a language. For example, in many Western European
languages the lemma of verb forms is the
infinitive. <Ref> shows several of the forms of the
Spanish verb poner (“to put”) organized around the lemma
form. In what follows, we use the lemma to identify 
lexemes, and wherever a probability distribution would condition on the
abstract lexeme ℓ we instead condition on the lemma
.σ̌.

Our probabilistic model of string transduction  is a
monotonic model with hard attention described in <cit.> and
can be viewed as a graphical model over strings like the one shown in
<ref>. It is expressed as follows.



    (w |.σ̌, σ, ℒ_-ℓ) =
      ∑_∈(w, .σ̌)(w, |.σ̌, σ, ℒ_-ℓ).


The definition of the model includes a sum over all monotonic
(non-crossing) alignments (w, .σ̌) between the
lemma .σ̌ and the output surface form w. The inner
term of this sum is estimated using a sequence to sequence model. The
sum itself is computable in polynomial time using a variant of the
forward algorithm <cit.>.
The model achieves state-of-the-art performance on the SIGMORPHON 2017
shared task on morphological reinflection <cit.>. We follow the hyperparameter used by <cit.>.



 §.§ Handling Syncretism



Many inflectional systems display syncretism—the morphological
phenomenon whereby two slots with distinct morpho-syntactic tags may
have an identical surface form.  In contrast to many models of
inflectional morphology, we collapse syncretic forms of a word into
a single paradigm slot, thereby assuming that every every surface form
w in a paradigm is distinct. An example of such a collapsed paradigm
in German is given in <ref>. Our formalization includes
a slot that merges the genitive, accusative and dative singular into a
single slot due to the word Herr.



To accomplish this we assume that each lexeme is associated with a set
of syncretism classes denoted by ^ℓ.
^ℓ: ℳ→ℳ is a map from a slot
σ to a citation form slot σ^' which indexed
the canonical suface citation form for that combination of
features. ^ℓ is used to collapse paradigm cells with
identical surface forms. For instance, all forms of the lexeme
go are realized as went in the English past tense,
regardless of person and number features; thus, for example,
^go([tns=past,per=3rd,
num=sing])=^go([tns=past,per=2nd,num=plural]). We
say that two lexemes ℓ and ℓ' are syncretically
  equivalent if ^ℓ(σ) = ^ℓ'(σ) for all
σ.  We assume the mappings ^ℓ are known and given in
advance in what follows.

We will use this syncretism-collapsed representation for all
simulations below. In particular, this assumption will allow us to
simply count the surface forms of each word in Wikipedia without
dealing with the tricky issue of assigning individual words to the
correct combination of morphosyntactic features <cit.>.



 §.§ Handling Derived Forms


As discussed above, we hold out whole lexemes, including all of their
inflected forms during training. However, derivational morphology
presents a potential challenge for this approach.  Consider the
irregular verb do/did/done. This verb appears in
a number of derived prefixed forms such as redo and
undo. These forms all inflect identically to the base form
do—for example, redo/redid/redone.[An anonymous reviewer points out that in some languages, such as Dutch, forms derived from irregular verbs become regular (e.g., zeggen/zei but toezeggen/toezegde). In those languages, it should be unnecessary to apply our heuristic approach.]  If
we train our probability model on such derived forms, it is likely to
estimate too high a wug-test probability for all forms which
are built from the shared stem.

To obviate this problem, we remove all derived forms from the data we
consider. To do so we develop a heuristic approach to isolate all
words that may have been derived from another. Note that a key
desideratum of heuristic is that it should be high precision with
respect to finding derivational transformation—we would rather
overexclude forms as potentially derivative of another, rather than
leave a derived form in the data.

We consider a lexeme ℓ^' to be derived from a lexeme ℓ
if and only if there is a string s ∈Σ^+ such that
(∀σ)[^'.σ = .σ· s] or
(∀σ)[^'.σ = s ·.σ] where
s · t denotes string concatenation of strings s and t. For
example, do and redo satisfy this condition, while
sing and ring do not. We perform a search for candidate
s for all pairs of lexemes in each language and remove all
ℓ^' that meet this criterion.



 §.§ Measuring Irregularity

With the above definitions in place, we can define an approximation to
our degree of irregularity ι.


    ι(w) =  -log(w |.σ̌, σ, ℒ_-ℓ)/1 -
          (w |.σ̌, σ, ℒ_-ℓ)


  In our analyses below, we will also wish to measure the irregularity
  of lexemes as a whole. To do this, we take the average irregularity
  score over the entire paradigm .
  
  
    ι(ℓ) = ∑_{(w, σ, ℓ) ∈  |   w ≠.σ̌} -log(w |.σ̌, σ, ℒ_-ℓ)/1 - (w |.σ̌, σ, ℒ_-ℓ)/||-1




§ STUDIES OF IRREGULARITY

The empirical portion of our work consists of three studies. We first
validate and examine the accuracy of the model
(<ref>). Second, we examine the distribution of
irregularity across the languages in our sample
(<ref>). Finally, we examine the correlation
between irregularity and frequency (<ref>). Before
presenting these studies we first give an overview of the data and
simulations common to all of them.



 §.§ Simulations




  
Data Provenance.
All word forms, paradigms, and morphosyntactic features are taken from
the UniMorph project <cit.>. Specifically, we
examine the following 28 languages: Albanian, Arabic, Armenian,
Basque, Bulgarian, Czech, Danish, Dutch, English, Estonian, French,
German, Hebrew, Hindi, Irish, Italian, Latvian, Persian, Polish,
Portuguese, Romanian, Russian, Spanish, Swedish, Turkish, Ukrainian,
Urdu, and Welsh. The languages come from 4 stocks (Indo-European,
Afro-Asiastic, Finno-Urgic and Turkic) with Basque, a language
isolate, included as well. Although this sample represents a
reasonable degree of typological diversity, the Indo-European family
is overrepresented in the UniMorph dataset, as is the case for most
current multilingual corpora. However, within the Indo-European
family, we consider a diverse set of subfamilies: Albanian, Armenian,
Slavic, Germanic, Romance, Indo-Aryan, Baltic, and Celtic. For each
subfamily, we sample subset of languages randomly.

All of our form-level frequencies were computed from
Wikipedia.[Wikipedia data retrieved on Feb 1^st,
  2019.]  Lexeme counts are achieved by summing over all entries in
the paradigm associated with a lexeme. In all simulations, we predict
the orthographic form of the target word w from the orthographic form
of the lemma .σ̌ as a proxy for phonological
transcriptions which do not exist for our all languages in UniMorph.



  
Lexeme-Based Cross Validation.
In the studies that follow, we train a separate instance of our model
on the forms in each language using the following procedure. We first
remove morphologically-complex forms that are derived from other
lemmas in the corpus using the heuristic technique described in
<ref>. We then randomly assign the remaining lexemes of each
language to one of ten splits. Note that each split will contain all
of the forms associated with each lexeme and a lexeme will never be
divided across splits.  We then perform 10-fold cross-validation,
training the model  on 8 splits, tuning on one of the
remaining two splits, and testing on the final remaining split. Note
that this approach to cross-validation allows us to approximate
ℒ_-ℓ without the costly procedure of retraining for
every held-out lexeme.  However, also note that this approach has a
potential confound. Lexemes can often be grouped into inflectional
  classes in which all lexemes mark different slots in the same
way. For example, verbs such as sing/sang/sung
and ring/rang/rung form an inflectional class in
English. Inflectional classes vary in their size and regularity
<cit.>. If all or most lexemes in the same irregular
inflectional class end up together in the test split under our
approach, we may systematically overestimate their irregularity.



 §.§ Validation and Accuracy



  §.§.§ Validation on English Verbs


  
  The first question we wish to ask is whether the irregularity
  predictions made by our model are consistent with human
  intuitions. To answer this question, we examine the predictions of
  our model on the English past tense—a morphological system which
  has been intensely studied for decades <cit.> and for which there is general agreement
  about which forms are regular or irregular. We make use of the
  databases of <cit.> which consists of 4039
  English verb forms and the dataset of <cit.>
  which consists of 15202 verb forms, both hand-annotated for
  irregularity by experts.

  We present our results in <ref>. We find that our
  measure of irregularity strongly correlates with human intuitions on
  English verbs.  We take this as tentative validation of our
  metric. Future work will investigate the linguistic plausibility of
  our metric on a greater diversity of languages.



  §.§.§ Wug-Test Accuracy




Our lexeme-based cross-validation setup differs substantially from the
form-based setup typically used to evaluate models of inflectional
morphology <cit.>.  In the typical evaluation
setup, individual surface word forms are heldout, rather than all of
the forms associated with entire lexemes. This means, amongst other
things, that words from irregular lexemes will often be split between
test and train, giving models an opportunity to learn partially
productive and semi-regular patterns of inflection. Our approach
however makes this impossible by strictly assigning all forms from
each lexeme to either train or test.

It is important to ask, therefore, how well does our model predict the
forms of heldout lexemes given this stricture? The results are
displayed in <ref>. This table displays the average
accuracy for each language in our sample as well as the number of
lexemes for that language, the total number of forms, and the average
number of forms per lexeme. The majority of languages show very high
generalization accuracy to our lexeme-based wug-tests: 21 out
of 28 have an average accuracy of 75% or higher. Three languages
stand out in terms of their low accuracy and are highlighted in
<ref>: Basque, Urdu, and Welsh. These languages,
Basque especially, are characterized by smaller numbers of lexemes and
larger numbers of forms per lexeme.

In the <ref>, we discuss the correlation between
irregularity and frequency. The interpretation of these results relies
on the ability of our model to accurately capture regular structure in
the inflectional systems of the languages that we study. For this
reason, we make the conservative choice to exclude all languages whose
average accuracy was below 75% from all further analyses below.



 §.§ Irregularity across Languages




It is often observed  that there are differences in the
prevalence of irregularity across languages
<cit.>. On one end of the spectrum, some languages
have widespread (often suppletive) allomorphy in their marking of
inflectional features. For example, Arabic marks plurality on nouns in
one of more than a dozen different ways and these are idiosyncratic to
the noun stem.  Similarly, Georgian verbs often have different roots
depending on their tense, aspect, or mood marking. On the other end of
the spectrum, it is sometimes claimed that agglutinative languages like
Turkish exhibit no irregularity whatsoever.

<Ref> displays the average irregularity
score per language for the 21 languages remaining after our 75%
accuracy criterion. Recall from <ref> that the degree of
irregularity ι is positive when the majority of predicted
probability mass falls on forms that are not the correct target form (i.e.,
the form is irregular), and negative when the majority of probability
mass falls on the predicted form (i.e., the form is regular). As can
be seen from the figure, average irregularity is negative across
languages. This is expected—most forms in these languages are
predicted accurately by the model. However, there is wide variability
in the average irregularity score between languages. In particular,
in the most regular language, Portuguese, correct forms are about
25,000 times more likely on average than alternative forms. In the
most irregular language, Hebrew, correct forms are only about 16 times
more likely on average than alternative forms.  We leave it to future
work to validate and further study these cross-linguistic differences
in irregularity predictions.



 §.§ Irregularity and Frequency


In some morphological systems, such as the English past tense, there is a 
strong and well-known correlation between irregularity and frequency  <cit.>. In such systems, the most
frequent past forms tend to be irregular and irregular forms tend to
come from the most frequent verbs. Based on cases like this,
it is widely believed in linguistics and psycholinguistics that
there is an association between frequency and irregularity
<cit.>. However, to
our knowledge, this relationship  has
never been explicitly tested quantitatively across many languages at once.

Recently, several authors have questioned the received wisdom that
irregularity and frequency are related
<cit.>.[But see
  <cit.>.] Thus, it has become important to test
this relationship empirically.  An example of such a challenge to the
standard assumption comes from <cit.> who proposed an
influential theory of morphological productivity known as the
tolerance principle. The mathematical derivation of the tolerance
principle relies on the assumption that irregular forms are uniformly
distributed throughout the frequency range
<cit.>.[Yang tentatively proposes that the
  correlation between frequency and irregularity might be accidental
  in languages such as English. He argues, however, that his theory is
  not contingent on this being the case
  <cit.>.]



Here we present the first study to probe the relationship between
irregularity and frequency at scale.  We first examine the
relationship between the degree of irregularity ι and the
frequency of individual word forms. To study this question, we
examined the Pearson correlation between the log-transformed frequency
of word forms in each language and their predicted irregularity scores
ι(w). Because word occurrences fall into the class of
large number of rare event distributions, finite samples will
tend to underestimate the probability of infrequent words—word forms
that appear 0 times in some sample often differ by orders of
magnitude in their true probability
<cit.>. For this reason, we
chose to exclude all frequency 0 forms from our analyses.

The correlations for the 21 languages considered in this study are
shown in <ref> with significant correlations
(p<0.05) marked in blue. Overall, a slight trend towards a positive
correlation between irregularity and frequency is discernible in this
set of word forms. Following <cit.>, we tested
this by fitting a mixed-effect model with irregularity as the
dependent variable, language as a random effect (slopes and
intercepts) and log count as a fixed effect
<cit.>. The results give a positive coefficient of
0.064 for the log count factor. The AIC-corrected log-odds ratio in
favor of the model with a fixed effect of count (compared to a model
with just random effects) is 3.44. A nested-model likelihood-ratio
χ-squared test shows that the log factor is significant with
p<0.04.



An important question about irregularity is whether it is a property
of individual forms, or rather whether it inheres to whole paradigms
<cit.>. To examine this
question more closely, we ran an alternative correlational analysis
examining the correlation between the sum of the counts of all forms
associated with a lexeme and the average irregularity score for all
forms associated with the lexeme (as in
<ref>). <Ref> shows the
results. Overall, a stronger trend towards a positive correlation
between irregularity and frequency is discernible at the lexeme level
than at the word-form level. We tested this by fitting a mixed-effect
model with irregularity as the dependent variable, language as a
random effect (slopes and intercepts) and log count as a fixed effect.
The models gives a positive coefficient of 0.14 for the log count
factor. The AIC-corrected log-odds ratio in favor of the model with a
fixed effect of count (compared to a model with just random effects)
is 11.8. A nested-model likelihood-ratio χ-squared test 
shows that the log count factor is significant with p<0.001. Thus,
the correlation between irregularity and frequency is considerably
more robust when considered at the lexeme level.



§ CONCLUSION

In this paper, we have introduced a measure of irregularity based on
wug-testing a model of morphological inflection. In
<ref>, we showed that this measure produces results
that are consistent with human judgements. Focusing on a subset of the
languages for which the model was able to recover the correct
inflected forms at a high rate (<ref>), we showed that
average irregularity varies a good deal between languages. This result
is consistent with the findings of <cit.> which
gave large scale empirical evidence of a tradeoff between the size of
morphological paradigms and the predictability of individual forms
within each paradigm.

The main novel empirical result of our paper was presented in
<ref> which showed that irregularity is correlated with
frequency both at the level of individual forms as well as at the
level of lexemes. To our knowledge, this is the first large-scale
empirical demonstration of this piece of linguistic folk wisdom and
provides evidence relevant to recent proposals questioning this
generalization <cit.>.

Perhaps of greater interest than this positive result is the
difference in the strength of the correlation between the level of
individual forms and the level of lexemes. This difference appears to
be driven by the fact that, in many cases, lexemes that contain
high-frequency forms will also contain a few low frequency forms as
well. Adopting the terminology of <cit.>, we can say
that low frequency forms free-ride on the higher frequency
members of the lexeme.

This finding lends credence to models of linguistic structure which
group words together by their lexeme or stem. Such models seem
necessary to account for paradigmatic structure cross linguistically
and to deal with phenomena such as the existence of defective
  paradigms—the phenomenon whereby certain inflected forms of a
word seem to be impossible for speakers <cit.>. A
canonical example is the past participle of stride (e.g.,
^*strode/^*stridden/^*strided). In these
cases, the problem seems to be that the irregularity of the overall
lexeme is known, but the particular word form has never been
observed. Our results provide further support for the view that
inflected forms represent surface exponence of common underlying
morphological objects.

More generally, we observe that our wug-test techniques
provides a general way of studying regularity and predictability
within languages and may prove useful for attacking other difficult
problems in the literature, such as detecting inflectional classes. By
measuring which words or lexemes are most predictable from one
another, a general picture of morphological relatedness within a
language can be built in a bottom-up way. 



§ ACKNOWLEDGMENTS

The third author gratefully acknowledges support from the Fonds de Recherche du Québec—Société et Culture and the Natural Sciences and Engineering Research Council of Canada.


acl_natbib

