abstract: The Langlands Programme, formulated by Robert Langlands in the 1960s and since much developed and refined, is a web of interrelated theory and conjectures concerning many objects in number theory, their interconnections, and connections to other fields. At the heart of the Langlands Programme is the concept of an L-function.
  .
  The most famous L-function is the Riemann zeta-function, and as well as being ubiquitous in number theory itself, L-functions have applications in mathematical physics and cryptography. Two of the seven Clay Mathematics Million Dollar Millennium Problems, the Riemann Hypothesis and the Birch and Swinnerton-Dyer Conjecture, deal with their properties. Many different mathematical objects are connected in various ways to L-functions, but the study of those objects is highly specialized, and most mathematicians have only a vague idea of the objects outside their specialty and how everything is related. Helping mathematicians to understand these connections was the motivation for the L-functions and Modular Forms Database (LMFDB) project. Its mission is to chart the landscape of L-functions and modular forms in a systematic, comprehensive and concrete fashion. This involves developing their theory, creating and improving algorithms for computing and classifying them, and hence discovering new properties of these functions, and testing fundamental conjectures.
  .
  In the lecture I gave a very brief introduction to L-functions for non-experts, and explained and demonstrated how the large collection of data in the LMFDB is organized and displayed, showing the interrelations between linked objects, through our website www.lmfdb.org. I also showed how this has been created by a world-wide open source collaboration, which we hope may become a model for others.
address: Warwick Mathematics Institute, Zeeman Building, University of Warwick, Coventry, CV4 7AL, UK
author: John Cremona
date: 18 September 2015, revised January 2016
title: The L-functions and modular forms database project

# What is the LMFDB?

Since the early days of using computers in number theory, computations and tables have played an important part in experimentation, for the purpose of formulating and proving (or disproving) conjectures.

Until the World Wide Web, such tables were hard to use, let alone to make, as they were only available in printed form, or on microfiche! An example relevant for the LMFDB is the 1976 Antwerp IV tables of elliptic curves, published as part of a conference proceedings in Springer Lecture Notes in Mathematics 476, as a computer printout with manual amendments and diagrams.

However, even since the WWW, tables and databases have been scattered among a variety of personal web pages (including my own ). To use them, you had to know who to ask, download data, and deal with a wide variety of formats. A few had more sophisticated interfaces, but there was no consistency.

In some areas of number theory, such as elliptic curves, the situation is now much better and easier: packages such as SageMath , Magma  and `Pari/gp`  contain elliptic curve databases (sometimes as optional add-ons, as they are large). Also, the internet makes accessing even "printed" tables much easier. But the data are still very scattered and incomplete.

The situation is now very much better: we have the **LMFDB**!

to

The LMFDB home page at www.lmfdb.org, January 2016

# L-functions and why they are important

**L-functions** are at the heart of the LMFDB. What are they? We will give a brief survey, referring to number theory textbooks for details.

The simplest L-function is the **Riemann zeta function** $\zeta(s)$. This

-   is a *complex analytic function* (apart from a pole at $s=1$);

-   has a *Dirichlet series* expansion over positive integers (valid when $\Re(s)>1$): $$\zeta(s)=\sum_{n=1}^{\infty} \frac{1}{n^s};$$

-   has an *Euler product* expansion over primes $p$ (when $\Re(s)>1$): $$\zeta(s)=\prod_p(1-p^{-s})^{-1};$$

-   satisfies a *functional equation*: $$\xi(s)=\pi^{-s/2}\Gamma(s/2)\zeta(s) = \xi(1-s);$$

-   has links to the *distribution of primes*.

## L-functions: a definition

The definition of an L-function encapsulates these properties: it is a **complex function** with a **Dirichlet series** and an **Euler product** expansion which satisfies a **functional equation**. There are other more technical axioms (by Selberg) which we omit here: refer to the LMFDB's own knowledge database for details: <http://www.lmfdb.org/knowledge/show/lfunction>.

Some of the defining properties have not in fact been proved for all the types of L-function in the database: this can be very hard! For example, Andrew Wiles proved Fermat's Last Theorem by proving the modularity of certain elliptic curves over ${\mathbb Q}$, which amounted to showing that the L-functions associated to elliptic curves really are L-functions in the above sense. This is not yet known in general for elliptic curves defined over other algebraic number fields.

Other expected properties of L-functions are not even known for $\zeta(s)$. For example, the **Riemann hypothesis** concerning the the zeros of $\zeta(s)$ has remained open since it was formulated by Riemann in 1859.

## The Riemann Hypothesis

The Riemann Hypothesis states that all the "non-trivial" zeros of $\zeta(s)$ (excluding those coming trivially from poles of $\Gamma(s)$) are on the "critical line" $\Re(s)=1/2$.

This was (part of) Hilbert's 8th problem and is also one of the Clay Mathematics Institute Millennium Prize Problems, so a million dollars awaits the person who proves it. There are similar conjectures about the location of the zeros of all L-functions, which are collectively known as the **Generalized Riemann Hypothesis** (GRH). These are not only of theoretical (or financial!) interest, but have important applications to the complexity of computing important quantities in number theory. For example, computing the class number of a number field is much faster if one assumes GRH for the number field's own L-function, its **Dedekind $\zeta$-function**.

What can a database say in relation to this problem?

It can give the object its own web page (<http://www.lmfdb.org/L/Riemann/>) which shows basic facts about it, and its graph along the critical line $1/2+it$ to "show" the first few zeroes. This is a pedagogical function of the database.

It can also store all the zeroes which have so far been explicitly computed: there are more than $10^{11}$ (that is *one hundred billion*) of them at <http://www.lmfdb.org/zeros/zeta/>, all computed to 100-bit precision by David Platt (Bristol), who in 2014 won a prize for his contributions to progress on the Goldbach Conjecture. This resource can then be used to study properties of the zeroes, such as their distribution, and connections to random matrices, showing that the database also serves as a research tool.

## Degrees of L-functions

The **Euler product** for a general L-function has the form $$L(s) = \prod_p 1/P_p(1/p^{s})$$ where each $P_p(t)$ is a polynomial, and the product is over all primes $p$. These polynomials all have the same degree, called the **degree** of the L-function, except for a finite number indexed by primes dividing an integer called the **conductor** of the L-function, where the degree is smaller. The zeros of these polynomials are also restricted in a way depending on another parameter, the **weight**.

For example, $\zeta(s)$ has $P_p(t)=1-t$ for all primes $p$; the degree is $d=1$, and the conductor is $N=1$.

## L-functions of degree $1$

There are other L-functions of degree $1$, with larger conductor $N$, which have been studied since the 19th century: **Dirichlet L-functions**. Their Dirichlet coefficients $a_n$ are given by the values of a **Dirichlet character** $a_n=\chi(n)$, meaning that they are **multiplicative** and **periodic** with period $N$.

An example with $N=4$ is $$L(\chi,s)=1^{-s}-3^{-s}+5^{-s}-7^{-s}+-\dots,$$ with all even coefficients $0$ and the odd coefficients alternating $\pm1$. Dirichlet used such L-functions to prove his celebrated theorem about primes in arithmetic progressions: for any integers $N\ge1$ and $a$, there are infinitely many primes $p\equiv
a\pmod{N}$, provided that $a$ and $N$ are coprime. The previous example can be used not only to show that there are infinitely many primes $p\equiv1\pmod4$ (for which $\chi(p)=+1$) and infinitely many primes $p\equiv3\pmod4$ (for which $\chi(p)=-1$) , but also to show that (in a precise sense) the primes are *equally distributed* between these two classes.

This is a *complete* list of all L-functions of degree $1$. For degrees greater than $1$, a complete classification has not yet been established, though a wide variety of sources of L-functions is known, and in some cases (such as in degree $2$, see below) we conjecture that all L-functions do arise from these known sources.

## Other sources of L-functions

A wide variety of mathematical objects have L-functions: algebraic number fields, algebraic varieties (including curves). There is a general term **motive** for objects which have L-functions.

In many cases, while we know how to define the L-function of a more complicated object, it has not yet proved that it actually satisfies the defining axioms for L-functions. Even for elliptic curves over ${\mathbb Q}$, this would have been true until the mid-1990s; for elliptic curves over real quadratic fields such as ${\mathbb Q}(\sqrt{2})$ it was true until 2013! Now, these elliptic curves are known to be modular .

## L-functions of number fields

An **algebraic number field**, or simply **number field**, is a finite extension of the rational field ${\mathbb Q}$, such as ${\mathbb Q}(\sqrt{2})$ or ${\mathbb Q}(i)$ or ${\mathbb Q}(e^{2\pi i/m})$. Every number field $K$ has an L-function called its **Dedekind zeta function** $\zeta_K(s)$, defined in a similar way to Riemann's $\zeta(s)=\zeta_{{\mathbb Q}}(s)$, and with similar analytic properties.

Just as the analytic properties of $\zeta(s)$ imply facts about the distribution of primes, from the analytic properties of $\zeta_K(s)$ we can deduce statements about prime factorizations in the field $K$. For example, taking $K={\mathbb Q}(e^{2\pi i/m})$ we can prove Dirichlet's Theorem on primes in arithmetic progressions using a combination of algebraic and analytic properties of $\zeta_K(s)$.

Also, just as some properties of $\zeta(s)$ are not yet proved (for example the Riemann Hypothesis), the same is true for $\zeta_K(s)$: the Generalized Riemann Hypothesis or GRH remains unsolved.

## L-functions of curves

Algebraic **curves** defined over algebraic number fields also have L-functions, whose degree depends on both the degree of the field over which the curve is defined, and the genus of the curve. So an elliptic curve over ${\mathbb Q}$, which is a curve of genus $1$ defined over a field of degree $1$, has a degree $2$ L-function, elliptic curves over fields of degree $d$ have L-functions of degree $2d$, and so on.

It is widely believed that all degree $2$ L-functions arise as follows: they are either products of two degree $1$ L-functions, or come from elliptic curves over ${\mathbb Q}$, or from (a special kind of) **modular form**. The insight of Weil, Taniyama, Shimura and others in the 1960s and 1970s was to realize that the latter two sources actually produce the *same* L-functions! This insight is behind the famous theorem of Wiles *et al.* that "every elliptic curve (over ${\mathbb Q}$) is modular", from which Fermat's Last Theorem was a consequence. But it is still an unsolved problem to show that those degree $2$ L-functions which are not products of degree $1$ L-functions do all arise from automorphic forms.

## Higher degree L-functions

For degrees $3$ and $4$, we do not yet even have a conjecture concerning all sources of L-functions, and for those which are known, not all the conjectured connections between them have been proved.

We mentioned above the recent result that elliptic curves defined over real quadratic fields (such as ${\mathbb Q}(\sqrt{5})$) are modular. This means that two sources of L-functions of degree $4$: on the one hand, elliptic curves over such a field, and on the other hand **Hilbert modular forms** over the same field, actually produce the *same* L-functions. Such results are extremely deep and require a vast amount of theory to establish, including real, complex and $p$-adic analysis and algebra, as well as some explicit computations (the ArXiV version of includes a number of Magma scripts).

By contrast, over *imaginary* quadratic fields (e.g. ${\mathbb Q}(\sqrt{-1})$) we conjecture, but cannot prove in general, that elliptic curves have L-functions also attached to a different kind of modular form, **Bianchi modular forms**. These can be computed, and work is in progress in entering many examples into the LMFDB, even though they are not all known to "be modular" and hence have genuine L-functions.

Modularity of *individual* elliptic curves over imaginary quadratic fields can be proved using the Serre-Faltings-Livné method (which uses Galois representations rather than analysis) as explained in a 2008 paper by Dieulefait, Guerberoff and Pacetti. We are currently using their method to prove modularity of *all* the curves in the database; at the same time we are developing enhancements to the algorithm to make it more efficient. A theoretical proof that all elliptic curves over these fields are modular seems very far off, so even in the world of L-functions of degree $4$ it is still important to carry out experiments and collect data.

## Showing connections through the LMFDB

The LMFDB shows connections between different objects with the same L-function, such as those described above, by linking its databases of (for example) elliptic curves over real quadratic fields, and Hilbert modular forms over the same field. The home page of each elliptic curve includes a link to the associated Hilbert modular form, and to the associated L-function, and (in progress) vice versa.

One difficulty we have encountered in setting up these links on the website, which is perhaps typical in a large project where many different individuals are providing data, is to maintain *consistency of labelling* of objects. Over the field ${\mathbb Q}(\sqrt{5})$, the Hilbert modular forms were computed (in Magma) by John Voight (Dartmouth College) and Steve Donnelly (Sydney) , while the elliptic curves were computed (in SageMath) by Jonathan Bober (Bristol), William Stein (Washington), Alyson Deines (CCR) and others . These groups used essentially the same naming convention, but we were careful to check that the labels of matching objects did match exactly, resulting in one set of data (the elliptic curves) requiring relabelling.

# The LMFDB database

The LMFDB consists of both a database, where the data collection itself is organized and stored, together with the website www.lmfdb.org. This provides a sophisticated user interface to the data, has home pages for individual objects in the database, showing links between related objects, and also provides an online repository of knowledge about L-functions and related objects, through its **knowledge database**.

Both database and website are currently hosted on servers at Warwick, funded by EPSRC; until 2013 they were hosted at the University of Washington on NSF-funded servers administered by William Stein. Plans are also underway to have mirror sites in other countries: this is an international project.

The LMFDB is also a group of mathematicians who collaborate to create and develop the database and its website. We will say more about this collaboration in the final section.

## The database and website software

We are using the open-source database software `MongoDB`. This currently holds nearly a terabyte of data and indices. This choice was made because `MongoDB` allows data to be organized in a completely flexible schema, rather than having to specify the schema for each item in advance as with SQL databases. It also has a powerful `Python` interface, `PyMongo`, which suits the project well, since it allows the website code to use other `Python` modules such as `Flask` (a web framework), and to have access to all the power of SageMath, another large `Python`-based open source mathematical software project. All of these are open source, which is another essential requirement. (Note, however, that not all of the data in the database have been computed using open source tools.) The website code is a collaborative open source project hosted at `GitHub` (see <https://github.com/LMFDB/lmfdb>).

Basing all the website code on `Python` has many advantages. It is relatively easy to learn to use, which is important since we want the barriers to new people joining our project to be as low as possible. And it is phenomenally powerful, giving access to a vast array of additional modules for interfacing with the database (`PyMongo`), running the web framework (`Flask`), web page templating (`Jinja`), testing and more.

Anyone contributing to the project who wants to do more than just donate data has to learn how to use this software. At project workshops we run tutorial sessions for newcomers, where code is written by beginners under the guidance of more experienced peers. All code is reviewed and tested before being adopted, as well as being subject to some automated testing.

## Database organization

The database as a whole consists of around 35 individual databases containing collections of mathematical objects (including ` elliptic_curves`, `hilbert_modular_forms`, and ` number_fields`) and other data such as the `knowledge` database, which holds the contents of **knowls** (see below).

The data are indexed in various ways for faster searching, and, of course, backed up regularly. Many parts of the database can also be recreated from plain text data files which are stored in separate `Git` repositories, also hosted on `GitHub`.

Each constituent database contains collections of records, and these records hold the data in a flexible format: additional data fields can be added later.

## Sample database entry

To take just one example, the database `number_fields` contains just one collection `fields`, for which a typical entry looks as follows (after being converted by `PyMongo` into a `Python` dictionary):

    {u'_id': ObjectId('4cb80fdb5009fb52db0946b6'),
     u'class_group': u'',
     u'class_number': 1,
     u'coeffs': u'1,0,-1,1',
     u'degree': 3,
     u'disc_abs_key': u'00123',
     u'disc_sign': -1,
     u'galois': {u'n': 3, u't': 2},
     u'label': u'184.108.40.206',
     u'ramps': [u'23'],
     u'signature': u'1,1',
     u'unitsGmodule': [[3, 1]]}

Here we see the coefficients of the minimal polynomial $x^3-x^2+1$ of a generator of the field stored as `coeffs`, and the label '126.96.36.199' which also uniquely determines the field. Invariants of the field which are easy to compute on the fly, and to which we do not need to provide direct access through database queries, need not be stored, while quantities which might be expensive to compute, or for which we may want to run searches, are stored and indexed.

This is only a simple example. The database entry for an individual elliptic curve over ${\mathbb Q}$ currently contains 33 fields, some very technical. The number of fields grows over time as new data are contributed. For example, in 2015 Jeremy Rouse offered to provide information concerning the $2$-adic Galois Representation attached to every elliptic curve over ${\mathbb Q}$, after developing and implementing an algorithm to determine this jointly with David Zureick-Brown (see ). He provided us with a Magma script of their implementation, we ran it and uploaded the data, and added a corresponding section on the home page of every curve showing these additional data.

## Software choices, pros and cons

Using off-the-shelf software has plenty of advantages but will never be perfect for a mathematical project.

Most mathematicians, even those with substantial computational experience and expertise, know almost nothing about databases or running websites, and many of the contributors to the LMFDB knew nothing at all about these before they joined the project. Decisions about the specific software used by the project was made by those who did have such experience, notably William Stein (lead developer of SageMath) and Harald Schilly (another key developer of the SageMathCloud project, <https://cloud.sagemath.com/>).

We have already seen some of the advantages of our choice of database, `MongoDB`. There are disadvantages too: `MongoDB` data consists of strings or integers or floating point values, with strings as keys. Values can also be lists of these, but a serious deficiency for number-theoretic data is that the integers cannot be larger than $2^{32}$. This means that most data fields which hold integers have to be stored as strings, and this limits functionality, such as searching for the value being in a certain range.

Similarly, rational numbers cannot be stored as such, or even as a pair of integers \[*numerator*,*denominator*\] if these could be large, so instead they are stored as strings such as '` 1728`' or '`-122023936/161051`'. Building on these, considerable thought has to be given as to how to store more complicated data, such as an element of a number field. Decisions such as these are made by consensus at LMFDB workshops, since they affect all developers, even though the effects of such decisions are hidden from users of the website.

# The LMFDB website

The LMFDB website serves several purposes. It provides

-   a shop window for the data;

-   a way to visualize the data, and the connections between different, linked mathematical objects;

-   a way to browse types of object;

-   a way to search for objects with specified properties;

-   a repository of knowledge through its "knowledge database";

-   a source of data for downloading for further work.

Catering for several different audiences at once is hard to get right!

## Technical support (or lack of)

The project would benefit greatly from having technical support staff. Our current grant from EPSRC does not provide this—it does support six postdoctoral researchers, who all have a certain amount of experience writing mathematical software, but not any dedicated software engineers. For this, we are currently relying on charitable contributions of time. We would not be where we are now, and indeed the website would never have been launched, without the enormous contributions of one person in particular: Harald Schilly , a doctoral student in Vienna and software consultant, who knows more than the rest put together about `Python`, `MongoDB`, `Flask`, and the rest.

From September 2015, through the Horizon 2020 European Research Infrastructure project OpenDreamKit (<http://opendreamkit.org/>), which provides substantial funding for the development of open source computational mathematics, we are currently seeking to employ a software engineer to provide support to the project.

## Homepages

A key organizing principle of the LMFDB is that every object has its own homepage. These have mathematically meaningful, permanent URLs which follow a carefully thought out schema. The home pages themselves are created on demand from templates, filled in with data partly retrieved directly from the database and partly computed on the fly. For example, the elliptic curve with label `5077a1` has URL <http://www.lmfdb.org/EllipticCurve/Q/5077/a/1>.

Each homepage gives a view of the object (depending on its nature), highlighting its most important properties, with *breadcrumbs* to show its position in the whole. In some cases, where the object has some interesting additional historical or mathematical significance, this can also be shown on its home page. For example, the elliptic curve `5077a1` was used by Dorian Goldfeld in 1985 to solve Gauss's class number problem effectively, by making use of a new connection between the problem and L-functions of elliptic curves, and this piece of historical information is shown on the curve's home page.

A **related objects** box on each homepage provides links between related objects. For example, from the pages of an elliptic curve, or a number field, or a modular form there are links to the associated L-function.

Where possible, on the home page of an object, we make it possible to see and download code which will re-create the object in one of the standard number-theoretical packages (SageMath, `Pari/gp` or Magma) and work with it there. In this way, the LMFDB can be used by students learning a subject who wish to work out their own examples, as well as researchers wishing to carry out larger-scale investigation starting from the LMFDB data. A more sophisticated programming interface through SageMath is also planned.

## Searching and browsing

Each class of objects in the LMFDB has its own **Browse and Search** page.

The **Browse** section is intended to be usable by people who know nothing of the underlying theory but want to browse through examples without having to type anything or have technical knowledge.

The **Search** section is more for experts looking for a specific object (possibly by its label), or for an object with certain properties: "a number field with Galois group $C_5$ ramified only at $p=5$", or "an elliptic curve with rank $2$ and non-trivial Tate-Shafarevich group", or "a classical modular form of weight $12$ and level $12$". This leads to a **Search Results** page listing all database entries which match (if any), with links to the home pages of each individual matching object.

## Knowledge and knowls

The knowledge aspect of the LMFDB exists in the first place as a glossary of technical terms used on the web pages, so the pages themselves do not get cluttered up, and there is consistency between pages on basic definitions.

The mechanism which serves these is the **knowl**, created by Harald Schilly and first demonstrated at an LMFDB workshop. The text expands within the page and can be dismissed after reading, without any need for "pop-ups" or new pages.

Knowls can be used anywhere on the web—for example, I use them on my own web page of preprints and publications to display abstracts of papers. Another good example of their use is in the online undergraduate textbook on Linear Algebra by Robert Beezer . For more about knowls and how to use them, see the knowl on the LMFDB itself, <http://www.lmfdb.org/knowledge/show/doc.knowl>, or the page <http://aimath.org/knowlepedia/>.

The content of knowls can be edited by any project member (someone who has a login account), and is itself stored in the database.

# The LMFDB project

## The LMFDB as a collaborative project

The LMFDB was first conceived at an AIM workshop in 2007. It holds regular workshops, which are run along the lines of AIM workshops: few talks and a lot of hard work. As well as individual workshops of around 30 people, there are smaller groups who meet to work together on specific projects, and there have also been longer periods of activity hosted at MSRI, during the semester programme "Arithmetic Statistics" in 2011, and ICERM, during the semester programme "Computational Aspects of the Langlands Program" in late 2015 (see <http://icerm.brown.edu/sp-f15/>). All members of the organizing committee for the latter are LMFDB contributors, and we expect that the LMFDB will make a substantial leap forward during the semester.

The AIM connection remains strong: both Brian Conrey (Director of AIM) and David Farmer (Director of Programs at AIM) are number theorists who have been intimately connected with the project from the start.

We have Editorial and Management Boards, but essentially all decisions are made by consensus at workshops.

## Funding

During 2008–2012, the LMFDB was funded by NSF FRG grant DMS:0757627; currently (2013-2019) it is supported by Programme Grant EP/K034383/1 from the UK research council EPSRC. The investigators on this are the author and Samir Siksek (Warwick) and Brian Conrey (AIM and Bristol), and Andy Booker and Jon Keating (Bristol). David Farmer (AIM) is a project partner, as are Fernando Rodriguez-Villegas (ICTP), William Stein (Washington) and Mike Rubinstein (Waterloo).

These research grants provide funding both for LMFDB workshops and for servers hosting the database and website; the NSF FRG grant also paid for some technical software support.

## Collaboration

The LMFDB encompasses such a wide range of mathematics, and it is essential to have an equally wide range of mathematical expertise contributing to the project. Many of the collaborators on the LMFDB project, who are all listed at <http://www.lmfdb.org/acknowledgment>, have contributed not by coding for the website but by providing the data (without which the project would be nothing!). More contributors are always welcome.

# Acknowledgements

Funding for the LMFDB project as a whole has been mentioned above. For a full list of contributors to the LMFDB project, financial and other support, see <http://www.lmfdb.org/acknowledgment>. The author personally acknowledges support from EPSRC Programme Grant EP/K034383/1 "LMF: L-Functions and Modular Forms" and (since 1 September 2015) from the OpenDreamKit Horizon 2020 European Research Infrastructures project (#676541).


abstract: Several audio-visual speech recognition models have been recently proposed which aim to improve the robustness over audio-only models in the presence of noise. However, almost all of them ignore the impact of the Lombard effect, i.e., the change in speaking style in noisy environments which aims to make speech more intelligible and affects both the acoustic characteristics of speech and the lip movements. In this paper, we investigate the impact of the Lombard effect in audio-visual speech recognition. To the best of our knowledge, this is the first work which does so using end-to-end deep architectures and presents results on unseen speakers. Our results show that properly modelling Lombard speech is always beneficial. Even if a relatively small amount of Lombard speech is added to the training set then the performance in a real scenario, where noisy Lombard speech is present, can be significantly improved. We also show that the standard approach followed in the literature, where a model is trained and tested on noisy plain speech, provides a correct estimate of the video-only performance and slightly underestimates the audio-visual performance. In case of audio-only approaches, performance is overestimated for SNRs higher than -3dB and underestimated for lower SNRs.
address:  $^1$Imperial College London
  $^2$Samsung AI Center, Cambridge
bibliography: mybib.bib
title: Investigating the Lombard Effect Influence on End-to-End Audio-Visual Speech Recognition

**Index Terms**: Audio-Visual Speech Recognition, Lombard Speech, End-to-End models

# Introduction

It is well known that speakers adapt their speaking style in noisy backgrounds in order to make their speech more intelligible. This is known as the Lombard effect and it is acoustically characterised by an increase in the sound intensity, fundamental frequency, vowel duration and a shift in the formant frequencies . Visually, it is characterised by hyper-articulation and more pronounced rigid-head motion .

Recently several audio-visual speech recognition models have been presented which aim to augment the performance of acoustic speech recognisers. The main application of such systems is in noisy acoustic environments since the main assumption is that the visual signal is not affected by noise and can therefore enhance the performance of speech recognition systems. However, this assumption is not true due to the Lombard effect which also affects the lip movements. In addition, such models are usually trained with plain[^1] speech which is artificially mixed with additive noise. This approach does not correspond to a realistic scenario where Lombard (and not plain) speech is mixed with noise. This mismatch can potentially harm the performance of audio-only, video-only and audio-visual speech recognisers.

Few works have investigated the impact of the Lombard effect on audio-only speech recognition . The main finding is that the performance of a model trained on plain speech mixed with noise is significantly degraded when tested on noisy Lombard speech. This is true even when compensated Lombard speech is used, i.e., the Lombard utterances are normalised to the same energy as the plain speech utterances, although the performance drop is smaller in this case . A similar performance degradation has also been reported for speaker recognition . However, if noisy Lombard speech is used for training then a significant improvement is reported. It is also worth pointing out that the performance of a model trained and tested on noisy Lombard is higher than a model trained and tested on noisy plain speech .

Even fewer works have investigated the effect of the Lombard reflex on visual and audio-visual speech recognition and the results are not conclusive. Marxer et al. report an improvement on the recognition of visual Lombard speech no matter if the model is trained on plain or Lombard speech. As expected the improvement is higher when visual Lombard speech is used for training. On the other hand, Heracleous et al. reported a performance drop when there is a mismatch between training and testing conditions. The same conclusion was also reached when an audio-visual speech recognition system was used. Finally, it has recently been shown that the mismatch between plain and Lombard speech can also affect the performance of audio-visual speech enhancement models .

In this work, we investigate the impact of the Lombard effect on end-to-end audio-only, video-only and audio-visual speech recognition. To the best of our knowledge, this is the first work that studies the Lombard effect within the framework of deep end-to-end models which learn to extract features directly from the raw images and audio waveforms. This is in contrast with the majority of previous works which used hand-crafted features in combination with Gaussian Mixture Models-Hidden Markov Models (GMM-HMMs).

In addition, we also consider both multi-speaker and subject-independent scenarios. The former has been extensively studied in previous works and offers an insight on the impact of the Lombard effect. However, in a real scenario we are mainly interested in the performance on unseen speakers. Hence, we first conduct multi-speaker experiments in order to test the claims made by prior works. Then we also conduct subject-independent experiments in order to investigate the performance on unseen speakers which has not been explored before. This is of particular interest since it is known that the degree of the Lombard effect is highly speaker dependent .

Finally, we report results on sentence-level speech recognition. This is in contrast to previous works which mainly focus either on isolated words or on specific words within a sentence . We believe that the conclusions reached by this approach can be more useful for a practical speech recognition system where the goal will most likely be to recognise all words in a sentence rather than recognise just isolated words.

We show that properly modelling Lombard speech during training leads to improved performance for audio-only, video-only and audio-visual speech recognition models in all experiments. We also show that in subject-independent experiments, including even a relatively small set of Lombard speech during training can significantly improve the performance of an audio-visual speech recogniser in real conditions, i.e., when testing on noisy Lombard speech. Finally, we show that the standard approach followed in the literature, where noise is mixed with plain speech for training and testing, overestimates the actual performance of audio-only models on noisy Lombard speech for Signal-to-Noise ratios (SNRs) higher than -3dB but underestimates it for lower SNRs. On the other hand, the visual performance is correctly estimated in all scenarios and the audio-visual performance is slighlty underestimated.

# Lombard Grid Database

For the purpose of this study, we use the Audio-Visual Lombard Gird corpus . The corpus consists of 5400 utterances from 54 speakers (30 females and 24 males), with 100 utterances (50 Lombard and 50 plain) per speaker. Each utterance is composed of a six word sequence from the combination of the following components: \<command: 4\>\<colour: 4\>\<preposition: 4\>\<letter: 25\>\<digit: 10\>\<adverb: 4\>, where the number of choices for each component is indicated in the angle brackets.

During speaking, both frontal and profile faces were simultaneously recorded at 25 frames per second (fps) and audio was recorded at 48kHz and downsampled to 16kHz. Recordings for each utterance were collected in two conditions, Lombard (L) and Non-Lombard (NL). The NL condition was performed by reading sentences to a condenser microphone placed 30cm in front of the participants, in which the own-voice attenuation was compensated. The L condition follows the same setting, but speech-shaped noise at 80dB sound pressure level was presented to participants via headphones.

# Architecture

The end-to-end audio-visual speech recognition architecture is shown in Fig. and is similar to the one proposed in . A CTC loss is added so the model can recognise continuous speech.

## Visual Stream

The visual stream consists of a spatiotemporal convolutional layer, followed by a ResNet-18 and a 2-layer BGRU. Specifically, the temporal-wise 3D convolutional layer has a kernel size of 5 frames. Then, frame-level features are extracted by ResNet-18. The output of ResNet-18 is fed to a 2-layer BGRU to model the temporal dynamics of visual features. Note that the outputs of the forward and backward GRU are concatenated together instead of added together. This means that although there are 128 GRU cells, the features produced by the GRU have a dimensionality of 256.

## Audio Stream

The audio stream consists of 5 temporal convolutional blocks, followed by a 2-layer BGRU and an average pooling layer. Each convolutional block includes a temporal convolutional layer, ReLU activation and batch normalisation. The first temporal convolutional layer uses a kernel of 5ms and a stride of 0.25ms to extract fine-scale spectral information. The output of the convolutional layers is fed to a 2-layer BGRU. Similarly to the visual stream, the outputs of the forward and backward BGRUs are concatenated. Finally, an averaging pooling layer is used to reduce the audio frame rate to the visual frame rate.

## Fusion Layers

Once the 256 audio features and 256 visual features are extracted, they are concatenated and fed into a 2-layer BGRU to model their temporal dynamics. Then a softmax layer follows which provides the characters probabilities for each frame.

## Connectionist Temporal Classification

The CTC loss is used to transcribe directly between inputs and target outputs without any intermediate annotation. Given an input sequence ${\mathbf x}=(x_1, ..., x_T)$, CTC sums over the probabilities of all valid alignments with length $T$ to obtain the posterior of the target sequence ${\mathbf y}=(y_1, ..., y_L)$: $$p({\mathbf y}|{\mathbf x}) = \sum_{valid Alignments}\prod_{t=1}^Tp_t(a_t|{\mathbf x})$$ where $p_t(a_t|{\mathbf x})$ is the per time-step probability and the product computes the probability of a single valid alignment. The CTC loss is computed by the negative log likelihood of the posterior probability.

# Experimental Setup

## Preprocessing

### Video Preprocessing

We use dlib to detect and track facial landmarks for frontal faces and the face alignment library proposed in for profile faces. The faces are first aligned using a neutral reference frame in order to normalise them for rotation and size differences. This is performed using an affine transform using 5 stable points, two eyes corners in each eye and the tip of the nose. Then the centre of the mouth is located based on the tracked points and a bounding box of 140 by 200 and 80 by 60 is used to extract the mouth region of interest (ROI) on frontal and profile faces, respectively.

### Audio Preprocessing

Lombard utterances have greater energy than plain speech utterances so for a given noise level their SNR is higher than noisy plain speech. So similarly to we also generate 'compensated' Lombard speech, where the energy of Lombard speech is normalised to the same energy as plain speech. In this case, the SNR between Lombard and plain utterances is the same for a given noise level.

To remove the artificial variability of the signals caused by the speaker-to-microphone distance, we follow the approach suggested in . We normalise the non-Lombard and 'compensated' Lombard signals to the same root mean square (RMS) of 0.05. For the Lombard signals, we set the RMS to $0.05\cdot \bar{x}_{rms}^L/\bar{x}_{rms}^{NL}$, where $\bar{x}_{rms}^L$ and $\bar{x}_{rms}^{NL}$ are the average RMS value on Lombard speech and non-Lombard speech corpus.

## Data Augmentation

During training, two data augmentation methodologies are performed in raw images, random cropping and horizontal flipping. Specifically, each frontal mouth ROI is randomly cropped to a size of 130 by 190 and each profile mouth ROI is randomly cropped to a size of 75 by 55. During testing, the central patch is cropped. Horizontal flipping with a probability of 0.5 is used to increase the variation on training samples.

Babble noise at different levels is added into the audio waveforms during training. The SNR levels range from -15dB to 6dB with an interval of 3dB. One of the noise levels or the clean signal is selected under a uniform distribution, which enhance robustness to different noise levels.

### Training

We first train each stream from scratch. An initial learning rate of 0.001 and a mini-batch of 64 are used for the audio stream and an initial learning rate of 0.0003 and a mini-batch of 10 are used for the visual stream. We train the audio stream for 400 epochs and the visual stream for 120 epochs separately. Once the audio and visual streams have been trained, their weights are fixed and the 2-layer BGRU used for fusion is trained with an initial learning rate of 0.0003 and a mini-batch of 10. Finally, the entire audio-visual model is fine-tuned for another 40 epochs.

# Results

| Views         |  L-L  | NL-L  | NL-NL |
|:--------------|:-----:|:-----:|:-----:|
| WER (Frontal) | 23.57 | 26.05 | 25.59 |

Video-only results on a multi-speaker scenario. L: Lombard, NL: non-Lombard. X-Y indicates a model trained on X (L or NL) speech and tested on Y (L or NL) speech.

## Multi-speaker experiments

In this set of experiments, we investigate the impact of the Lombard effect in a multi-speaker scenario when end-to-end deep models are used for speech recognition. For the purpose of this study, we use 30, 10 and 10 utterances from each subject for training, validation and testing, respectively. A similar study has been conducted in , but a traditional GMM-HMM approach was followed and SNR-specific models were trained.

- 1 For comparison purposes we first train SNR-specific audio-only models for non-Lombard and Lombard speech similarly to . Results are shown in Fig. and overall are consistent with the results presented in . We notice that when we train a model on non-Lombard speech and test it on Lombard speech (red solid line), a significant drop in performance compared to testing on non-Lombard speech (orange solid line) is observed between -9dB and 6dB. This is mainly the consequence of the SNR mismatch between Lombard and plain speech. However, between -12dB and -15dB, there is no difference between the two training approaches. When we test on 'compensated' Lombard speech (blue solid line), the results are still worse than non-Lombard speech (up to 4%). This indicates that not only the SNR mismatch affects the performance but also the difference in acoustic characteristics between Lombard and non-Lombard speech, to a smaller extent though.

Results for multi-speaker experiments where a single model is trained using the SNR augmentation approach from section are shown in Fig. . The main difference with the previous set of experiments is that the performance on Lombard speech (red solid line), for a model trained on non-Lombard speech, is better than the performance on non-Lombard speech (orange solid line) between -15dB and -6 dB. This is probably due to the fact that during training all SNR levels are seen so the influence of the SNR mismatch between Lombard and plain speech is minimised. The same pattern is also observed for 'compensated' Lombard speech (blue solid line). This indicates that although at higher SNRs the performance of a model trained and tested on non-Lombard speech, which is the usual approach in the literature, overestimates the actual performance, in lower SNRs it actually underestimates it. It is also worth pointing out that when we train on Lombard speech, a significant improvement in performance is observed when we test on Lombard speech (green solid line) compared to training on non-Lombard speech and testing either on Lombard (red solid line) or non-Lombard speech (orange solid line).

- 1 The results of video-only models are reported in Table  . A slight improvement of 0.45% is reported in the case of NL-L over NL-NL. This is not entirely consistent with who reported a greater improvement of 4.6%. We also notice that L-L has an absolute improvement of 2.48% compared to NL-L, which shows the benefit of properly modelling Lombard speech.

The results of audio-visual models are shown in Fig. . As expected the audio-visual models have a lower WER compared to audio-only models across all noise levels. It is worth pointing out again that when Lombard speech is properly modelled then a better performance is achieved (green dashed line vs red dashed line).

## Subject-independent experiments

Previous experiments considered multi-speaker models. However, in real scenarios, we would like to have a model that works on unseen subjects. To better investigate the impact of Lombard effect in subject-independent experiments, the training, validation and test sets are divided into 36, 6 and 12 subjects, respectively. It is important to note that the same number of female and male speakers are included on validation and test sets.

The results of audio-only experiments are shown in Fig. . Similar conclusions to the ones drawn for multi-speaker experiments in Section can be drawn. The performance on Lombard speech (red solid line), for a model trained on non-Lombard speech, is better than the performance on non-Lombard speech (orange solid line) between -15dB and -6 dB. The same pattern is also observed for 'compensated' Lombard speech (blue solid line). Again, this demonstrates that the approach followed in the literature, i.e., training and testing on non-Lombard speech, overestimates the actual performance at higher SNRs but underestimates it in lower SNRs.

| Views         |  L-L  | NL-L  | NL-NL |
|:--------------|:-----:|:-----:|:-----:|
| WER (Frontal) | 25.00 | 27.84 | 27.66 |
| WER (Profile) | 39.45 | 47.61 | 47.47 |

Video-only results on subject-independent experiments. L: Lombard, NL: non-Lombard. X-Y indicates a model trained on X (L or NL) speech and tested on Y (L or NL) speech.

The video-only results are reported in Table . When we train and test a model on Lombard speech, an absolute improvement of 2.84% and 8.16% is observed in frontal faces and profile faces, respectively, over the NL-L scenario. The performance of NL-L is very similar to NL-NL which reveals that the approach followed in the literature (NL-NL) provides a correct estimate of the actual performance (NL-L). We also notice that the performance on profile faces is much worse due to less information being available as well as inaccurate tracking in profile videos.

The results of audio-visual models are shown in Fig. . Similarly to the multi-speaker scenario, the best performance is achieved when a model is trained and tested on Lombard speech (green dashed line). It is also clear that training and testing on plain speech (orange dashed line) slightly underestimates the performance of the real scenario, where Lombard speech is used for testing (red dashed line).

Fig. shows the performance of an audio-visual model as a function of the percentage of Lombard speech combined with plain speech for training. It is clear that even when the Lombard speech utterances added to the training set account for 25% of plain speech the gap between NL-L and L-L is reduced to half. Also, when Lombard speech accounts for 50% of plain speech similar performance to the L-L scenario is achieved for very low SNRs.

# Conclusions

In this work, we investigate the impact of the Lombard effect on audio-only, video-only and audio-visual speech recognition. We show that it is always beneficial to properly model Lombard speech. We also show that training and testing on noisy plain speech, which is commonly used in the literature, is a good estimate for the performance on visual Lombard speech but a bad estimate for the performance of audio-only speech recognition. It would be interesting to investigate in future works how different types of background noise affect the performance of audio-visual speech recognition models.

# Acknowledgements

We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan V GPU used for this research. The work of Pingchuan Ma has been funded by the Honda Research Institute.

[^1]: The terms plain and non-Lombard speech are used interchangeably in this work.


abstract: A lot of mathematical knowledge has been formalized and stored in repositories by now: different mathematical theorems and theories have been taken into consideration and included in mathematical repositories. Applications more distant from pure mathematics, however — though based on these theories — often need more detailed knowledge about the underlying theories. In this paper we present an example Mizar formalization from the area of electrical engineering focusing on stability theory which is based on complex analysis. We discuss what kind of special knowledge is necessary here and which amount of this knowledge is included in existing repositories.
author: Agnieszka Rowinska-Schwarzweller; Christoph Schwarzweller
title: On Building a Knowledge Base for Stability Theory [^1]

# Introduction

The aim of mathematical knowledge management is to provide both tools and infrastructure supporting the organization, development, and teaching of mathematics with the help of effective up-to-date computer technologies. To achieve this ambitious goal it should be taken into account that the predominant part of potential users will not be professional mathematicians themselves, but rather scientists or teachers that apply mathematics in their special domain. To attract this group of people it is essential that our repositories provide a sufficient knowledge base for those domains. We are interested in how far existing mathematical repositories are from meeting this precondition yet, or in other words: How big is the gap between the knowledge already included in repositories and the knowledge necessary for particular applications?

This problem, however, concerns not only the simple question how much knowledge of a domain is available in a repository. We believe, that in order to measure this gap, it is equally important to consider the basic conditions for a successful formalization of applications on top of existing knowledge, that is on top of a mathematical repository: The more easy such a formalization is, the more attractive is a mathematical repository. To describe attractiveness of a repository for an application one can identify three major points:

-   Amount of knowledge
    This is the obvious question, how much knowledge of a particular domain already has been formalized and included in the repository. Basically, the more knowledge of a domain is included the more attractive a repository is for applications.

-   Representation of knowledge
    This concerns the question of how the knowledge has been defined and formalized: Often mathematicians use more abstract constructions than necessary — and attractive — for applications. An example is the construction of rational functions from polynomials.

-   Applicability of knowledge
    This point deals with both how the knowledge of a domain is organized in a repository and the question of how easy it is to adapt available knowledge to one's own purposes.

In this paper we focus on electrical engineering, in particular on network stability . Network theory deals with the mathematical description, analysis and synthesis of electrical (e.g. continuous, time-discrete or digital) networks. For a reliable application such systems have to be (input/output-) stable, that is for an arbitrary bounded input the output have to be bounded again. In practice it is impossible to verify responses for all input signals. In this situation there is, however, a number of theorems permitting easier methods to decide whether a network is stable. We shall introduce the mathematical fundamentals and prequisites of one example theorem and present a Mizar formalization of this theorem. After that we discus our formalization in the spirit of the three points from above: How far is the Mizar sytem from providing a suitable mathematical repository for applications in stability theory?

# Networks and their Stability

As mentioned in the introduction the (input/output-) stability of networks is one of the main issues when dealing with the analysis and design of electrical circuits and systems. In the following we briefly review definitions and properties of electrical systems necessary to understand the rest of the paper. In electrical engineering stability applies to the input/output behaviour of networks (see figure 1). For (time-) continuous systems one finds the following definition. For discrete systems an analogous definition is used.

Physically realizable, linear time-invariant systems (LTI systems) can be described by a set of differential equations . The behaviour of a LTI system then is completely characterized by its impulse response $h(t)$.[^2] If the impulse response of auch a system is known, the relation between the input $x(t)$ and the output $y(t)$ is given by the convolution integral $$\begin{aligned}
y(t) \ = \ \int \limits _{-\infty}^{\infty} x(\tau) h(t-\tau) \mbox{d}\tau .
\end{aligned}$$ Furthermore, a LTI system is stable, if and only if its impulse response $h(t)$ is absolute integrable, that is there exists a constant $K$ such that $$\begin{aligned}
\int \limits_{-\infty}^{\infty} \left | h(\tau) \right | \mbox{d}\tau \ \leq \ K \ < \ \infty .
\end{aligned}$$ In network and filter analysis and design, however, one commonly employs the frequency domain rather than the time domain. To this end the system is described based on its transfer function $H(s)$. In case the Laplace transformation is used we have[^3] $$\begin{aligned}
H(s) \ = \ \int \limits_{-\infty}^{\infty} h(t)e^{-st} \mbox{d}t . \\ \nonumber
\end{aligned}$$ where $s = \sigma + j\omega$ is a complex variable with $\Re\{s\} = \sigma$ and $\Im\{s\} = \omega$.

The evaluation of $H(s)$ for $s = j\omega$ — in case of convergence[^4] — enables the qualitative understanding of how the system handles and selects various frequencies $\omega$, so for example whether the system describes a high-pass filter, low-pass filter, etc. Now the necessary condition to demonstrate the stability of LTI systems in the frequency domain reduces to show, that the $j\omega$-axis lies in the Laplace transformation's region of convergence (ROC).

For physically realizable LTI systems, such as the class of networks with constant and concentrated parameters, $H(s)$ is given in form of a rational function with real coefficients, that is $$\begin{aligned}
H(s) \ = \ \frac{a_n s^n + \ldots + a_0}{b_m s^m + \ldots + b_0}, \ \ \ a_i,b_i \in \mathbb{R}.
\end{aligned}$$ In this case the region of convergence can be described by the roots of the denominator polynomial: If $s_i = \sigma_i + j\omega_i$ for $i = 1,\ldots m$ are the roots of $b_m s^m + \ldots + b_0$, the region of convergence is given by $$\Re \{ s \} \ > \max \{ \sigma_i,\ i = 1, \ldots m \} .$$ To check stability it is therefore sufficient, to show that the real part $\Re \{ s \}$ of all poles of $H(s)$ is smaller then $0$. The denominator of $H(s)$ is thus a so-called Hurwitz polynomial.

Note that the stability problem for discrete-time signals and systems can be analized with the same approach. For a given discrete-time transfer function $H(z)$ in the $Z$- domain, it has to be checked whether the unit circle is contained in the region of convergence. Hence for all poles $z_i$ of $H(z)$ we must have $|z_i| < 1$. Using bilinear transformations $$\begin{aligned}
z \ := \ \frac{1+s}{1-s}.
\end{aligned}$$ it is thus sufficient to check whether the denominator of $$\begin{aligned}
\left. H(z) \right | _{z := \frac{1+s}{1-s}}
\end{aligned}$$ is a Hurwitz polynomial.

The practical proof of stability of high-precision filters, however, turns out to be very hard. In practical applications the poles of concern are usually close to the axis $s = j\omega$ or the unit circle $|z| = e^{j\omega}$ respectively. Thus numerical determination of the poles is highly error-pruning due to its rounding effects. In digital signal processing in addition degrees of transfer functions tend to be very high, for example 128 and higher in communication networks.

An interesting and in practice often used method to check the stability of a given network is based on the following theorem.
**Theorem 1.** ()
Let $f(x)$ be a real polynomial with degree $n \geq 1$. Furthermore let all coefficients of $f(x)$ be greater than $0$. Let $f_e(x)$ and $f_o(x)$ denote the even part resp. the odd part of $f(x)$. Assume further that $$Z(x) \ = \ \frac{f_e(x)}{f_o(x)}$$ or the reciprocal of $Z(x)$ is a reactance one-port function of degree $n$. Then $f(x)$ is a Hurwitz polynomial.
The concept of reactance one-port function stems from electrical network theory: In arbitrary passive (that is RLC-) networks we find the following relations between the complex voltage $U_{\nu}(s)$ and the complex current $I_{\nu}(s)$:

$$\begin{array}{lclll}
U_{r}(s) & = & R_r \cdot I_{r}(s) & & \mbox{for a resistor } R_r \\ \\
U_{l}(s) & = & s \cdot L_l \cdot I_{l}(s) & & \mbox{for an inductance } L_l \\ \\
U_{k}(s) & = & \frac{1}{s\cdot C_k} \cdot  I_{k}(s) & & \mbox{for a capacity } C_k \\ \\
\end{array}$$

An impedance (complex resistor) or admittance (complex conductance) composed of network elements R, L and C only is called a (RLC-) one-port function, an impedance or admittance composed of network elements L and C only is called a reactance one-port function. Conversely, for every one-port function $Z(s)$ there exists at least one one-port, whose impedance or admittance is equal to $Z(s)$:

Hence, theorem 1 reduces stability checking to the considerable easier task to synthesize a one-port solely using inductors (L) and capacitors (C), that is to synthesize a reactance one-port. To this end there exist easy procedures like for example Routh's method to construct a chain one-port .
It turns out that one-port functions $Z(s)$ are exactly the real positive rational functions. For a real function we have that for real $s$ also $Z(s)$ is real[^5], and a positive function means that $\Re\{s\} > 0$ implies $\Re\{Z(s)\} > 0$. A reactance one-port function is a one-port function, that is in addition odd. The property of being positive is closely connected to Hurwitz polynomials:
**Theorem 2.** ()
Let $f(x)$ be a real polynomial with degree $n \geq 1$. Furthermore let all coefficients of $f(x)$ be greater than $0$. Let $f_e(x)$ and $f_o(x)$ denote the even part resp. the odd part of $f(x)$. Assume that$f_e(x)$ and $f_o(x)$ have no common roots and that $Z(x) = f_e(x)/f_o(x)$ is positive. Then

-   $\Re\{Z(x)\} \geq 0$ for all $x$ with $\Re\{x\} = 0$

-   $f_e(x) + f_o(x)$ is a Hurwitz polynomial.

In section we will see that this theorem is also the key to prove that stability checking can be reduced to synthesizing LC-one-ports, in other words to prove theorem 1 from above.

# The Mizar System

The logical basis of Mizar is classical first order logic extended, however, with so-called schemes. Schemes introduce free second order variables, in this way enabling among others the definition of induction schemes. In addition Mizar objects are typed, the types forming a hierarchy with the fundamental type `set`. The user can introduce new (sub)types describing mathematical objects such as groups, fields, vector spaces or polynomials over rings or fields. To this end the Mizar language provides a powerful typing mechanism based on adjective subtypes .

The current development of Mizar relies on Tarski-Grothendieck set theory — a variant of Zermelo Fraenkel set theory using Tarski's axiom on arbitrarily large, strongly inaccessible cardinals which can be used to prove the axiom of choice —, though in principle the Mizar language can be used with other axiom systems also. Mizar proofs are written in natural deduction style as presented in the calculus of . The rules of the calculus are connected with corresponding (English) natural language phrases so that the Mizar language is close to the one used in mathematical textbooks. The Mizar proof checker verifies the individual proof steps using the notion of obvious inferences to shorten the rather long proofs of pure natural deduction.

Mizar objects are typed, the types forming a hierarchy with the fundamental type `set` . New types are constructed using type constructors called modes. Modes can be decorated with adjectives — given by so-called attribute definitions — in this way extending the type hierarchy: For example, given the mode `Ring` and an attribute `commutative` a new mode `commutative Ring` can be constructed, which obeys all the properties given by the mode `Ring` plus the ones stated by the attribute `commutative`. Furthermore, a variable of type `commutative Ring` then is also of type `Ring`, which implies that all notions defined for `Ring` are available for `commutative Ring`. In addition all theorems proved for type `Ring` are applicable for objects of type `commutative Ring`; indeed the Mizar checker itself infers subtype relations in order to check whether notions and theorems are applicable for a given type.

# Mizar Formalization of the Theorem

## Some Preliminaries About Rational Functions

Although the theory of polynomials in Mizar is rather well developed, rational functions have not been defined yet. Rational functions can — analogously to polynomials — be defined over arbitrary fields: Rational functions are simply pairs of polynomials whose second component is not the zero polynomial.[^6] These can be easily introduced as a Mizar type `Rational_function of L`, where `L` is the underlying coefficient domain.

    definition
    let L be non trivial multLoopStr_0;
    mode rational_function of L means
      ex p1 being Polynomial of L st
      ex p2 being non zero Polynomial of L st it = [p1,p2];
    end;

In Mizar the result types of the pair constructor `[ , ]` and the projections `` `1 `` and `` `2 ``, that in the original definition are simply `set`, then can be modified into `Rational_function` and `(non zero) Polynomial` respectively. In addition one can introduce the usual functions `num` and `denom` as synonyms for the corresponding projections.

    definition
    let L be non trivial multLoopStr_0;
    let p1 be Polynomial of L;
    let p2 be non zero Polynomial of L;
    redefine func [p1,p2] -> rational_function of L;
    end;

    definition
    let L be non trivial multLoopStr_0;
    let z be rational_function of L;
    redefine func z`1 -> Polynomial of L;
    redefine func z`2 -> non zero Polynomial of L;
    end;

    notation
    let L be non trivial multLoopStr_0;
    let z be rational_function of L;
    synonym num(z) for z`1;
    synonym denom(z) for z`2;
    end;

Now that `num` and `denom` — applied to rational functions — have result type `Polynomial`, operations for rational functions can straightforwardly be defined by employing the corresponding functions for polynomials. So, for example, the evaluation of rational functions can be defined using evaluation of polynomials and the division operator `/` defined for arbitrary fields `L`.

    definition
    let L be Field;
    let z be rational_function of L;
    let x be Element of L;
    func eval(z,x) -> Element of L equals
      eval(num(z),x) / eval(denom(z),x);
    end;

Note that according to the definition of `eval` for polynomials the type of the first argument — that is of `num(z)` and `denom(z)` — has to be `Polynomial`. This is ensured by the redefinitions from above, which in this sense allow for reusing the operations defined for polynomials in the case of rational functions. Other necessary operations for rational functions such as the degree or arithmetic operations can be defined the same way.

## The Theorem

Using the general Mizar theory of polynomials and rational functions for our purposes, that is for complex numbers, is straightforward. We just instantiate the parameter `L` describing the coefficient domain with the field of complex numbers `F_Complex` from . So an object of type $$\verb@rational_Function of F_Complex@$$ combines the theory of rational functions with the one of complex numbers.

Further properties necessary to state the main theorem are introduced by defining appropriate attributes for polynomials and rational functions resp. Note that these definitions apply to polynomials and rational functions over the complex numbers only given by the instantiated Mizar types mentioned above.

    definition
    let p be Polynomial of F_Complex;
    attr p is real means
      for i being Element of NAT holds p.i is real number;
    end;

    definition
    let p be rational_Function of F_Complex;
    attr Z is positive means
      for x being Element of F_Complex
      holds Re(x) > 0 implies Re(eval(Z,x)) > 0;
    end;

Using these attributes — and the attribute `odd` describing odd functions — we can then introduce one-ports and reactance one-ports in Mizar by the following mode definitions.

    definition
    mode one_port_function is real positive rational_function of F_Complex;
    mode reactance_one_port_function is
                          odd real positive rational_function of F_Complex;
    end;

We also needed to define the odd and the even part of a polynomial `f`. This is accomplished by two Mizar functors `even_part(f)` and `odd_part(f)`, which however can be defined straightforwardly. Finally we formalize the condition from the theorem, that all the coefficients of the given polynomial $f$ should be greater than $0$ as usual as a Mizar attribute:

    definition
    let f be real Polynomial of F_Complex;
    attr f is with_positive_coefficients means
      for i being Element of NAT st i <= deg p holds p.i > 0;
    end;

Note that for a real polynomial $f$ with positive coefficients and deg$(f) \geq 1$ both the even and the odd part of $f$ are not $0$, hence both can appear as the denominator of a rational function. Thus prepared we can state theorem 1 from section in Mizar. Note again that due to the redefinitions of section the functor `[ , ]` returns a rational function.

    theorem
    for p be non constant with_positive_coefficients
            (real Polynomial of F_Complex)
    st [even_part(p),odd_part(p)] is reactance_one_port_function &
       degree([even_part(p),odd_part(p)]) = degree p
    holds p is Hurwitz;

The proof of the theorem, as already indicated, basically relies on theorem 2 from section , which connects rational functions with the property of being a Hurwitz polynomial. Based on our development from above one can formulate this theorem as follows.

    theorem
    for p be non constant with_positive_coefficients
            (real Polynomial of F_Complex)
    st [even_part(p),odd_part(p)] is positive &
       even_part(p),odd_part(p) have_no_common_roots
    holds (for x being Element of F_Complex
           st Re(x) = 0 & eval(odd_part(p),x) <> 0
           holds Re(eval([even_part(p),odd_part(p)],x)) >= 0) &
          even_part(p) + odd_part(p) is Hurwitz;

The corresponding Mizar proof is rather technical. The basic idea consists of considering in addition to $$\begin{aligned}
 Z(x) \ = \  \frac{f_e(x)}{f_o(x)}
\end{aligned}$$ the rational function $$\begin{aligned}
 \label{w}
 W(x) \ = \  \frac{Z(x)-1}{Z(x)+1}
      & = &  \frac{\mbox{num}(Z)(x) - \mbox{denom}(Z)(x)}
                  {\mbox{num}(Z)(x) + \mbox{denom}(Z)(x)}.
\end{aligned}$$ and to analyze the absolute values $|W(x)|$. If $Z(x)$ is positive, then $|W(x)| \leq 1$ for all $x$ with $\Re(x) \geq 0$, which implies that $W(x)$ has no poles for $\Re(x)\geq 0$. Thus the denominator polynomial can have roots only for $\Re(x) < 0$, so $\mbox{num}(Z)(x) + \mbox{denom}(Z)(x)$ is a Hurwitz polynomial.

The main theorem now easily follows from theorem 2, because the degree condition implies that `even_part(p)` and `odd_part(p)` have no common roots.

# Discussion — Lessons Learned

In the following we discuss our formalization from the last section with respect to the tree criteria developed in the introduction. Though restricted to Mizar we claim that the situation in other repositories is similar, so that most of our results hold in a more general context also.

In we already presented a Mizar formalization of Schur's theorem, another helpful criterion for stability checking. Based on polynomials only its formalization was rather harmless. The only missing point that caused some work was division of polynomials. However, as we will see, stability checking in general needs definitely more extension than in this case.

## Amount of Knowledge

Complex numbers and polynomials (over arbitrary rings) are included in MML. A lot of theorems have been proved here, so that almost all we needed could be found in the repository. Interestingly rational functions — a rather basic structure — had not been defined, yet. The reason might be that rational functions are mathematically rather simple and this is the first time that a theorem relying on rational functions has been formalized.

Though even and odd functions were already included in MML, the even and odd part of a polynomial was not. This, however, comes with no surprise, just because these polynomial operations are rather seldom used. We hence had to prove a number of theorems dealing with these polynomials, most of them however being elementary like for example

    theorem
    for p being real Polynomial of F_Complex
    for x being Element of F_Complex st Re(x) = 0
    holds Re(eval(odd_part(p),x)) = 0;

Summarizing, besides the lack of rational functions, MML provides the amount of knowledge for stability checking one could expected.

## Representation of Knowledge

The construction of rational functions can be performed in different ways. Of course, one can define rational functions as pairs of polynomials. On the other hand there is the possibility to construct (the field of) rational functions as the completion of polynomial rings. Though the second version is mathematically more challenging we decided to use pairs. We wanted to emphasize the contribution to applications by concentrating on prior knowledge of potential users: Electrical engineers are probably not interested in (working with!) abstract algebra, their interests and needs are different.

In the same context there is another representational problem: In MML we find both the complex numbers and the field of complex numbers. Not a problem in itself, this may cause some confusion when searching for notions and theorems: The functors R͡e and `Im` giving the real and imaginary part, for exmaple, are defined for complex numbers only, thus — theoretically — not applicable to elements of a field. In Mizar, however, this is not necessarily the case: Using a special registration — `identify` — the user can identify terms and operations from different structures, here complex numbers with elements of the field of complex numbers:

    registration
      let a,b be complex number;
      let x,y be Element of F_Complex;
      identify x+y with a+b when x=a, y=b;
      identify x*y with a*b when x=a, y=b;
    end;

In effect, after this registration functors R͡e and `Im` are applicable to elements of the field of complex numbers.

In general, different views on mathematical objects — here, complex numbers as numbers or elements of a field — have to be handled carefully in mathematical repositories in order to not confuse possible users. Even the rudimentary difference between a polnomial and its polynomial function can lead to surprises and incomprehension for people not familiar with the formal treatment of mathematics in repositories.

## Applicability of Knowledge

As we have already seen, the adaption of general knowledge in MML to special cases is straightforward: One just instantiates parameters describing the general domain with the special one, so for example `Polynomial of F_Complex` for polynomials over the (field of) complex numbers.

This, on the other hands, means that to work with such instantiations the user has to apply theorems about the general structure. Though highly desirable from the mathematical point of view, it is not clear whether this is really convenient for application users: To work in the special field of complex numbers, for example, then means to search for helpful theorems in the theory of fields, rings or even groups and semigroups. Maybe here a search tool that generates and collects theorems for special instances of theories would be a reasonable help.

The organization of MML is mainly by articles in which authors prove not only their main theorems, but also whatever is necessary and not found in MML. As a consequence theorems of the same topic, e.g. polynomials, can be spread over the repository. A step to overcome this shortcoming is an ongoing project called Mizar encyclopedia building articles with monographic character whose contents is semi-automatically extracted from contributed Mizar articles. Unfortunately polynomials have not been considered in this project, yet.

Summarizing the Mizar system though flexible in order to support special applications lacks an organization of its corresponding repository to support application users in their efforts.

# Conclusions

We have presented a Mizar formalization of a theorem for stability checking and have discussed how the knowledge contained in MML supported the process from an application user's point of view. Here we want to emphasize two points.

First, when building a knowledge base for an application area, it is hardly foreseeable what knowledge is necessary. We have seen that the formalization of Schur's theorem went through without major problems, while the present theorem caused definitely more work and preparation. Furthermore, there are theorems on stability checking using even involved mathematical techniques such as, e.g., analytic functions and the maximum principle.

Second, attractiveness of mathematical repositories does not only depend on the amount of knowledge included. Equally important are a clear representation and organization of knowledge in the sense that it stays familiar for users outside the mathematical community.

Consequently is it essential to communicate with experts from the application area. If we want our repositories to be widely used we have both to provide a reasonable knowledge base and to take care of the fact that application users might represent mathematical knowledge in a different way we are used to.

[^1]: The final publication of this paper is available at www.springerlink.com.

[^2]: $h(t)$ is the output of the system, when the input is the Dirac delta function $\delta(t)$.

[^3]: Note that this is a generalization of the continuous-time Fourier transformation.

[^4]: In this case $H(j\omega)$ equals the Fourier transform.

[^5]: This condition implies that the coefficients of $Z(s)$ are real. In network theory, however, this definition is used.

[^6]: Of course rational functions can be introduced "more algebraically" as the quotient field of a polynomial ring. Here we decided to use pairs to concentrate on application issues; see the discussion in section .


address: Department of Physics, University of Notre-Dame, Notre-Dame, IN 46556
author: Albert-László Barabási[^1] and Réka Albert
title: Emergence of Scaling in Random Networks

Systems as diverse as genetic networks or the world wide web are best described as networks with complex topology. A common property of many large networks is that the vertex connectivities follow a scale-free power-law distribution. This feature is found to be a consequence of the two generic mechanisms that networks expand continuously by the addition of new vertices, and new vertices attach preferentially to already well connected sites. A model based on these two ingredients reproduces the observed stationary scale-free distributions, indicating that the development of large networks is governed by robust self-organizing phenomena that go beyond the particulars of the individual systems.

The inability of contemporary science to describe systems composed of non-identical elements that have diverse and nonlocal interactions currently limits advances in many disciplines, ranging from molecular biology to computer science (*1*). The difficulty in describing these systems lies partly in their topology: many of them form rather complex networks, whose vertices are the elements of the system and edges represent the interactions between them. For example, living systems form a huge genetic network, whose vertices are proteins and genes, the edges representing the chemical interactions between them (*2*). At a different organizational level, a large network is formed by the nervous system, whose vertices are the nerve cells, connected by axons (*3*). But equally complex networks occur in social science, where vertices are individuals or organizations, and the edges characterize the social interaction between them (*4*), or describe the world wide web (www), whose vertices are HTML documents connected by links pointing from one page to another (*5, 6*). Due to their large size and the complexity of the interactions, the topology of these networks is largely unknown.

Traditionally, networks of complex topology have been described using the random graph theory of Erdős and Rényi (ER) (*7*), but in the absence of data on large networks the predictions of the ER theory were rarely tested in the real world. However, driven by the computerization of data acquisition, such topological information is increasingly available, raising the possibility of understanding the dynamical and topological stability of large networks.

In this paper we report on the existence of a high degree of self-organization characterizing the large scale properties of complex networks. Exploring several large databases describing the topology of large networks that span as diverse fields as the www or the citation patterns in science we show that, independent of the system and the identity of its constituents, the probability $P(k)$ that a vertex in the network interacts with $k$ other vertices decays as a power-law, following $P(k)\sim k^{-\gamma}$. This result indicates that large networks self-organize into a scale-free state, a feature unexpected by all existing random network models. To understand the origin of this scale invariance, we show that existing network models fail to incorporate growth and preferential attachment, two key features of real networks. Using a model incorporating these two ingredients, we show that they are responsible for the power-law scaling observed in real networks. Finally, we argue that these ingredients play an easily identifiable and important role in the formation of many complex systems, implying that our results are relevant to a large class of networks observed in Nature.

While there are a large number of systems that form complex networks, detailed topological data is available only for a few. The collaboration graph of movie actors represents a well documented example of a social network. Each actor is represented by a vertex, two actors being connected if they were cast together in the same movie. The probability that an actor has $k$ links (characterizing his or her popularity) has a power-law tail for large $k$, following $P(k) \sim k^{-\gamma_{actor}}$, where $\gamma_{actor}=2.3\pm 0.1$ (Fig.$\,$`<!-- -->`{=html}1A). A more complex network with over $800$ million vertices (*8*) is the www, where a vertex is a document and the edges are the links pointing from one document to another. The topology of this graph determines the web's connectivity and, consequently, our effectiveness in locating information on the www (*5*). Information about $P(k)$ can be obtained using robots (*6*), indicating that the probability that $k$ documents point to a certain webpage follows a power-law, with $\gamma_{www}=2.1\pm 0.1$ ( Fig.$\,$`<!-- -->`{=html}1B) (*9*). A network whose topology reflects the historical patterns of urban and industrial development is the electrical powergrid of western US, the vertices representing generators, transformers and substations, the edges corresponding to the high voltage transmission lines between them (*10*). Due to the relatively modest size of the network, containing only 4941 vertices, the scaling region is less prominent, but is nevertheless approximated with a power-law with an exponent $\gamma_{power}\simeq 4$ (Fig.$\,$`<!-- -->`{=html}1C). Finally, a rather large, complex network is formed by the citation patterns of the scientific publications, the vertices standing for papers published in refereed journals, the edges representing links to the articles cited in a paper. Recently Redner (*11*) has shown that the probability that a paper is cited $k$ times (representing the connectivity of a paper within the network) follows a power-law with exponent $\gamma_{cite}=3$.

The above examples (*12*) demonstrate that many large random networks share the common feature that the distribution of their local connectivity is free of scale, following a power-law for large $k$, with an exponent $\gamma$ between 2.1 and 4 which is unexpected within the framework of the existing network models. The random graph model of ER (*7*) assumes that we start with $N$ vertices, and connect each pair of vertices with probability $p$. In the model the probability that a vertex has $k$ edges follows a Poisson distribution $P(k)=e^{-\lambda}{\lambda}^k/k!$, where $\lambda=N\left(\begin{array}{c}N-1\\k\end{array}\right)p^k(1-p)^{N-1-k}$. In the small world model recently introduced by Watts and Strogatz (WS) (*10*), $N$ vertices form a one-dimensional lattice, each vertex being connected to its two nearest and next-nearest neighbors. With probability $p$ each edge is reconnected to a vertex chosen at random. The long-range connections generated by this process decrease the distance between the vertices, leading to a small-world phenomenon (*13*), often referred to as six degrees of separation (*14*). For $p=0$ the probability distribution of the connectivities is $P(k)=\delta(k-z)$, where $z$ is the coordination number in the lattice, while for finite $p$, $P(k)$ is still peaked around $z$, but it gets broader (*15*). A common feature of the ER and WS models is that the probability of finding a highly connected vertex (that is, a large $k$) decreases exponentially with $k$, thus vertices with large connectivity are practically absent. In contrast, the power-law tail characterizing $P(k)$ for the studied networks indicates that highly connected (large $k$) vertices have a large chance of occurring, dominating the connectivity.

There are two generic aspects of real networks that are not incorporated in these models. First, both models assume that we start with a fixed number ($N$) of vertices, that are then randomly connected (ER model), or reconnected (WS model), without modifying $N$. In contrast, most real world networks are open, they form by the continuous addition of new vertices to the system, thus the number of vertices, $N$, increases throughout the lifetime of the network. For example, the actor network grows by the addition of new actors to the system, the www grows exponentially in time by the addition of new web pages (*8*), the research literature constantly grows by the publication of new papers. Consequently, a common feature of these systems is that the network continuously expands by the addition of new vertices that are connected to the vertices already present in the system.

Second, the random network models assume that the probability that two vertices are connected is random and uniform. In contrast, most real networks exhibit preferential connectivity. For example, a new actor is cast most likely in a supporting role, with more established, well known actors. Consequently, the probability that a new actor is cast with an established one is much higher than casting with other less known actors. Similarly, a newly created webpage will more likely include links to well known, popular documents with already high connectivity, or a new manuscript is more likely to cite a well known and thus much cited paper than its less cited and consequently less known peer. These examples indicate that the probability with which a new vertex connects to the existing vertices is not uniform, but there is a higher probability to be linked to a vertex that already has a large number of connections.

We next show that a model based on these two ingredients naturally leads to the observed scale invariant distribution. To incorporate the growing character of the network, starting with a small number ($m_0$) of vertices, at every timestep we add a new vertex with $m$($\leq m_0$) edges that link the new vertex to $m$ different vertices already present in the system. To incorporate preferential attachment, we assume that the probability $\Pi$ that a new vertex will be connected to vertex $i$ depends on the connectivity $k_i$ of that vertex, such that $\Pi(k_i)=k_i/\sum_j k_j$. After $t$ timesteps the model leads to a random network with $t+m_0$ vertices and $mt$ edges. This network evolves into a scale-invariant state with the probability that a vertex has $k$ edges following a power-law with an exponent $\gamma_{model}=2.9\pm 0.1$ (Fig.$\,$`<!-- -->`{=html}2A). As the power-law observed for real networks describes systems of rather different sizes at different stages of their development, it is expected that a correct model should provide a distribution whose main features are independent of time. Indeed, as Fig.$\,$`<!-- -->`{=html}2A demonstrates, $P(k)$ is independent of time (and, subsequently, independent of the system size $m_0+t$), indicating that despite its continuous growth, the system organizes itself into a scale-free stationary state.

The development of the power-law scaling in the model indicates that growth and preferential attachment play an important role in network development. To verify that both ingredients are necessary, we investigated two variants of the model. Model A keeps the growing character of the network, but preferential attachment is eliminated by assuming that a new vertex is connected with equal probability to any vertex in the system (that is, $\Pi(k)=const=1/(m_0+t-1)$). Such a model (Fig.$\,$`<!-- -->`{=html}2B) leads to $P(k)\sim \exp(-\beta k)$, indicating that the absence of preferential attachment eliminates the scale-free feature of the distribution. In model B we start with $N$ vertices and no edges. At each time step we randomly select a vertex and connect it with probability $\Pi(k_i)=k_i/\sum_j k_j$ to vertex $i$ in the system. While at early times the model exhibits power-law scaling, $P(k)$ is not stationary: since $N$ is constant, and the number of edges increases with time, after $T\simeq N^2$ timesteps the system reaches a state in which all vertices are connected. The failure of models A and B indicates that both ingredients, namely growth and preferential attachment, are needed for the development of the stationary power-law distribution observed in Fig.$\,$`<!-- -->`{=html}1.

Due to the preferential attachment, a vertex that acquired more connections than another one will increase its connectivity at a higher rate, thus an initial difference in the connectivity between two vertices will increase further as the network grows. The rate at which a vertex acquires edges is $\partial k_i/{\partial t}=k_i/{2t}$, which gives $k_i(t)=m(t/t_i)^{0.5}$, where $t_i$ is the time at which vertex $i$ was added to the system (see Fig.$\,$`<!-- -->`{=html}2C), a scaling property that could be directly tested once time-resolved data on network connectivity becomes available. Thus older (smaller $t_i$) vertices increase their connectivity at the expense of the younger (larger $t_i$) ones, leading with time to some vertices that are highly connected, a "rich-gets-richer" phenomenon that can be easily detected in real networks. Furthermore, this property can be used to calculate $\gamma$ analytically. The probability that a vertex $i$ has a connectivity smaller than $k$, $P(k_i(t)<k)$, can be written as $P(t_i>m^2t/k^2)$. Assuming that we add the vertices at equal time intervals to the system, we obtain that $P(t_i>m^2t/k^2)=1-P(t_i\leq m^2t/k^2)=1-m^2t/k^2(t+m_0)$. The probability density $P(k)$ can be obtained from $P(k)=\partial P(k_i(t)<k)/\partial k$, which, at long times, leads to the stationary solution $$P(k)=\frac{2m^2}{k^3},$$ giving $\gamma=3$, independent of $m$. While it reproduces the observed scale-free distribution, the proposed model cannot be expected to account for all aspects of the studied networks. For this we need to model these systems in more detail. For example, in the model we assumed linear preferential attachment, that is $\Pi(k)\sim k$. However, while in general $\Pi(k)$ could have an arbitrary nonlinear form $\Pi(k)\sim k^\alpha$, simulations indicate that scaling is present only for $\alpha=1$. Furthermore, the exponents obtained for the different networks are scattered between $2.1$ and $4$. However, it is easy to modify our model to account for exponents different from $\gamma=3$. For example, if we assume that a fraction $p$ of the links is directed, we obtain $\gamma(p)=3-p$, which is supported by numerical simulations (*16*). Finally, some networks evolve not only by adding new vertices, but by adding (and sometimes removing) connections between established vertices. While these and other system-specific features could modify the exponent $\gamma$, our model offers the first successful mechanism accounting for the scale-invariant nature of real networks.

Growth and preferential attachment are mechanisms common to a number of complex systems, including business networks (*17, 18*), social networks (describing individuals or organizations), transportation networks (*19*), etc. Consequently, we expect that the scale-invariant state, observed in all systems for which detailed data has been available to us, is a generic property of many complex networks, its applicability reaching far beyond the quoted examples. A better description of these systems would help in understanding other complex systems as well, for which so far less topological information is available, including such important examples as genetic or signaling networks in biological systems. We often do not think of biological systems as open or growing, since their features are genetically coded. However, possible scale-free features of genetic and signaling networks could reflect the evolutionary history dominated by growth and aggregation of different constituents, leading from simple molecules to complex organisms. With the fast advances in mapping out genetic networks, answers to these questions might not be too far. Similar mechanisms could explain the origin of the social and economic disparities governing competitive systems, since the scale-free inhomogeneities are the inevitable consequence of self-organization due to the local decisions made by the individual vertices, based on information that is biased towards the more visible (richer) vertices, irrespective of the nature and the origin of this visibility.

**References and Notes**

1\. R. Gallagher and T. Appenzeller, *Science* **284**, 79 (1999); R. F. Service, *Science* **284**, 80 (1999).

2\. G. Weng, U. S. Bhalla, R. Iyengar, *Science* **284**, 92 (1999).

3\. C. Koch and G. Laurent, *Science* **284**, 96 (1999).

4\. S. Wasserman and K. Faust, *Social Network Analysis*, (Cambridge University Press, Cambridge, 1994).

5\. Members of the *Clever* project, *Sci. Am* **280**, 54 (June 1999).

6\. R. Albert, H. Jeong and A.-L. Barabási, Nature **401**, 130 (1999), see also http://www.nd.edu/ networks.

7\. P. Erdős, and A. Rényi, *Publ. Math. Inst. Hung. Acad. Sci* **5**, 17 (1960); B. Bollobás, *Random Graphs* (Academic Press, London, 1985).

8\. S. Lawrence and C. L. Giles, *Science* **280**, 98 (1998); *Nature* **400**, 107 (1999).

9\. Note that in addition to the distribution of incoming links, the www displays a number of other scale-free features, characterizing the organization of the webpages within a domain \[B. A. Huberman and L. A. Adamic, Nature **401**, 131 (1999)\], the distribution of searches \[B. A. Huberman, P. L. T. Pirolli, J. E. Pitkow and R. J. Lukose, *Science* **280**, 95 (1998)\], or the number of links per webpage (*6*).

10\. D. J. Watts and S. H. Strogatz, *Nature* **393**, 440 (1998).

11\. S. Redner, *European Physical Journal B* **4**, 131 (1998).

12\. We also studied the neural network of the worm Caenorhabditis elegans (*3, 10*) and the benchmark diagram of a computer chip
(http://vlsicad.cs.ucla.edu/$\sim$cheese/ispd98.html). We find that $P(k)$ for both is consistent with power-law tails, despite the fact that for *C. elegans* the relatively small size of the system ($306$ vertices) limits severely the data quality, while for the wiring diagram of the chips vertices with over $200$ edges have been eliminated from the database.

13\. S. Milgram, *Psychol. Today* **2**, 60 (1967); M. Kochen (ed.) *The Small World* (Ablex, Norwood, NJ, 1989).

14\. J. Guare, *Six Degrees of Separation: A play* (Vintage Books, New York, 1990).

15\. M. Barthélémy and L. A. N. Amaral, *Phys. Rev. Lett.* **82**, 15 (1999).

16\. Note that for most networks the connectivity $m$ of the newly added vertices is not constant. However, choosing $m$ randomly will not change the exponent $\gamma$ \[Y. Tu, private communication\].

17\. W. B. Arthur, *Science* **284**, 107 (1999).

18\. Note that preferential attachment was also used to model correlations between stock prices \[L. A. N. Amaral and M. Barthélémy, private communication\].

19\. J. R. Banavar, A. Maritan and A. Rinaldo, *Nature* **399**, 130 (1999).

20\. We thank D. J. Watts for providing the C. elegans and the power grid data, B. C. Tjaden for supplying the actor data, H. Jeong for collecting the data on the www and L. A. N. Amaral for helpful discussions. This work was partially supported by NSF Career Award DMR-9710998.

[^1]: firstname.lastname@example.com