abstract: Computational Logic is the use of computers to establish facts in a logical formalism. Originating in 19th-century attempts to understand the nature of mathematical reasoning, the subject now comprises a wide variety of formalisms, techniques and technologies. One strand of work follows the "LCF approach" pioneered by Robin Milner FRS, where proofs can be constructed interactively or with the help of users' code (which does not compromise correctness). A refinement of LCF, called Isabelle, retains these advantages while providing flexibility in the choice of logical formalism and much stronger automation. The main application of these techniques has been to prove the correctness of hardware and software systems, but increasingly researchers have been applying them to mathematics itself.
address: Computer Laboratory, University of Cambridge, England,
author: Lawrence C. Paulson FRS
subject: Theory of computing, mathematical logic
title: Computational Logic: Its Origins and Applications

# Introduction

Computers control everything from heart pacemakers to aircraft, putting everyone at risk when they fail. Computer-related failures regularly cause disruption, damage and occasionally death. A recent example is the WannaCry ransomware attack, which hit computers around the world, notably in the National Health Service . This attack was made possible by a specific defect, MS17-010, affecting several versions of the Windows operating system. In 1994, a floating-point division error in the Pentium processor forced Intel to recall millions of chips at a cost of \$475 million . Many such failures are caused by carelessness and could be prevented by the introduction of more disciplined development methods. The ultimate discipline is to use formal logic and mathematics to prove the correctness of designs. This is called *formal verification*.

Formal verification typically relates a *formal specification* to a *model implementation*. That is, we start with a complete description of all services to be delivered by the system (the *specification*) and prove that it is satisfied by the model implementation, where both have been expressed in some logical formalism. A proof that all services will be delivered includes a proof that no crashes or other failures will prevent this. However, any formalised implementation must leave out some details of the actual implementation: for example, finite-precision arithmetic may be assumed to be exact, encryption to be unbreakable or hardware components to be infinitely fast. (Readers may remember schoolbook physics problems involving frictionless surfaces, or when friction is included, it is assumed to be linear.) If the model of the implementation is too unrealistic, proofs about it will overlook certain failure modes. No model can include all real-world details, but it is crucial to include the aspects of the implementation that we are most concerned about. For example, in computer security, many systems can be broken without defeating the encryption itself . Then it makes sense to assume encryption to be unbreakable if that simplifies the proofs. Validating the model is complementary to the formal proofs and could involve a variety of methods, ranging from professional judgement to systematic testing. (It is a misconception to think that formal proof eliminates the need for testing.)

Formal verification requires the support of specialised computer software. *Computational Logic* is concerned with formal logic as implemented on computers, where it is frequently applied to solve problems related to computation. However, formal verification is increasingly being applied to systems that interact with the real world and deal with phenomena modelled by differential equations, or that have to achieve goals expressed in terms of probability. Therefore, it has become necessary to formalise mathematical knowledge.

# A Brief History of Formal Logic

A *logical formalism* is a precisely defined symbolic language that includes logical primitives such as "and" ($\land$), "or" ($\lor$), "not" ($\neg$), "implies" ($\to$), "for all" ($\forall$) and "there exists" ($\exists$). It may include the ability to define new symbols as abbreviations for other expressions. It will include rules for making simple logical deductions: for example, from $\phi\land \psi$, the conclusions $\phi$ and $\psi$ can both be derived; from $\phi$ we can derive $\phi\lor \psi$. From such basic primitives, vast swathes of mathematics can be derived.

Formal logic emerged from philosophy in the 19th century. Gottlob Frege's *Begriffschrift* (concept language) was the first recognisably modern treatment of logic. His work introduced the project of reducing the whole of mathematics to logic . But Frege's work ended with the discovery of Russell's paradox:

> Let $R$ be the set of all sets that are not members of themselves. Then is $R$ a member of itself?

If $R$ is a member of itself (written $R\in R$) then it cannot be a member of $R$, and therefore $R\not\in R$; on the other hand, if $R\not\in R$ then $R\in R$. We get a contradiction either way. Note that as long as we confine ourselves to English, Russell's paradox may resemble amusing puzzles such as "This statement is false", but once we use a formal language with strict rules, we are in trouble. For $R\not\in R$ and $R\in R$ are both true, from which all other statements can be proved using basic logic. There are many other paradoxes , e.g. Burali-Forti, which involves constructing the greatest ordinal number $\Omega$, but then $\Omega+1 < \Omega$. As Gödel remarks,

> By analysing the paradoxes to which Cantor's set theory had led, \[Russell\] freed them from all mathematical technicalities, thus bringing to light the amazing fact that our logical intuitions (i.e. intuitions concerning such notions as truth, concept, being, class) are self-contradictory.

From that moment, the development of formal logic was determined by the approach taken to solving the paradoxes. *Set comprehension* — the idea that every property $\phi$ created a set, $\{x \mid \phi(x)\}$ — somehow had to be constrained to deny the existence of $\{x \mid x\not\in x\}$, the Russell set. One approach was to restrict comprehension to some *already existing* set, say $A$, yielding the smaller set $\{x\in A \mid \phi(x)\}$: the elements of $A$ satisfying the property $\phi$. The universe of sets is not itself a set, and it is built up from the empty set using a few primitives such as union and powerset. This approach leads to modern axiomatic set theory .

Another solution to the paradoxes involves the notion of *types*. Whitehead and Russell's original conception of type was rather unclear , but by 1940, Church had developed types to a form that survives today. Rather than postulating one universal collection of everything that exists, mathematical objects are classified into distinct types, representing smaller collections. Every construction has a fixed type. When forming a set, all the elements must have the same type, say $\tau$, and the set itself will have a different type, which we may write as $\tau\,\text{set}$. Thus the fatal proposition $R\in R$ cannot even be written, because if $R$ has type $\tau$ then it cannot have type $\tau\,\text{set}$ at the same time.

Types seem to be unpopular with mathematicians, and it is usual to assert that mathematics is founded on set theory. Whitehead and Russell , provided no notation or symbols for types, making them invisible, while Church had one single type $\iota$ of individuals encompassing all objects that were neither sets nor functions. However, Church's ideas entered into computer science, particularly through his collaboration with Alan Turing, and by the early 1960s, the logical notion of type was starting to get conflated with programming language features (previously called "modes") that distinguish between integer and floating-point arithmetic, for example. The ubiquity and utility of types in computer programming may be one reason why computer scientists tend to prefer elaborately typed formalisms.

Yet another solution to the paradoxes was the philosophy of *constructive mathematics* or *intuitionism*  . Briefly, this is the view that mathematical objects exist only in our imaginations, in contrast to the Platonist or realist view that mathematical objects have their own reality . Intuitionism demanded a new treatment of logic, in which the disjunction $\psi\lor\phi$ could be true only if we *knew* which of the two statements, $\psi$ or $\phi$, was true. Many of the basic tautologies of Boolean logic must then be rejected, starting with $\psi\lor\neg\psi$ (as we have no general way of knowing which alternative holds) and with $\neg\neg\phi$ a weaker statement than $\phi$. At the level of quantifiers, $\exists x\, \phi(x)$ could be true only if we *knew* some specific value $a$ for which $\phi(a)$ held. For that reason, $\exists x\, \phi(x)$ was a stronger claim than $\neg[\forall x\, \neg\phi(x)]$, although in classical logic the two formulas are equivalent.

Intuitionism has not caught on with mainstream mathematicians. However, there are strong links between intuitionistic logic and computation: the knowledge mentioned in the previous paragraph had to be computable. Recent decades have seen the introduction of formalisms  that identify intuitionistic formulas with elaborate types: every logical symbol has an equivalent at the type level, and to prove a proposition amounts to exhibiting an element of the corresponding type. These formalisms are highly expressive and typically, all expressions denote computable functions. And there is a direct identification between logical propositions and types . For example, proving the formula $(\forall x\in A) (\exists y\in B)\, \phi(x,y)$ yields a computable function $f$ such that if $x\in A$ (that is, $x$ belongs to type $A$) then $f(x)$ returns some $y\in B$ paired with some sort of "evidence" affirming that the property $\phi(x,y)$ holds. [^1] Then we would have shown that $f$ had the type $(\prod x\in A) (\sum y\in B)\, \phi(x,y)$, which we can identify with the formula above. Here, $\prod$ and $\sum$ are the type-level counterparts of the quantifiers $\forall$ and $\exists$. The elements of $(\prod x\in A)B(x)$ are functions $f$ that map any given $a\in A$ to some $f(a)\in B(a)$, generalising the function type $A\to B$ by allowing the result type to depend on the value of the argument; similarly, the elements of $(\sum x\in A)B(x)$ are pairs $\langle a,b\rangle$ where $a\in A$ and $b\in B(a)$, generalising the Cartesian product $A\times B$. This "propositions as types" paradigm is the focus of much current research; see Wadler for a detailed historical introduction.

Formal verification today can therefore be done within a wide variety of logical formalisms, which include

1.  those based on axiomatic set theory, and with no concept of type;

2.  the *simple* type theory of Church, where a few primitive types can be combined using operators such as $\times$ (Cartesian product), $+$ (disjoint sum), $\to$ (function space) and the set type constructor mentioned above;

3.  *dependent* type theories where types can take parameters, as in $B(x)$, and with variable binding as in $(\prod x\in A)\, B(x)$, thereby allowing the type of $n\times n$ matrices, for example.

The typed approaches have received the most attention in computational logic, and below we shall focus on simple type theory. The third approach is also the focus of much research.

There are many other formalisms (for example, modal and relevance logics) and computational logic techniques not covered below. In particular, *model checking* techniques  are widely used. They can automatically verify system properties expressed in temporal logics (for reasoning about time). However, they rely on state enumeration rather than proof construction, so are out of scope here.

# Mechanised Logic: the LCF Tradition

If we want to prove theorems in a formal logic, computer power is necessary. Whitehead and Russell's development of basic logic, classes and relations only reaches (a precursor to!) 1+1=2 after hundreds of pages . Mathias has pointed out that the representation of the number 1 as defined by Bourbaki expands to 4,523,659,424,929 symbols. Better choices can mitigate such issues, but the fact remains that formal proofs are extremely long and detailed.

Ideally, we would like the computer to prove our theorems automatically. If that is impossible, then we would like to give only strategic advice, and let the machine work out the details. As a general rule, greater automation requires the choice of a less expressive formalism. *Boolean logic* (where we only have and, or, not) is decidable, so in principle, all questions can be settled automatically. The problem is known as SAT (for Boolean SATisfiability) and although a general solution is thought to be exponential in the number of Boolean variables, highly efficient heuristic approaches exist. In a dramatic demonstration of this technology, Heule and Kullmann solved the Boolean Pythagorean Triples problem, an open problem in Ramsey Theory, generating a 200TB proof.

Our logic becomes more expressive if we introduce the quantifiers "for all" and "there exists". If we forbid quantification over functions and sets, this is called *first-order logic*. This theory is *complete*: all theorems can be proved, given unlimited resources. But there is no general way to determine how much effort might be required or whether the claim is false. Automatic theorem proving in first-order logic is a significant field in its own right; one of its main achievements, though 20 years ago, is the solution of the Robbins Conjecture . Incidentally, two early titans of computational logic — Hilary Putnam and J. Alan Robinson — were trained as philosophers, and Putnam is chiefly remembered for his essays.

Unfortunately, fully automatic theorem proving cannot meet the needs of verification. Arithmetic, functions and sets are necessary for specifying even simple systems. No automatic procedure can decide arbitrary statements involving integer arithmetic: this is Hilbert's famous *Entscheidungsproblem*, which was settled by Turing . Another reason is that specifications may extend to hundreds of pages, and fully automatic theorem proving does not scale well. Interactive methods are the only feasible way to prove theorems based on huge specifications. [^2] And the name most associated with interactive theorem proving is Robin Milner (FRS 1988).

Milner was interested in an obscure formalism called Logic for Computable Functions (LCF), and built a simple proof checker while at Stanford University. It had two major drawbacks: proofs often required long sequences of repetitive, obvious steps, and the proofs themselves took up too much memory. Upon taking a position at the University of Edinburgh, he created an ambitious new system called Edinburgh LCF , incorporating several innovations. One was to provide the user with a specialised programming language, called ML (for metalanguage) so that users could extend the system with their own code. That would enable them to deal with any issue of repetitive steps by coding their own automation. Another was to eliminate the storage of proofs through a software technique known as *abstract data types*. By reserving the privilege of creating theorems to a small *kernel* of code, the need to store proofs was eliminated. User-supplied automation, however faulty, could never cause Edinburgh LCF to proclaim an invalid theorem. Edinburgh LCF also introduced an entire framework of techniques and terminology:

-   *theory files* containing specifications and definitions of the concepts of interest: theories can build on other theories, allowing hierarchical developments of any size.

-   *backward proof*: one begins with a statement of the desired result and successively reduces it (using *proof tactics*) to simpler statements until all have been proved.

-   *tacticals* for combining proof tactics in simple ways, for example, repetition or executing a series of tactics

-   built in *automation*, for example to perform routine simplification

-   along with the ability to *extend* this automation by writing additional ML code.

Today, almost all of the main interactive theorem provers incorporate these techniques, and in many cases, actual code from Edinburgh LCF. ML turned out to be useful for programming in general, and has had a huge impact on the design of modern programming languages such as Haskell, OCaml and Scala. The meanings of LCF and ML go well beyond their literal meanings as acronyms. [^3]

Another significant figure in interactive theorem proving was Michael JC Gordon (FRS 1994). He created the HOL system , which survives to this day in several variants, including HOL4 and HOL Light . All of these are descendants of Edinburgh LCF, which he also helped to create . Even more significant is Gordon's development of techniques for verifying computer hardware; when he started working on this topic in the early 1980s, essentially all verification research was focused on software. Gordon developed techniques that scaled from a single-transistor device to an entire computer. His key insight was that the right formalism for this task was *simple type theory*, also known as *higher-order logic*, hence HOL.

The choice of higher-order logic was radical at the time. First-order logic was dominant, thanks perhaps to its strong theoretical properties such as completeness. Gordon noted that such properties were irrelevant to verification, and recognised that the additional expressiveness of higher-order logic was necessary . His choice represented a return to the stronger logics of the early twentieth century. With his students, Gordon extended his techniques to cover a variety of digital systems, including floating-point hardware, probabilistic algorithms, and many other applications. Typical of these is Harrison's verification of a floating-point algorithm for the computation of the exponential function . The proof required a complete formalisation of the IEEE floating-point standard, in all its complexity, as well as the error analysis for the algorithm itself.

While on the topic of radical choices, we must not overlook research into constructive type theories. Inspired originally by the work of Martin-Löf and by Coquand and Huet , researchers built a series of interactive theorem provers, all following the Edinburgh LCF paradigm. For reasons of space, I will only mention Coq , which has become perhaps the most popular interactive theorem prover in the world. Typical of all these approaches is that through dependent types, the type system carries out much of the reasoning burden that would otherwise be done by explicit proof steps. Among the landmark achievements carried out using Coq are the formal verification of a C compiler and the formalisation of the odd order theorem . A full treatment of this line of work would require a separate article.

# A New Theorem-Proving Architecture: Isabelle

During the 1980s, many different logical formalisms were competing for researchers' attention. In addition to LCF, type theories were evolving rapidly, along with a wide variety of formalisms for proving software correct. One drawback of the LCF approach was its requirement that every primitive inference rule had to be implemented as program code, and then a second time as a proof tactic. An error in the latter code could prevent the user from obtaining the theorem they thought they had proved, while an error in the former code could allow the system to assert false statements as true.

To understand this drawback in more detail, consider one of the simplest logical inference rules: if $\phi$ and $\psi$ are theorems, then so is their conjunction, $\phi\land\psi$. It is often written as $$\frac{\phi \quad \psi}{\phi\land\psi.}$$ However, note that mathematical proofs frequently involve temporary assumptions, such as when we consider the two cases of whether some integer $n$ is even or odd. Each of the two cases is proved with the help of the corresponding assumption about $n$. So people generally adopt a so-called *natural deduction* calculus, where all reasoning takes place within a *context*: a set of formulas $\theta_1$, …, $\theta_n$ temporarily assumed to be true. [^4] In order to minimise the need for subscripting, we typically use the large Greek letters $\Gamma$ and $\Delta$ to stand for contexts. If we chose to make this explicit, our inference rule could be written (forming what is known as a *sequent calculus*) $$\frac{\Gamma \Longrightarrow \phi \qquad \Delta \Longrightarrow \psi}{\Gamma,\Delta \Longrightarrow \phi\land\psi.}$$ Note that the two contexts are combined when we form the conclusion, since it depends on everything assumed by the premises. And this is the simplest possible example. A number of rules modify contexts by "discharging" assumptions. The treatment of quantified variables introduces further technicalities.

To implement this rule using the LCF architecture (which is also adopted by the HOL family and Coq), we begin by designing data structures that can represent the full syntax of our formalism. Then we collect all the inference rules that need to be implemented; taken together, they will define the abstract data type `thm`, the type of theorems. Each rule will take the form of a function coded in ML. The function for the rule above will take as its arguments the two premises, namely $\Gamma \Longrightarrow \phi$ and $\Delta \Longrightarrow \psi$. This rule is simple, but in other cases, the code must make syntactic checks on the premises to ensure that they have the correct form, and if not, signal an error. Finally, the function returns $\Gamma,\Delta \Longrightarrow \phi\land\psi$ as a value of type `thm`, thus certifying it to be a theorem. This particular function is called `CONJ` in LCF and its descendants.

Defining type `thm` as outlined above implements a logic in ML, but a usable system must also support backward proof: the ability to prove a statement of the form $\phi\land\psi$ by proving the two parts separately. So we need to write another function, called a *tactic*, which essentially executes the inference rule in reverse. Given the goal of the form $\Gamma \Longrightarrow \phi\land\psi$, it returns the two separate subgoals $\Gamma \Longrightarrow \phi$ and $\Gamma \Longrightarrow \psi$ (and note that here we have one single context, $\Gamma$, since we are working backwards).

These subgoals are not theorems but merely statements to work on next. In order to yield a theorem at the end, the tactic also needs to return a piece of code involving `CONJ`. This theorem-yielding function is called the *validation*, and the design allows most of the code to lie outside the abstract data type `thm`, on which the soundness of the whole system depends. If the tactic or its validation is somehow wrong, the system's soundness will not be compromised. However, the user is likely to be disappointed, making the effort to prove the theorem but never getting the theorem itself. (Instead of an error will be signalled, or possibly some other theorem delivered.) Gordon's retrospective on these techniques is informative , as is an early exposition of mine .

So we see that to implement any formal calculus using the LCF architecture, every inference rule must be implemented in code twice. There could be dozens of rules, some of them complicated, and any error would be damaging. Modifying an LCF-style system to implement a new formalism required months of work.

The solution to these problems is to use the LCF idea in a different way. Instead of focusing on the theorems of the given formalism, we should focus on *contextual implications* of the form $\Gamma \Longrightarrow \phi$. In particular, such implications can be joined end-to-end, creating proof trees. This approach has a number of advantages:

-   Contexts are managed in the same way for every formalism being supported. Meanwhile, formalisms that manage the context explicitly can still be handled.

-   The incremental buildup of a proof tree can express both forward and backward proof.

-   Subgoals can even contain shared "placeholder" variables, as may arise when proving something like $\exists x\, (\phi(x)\land\psi(x))$. That is to say, we can try to prove something of the form $\phi(-)$ and if we manage to prove $\phi(f(1))$, then the other subgoal will become $\psi(f(1))$.

But the main advantage is simply that inference rules can now be written as assertions. For example, the conjunction rule above becomes $$\phi, \psi \Longrightarrow \phi\land\psi.$$ The rule to eliminate a disjunction (proof by case analysis) becomes $$\phi\lor\psi, (\phi \Longrightarrow\theta), (\psi\Longrightarrow\theta) \Longrightarrow \theta.$$ The rule to introduce an existential quantifier becomes $$\phi(a) \Longrightarrow \exists x\,\phi(x).$$ This architecture allows primitive inferences of a formalism to be entered more or less as they might appear in a textbook.

Isabelle is the world's first *generic* theorem prover: designed to support logical inference in general as opposed to any single formalism. Isabelle takes some ideas from higher-order logic to represent formal syntax and logical inference rules . Notational features such as variable binding (the $i$ in $\sum_{i\in I}\,t_i$) and principles such as mathematical induction could be expressed straightforwardly, and conformance to traditional presentations could be proved . Such meta-logical systems are known as *logical frameworks* , the first of which was the AUTOMATH language created by N. G. de Bruijn .

Another fundamental Isabelle principle is to provide the basic components of automatic proof search at the lowest level. The resolution techniques of first-order logic had already shown promise. It seemed right to adopt some of those techniques to provide automation for other formalisms. The joining together of implications, which is Isabelle's core inference mechanism, is itself a form of resolution. Over the years, increasingly sophisticated forms of automatic proof search were added to Isabelle .

Have these ideas stood the test of time? Certainly Isabelle is the only widely-used interactive theorem prover to be based on a logical framework. Meanwhile, efforts to exploit the power of resolution theorem proving finally resulted in strong integration of the two technologies: the *sledgehammer* tool  applies a battery of resolution theorem provers to a problem. It often finds a proof with the help of obscure facts from Isabelle's extensive libraries. Some of the most impressive work done using Isabelle—from the verification of security protocols  to a verified operating system kernel  to formalised mathematical theorems —was only possible thanks to Isabelle's automation.

On the other hand, the idea of generic theorem proving is less successful. It's true that Isabelle has been used to support a great many formalisms , including a version of Martin-Löf's intuitionistic type theory, first-order logic, set theory and some modal logics. Isabelle/ZF develops a substantial amount of Zermelo-Fraenkel set theory . One can argue that, even in the case of a single formalism, the use of a logical framework makes for a more flexible notion of proof construction than that found in traditional LCF-like systems. However, Isabelle's facility to support multiple formalisms is hardly used. Its most popular instantiation by far is Isabelle/HOL, which at bottom is yet another implementation of Church's higher-order logic . Constructive type theories tend to be demanding syntactically; that research community has always preferred to build its own tools.

Isabelle's core ideas were in place by 1989 . Since then, Isabelle development has increasingly moved to the Technical University of Munich, under the leadership of Prof. Tobias Nipkow. Of the innumerable contributions made there, the most important one is the Isar structured proof language , which although completely formal includes many mathematical idioms and often allows legible, natural-looking proofs. (Hitherto, Isabelle followed LCF in expecting users to generate proofs directly through the ML programming language.) Extensions to Isabelle's core ideas to support higher-order logic are also due to Nipkow and his colleagues, in particular, the notion of type classes . The years have seen a long series of refinements made and case studies undertaken. One notable example is counterexample detection , whereby Isabelle can frequently inform the user that the statement under consideration is not a theorem.

# Formalising Mathematics

The idea of performing mathematics by machine goes back at least as far as Leibniz, who invented a calculating machine. The first mathematician to properly realise this dream was de Bruijn , whose AUTOMATH project started in the 1960s and culminated in the formalisation of a monograph on the construction of the reals . De Bruijn's approach was based on dependent type theories of his own devising. At around the same time, Andrzej Trybulec had the idea of formalising mathematics using a form of typed set theory; a substantial amount of mathematics was translated into his Mizar formalism, with a particularly elegant treatment of algebraic structures .

However, the main impetus for formalised mathematics has arisen from the requirements of verification, which has been a priority area for research funding. Harrison's early work  to verify a floating-point algorithm for the exponential function naturally led to the formalisation of increasing amounts of analysis. In order to verify probabilistic algorithms (which are controlled by virtual coin tosses), Hurd  formalised some probability theory, including measure theory and Lebesgue integration. Recent work in this direction includes Hölzl's comprehensive formalisation  of Markov chains, an abstract model of probabilistic systems that has numerous applications. Such work has generally been done using versions of HOL and Isabelle/HOL, but floating-point algorithms have also been verified using Coq .

Over time, competition arose between the proponents of different verification systems on the basis of how many well-known theorems had been formalised in each. Harrison took the lead here, formalising numerous results in his HOL Light system . These included much of complex analysis  as well as a proof of the prime number theorem  and many other results. Later, Harrison played a major role in the effort to formally verify Hales's 1998 proof of the Kepler Conjecture, a 400-year-old problem concerning the optimal packing of spheres. Referees had objected to this proof because of its reliance on extensive computer calculations. In response, Hales launched the Flyspeck project  to formally verify his proof. Flyspeck was ultimately successful, confirming and simplifying Hales's argument . Some of the formal proofs were done using Isabelle .

This was not the first time a mathematical result was formalised in order to confirm a contested proof. In 2005, Gonthier  formalised the Four Colour Theorem within Coq. The 1976 proof by Appel and Haken was notorious for its the use of a bespoke computer program to check nearly 2000 cases. Gonthier used a similar strategy, with a crucial difference: the cases were checked by algorithms that had been verified using Coq.

Other work of this sort concerned proofs that, while enjoying the full confidence of mathematicians, had other issues. My own formalisation of Gödel's second incompleteness theorem  falls into this category: most published proofs are sorely lacking in detail. Fleuriot's formalisation  of some material from Newton's *Principia* is remarkable for embracing Newton's use of infinitesimals: Fleuriot formally justifies the original proofs through *non-standard analysis*, which gives a rigorous foundation to infinitesimal reasoning. Above all, we have Gonthier's monumental project to verify the odd order theorem ; the issue here is simply size, as the theorem was originally proved in a 255-page journal article.

Generally speaking, the point of formalised mathematics is to clear up doubts, resolve ambiguities and identify errors. Mathematicians regularly make mistakes, as examined by Lakatos  in his history of Euler's polyhedron formula; he points out that even definitions can be wrong. Famous modern mathematicians can also be fallible:

> When the Germans were planning to publish Hilbert's collected papers and to present him with a set on the occasion of one of his later birthdays, they realised that they could not publish the papers in their original versions because they were full of errors, some of them quite serious. Thereupon they hired a young unemployed mathematician, Olga Taussky-Todd, to go over Hilbert's papers and correct all mistakes. Olga laboured for three years. .

Increasingly, the impetus for formalising mathematics comes from mathematicians themselves. Fields medallist Vladimir Voevodsky wrote of his horror at finding errors in his own work:

> And who would ensure that I did not forget something and did not make a mistake, if even the mistakes in much more simple arguments take years to uncover?

Voevodsky's *homotopy type theory*  is a new approach to the formalisation of mathematics. It gives Martin-Löf's intuitionistic type theory a new interpretation involving topological spaces, where types denote homotopy types. [^5] Moreover, this interpretation suggests the new *univalence axiom*, which says that isomorphic constructions can be identified . These ideas have generated widespread excitement, and experiments are under way using the Coq proof assistant.

In the opposite direction, axiomatic set theory has also been used to formalise mathematics. Sets and types both denote collections, with one great difference: typically a variable can have only one type but can belong to any number of sets. So set theory may offer the greatest flexibility in formalising mathematics. Work of my own concerns proving a number of properties to be equivalent to the axiom of choice  as well as formalising Gödel's proof of the relative consistency of the axiom of choice . This work can be criticised as being only about set theory itself. However, recent work by Zhan shows that set theory can also be used to formalise traditional mathematical results from group theory, point-set topology and real analysis .

# Obstacles to Formalising Mathematics

The obstacles to the formalisation of mathematics are perhaps not what people imagine. Gödel's theorem is not especially relevant to mathematics in practice. Even the *Entscheidungsproblem* is not that relevant: when we do have decision procedures, they can be too expensive to actually run. Anyway, our ambition is to *assist* mathematicians, not to replace them by computers. The obstacles we face take a variety of forms.

*Foundational*. A mathematician's imagination is unconstrained, but every formal calculus is a rigid framework and ultimately a prison. For example, axiomatic set theory begins by creating a world big enough for 19th-century mathematics, then escalates it through powerful principles to create an inconceivably large universe of mathematical objects. It contains such oddities as the limit of $\{\aleph_0, \aleph_{\aleph_0}, \aleph_{\aleph_{\aleph_0}}, \ldots\}$, a cardinal $\lambda$ satisfying the equation $\lambda = \aleph_\lambda$. (The $\aleph$-sequence enumerates the infinite cardinalities, starting with $\aleph_0$: the cardinality of the integers.) Boolos frankly states that he can't believe that such an object exists. And yet, category theory begins by creating the category of sets and functions, assuming at a stroke the existence of a mathematical object incorporating the entire universe of sets, and building upon it. So one moment our universe seems far too large, and a moment later, it has become too small.

*Conceptual*. Intuition plays a huge role in mathematics. A mathematician easily sees that a certain function — perhaps defined by a complicated integral — is continuous. Many convergence properties are obvious. But in some cases, the formal proofs are enormous . There are a variety of responses to this situation. One is to provide additional automation to deal with specific tasks, such as proving asymptotic properties . Another is to accept that some apparently obvious facts, such as the Jordan curve theorem, [^6] really are extremely difficult to prove rigorously . Another is to accept that many apparently obvious statements are actually false . But for all that, the use of any logical formalism requires immense patience.

*Notational*. The language of mathematics has evolved over centuries. It can be inconsistent and ambiguous — compare $\sin^2 x$ with $y^2x$ — but it is familiar and reasonably effective. Any formalism implemented on a computer will be code, and look like it. For example, the fundamental theorem of algebra states that if $n$ is a positive integer and $a_0$, $a_1$, …, $a_n$ are complex numbers with $a_n\not=0$, then $a_nz^n +\cdots+a_1z+a_0 = 0$ for some complex $z$. The HOL Light version relaxes the precondition to $a_0=0$ or otherwise that the $a_k$ are not all zero for $k=1,\ldots,n$:

    !a n. a(0) = Cx(&0) \/ ~(!k. k IN 1..n ==> a(k) = Cx(&0))
             ==> ?z. vsum(0..n) (\i. a(i) * z pow i) = Cx(&0)

The elegance of the mathematical notation is completely lost. Isabelle does a bit better, allowing the Greek alphabet and many mathematical symbols, but all such languages are still code.

One further complication deserves a mention. Many expressions of mathematics can be *undefined*, e.g. some limits, derivatives, integrals or simply $x/y$ when $y=0$. A formal calculus can deal with this problem in a number of different ways . So we may find that $x/y=x/y$ (when $x/0$ denotes some arbitrary value) or that $x/y=x/y$ if and only if $y\not=0$ (when we regard primitive assertions about undefined values as necessarily false) or we may find that the very act of writing $x/y$ is forbidden unless we can prove $y\not=0$. This last approach can be inconvenient , indeed each approach has its good and bad points. The idea that $x/y$ is always something horrifies some mathematicians (especially if we go further, as in Isabelle, and actually stipulate that $x/0=0$). Elaborate treatments of definedness yield greater expressive power: paid for, as usual, by more complicated notations and harder proofs.

# The Way Forward

In the course of this paper we have narrowed our focus several times, deliberately seeking out research as opposed to applications. This is necessary because computational logic tools, interpreted broadly, are now widely used in the development of hardware, software and in the provision of online services. We are in danger, perhaps, of overlooking the main motivation for computational logic: proving the correctness of computer systems. So let's take a moment to consider a basic problem in computer science: compiler correctness.

A compiler is a piece of software that translates program code (written in a programming language such as C or Java) into some sort of binary form that can be executed on computer hardware. Compilers are complex, not least because they use a wide variety of sophisticated techniques to generate binary code that will run as fast as possible. If the compiler is somehow faulty then it could introduce errors into the code that is actually executed. People have sought to prove compilers correct using computational logic techniques . But even if we finally obtain a correct compiler, how do we translate it into executable binary code? A further complication is that the semantics of a language like C has little in common with the way C is used in real software, which relies on innumerable undefined behaviours.

Kumar et al.  have solved this chicken-and-egg problem. They have designed their own programming language, CakeML, with a clean semantics. They express a simple compiling algorithm as a mathematical function in HOL4 (a descendant of Gordon's original HOL system). They translate the same algorithm to CakeML and compile it to binary (for the popular x86 architecture) *using HOL4 itself*, giving high assurance of correctness. This yields a trustworthy version of the compiler in binary form. Their approach is an instance of the well-known technique of *bootstrapping*, where a compiler is built for a simple but adequate language and then used as a tool to further its own development until the full language is supported. In this case, the bootstrapping process includes verification, and is possible because HOL4 can cope with the full binary code of the CakeML compiler. This is one representative of a large body of research concerned with treating mathematical functions defined within a theorem prover as executable code.

Readers who want to pursue these topics further will find surveys available on formally verified mathematics and on the history of interactive theorem proving . Isabelle can be downloaded via the URL <https://isabelle.in.tum.de>.

#### Data accessibility.

No experimental data involved. There is a software download link above.

#### Competing interests.

Not applicable.

#### Funding.

Much of the research reported here was supported by the EPSRC or its predecessors, or by German and EU funding agencies, going back 40 years.

#### Acknowledgements.

Angeliki Koutsoukou-Argyraki commented on a draft of this paper. The referees also made insightful comments. I would also like to thank Mike Gordon FRS and Robin Milner FRS for their mentorship.

[^1]: Thus, a form of the axiom of choice is actually provable in this system .

[^2]: Interactive theorem provers are also called *proof assistants*.

[^3]: ML can also refer to machine learning.

[^4]: Mathematical proofs also introduce temporary quantities, as when we allow $x$ to denote a root of some polynomial. Therefore contexts may include a string of bound variables, but the details  are too complicated to present here.

[^5]: A *homotopy* between two continuous functions $f$ and $g$ is a continuous "deformation" of $f$ into $g$. With this concept one can define an equivalence relation on topological spaces, and the resulting equivalence classes are called *homotopy types*.

[^6]: which states that every simple closed curve partitions the plane into an interior and an exterior region


author: Michael Felderer, Barbara Russo and Florian Auer
bibliography: references.bib
title: On Testing Data-Intensive Software Systems

# Introduction

Information technology is evolving towards a kind of software-intensive systems that, more and more extensively, collect, integrate, process and analyze large data volumes that have to be processed with high velocity coming from a number of diverse data sources, so called "big data". Examples for such data-intensive systems can be found in all domains (e.g., finance, automotive, production) and all types of systems (e.g., information systems and cyber-physical systems). Especially also large and long-running cyber-physical systems that can be found in manufacturing plants are today data- and software-intensive as they are controlled by software and have to process large volumes and different types of data in real-time (see Section ). The rapid growth of such systems is generating a paradigm shift in the field of software engineering. We are moving from traditional software engineering, which is functionality-centric, towards modern software and data engineering, which is data-centric and where the functionality is driven by the availability of data. Modern systems collect raw data from the users and their personal devices (like smart watches), from the environment and its smart objects (like sensors in industrial plants), as well as higher-level data coming from information providers like social platforms, open-data sites, and data silos. Also different types of data like image, sound, video or textual data have to be taken into account. Beyond functionality, the success (and added value) of such systems is tied to the availability of the data that is processed as well as its quality.

Data-intensive (software) systems are therefore becoming increasingly prominent in today's world, where the collection, processing, and dissemination of ever-larger volumes of data has become a driving force behind innovation. However, data-intensive systems also pose new challenges to quality assurance  and especially testing. Testing software-intensive systems has traditionally focused on verifying and validating compliance and conformance to specifications, as well as some general nonfunctional requirements. Data-intensive systems require a different approach for testing and analysis, moving more towards exploring the system, its elements, behavior and properties from a big data and analytics perspective to help decision makers respond in real-time (e.g., for cyber-physical systems). The behavior and, therefore, the expected test result can often not be specified precisely, but only with a statistical uncertainty. For instance, the movement of a robot to assemble a part of a car can often not sufficiently be specified in an explicit way, but based on machine learning algorithms with uncertainty.

As the area of data-intensive (software) systems and their testing is not well investigated so far, this chapter contributes by defining the main terminology and exploring existing approaches and promising future directions.

The chapter is structured as follows. Section  presents background on software testing and data quality. Section  defines what a data-intensive (software) system is and what main challenges to its quality assurance exist. Section  discusses cyber-physical production systems as an example for data-intensive systems. Section  presents an exploratory literature study on testing data-intensive software systems. Section  discusses existing approaches to testing data-intensive systems and future directions. Finally, Section  concludes the chapter.

# Background

In this section, we cover background on software testing and data quality.

## Software Testing

Software testing (or software-intensive system testing) is a part of the overall engineering process of cyber-physical systems and can be defined as the process consisting of all lifecycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software-intensive products, systems and services and related artifacts to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects . This broad definition of testing comprises dynamic testing activities like classic unit or system testing, but also static testing activities, where artifacts are not executed, like static analysis or reviews. The tested software-based system is called *system under test* (SUT). Traditional dynamic testing evaluates whether the SUT behaves as expected or not by executing a *test suite*, i.e., a *finite* set of test cases suitably *selected* from the usually infinite execution domain  . In a generic sense, dynamic testing can also be defined as evaluating software by observing its execution , which may also subsume *runtime monitoring* of live systems.

After executing a test case, the observed and intended behaviors of a SUT are compared with each other, which then results in a *verdict* . Verdicts can be either of *pass* (behaviors conform), *fail* (behaviors don't conform), and *inconclusive* (not known whether behaviors conform) . A *test oracle* is a mechanism for determining the verdict. The observed behavior may be checked against user or customer needs (commonly referred to as testing for *validation*) or against a specification (testing for *verification*). So the oracle compares an expected output value with the observed output. But this may not always be feasible, especially in the context of data-intensive software systems. For instance, consider data-intensive systems that produce complex output, like complicated numerical simulations or varying output based on a prediction model, where defining the correct output for a given input and then comparing it with the observed output may be non-trivial and error-prone. This problem is referred to as the *oracle problem* and it is recognized as one of the fundamental challenges of software testing . *Metamorphic testing*  is a technique conceived to alleviate the oracle problem. It is based on the idea that often it is simpler to reason about relations between outputs of a program, than it is to fully understand or formalize its input-output behavior.

A *failure* is an undesired behavior. Failures are typically observed (by resulting in verdict fail) during the execution of the system under test. A *fault* is the cause of a failure. It is a static defect in the software, usually caused by human error in the specification, design, or coding process. During testing, it is the execution of faults in the software that causes failures. Differing from active execution of test cases, passive testing only monitors running systems without interaction.

Refining previous classifications , testing can be classified utilizing the three dimensions: test objective, test level and execution level (see Fig. ).

*Test objectives* are reasons or purposes for designing and executing a test. The reason is either to check the functional behavior of the system or its non-functional properties. *Functional testing* is concerned with assessing the functional behavior of an SUT, whereas *nonfunctional testing* aims at assessing nonfunctional requirements with regard to quality characteristics like security, performance, conformance, reliability, maintainability or usability.

The *test level* addresses the granularity of the SUT and can be classified into component, integration and system testing. It also determines the *test basis*, i.e., the artifacts to derive test cases. *Component testing* (also referred to as *unit testing*) checks the smallest testable component (e.g., a class in an object-oriented implementation or a single electronic control unit) in isolation. *Integration testing* combines components with each other and tests those as a subsystem, that is, not yet a complete system. *System testing* checks the complete system, including all subsystems. A specific type of system testing is *acceptance testing* where it is checked whether a solution works for the users of a system. *Regression testing* is a selective retesting to verify that modifications have not caused side effects and that the SUT still complies with the specified requirements .

The *execution level* addresses whether and how the SUT is executed. *Static testing* checks software development artifacts (e.g., requirements, design or code) without execution of these artifacts. *Dynamic testing* actively executes a system under test and evaluates its results.

The process of testing comprises the core activities test planning, design, automation, execution, and evaluation . According to , *test planning* is the activity of establishing or updating a test plan. A test plan is a document describing the objectives, scope, execution levels, approaches, resources, and schedule of intended test activities. It identifies, among others, concrete objectives, the features to be tested, the test design techniques, and exit criteria to be used and the rationale of their choice. A test objective is a reason, which can either be to check the functional behavior of a system or its nonfunctional or (software) quality properties, for designing and executing a test. Exit criteria are conditions for permitting a process to be officially completed. They are used to report against and to plan when to stop testing. Adequacy criteria like coverage criteria aligned with the tested feature types and the applied test design techniques are typical exit criteria. Once the test plan has been established, test control begins. It is an ongoing activity in which the actual progress is compared against the plan which often results in concrete measures.

During the *test design* phase the general testing objectives defined in the test plan are transformed into tangible test conditions and abstract test cases. In *criteria-based test design*, one designs test cases that satisfy specific engineering goals such as coverage criteria. In *human-based test design*, one designs test cases based on domain knowledge of the system and human knowledge of testing. *Test automation* comprises tasks to make the abstract test cases executable. This includes tasks like preparing test harnesses and test data, providing logging support or writing test scripts which are necessary to enable the automated execution of test cases. In the *test execution* phase, the test cases are then executed and all relevant details of the execution are logged and monitored. Finally, in the *test evaluation* phase, the exit criteria are evaluated and the logged test results are summarized in a test report.

## Data Quality

Quality, in general, has been defined as the totality of characteristics of a product that bear on its ability to satisfy stated or implied needs . This generic definition can be instantiated to software and data quality, as capability of a software and data product, respectively, to satisfy stated and implied needs when used under specified conditions. For software systems, according to ISO/IEC 25010 , these quality characteristics are: functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability and portability for product quality as well as effectiveness, efficiency, freedom from risk and context coverage for quality in use. According to ISO/IEC 25012  data quality characteristics in the context of software development can be classified into inherent and system-dependent data characteristics.

*Inherent data quality* refers to the degree to which data quality characteristics have the intrinsic potential to satisfy stated and implied needs when data is used under specified conditions. From the inherent point of view, data quality refers to data itself, in particular to data domain values and possible restrictions (e.g., business rules governing the quality required for the characteristic in a given application), relationships of data values (e.g., consistency) and meta-data.

*System-dependent data quality* refers to the degree to which data quality is reached and preserved within a system when data is used under specified conditions. From this point of view, data quality depends on the technological domain in which data are used and it is achieved by the capabilities of systems' components such as hardware devices or sensors (e.g., to make data available or to obtain the required precision) as well as system and other software (e.g., backup software to achieve recoverability or data processing software).

According to ISO/IEC 25012 , inherent data quality characteristics are:

-   *Accuracy*, i.e., the degree to which data has attributes that correctly represent the true value of the intended attribute of a concept or event in a specific context of use.

-   *Completeness*, i.e., the degree to which data has attributes that are free from contradiction and are coherent with other data in a specific context of use. It can be either or both among data regarding one entity and across similar data for comparable entities.

-   *Credibility*, i.e., the degree to which data has attributes that are regarded as true and believable by users in a specific context of use. Credibility includes the concept of authenticity (the truthfulness of origins, attributions, commitments).

-   *Currentness*, i.e., the degree to which data has attributes that are of the right age in a specific context of use.

According to ISO/IEC 25012 , inherent and system-dependent characteristics are :

-   *Accessibility*, i.e., the degree to which data can be accessed in a specific context of use, particularly by people who need supporting technology or special configuration because of some disability.

-   *Compliance*, i.e., the degree to which data has attributes that adhere to standards, conventions or regulations in force and similar rules relating to data quality in a specific context of use.

-   *Confidentiality*, i.e., the degree to which data has attributes that ensure that it is only accessible and interpretable by authorized users in a specific context of use.

-   *Efficiency*, i.e., the degree to which data has attributes that can be processed and provide the expected levels of performance by using the appropriate amounts and types of resources in a specific context of use.

-   *Precision*, i.e., the degree to which data has attributes that are exact or that provide discrimination in a specific context of use.

-   *Traceability*, i.e., the degree to which data has attributes that provide an audit trail of access to the data and of any changes made to the data in a specific context of use.

-   *Understandability*, i.e., the degree to which data has attributes that enable it to be read and interpreted by users, and are expressed in appropriate languages, symbols and units in a specific context of use.

According to ISO/IEC 25012 , system-dependent characteristics are :

-   *Availability*, i.e., the degree to which data has attributes that enable it to be retrieved by authorized users and/or applications in a specific context of use.

-   *Portability*, i.e., the degree to which data has attributes that enable it to be installed, replaced or moved from one system to another preserving the existing quality in a specific context of use.

-   *Recoverability*, i.e., the degree to which data has attributes that enable it to maintain and preserve a specified level of operations and quality, even in the event of failure, in a specific context of use.

Big data is a term used to refer to data sets that are too large or complex for traditional data-processing software to adequately process them. It is high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making. Big data can be described by the following characteristics:

-   *Volume* refers to the quantity of generated and stored data that determined the value and potential insight

-   *Velocity* refers to the speed at which the data is generated and processed to meet the demands and challenges

-   *Variety* refers to the different types of data that have to be processed, i.e., text, images, audio and video

-   *Veracity* refers to the refers to the data quality and the data value that can be measured based on the quality criteria highlighted before

# Data-Intensive Software Systems

Systems that process large volumes of data have commonly been referred to as "data-intensive". However, the amount of data that is considered as large is relative and changes over time. Thus, information systems that stored and retrieved large volumes of data at their time, may be no longer considered as "data-intensive" today. This made the concept of "data-intensive" systems relative to time and recent developments in storage capacities and processing capabilities.

Today, data-intensive systems do not only process large volumes of data, but differ from other systems with respect to the following aspects:

-   The data is stored and retrieved but depending on the context also collected, generated, manipulated and redistributed (i.e., processed).

-   The data is "big" data and fulfills the characteristics described in the previous section (i.e., volume, velocity, variety, and veracity).

-   The data does influence in addition to the operation phase also the analysis, design, implementation, testing, and maintenance phase of the system lifecylce.

These data-intensive systems are more than large data stores that allow the storage and retrieval of large volumes of data. Data-intensive systems additionally comprise the collection of data from different sources, the generation of new data, the manipulation of existing data and the redistribution of data to data consumers.

The data itself fulfills the characteristics of big data. The different data sources that are used by data-intensive systems provide large volumes of data (volume) in varying formats (variety). Furthermore, the data is processed timely to meet the demands of its users (velocity). Finally, the high dependency on data requires the maintenance of high data quality (veracity) throughout the system.

With respect to the central role that data has in all phases of the system lifecycle (with the consequence that the system behavior is determined by the data), data-intensive systems can be opposed to specification-driven systems where the behavior is explicitly specified and coded in (deterministic) algorithms.

To summarize, data-intensive systems are defined as follows:
*Data-intensive systems are systems that may handle, generate, and process a large volume of data of different nature and from different sources over time orchestrated by means of different technologies and with the goal to extract value from data for different types of businesses. They pose specific challenges in all phases of the system lifecycle and over time they might evolve to very large, complex systems as data, technologies and value evolve.*
In the following sections, the architecture and quality assurance of data-intensive systems are discussed.

## Architecture of Data-Intensive Systems

Recent years have seen the rapid growth of interest on developing data-intensive systems. These systems include technologies like Hadoop/MapReduce, NoSQL, or systems for stream processing . Specific technologies like databases, caches, search indexes, stream processing, or batch processing enable the development of data-intensive systems. They provide services like decision support, system behavior monitoring or reporting applicable in different business domains. Thus, a data-intensive system is typically built from standard building blocks that provide commonly needed functionality. For example, many systems need databases, search indexes, or batch processing. As there are various options for database systems, or approaches to caching, the most appropriate technologies for the respective system have to be chosen among the numerous possibilities. Key quality attributes for these decisions are, according to Kleppmann , reliability, scalability, and maintainability.

The building blocks that assemble a data-intensive system are visualized in the conceptual structure of a data-intensive system given in Fig. , which is based on the system lifecycle of data-intensive systems presented in . Following the path of the data through the system, first the data is collected from the different data sources with possible varying protocols (e.g., ftp, http) and data formats like domain-specific exchange formats ("Data sources").

Next, in the staging area, it is ensured that the data conforms to an unified interpretation, sharing and preservation (curation). This includes describing the data with meta-data to ease further processing. The meta-data are created, organized and classified according to the information modeling of the data-intensive system. Thereafter, the data and its meta-data are stored into an archive system ("Data management"). This part of the system supports searching on the data and further prepares the data for its later usage in the workflow management component. Following the data management, the workflow management is responsible for processing the data. It includes tasks to process the data (e.g., calculations) and workflows that align tasks into a processing pipeline that can make data-based decisions on which concrete tasks to execute. In the last step, the data leaves the data-intensive system as part of a data product. Examples therefore are reports, prediction models, analyses and recommendations or generated data. It represents the data output of the data-intensive system that may be further processed within a system. The four big data characteristics (see Section ) that are next to the system components highlight that volume, velocity, variety and veracity are inherent in every part of the system. Each component, and activity (e.g., collection, curation or processing) have to take these characteristics into account. This reflects the intensive dependency of these systems on data.

## Quality Assurance of Data-Intensive System

Data-intensive systems process large volumes of data and provide through the processing of it high value to the business. Thus, failures and other quality issues in data-intensive systems are incredibly costly (e.g., production failures), because of their affect on the business. As a result, new types of quality assurance activities and concerns raise. For example, data debugging i.e., discovering system failures caused by well-formed but incorrect data is a primary issue in the quality assurance of data-intensive systems.

Hummel et al.  identified eight challenges of quality assurance in the context of data-intensive systems.

-   *Challenging Visualization and Explainability of Results*. Data needs to be visualized with the right balance between data dimensions and resolution, in order to support the user to understand and assess the validity of the data. Furthermore, the processing that lead to a particular result is difficult to explain (e.g., in deep learning). Thus, trustworthiness and understandability are important challenges for data-intensive systems.

-   *Non-Intuitive Notion of Consistency*. The large volume of data that is processed by data-intensive systems requires to weaken the notion of data consistency for performance reasons. These inconsistencies are confusing for users not used to softened consistency notions.

-   *Complex Data Processing and Different Notions of Correctness*. The numerous processing steps and interconnections between data makes the processing of data in data-intensive systems complex. The notion of correctness becomes difficult to define, which complicates the testability of such systems.

-   *High Hardware Requirements for Testing*. Data-intensive systems are expected to process large volumes of data. Thus, testing requires to consider issues like performance or scalability that are related to processing of big data. However, this requires hardware similar to its later application environment, which may be not possible because of involved operating costs of such environments.

-   *Difficult Generation of Adequate, High-Quality Data*. The data provided for testing should represent realistic and application-specific data in order to meaningfully test a data-intensive system. Thus, all big data characteristics described in Section should be covered, which makes test data generation and management a challenging task, especially when also taking data quality aspects into account.

-   *Lack of Debugging, Logging, and Error-Tracing Methods*. The architecture of data-intensive systems results into a distributed system that complicates debugging. Furthermore, logging is scattered over the different components of a system. As a result, tracing errors back to their origin or understanding system behavior based on the logs become inherently difficult.

-   *State Explosion in Verification*. The processing of large volumes of data that requires distributed computing to process its requests, results into an exponential state explosion that makes verification approaches complex to apply.

-   *Ensuring Data Quality*. The data quality of the large volumes of data that are collected, processed and aggregated by data-intensive systems is difficult to assess computationally and semantically.

# Cyber-Physical Production Systems as Data-Intensive Systems

Industrial production systems like robots, steel mills or manufacturing plants are large and long-running cyber-physical systems that generate and process large volumes of data from various sources (see Fig. ) and types (e.g., order data, personnel data or machine data). Data sources of typical production systems are among others technical production data (like machine or process data), organizational production data (like order or personnel data) and messages from the ERP system. Thus, cyber-physical production systems represent data-intensive systems that are of high value to the business and require testing. The production is tightly coupled to the data-intensive systems, which puts high requirements on the standard of quality assurance of the system and the data it operates on.

The architecture of a large cyber-physical production system as used in manufacturing plant is conceptually consistent with the architecture described in Section . Data of various sources (e.g., sensors, orders, processes) is collected, unified and organized according to the later information needs of the production system (e.g., order status). Employees or intelligent systems (as envisioned by Industry 4.0 ) can search within the data and extract relevant information for their specific data needs. Finally, every production system has some kind of workflow management that processes the data in order to generate data products like aggregated reports or prediction models (e.g., for predictive maintenance) to support decision making.

Production failures or unintended production stops are possible results of software faults in the cyber-physical production system and consequence of inappropriate testing. A cyber-physical production system in a manufacturing plant, which represents a complex and large production system that processes large volumes of data, is a critical business asset that requires careful testing to mitigate production related risks. Furthermore, the software and data quality of the system has a strong affect on the production itself. Higher quality standards can lead to optimizations and improvements of the overall production, whereas quality problems can lead to a decline in production. Challenges to the quality assurance (see Section ) like high hardware requirements for testing (e.g., related to robots or production line) or the difficulty to ensure the quality of the data from various sources (e.g., machines, orders, processes) are specifically present in production systems. Moreover, the trend to smart factories, that, for example, encourage the intelligent communication between robots, further increases the requirements on quality assurance. As a result, the research on testing of data-intensive software systems (see Section and Section ) is of great importance for large cyber-physical production systems in general and their quality assurance in particular.

# Testing Data-Intensive Software Systems

Research on testing data-intensive systems is becoming of high academic and practical importance. Only few recent papers (e.g., Hummel et al.  as presented in Section ) discussed challenges and derived open questions to start guiding research in that field. However, primary studies contributing approaches on testing data-intensive systems are still rare. In this section, we report the result of our literature study on existing research on testing data-intensive systems. The study is not intended to be systematic, but rather exploratory to get an initial insight on the aspects of the research that are currently under investigation.

## Literature Study on Testing Data-Intensive Systems

We performed an exploratory and preliminary literature study to get initial insights into the research on testing data-intensive systems. Papers have been collected from Google Scholar[^1] and the IEEE Xplore Digital Library[^2] by searching with keywords "Data-Intensive System" (OR "Data-Intensive Systems") AND "Software Testing". Included papers are written in English and are journal articles, conference or workshop papers, book chapters, technical reports or thesis works that cover at least one testing activity as well as big data or data processing aspects. The search initially returned 133 papers. After reading title and abstract and eventually the body of the work, papers were removed if they:

-   Misused the search terms (e.g., "Data-Oriented Systems" as "Data-Intensive System"),

-   Only used the search term "Data-Intensive System" or "Software Testing" in their citations,

-   Only referred to data-intensive systems as one of the systems under analysis with no specific discussion or exploitation of the characteristics of data-intensive systems, or

-   Mention "Data-Intensive System" only in the related work section or as an example context in which an approach can eventually work.

Taking these criteria into account, we could finally include 16 relevant papers. This indicates that software testing of data-intensive systems is not yet an established and clearly-defined term. The 16 papers were further classified according to three major areas, (i.e., Testing, System, Data), related categories (e.g., Level of Testing), and factors (e.g., Component, Integration, and System for Level of Testing) based on the following criteria:

-   Testing

    -   Level of Testing (Component, Integration, System)

    -   Test Activity (Test Management, Test-Case Design (Criteria-based), Test-Case Design (Human Knowledge-Based), Test Automation, Test Execution, Static Analysis, Test Evaluation (Oracle))

-   System

    -   System Quality (Security, Performance, Conformance, Usability etc.)

    -   System Artifact (SQL Query, System Code, User Form etc.)

-   Data

    -   Big Data Aspect (Volume, Velocity, Variety, Veracity, Value)

    -   Data Processing Aspect (Collection, Generation, Manipulation, Redistribution)

    -   Data Quality Aspect (Integrity etc.)

    -   Data Technology (Database, Cache etc.)

The applied classification scheme was derived from established classifications as presented in Section  and Section , experience of the authors and an analysis of relevant papers. Open coding (to identify the categories) and axial coding (to relate categories) of grounded theory  on titles and abstracts of such papers have been used to verify and refine the initial categorization.

## Classification of Papers

The 16 relevant papers were classified according to the categories and factors defined in Section . The classification has been performed by reading the body of each paper. Papers may be classified into multiple factors of the same category or multiple categories of the same area.

Figure  illustrates the resulting classification of the included papers by comparing categories and factors characterizing software testing with the ones of data-intensive system (i.e., categories classified as *System* and *Data* in Section ). One may immediately recognize that not all factors are included in the figure. For instance, Velocity for the Big Data aspects or Cache for the Data Technology aspects are never considered when it comes to software testing in the investigated set of papers. For each factor, the maximum count is four (i.e., a quarter of the investigated papers), which is attained for (1) Database in the Data Technology category at Component and System Level of Testing, (2) Test-Case Design (Human Knowledge-Based) and Test Automation in the Test Activity category for what concerns SQL Query in System Artifacts and at the System Level Testing and Security in System Quality, respectively, (3) Volume in Big Data aspects concerning System Level of Testing.

# Discussion

In this section, we discuss the current state of testing data-intensive software systems and present directions of research that have the potential to significantly contribute to solving key challenges of testing data-intensive software systems.

The initial literature survey presented in the last section revealed that the automatic search in digital libraries returns a large number of false positives (i.e., 117 out of 133 papers) where the terms of the search are used only marginally, often to mention only future application or related work of a proposed approach. However, we also need to acknowledge that our approach is just exploratory and may have missed some relevant papers. In particular, we decided not to perform snowballing from the papers retrieved from the digital libraries as research on data-intensive system is rather new and backward snowballing seemed not to be promising after a first check of the references of the retrieved papers. With the classification of the relevant papers, we found that some aspects that commonly refer to testing or data-intensive software system are not yet explored when it comes to testing of data-intensive systems. For instance, for velocity as a big data aspect suitable testing approaches for data-intensive systems are not available. Velocity is of paramount importance in modern data-intensive systems, especially in the context of real time analytics, where results are delivered in a continuous fashion and results are only useful if the delay is very short. However, before focusing on testing for velocity, the research community is still struggling with the uncertainty of the output of test cases and the definition of suitable oracles () and the volume and variety of data that data-intensive systems can potentially handle (e.g., , ). A certain number of papers concern testing the correctness and the integrity of the data stored in databases by data-intensive systems. For instance, static analysis has been used to test SQL statements embedded in the code of data-intensive systems to ensure database integrity .

However, there are several promising directions of research and new frontiers that are highly relevant in the context of data-intensive systems that have not yet been fully exploited.

Testing of data-intensive software systems is problematic because these systems belong to the class of "non-testable" software, where typically no test oracles exist for verification . A proven approach for testing such systems is metamorphic testing , which requires the discovery of metamorphic relations that can be used to test the validity of machine learning algorithms. For instance, Xie et al.  provide an approach to metamorphic testing of machine-learning-based classification algorithms. The authors enumerate the metamorphic relations that classifiers would be expected to demonstrate and then determine for a given implementation whether each relation is a necessary property for the corresponding classification algorithm. If this is the case, then failure to exhibit the relation indicates a fault.

Testing of machine learning algorithms, especially for sophisticated algorithms like deep learning systems, requires specific testing approaches. To that end, recently several adequacy criteria for deep learning systems like neuron coverage , surprise adequacy  or criteria derived from modeling deep learning systems as abstract state transition systems  were defined. However, testing algorithms is not sufficient as the integration of algorithms into systems can be complex, leading to problems and defects being injected along the way. Specific concerns mentioned in the literature include untrustworthy data, bad assumptions, and incorrect mathematics among others . Tian et al.  propose an approach based on metamorphic testing to test deep learning systems in the context of autonomous driving. As especially the environment is very complex, simulation also plays an important role for testing data-intensive systems as used in the context of autonomous driving, where large, complex, and independent system interact and form so called Systems-of-Systems .

One step further, testing can even be shifted to the running system. This novel approach to testing of data-intensive systems performs monitoring of the runtime environment to assess the behavior and changes in the system. System changes under test are exposed to a limited number of users. The measured runtime data generated through their interaction with the changes are compared to users that were not exposed to the changes. The analysis of the collected experimentation data reveals the influence of the chance on predefined characteristics like performance or usability. The approach of system change assessment is also continuous experimentation () and promising especially also in the context of testing data-intensive systems .

Observing unexpected execution behavior is also used to build self-assessment oracles, a new type of oracles introduced in the anticipatory testing approach . Such an approach aims at detecting failures before they even occur. Its oracles can be crucial for data-intensive systems where the volume of the input and the uncertainty of the output are key challenges in designing test cases .

Taking the discussion into account, the categories on the testing dimensions for traditional systems shown in Fig.  have to be extended for testing data-intensive software systems. The testing dimensions and their values for data-intensive software systems are shown in Fig. . The added categories are shown in italic.

The dimension test level is extended by the values *Algorithm* and *System-of-Systems*. On the one hand, the testing of algorithms in isolation becomes essential (e.g., for classification algorithms or deep learning algorithms). On the other hand, also *Systems-of-Systems*, for instance in the context of autonomous driving, require new testing approaches where especially simulation of the environment plays an important role. The dimension execution level is extended by *Runtime Monitoring*, where a running system is passively observed to test changes or unspecified system behavior, in contrast to active penetration in case of dynamic testing. Finally, the dimension test objective is extended by *Data Quality*, which comprises testing of data and big data quality properties like correctness or velocity.

# Conclusion

Software engineering research is needed to fully understand the role of testing in data-intensive software systems (like large cyber-physical production systems) and to provide approaches and frameworks to properly address this challenging and critical issue. The goal of this chapter is to contribute into this direction by providing basic terminology, an overview of the literature, a discussion, and future directions of research.

In this chapter, we first presented the background as well as the basic terminology and then we discuss the current state and new frontiers on testing data-intensive software systems that are also relevant for testing cyber-physical production systems. To this end, we first provided a definition of data-intensive system (Section ) that clearly distinguishes them from other types of software systems and characterized cyber-physical production systems as data-intensive systems. The definition also helped us to derive the major factors characterizing such systems and drive the investigation on the existing research (Section ). We further discussed the current state and new frontiers on testing data-intensive software systems (Section ). The frontiers include metamorphic testing, algorithm testing, testing systems-of-systems, data quality aspects, runtime monitoring, and anticipatory testing. Finally, we extended the testing dimensions for data-intensive software systems accordingly.

[^1]: <http://scholar.google.at>

[^2]: <https://ieeexplore.ieee.org>