abstract: We consider the effects of social learning on the individual learning and genetic evolution of a colony of artificial agents capable of genetic, individual and social modes of adaptation. We confirm that there is strong selection pressure to acquire traits of individual learning and social learning when these are adaptive traits. We show that selection pressure for learning of either kind can supress selection pressure for reproduction or greater fitness. We show that social learning differs from individual learning in that it can support a second evolutionary system that is decoupled from the biological evolutionary system. This decoupling leads to an emergent interaction where immature agents are more likely to engage in learning activities than mature agents.
author: Chris Marriott$^{1}$; Jobran Chebib$^2$

$^1$University of Washington, Tacoma, WA, USA 98402
$^2$University of Calgary, Calgary, AB, Canada T2N 1N4
email@example.com
bibliography: example.bib
title: The Effect of Social Learning on Individual Learning and Evolution

# Introduction

When agents possess both genetic adaptation operating on a generational time scale and learning operating on the agent's time scale there is the potential for interaction between these adaptive mechanisms. have surveyed studies into this interaction and show conditions under which the presence of learning can accelerate or decelerate genetic adaptation.

In particular found that if the fitness function in the direction of adaptation is concave and the step size of the learning algorithm is small genetic adaptation can occur at an increased rate. This acceleratory effect was seen in a number of different simulations (for instance ).

On the other hand if this condition is not upheld, or indeed the opposite conditions exists, then the learning can slow the natural genetic adaptation (see ). Some simulations have shown both acceleratory and deceleratory effects under different conditions ().

Social learning is a form of learning that arises from social situatedness () and is characterized by agents interacting with one another in order to learn. Social learning can accelerate learning beyond that of individual learning strategies (see ) and most notably is its ability to support cumulative cultural evolution () as witnessed in human culture. While a wide range of mechanisms of learning have been studied in conjunction with evolution we have found little evidence of the study of social learning mechanisms on evolution. Further, since social learning is a distinct form of learning identified by its social as opposed to individual nature, it may also interact with the individual learning processes. In this study we explore the interactions between social learning, individual learning and genetic adaptation.

We have designed a model that allows for evolution of reproductive, individual learning and social learning abilities in conjunction with traits for fitness. These abilities are not just on/off for our agents but instead can be participated in more or less than others. The abilities also come at a cost to the agent (in time) which limits the maximum fitness achievable by the agent.

The model we have developed allows for the genetic information and the learned information to be expressed in the same format, allowing for direct comparison between genetic adaptation and the individual and social learning processes.

# Experimental Setup

### Learning Task

Our simulation involves agents that gather resources from a number of different resource sites. Each site has five locations where resources might be found. Only one, two, or three of the locations actually have resources while the others are empty. We call this value the reward of the site. Say an agent is at a resource site with three resources hidden in the five locations. The agent must select the order it will check the locations. Once the agent has all three resources it can stop checking, and the cost of this gather attempt is the number of locations checked. This cost is measured in time units.

Agents will have to select which sites to gather from each day and what order to check locations at those sites. The resource sites may not be repeated in a day, but can be repeated again the next day. The next day the resource site will have the same number of resources and they will be stored at the same locations. This allows for the agent to adapt its strategy on future days.

More formally we can think of a resource site as a subset of $\{1,2,3,4,5\}$ with size equal to the reward. For instance with reward equal to 3 the resource site might be the set $\{1,2,4\}$. An attempt at this site by the agent is characterized by a permutation of the set $\{1,2,3,4,5\}$, for instance, $[5,4,2,1,3]$. The cost to the agent (in time units) at this site is equal to the number of entries in the permutation that must be processed before the subset representing the site has been covered by the elements of the permutation. In our example it requires the first 4 elements of $[5,4,2,1,3]$ to cover the set $\{1,2,4\}$ resulting in a cost of 4 time units spent to gather 3 resources. Notice the maximum cost is always 5 and the minimum cost is equal to the reward.

An agent is allocated 50 time units in a day and may gather from as many sites as it has time units. Thus optimally an agent can gather 50 rewards from resource sites in one day if all time units are dedicated to gathering. An agent may also spend its time units on activities that may increase its long term fitness. Specifically it may spend units attempting to reproduce, learn individually, or learn socially. These tasks each take one time unit and may be taken more than once in a day.

### Agent Design

The agent consists of a genome and a memome. The genome and the memome are internally identical, but play different roles in the agent. The genome is inert information that remains static for the duration of the agent's life. It is used during asexual reproduction to create a new agent. The memome is dynamic and may change over the lifetime of the agent. It is used to select daily behavior and may be spread socially.

Genomes and memomes are internally identical and in this paragraph we will describe these internals from the perspective of a genome. However, everything is equally true of the memome. A genome consists of 50 geneplexes, each describing a series of actions that could be carried out by an agent in a single day. A geneplex's total cost is the time cost of all actions in the geneplex and is always less than or equal to 50. A geneplex's fitness is equal to the total reward that its actions accrue (ties are broken favoring geneplexes that spend less time attaining the same total reward or spend some time reproducing or learning). When a genome is mutated the five geneplexes with greatest fitness replace the five geneplexes with least fitness and the copies are mutated. A geneplex mutates by changing the strategy at a randomly selected resource site, adding a randomly generated action, or removing a randomly selected action.

At birth an agent will copy its genome to create its memome. When a day's activities are to be selected the agent takes the memeplex with highest fitness and carries out its list of actions. Its daily reward is the total number of resources these actions accrue.

A new agent is created from a parent selected randomly but not uniformly from the population. An agent's chance of being selected is equal to the number of breeding tickets it has. It is rewarded one breeding ticket for every five resources it gathered today and one for each time unit the agent spent on reproducing today. So agents better at gathering and who spend more time on reproduction are more likely to be a parent. The new agent's genome is a mutated clone of its parent's genome.

When an agent is born the agent with least fitness dies to maintain a constant population size. An agent's fitness is equal to the average daily reward during its life minus its age. As a result agents that die are either poorer at gathering, older, or both.

During a day an agent may spend time learning. For each occurrence of an individual learning action in its daily action list the agent will mutate its memome once. This activity will change the memome but leaves the genome unchanged. The new memome is used to select the next day's actions.

An agent is selected for participation in a social gathering with probability proportional to the number of time units spent on socially learning. The selected agents then contribute their best five memeplexes into a pool. The best five in the pool are then redistributed to the agents replacing their five least fit memeplexes. Again this exchange affects only the memome leaving the genome unchanged.

### Population Design

We considered three isolated populations of 50 agents. Each agent in the initial population has a genome consisting of geneplexes with only a single random gathering action. No genome was seeded with reproducing, individual learning, or social learning actions.

The first population served as a control and agents were not capable of learning in any way. Their only means of behavioral adaptation is genetic evolution. Any actions spent by these agents on learning were wasted actions as they had no result and thus these actions were slightly maladaptive for these agents. The memome and genome were identical in these agents since their memome was incapable of change.

Agents in the second population were capable of learning from the environment but not from each other. Any actions spent on learning from others were wasted actions in this population and thus were slightly maladaptive. Agents had to evolve to learn before they would begin learning.

Agents in the third population were capable of all the learning strategies, and like the agents in the second population, they needed to evolve to learn individually or socially. These learning capabilities were separate and thus evolving one did not imply evolving the other.

We gathered data on the maximum fitness of agents in each population over time. This allowed us to track the optimization occurring in each population. We also tracked the maximum fitness of geneplexes in the genomes of agents in each population. This allowed us to isolate and track genetic optimization. We also gathered data on the frequency of actions for reproducing, individual learning, and social learning among our agents. We selected an epoch length of 10000 days with data samples every 20 days. We have averaged results over 159 runs for presentation. With such a large number of runs the confidence intervals are very small and are omitted for clarity.

# Observations and Discussion

### Overall Fitness

By day 1500 the average maximum memetic fitness of agents in the social learning group increases to a maximum of 48 (see Figure ). Agents cannot both attain the optimum and maintain their social learning behavior. In most runs agents converged on a shared memeplex with one social learning action and one individual learning action (and occasionally a reproduction action).

This behavior can be considered optimal in that a memeplex of higher fitness would have to replace either the social learning action or the individual learning action with a gathering action, and thus lose the benefit these actions give. Replacing the social learning action might make the agent more fit, but would prevent that action from being shared with others and the more fit memeplex would die with the agent. Replacing the individual learning action would stop individual optimization of the memeplex making it very unlikely to increase fitness. Unlikely as this event is in our model we do observe this occurring infrequently.

By day 1500 the average maximum fitness of agents from the individual learning group has increased to 45. By the end of the 10000 day epoch this has been further optimized to nearly 47.

At the 10000 day epoch individual learners have an average of four actions allocated to reproducing and learning in the memome. Individual learners are responsible for optimizing their own memeplexes whereas social learners can rely on the community to communicate the optimums to them. As a result the individual learners must maintain greater commitment to learning even when near the optimum fitness to be competitive.

The average maximum fitness of agents from the non-learning group increases to only 34 by day 1500. At the end of the 10000 day epoch the average fitness is nearly 44. This is still a few steps from the optimal and this is because we see at the end of the epoch the maximal memeplex had on average four reproduction actions (and one of the null actions).

This behavior should be seen as nearly optimal. Breeding tickets are awarded one for every five rewards achieved and one for every reproduction action. This would suggest that there is an equilibrium between selection pressures rewarding greater fitness and those rewarding time spent on reproduction. Since on average five time units are spent on non-gathering actions the fitness of these agents can theoretically be increased by five. However this would award the agents only a single additional breeding ticket (in exchange for four lost), greatly decreasing their odds of reproducing. On the other hand agents that spend much more time on breeding actions than four (or five) will reduce their fitness making them more vulnerable to early death due to low fitness. The midpoint between these opposing selective forces appears to be at five time units.

These average maximum fitness results support a common hypothesis regarding the rate of optimization utilizing these different modes of learning (). Specifically, genetic optimization is seen as slower than optimization through individual learning or learning from the social group. Furthermore, learning from the social group can maintain learned strategies in the memosphere and thus carry out cummulative learning. This leads to the hypothesis that social learning is the most rapid mode of optimization.

### Genetic Fitness

Our interest extends further to the effect that these modes of learning have on the genetics of the agents using them. Modern hypotheses () suggest that the learning mechanisms can both heighten selection, thus accelerating genetic adaptation, and shield it, thus decelerating genetic adaptation.

Both the individual learners and the social learners had less fit genomes compared to the control non-learning group for most of the simulation. At day 1500 both learning groups have a average maximum genome fitness of 30 while the non-learning groups was 34 (see Figure ). This indicates that the selection pressure operating on the genomes according to fitness levels was shielded by the learning of both kinds in the early days of the simulation. This is predicted in instances like ours where the learning can operate to flatten the fitness landscape.

Optimization is shielded for fitness levels in the range 15 to 30 during an "easy task" of growing the initially short geneplexes to ones that use all 50 time units. After this point the optimization problem increases in difficulty since the geneplexes then can only be optimized by improving the strategies of individual gathering attempts.

There does not appear to be any further shielding effect on the social learners during the "difficult task". That is the slope of improvement for non-learners and social learners is very close after day 1500 (both improve 10 fitness in 8500 days). In contrast the individual learners appear to optimize at a more rapid rate (12 fitness in 8500 days).

In individual learners differences between agents' average daily fitnesses becomes dependent on early performance and speed of convergence while solving the "difficult task". Agents that have less fit genomes perform worse early on and those that take longer to converge spend more time at lower reward levels. Thus, conditions seem met for an acceleration of adaptation of both the genes for learning and the genes for more efficient gathering.

That is, agents that have a more fit genome or occupy positions that allow them to more rapidly overcome genome deficiencies are selected for in the population. On the one hand this pressure rewards agents with more fit genomes because they both perform better early on and can optimize more rapidly from this better position. On the other hand this pressure would favor agents that can learn rapidly indicating selection for individual learning actions (see discussion below). This seems to be a case of Baldwin's initial hypothesis where the selection pressure accelerates adaptation of plasticity simultaneously with greater fitness.

Noting this we might expect also that social learners should benefit from this same acceleration though we did not find this in our data. We did find strong evidence of increased selection for social learning actions (see discussion below) in these agents and we hypothesize the benefits of these actions greatly outweigh benefits to early performance.

### Reproductive Actions

Non-learning agents converged on an average of four reproduction actions by the end of the epoch. This is due to the selection pressure that is applied related to the increased chance of reproduction for these actions. We expect the same selection pressure to exist for the learning agents as well and we see evidence for that in the genomes of the learners (see Figure ). The individual learners steadily increase to an average of about 2.75 reproductive actions per agent. Notably this average is lower than that in the non-learners indicating that this selection pressure is shielded by the learning. We also see evidence of this shielding effect in the social learners who increase to an average of about 1.6 reproductive actions per agent.

A contributing factor to the shielding effect is that the memome is used to determine the number of breeding tickets issued, not the genome. As a result the number of reproductive actions in the genome do not contribute long term to the number of breeding tickets issued to the agent, as the memome is assumed to change from its initial state in learners. As such the reproductive actions in the genome contribute most to the chance of breeding early in life (before learning potentially replaces these actions). This also means that an early genetically determined reproductive behavior can be learned away. This provides fewer opportunities for more fecund genomes to express their greater fecundity resulting in a shielded selection pressure.

We see that in individual learners the number of reproduction actions in the memome is coupled to the number in the genome. While prior to day 3000 there are more learning actions in the memome than the genome, this changes after about day 3000. The reason for this change is as the genetic fitness approaches the optimum the pressure to learn decreases.

In the social learners we see a different dynamic. The average reproductive actions in the memome seem uncoupled from the genome. The reproductive actions in the genome increase due to pressure (if shielded) for increased reproduction. In the memome we have a depreciating curve, indicating selection pressure against breeding in the memome. Breeding actions do not help spread the memeplex, but they do help spread the geneplex, and so we see divergent selection pressures here and a decoupling of the adaptive processes.

This decoupling is due to the independence of the evolution of memeplexes from the evolution of geneplexes and this independence results in cumulative cultural evolution. The individual learning of each individual learner must begin optimizing from the initial condition coded in its genome (hence a coupling of these processes). The social learner in contrast can begin from whatever state is provided by its community. Thus it need not "reinvent the wheel" and repeat learning steps that others have already carried out. This leads to a memosphere whose evolution is independent of (decoupled from) the co-evolving genosphere. As a result the selection pressure that occurs in the genosphere for reproductive actions does not exist in the memosphere. This may also be the case for individual learners, but only the social learners have decoupled the memosphere's evolutionary path so the selection pressure can be effective.

Nonetheless the genetic selection pressure still penetrates into the genome. Since agents still have early performance based on their genome, early reproductive performance is still determined by the reproductive actions in the genome. In our simulations the expected number of days before a new agent participates in its first social learning gathering is quite small. After this point in the agent's life the selection pressure is shielded by the memome but before this point it can favor agents that have more reproductive actions in the genome.

### Individual Learning Actions

Learning actions in the non-learning group were null actions and were slightly maladaptive. The maladaptivity came from the fact that the action had a temporal cost of one time unit that could have been better spent on harvesting. This would suggest that there would be selection against having this trait in the genome. We see that the average memome (and thus genome) in the non-learning group has around 0.5 occurrences of each null learning action or about 1 full occurrence of a null learning action (see Figure ). In our simulation null actions and no actions are rewarded the same in fitness.

Then the frequency of null learning actions in the memome of our non-learners speaks to the average number of time units not spent on harvesting. Specifically we'd expect an equal number of empty spaces to null learning actions (since they are equally favored by the fitness function) so an average of one null learning action suggests an average of two time units not spent on harvesting in the genome of non-learners. These unused time units are integral to the formation of more fit variants as they are replaced with new harvesting actions.

Individual learning is adaptive for the individual learning agents. We see a very rapid assimilation of the individual learning action into the population after its first appearance in the genosphere. Since with our settings it takes about 100 days to replace the entire population we expect it takes about this minimum time for the individual learning action to be represented in every living agent. That is, non-learning agents are quickly replaced by individual learning agents in the population once learning agents evolve.

This selection for individual learning continues after this initial assimilation in both the memome and the genome. For instance we see early on that the memome's average frequency of individual learning actions exceeds that of the genome's average frequency. The first agents that evolved to learn then quickly learned to learn more (their memome's acquired a second or third occurrence of the individual learning action to accelerate learning). In addition, selection for learning actions in the genome continues at a rapid rate, eventually overtaking the frequency in the memome at around two individual learning actions per agent on average.

This switch is best explained with reference to when individual learning is the most advantageous to an agent. Young agents have lower fitnesses (since their daily reward is tied to their genome) and thus to be competitive they need to rely on their individual learning actions to optimize their memome rapidly. The more rapidly they do this the smaller overall hit they will take to their fitness. Thus selection strongly favors agents that have more individual learning actions in their genomes.

This selection does not work similarly in the memome. Older agents will have optimized their memome to a higher fitness and gain less benefit from individual learning. Indeed we see that some rare individuals learn to stop learning near the end of their lives if they can swap out that action with a more fit harvesting action. More commonly we see that agents that at birth had two or three individual learning actions in their genome will optimize at least one of those actions away by their old age.

Another interesting trend in the individual learners is the steady decrease of the of the number of individual learning actions in both the genome and the memome after peaking around 3000 days. This decrease suggests that as the simulation progresses the selection pressure that once favored higher numbers of individual learning actions is lessening. This occurs as the genome optimizes and approaches the memome's fitness levels. Since the genome represents the starting conditions of the individual learning process the more fit this initial state is the less in need of learning the agents are. This happens both within the agent's life (making individual learning actions less common in memomes of older agents than younger) and within the larger population in generational time (making individual learning actions less common in the genomes of agents born later than those born earlier).

Individual learning is still an important skill in the social learning group. While social learning facilitates sharing of memeplexes and the maintenance of a cross-generational culture, it has no means of optimization. All optimization in the social learners is still due to the individual learning actions in their memomes. We see that selection for individual learning in the genomes is rapid in the social learners.

Interestingly we also see a more rapid increase in the presence of individual learning actions in the social learners' memomes in the early stages paralleling the individual learners learning to learn more. However, in the social learners this rapid increase is very quickly countered when the fitness of the shared memeplex reaches near optimal and a new pressure to learn less appears. The consequence of this is that the shared memeplex of the social learners tends to have exactly one individual learning action. Notice this is the minimum number to ensure continued optimization allowing it to satisfy the pressure for continued optimization while minimizing the cost of that optimization.

Further in rare cases the optimization is capable of eliminating that last individual learning action from the memome since there is no need for further optimization when it is at the near optimal. The reason this is a rare occurrence is because for this to occur the agent would have to use an individual learning action to optimize its genome and then eliminate the action that did the optimization and result in a genome that is more fit. This can only occur with a very lucky sequence of mutations in our setup and so is very rare. Nonetheless this is the reason why the average number of individual learning actions in the memome is not exactly one but instead less than, indicating the number of runs in which the social learners were capable of achieving the lucky sequence of mutations to eliminate this action. About one fifth to one quarter of the simulations had eliminated this action by the end of the epoch.

Again the striking difference in shapes between the curves of the individual learning actions in the memome and genome of the social learners can be attributed to a decoupling of selection pressures on the memome and the genome. While young agents will be more likely to individually learn as soon as they socially learn their genetic proclivity is overridden by the shared memeplex's behavior.

### Social Learning Actions

Finally consider the social learning actions. We know these actions are null actions in both the non-learners and individual learners and as such we see very low frequencies in both the genomes and memomes of these agents. For the reasons discussed above regarding tie breaks we see non-zero occurrences of these actions.

However in the social learners the social learning action is efficacious. Further social learning is adaptive for these agents, but even more so if it is accompanied by an individual learning action. Without the individual learning action a collection of social learners will only share the most fit memeplex in their memomes. Since they are not engaged in individual learning this represents the most fit geneplex in their genome. As a result agents can still benefit from social learning in the absence of individual learning, though this benefit is negligible.

There is a parallel and independent selection pressure for individual learning. Both selection pressures are strong and so agents are expected to quickly evolve these actions, but very commonly they evolve roughly simultaneously. The two pressures are synergistic in that agents that evolve both traits outperform those with only one. The results of this synergistic selection is that the genomes of agents rapidly include at least one copy of each of the learning actions, with social learning actions being selected for more aggressively than individual learning actions.

We also see that selection for the social learning action in the genome continues to increase whereas in the memome this frequency is highly stable with an average of one per agent. Indeed the reason that this average is greater than one is that newborn agents are frequently born with more than one social learning action in their genome (and thus their initial memome) and this slight deviation from the norm of a few agents in the social community is enough to keep this average slightly over one. This is further evidence of the decoupling of the memosphere and the genosphere.

This decoupling leads to a very interesting dynamic. We see that social learners optimize to have on average about 2.5 social learning actions in the genome of an agent. A mature adult (one that has already engaged in social learning at least once) will have only one social learning action in the memome of the agent. This means that a newborn or immature agent will be 2.5 times more likely to engage in social learning than a mature adult.

This is an important feature since this means that younger, immature social learners are more likely to engage in the activity that will make them more fit, but once they have engaged in that activity they do not attempt to perform it as frequently anymore allowing for younger agents to have the opportunity. It is also still important for mature agents to engage in social learning but their role is as an instructor, not a learner in most of these meetings.

The pressures that lead to this state are once again independent and decoupled. Young agents, due to selection on early performance on their genome, face a strong selection pressure to socialize early. This is accommodated by having as many social learning actions as possible in their genome. Old agents, due to selection pressure related to fitness, face a strong selection pressure to socialize less to make room for more harvesting. This is accomodated by having the minimum number of socializaing actions as possible in the memome. The equilibrium found by this interaction is to make young agents socialize as soon as they can while older agents socializing only enough to ensure that young agents learn the dominant memeplexes in the memosphere.

Once again this is clear evidence of the decoupling of the memosphere from the genosphere in the social learners. Nonetheless, the early stages of a memome's development is still dependent on the genome of the agent. Once the agent learns from the community this dependence ends and the systems are decoupled.

# Conclusions

Our experiment shows how social learning, individual learning and genetic adaptation can co-exist in a population of agents and some of the interactive dynamics between these optimizing forces. Specifically we see both instances of accelerated and decelerated genetic adaptation in response to these learning mechanisms in our model.

We see clear selection shielding in the early stages of the simulation as a result of both individual learning and social learning. The cause of this shielding is that the learning mechanisms flatten the fitness function, thus equalizing selection pressure on all agents. Later in the simulation we see slight selection boosting for the individual learners. The cause of the boosting is selection for early performance and a Baldwin effect.

In both the learning populations we see shielding of selection pressures for reproductive actions in the genome. These selection pressures still exist but are weaker in agents capable of individual and/or social learning. Shielding is greatest in social learners. This indicates that individual learning and social learning in particular can weaken or even eliminate selection pressure for genetic reproductive behavior while simultaneously suppressing the same behaviors.

In both learning populations we also see boosting of selection pressures for learning actions in a genome. The fact that both learning mechanisms are highly adaptive leads to learners always outperforming non-learners and thus rapid assimilation of new learning actions into the genome occurs. Further we see a trend of increased selection for learning actions in the genome even when there is contrasted selection for removing those actions from the memome. This emerges from the opposite need of young and old agents for learning. The young agents are necessarily less fit and so can really benefit from learning, whereas the old agents tend to be highly optimized and thus further learning is wasted effort.

This trend appears in both individual learners and social learners but is most apparent in the social learners. This is due the evolution of the memosphere decoupling from the evolution of the genosphere made possible by the shared memosphere. Not only is this decoupling responsible for the rapid optimization of the social learning group but it results in divergent and uncorrelated behavior frequencies between genome and memome.

To be clear the evolution of the memosphere and genosphere in the individual learners is independent but the starting position of both optimization procedures begin with the genome, and thus the two procedures are coupled. In the social learners the genosphere and memosphere are coupled for a very short period before the memosphere reinitializes with the best memeplex from the community. This allows the memosphere's evolution to begin from a different point than the genosphere's evolution. This implies the decoupling is not $100\%$ but there is still a small link. We say that while decoupled the memosphere is still tethered to the genosphere.

The decoupling of the two evolutionary spheres allows for opposite selection pressures to be fully efficacious in each sphere. The best evidence of this we have is that while selection for reproductive, individual learning, and social learning actions is strong in the genosphere of social learning agents, it is weak, or even negative, in the memosphere of social learning agents. The memosphere attempts to eliminate the extraneous reproduction actions and learning actions, while the genosphere is selecting for greater numbers of these actions in the genome. The greater numbers in the genoshpere lead to the emergent behavior that young agents are more likely to reproduce, learn on their own, or learn from others than their older counterparts. This is beneficial to them since they are the least fit members of the population at this time.


abstract: Usability is a key quality attribute of successful software systems. Unfortunately, there is no common understanding of the factors influencing usability and their interrelations. Hence, the lack of a comprehensive basis for designing, analyzing and improving user interfaces. This paper proposes a 2-dimensional model of usability that associates system properties with the activities carried out by the user. By separating activities and properties, sound quality criteria can be identified, thus facilitating statements concerning their interdependencies. This model is based on a tested quality meta-model that fosters preciseness and completeness. A case study demonstrates the manner by which such a model aids in revealing contradictions and omissions in existing usability standards. Furthermore, the model serves as a central and structured knowledge base for the entire quality assurance process, e. g. the automatic generation of guideline documents.
author: Sebastian Winter; Stefan Wagner; Florian Deissenboeck
bibliography: dsvis07.bib
title: A Comprehensive Model of Usability

**Keywords:** usability, quality models, quality assessment

# Introduction

There is a variety of standards concerning the quality attribute *usability* or *quality in use* . Although in general all these standards point in the same direction, due to different intuitive understandings of usability, they render it difficult to analyze, measure, and improve the usability of a system. A similar situation also exists for other quality attributes, e. g. *reliability* or *maintainability*. One possibility to address this problem is to build a comprehensive model of the quality attribute. Most models take recourse to the decomposition of quality proposed by Boehm et al. . However, this decomposition is still too abstract and imprecise to be used concretely for analysis and measurement.

More comprehensive models have been proposed for product quality in general  or even usability . However, these models have three problems: First, they do not decompose the attributes and criteria to a level that is suitable for actually assessing them for a system. Secondly, these models tend to omit rationale of the required properties of the system. Thirdly, the dimensions used in these models are heterogeneous, e. g. the criteria mix properties of the system with properties of the user. The first problem constrains the use of these models as the basis for analyses. The second one makes it difficult to describe impacts precisely and therefore to convince developers to use it. The third problem hampers the revelation of omissions and inconsistencies in these models. The approach to quality modeling by Broy, Deissenboeck, and Pizka  is one way to deal with these problems. Using an explicit meta-model, it decomposes quality into system properties and their impact on activities carried out by the user. This facilitates a more structured and uniform means of modeling quality.

#### Problem.

Although usability is a key quality attribute in modern software systems, the general understanding of its governing factors is still not good enough for profound analysis and improvement. Moreover, currently there is no objective criteria for evaluating usability.

#### Contribution.

This paper proposes a comprehensive 2-dimensional model of usability based on a quality meta-model that facilitates a structured decomposition of usability and descriptions of the impacts of various facts of the system. This kind of model has proven to be useful for the quality attribute *maintainability* . Several benefits can be derived by using this type of model:

1.  The ability to reveal omissions and contradictions in current models and guidelines.

2.  The ability to generate guidelines for specific tasks automatically.

3.  A basis for (automatic) analysis and measurement.

4.  The provision of an interface with other quality models and quality attributes.

We demonstrate the applicability of the 2-dimensional model in a case study of the ISO 15005  which involves domain-specific refinements. By means of this model we are able to identify several omissions in the standard and suggest improvements.

#### Consequences.

Based on the fact that we can pinpoint omissions and inconsistencies in existing quality models and guidelines, it seems advisable to use an explicit meta-model for usability models, precisely to avoid the weaknesses of the other approaches. Furthermore, it helps to identify homogeneous dimensions for the usability modeling. We believe that our model of usability is a suitable basis for domain- or company-specific models that must be structured and consistent.

#### Outline.

In Sec.  we describe prior work in the area of quality models for usability and the advances and shortcomings it represents. In Sec. , using an explicit meta-model, we discuss the quality modeling approach. The 2-dimensional model of usability that we constructed using this approach is presented in Sec. . This model is refined to a specific model based on an ISO standard in the case study of Sec. . The approach and the case study are discussed in Sec. . In Sec.  we present our final conclusions.

# Related Work

This section describes work in the area of quality models for usability. We discuss general quality models, principles and guidelines, and first attempts to consolidate the quality models.

## Quality Models for Usability

Hierarchical structures as quality models which focus mainly on *quality assurance* have been developed. A model first used by Boehm  and McCall et al.  consists of three layers: factors, criteria, and metrics. Consequently, the approach is referred to as the factor-criteria-metrics model (FCM model). The high-level factors model the main quality goals. These factors are divided into criteria and sub-criteria. When a criterion has not been divided, a metric is defined to measure the criteria. However, this kind of decomposition is too abstract and imprecise to be used for analysis and measurement. In addition, since usability is not a part of the main focus, this factor is not discussed in detail.

In order to provide means for the operational measurement of usability several attempts have been made in the domain *human-computer interaction* (HCI). Prominent examples are the models from Shackel and Richardson  or Nielsen . Nielsen , for example, understands usability as a property with several dimensions, each consisting of different components. He uses five factors: *learnability*, *efficiency*, *memorability*, *errors*, and *satisfaction*. *Learnability* expresses how well a novice user can use the system, while the efficient use of the system by an expert is expressed by *efficiency*. If the system is used occasionally the factor *memorability* is used. This factor differentiates itself from *learnability* by the fact that the user has understood the system previously. Nielsen also mentions that the different factors can conflict with each other.

The ISO has published a number of standards which contain usability models for the operational evaluation of usability. The ISO 9126-1  model consists of two parts. The first part models the *internal* as well as the *external* quality, the second part the *quality in use*. The first part describes six characteristics which are further divided into sub-characteristics. These measurable attributes can be observed during the use of the product. The second part describes attributes for *quality in use*. These attributes are influenced by all six product characteristics. Metrics are given for the assessment of the sub-characteristics. It is important to note that the standard does not look beyond the sub-characteristics intentionally.

The ISO 9241 describes human-factor requirements for the use of software systems with user interface. The ISO 9241-11  provides a framework for the evaluation of a running software system. The framework includes the context of use and describes three basic dimensions of usability: *efficiency*, *effectiveness*, and *satisfaction*.

## Principles and Guidelines

In addition to the models which define usability operationally, a lot of design principles have been developed. Usability principles are derived from knowledge of the HCI domain and serve as a design aid for the designer. For example, the "eight golden rules of dialogue design" from Shneiderman  propose rules that have a positive effect on usability. One of the rules, namely *strive for consistency*, has been criticized by Grudin  for its abstractness. Grudin shows that consistency can be decomposed into three parts that also can be in conflict with each other. Although Grudin does not offer an alternative model, he points out the limitations of the design guidelines.

Dix et al.  argue as well that if principles are defined in an abstract and general manner, they do not help the designer. In order to provide a structure for a comprehensive catalogue of usability principles Dix et al. divide the factors which support the usability of a system into three categories: *learnability*, *flexibility*, and *robustness*. Each category is further divided into sub-factors. The ISO 9241-110  takes a similar approach and describes seven high-level principles for the design of dialogues: *suitability for the task*, *self-descriptiveness*, *controllability*, *conformity with user expectations*, *error tolerance*, *suitability for individualization*, and *suitability for learning*. These principles are not independent of each other and some principles have an impact on other principles. For example *self-descriptiveness* influences *suitability for learning*. Some principles have a part-of relation to other principles. For example, *suitability for individualization* is a part of *controllability*. The standard does not discuss the relations between the principles and gives little information on how the principles are related to the overall framework given in .

## Consolidated Quality Models for Usability

There are approaches which aim to consolidate the different models. Seffah et al.  applied the FCM model to the quality attribute *usability*. The developed model contains 10 factors which are subdivided into 26 criteria. For the measurement of the criteria the model provides 127 metrics.

The motivation behind this model is the high abstraction and lack of aids for the interpretation of metrics in the existing hierarchically-based models. Put somewhat differently, the description of the relation between metrics and high-level factors is missing. In addition, the relation between factors, e. g. *learnability* vs. *understandability*, are not described in the existing models. Seffah et al. also criticize the difficulty in determining how factors relate to each other, if a project uses different models. This complicates the selection of factors for defining high-level management goals. Therefore, in  a consolidated model that is called *quality in use integrated measurement model* (Quim model) is developed.

Since the FCM decomposition doesn't provide any means for precise structuring, the factors used in the Quim model are not independent. For example, *learnability* can be expressed with the factors *efficiency* and *effectiveness* .

The same problem arises with the criteria in the level below the factors: They contain attributes as well as principles, e. g. *minimal memory load*, which is a principle, and *consistency* which is an attribute. They contain attributes about the user (*likeability*) as well as attributes about the product (*attractiveness*). And lastly, they contain attributes that are similar, e. g. *appropriateness* and *consistency*, both of which are defined in the paper as capable of indicating whether visual metaphors are meaningful or not.

To describe how the architecture of a software system influences usability, Folmer and Bosch  developed a framework to model the quality attributes related to usability. The framework is structured in four layers. The high-level layer contains *usability definitions*, i. e. common factors like *efficiency*. The second layer describes concrete measurable *indicators* which are related to the high-level factors. Examples of indicators are *time to learn*, *speed*, or *errors*. The third layer consists of *usability properties* which are higher level concepts derived from design principles like *provide feedback*. The lowest layer describes the *design knowledge* in the community. Design heuristics, e. g. the *undo pattern*, are mapped to the *usability properties*. Van Welie  also approaches the problem by means of a layered model. The main difficulty with layered models is the loss of the exact impact to the element on the high-level layer at the general principle level when a design property is first mapped to a general principle.

Based on Norman's action model  Andre et al. developed the User Action Framework . This framework aims toward a structured knowledge base of usability concepts which provides a means to classify, document, and report usability problems. By contrast, our approach models system properties and their impact on activities.

## Summary

As pointed out, existing quality models generally suffer from one or more of the following shortcomings:

1.  *Assessability.* Most quality models contain a number of criteria that are too coarse-grained to be assessed directly. An example is the *attractiveness* criterion defined by the ISO 9126-1 . Although there might be some intuitive understanding of attractiveness, this model clearly lacks a precise definition and hence a means to assess it.

2.  *Justification.* Additionally, most existing quality models fail to give a detailed account of the impact that specific criteria (or metrics) have on the user interaction. Again the ISO standard cited above is a good example for this problem, since it does not provide any explanation for the presented metrics. Although consolidated models advance on this by providing a more detailed presentation of the relations between criteria and factors, they still lack the desired degree of detail. An example is the relationship between the criterion *feedback* and the factor *universality* presented in . Although these two items are certainly related, the precise nature of the relation is unclear.

3.  *Homogeneity.* Due to a lack of clear separation of different aspect of quality most existing models exhibit inhomogeneous sets of quality criteria. An example is the set of criteria presented in  as it mixes attributes like *consistency* with mechanisms like *feedback* and principles like *minimum memory load*.

# A 2-Dimensional Approach to Model Quality

To address the problems with those quality models described in the previous section we developed the novel two-dimensional quality meta-model Qmm. This meta-model was originally based on our experience with modeling maintainability , but now also serves as a formal specification for quality models covering different quality attributes like *usability* and *reliability*. By using an explicit meta-model we ensure the well-structuredness of these model instances and foster their preciseness as well as completeness.

## The 2-Dimensional Quality Meta-Model

This model is based on the general idea of hierarchical models like FCM, i. e. the breaking down of fuzzy criteria like *learnability* into sub-criteria that are tangible enough to be assessed directly. In contrast to other models, it introduces a rigorous separation of system *properties* and *activities* to be able to describe quality attributes and their impact on the usage of a software product precisely.

This approach is based on the finding that numerous criteria typically associated with usability, e. g. *learnability*, *understandability*, and of course *usability* itself, do not actually describe the properties of a system but rather the activities performed on (or with) the system. It might be objected that these activities are merely expressed in the form of adjectives. We argue, by contrast, that this leads precisely to the most prevalent difficulty of most existing quality models, namely to a dangerous mixture of activities and actual system properties. A typical example of this problem can be found in  where *time behavior* and *navigability* are presented as the same type of criteria. Where *navigability* clearly refers to the navigation activity carried out by the user of the system, *time behavior* is a property of the system and not an activity. One can imagine that this distinction becomes crucial, if the usability of a system is to be evaluated regarding different types of users: The way a user navigates is surely influenced by the system, but is also determined by the individuality of the user. In contrast, the response times of systems are absolutely independent of the user. A simplified visualization of the system property and activity decompositions as well as their interrelations is shown in Fig. . The activities are based on Norman's action model . The whole model is described in detail in Sec. .

The final goal of usability engineering is to improve the *usage* of a system, i. e. to create systems that support the activities that the user performs on the system. Therefore, we claim that usability quality models must not only feature these activities as first-class citizens, but also precisely describe how properties of the system influence them and therewith ultimately determine the usability of the system.

## Facts, Activities, Attributes, & Impacts

Our usability model does not only describe the product, i. e. the user interface, itself, but also comprises all relevant information about the situation of use (incl. the user). To render this description more precisely the model distinguishes between *facts* and *attributes*. Facts serve as a means to describe the situation of use in a hierarchical manner but do not contain quality criteria. For example, they merely model that the fact *user interface* consists of the sub-facts *visual interface* and *aural interface*.

*Attributes* are used to equip the facts with desired or undesired low-level quality criteria like *consistency*, *ambiguousness*, or even the simple attribute *existence*. Thus, tuples of facts and attributes express system properties. An example is the tuple \[Font Face$\vert$CONSISTENCY\] that describes the consistent usage of font faces throughout the user interface. Please note, that for clarity's sake the attributes are not shown in Fig. .

The other part of the model consists of a hierarchical decomposition of the activities performed by a user as part of the interaction with the system. Accordingly, the root node of this tree is the activity *interact* that is subdivided into activities like *execute* and *evaluate* which in turn are broken down into more specific sub-activities.

Similar to facts, activities are equipped with attributes. This allows us to distinguish between different properties of the activities and thereby fosters model preciseness. Attributes typically used for activities are *duration* and *probability of error*. The complete list of attributes is described in Sec. .

The combination of these three concepts enables us to pinpoint the impact that properties of the user interface (plus further aspects of the situation of use) have on the user interaction. Here impacts are always expressed as a relation between fact-attribute-tuples and activity-attribute-tuples and qualified with the direction of the impact (positive or negative):

\[Fact $\mathsf{f}$$\,\vert\,$`<!-- -->`{=html}ATTRIBUTE $\mathsf{a_1}$\] $\stackrel{+/-}{\longrightarrow}$ \[Activity $\mathsf{a}$$\,\vert$ ATTRIBUTE $\mathsf{a_2}$\]

For example, one would use the following impact description

\[Font Face$\,\vert\,$`<!-- -->`{=html}CONSISTENCY\] $\stackrel{-}{\longrightarrow}$ \[Reading$\,\vert$ DURATION\]

to express that the consistent usage of font faces has a positive impact on the time needed to read the text. Similarly the impact

\[Input Validity Checks$\,\vert\,$`<!-- -->`{=html}EXISTENCE\] $\stackrel{-}{\longrightarrow}$ \[Data Input$\,\vert$ PROBABILITY OF ERROR\]

is used to explain that the existence of validity checks for the input reduces the likelihood of an error.

## Tool Support

Our quality models are of substantial size (e. g. the current model for maintainability has $>$ 800 model elements) due to the high level of detail. We see this as a necessity and not a problem, since these models describe very complex circumstances. However, we are well aware that models of this size can only be managed with proper tool support. We have therefore developed a graphical editor, based on the Eclipse platform [^1] that supports quality engineers in creating models and in adapting these models to changing quality needs by refactoring functionality. Additionally, the editor provides quality checks on the quality models themselves, e. g. it warns about facts that do not have an impact on any activity.

For the distribution of quality models the editor provides an export mechanism that facilitates exporting models (or parts thereof) to different target formats. Supported formats are, e. g. , simple graphs that illustrate the activity and system decomposition, but also full-fledged quality guideline documents that serve as the basis for quality reviews. This export functionality can be extended via a plug-in interface.

# Usability Quality Model

Based on the critique of existing usability models described in Sec.  and using the quality modeling approach based on the meta-model from Sec. , we propose a 2-dimensional quality model for usability. The complete model is too large to be described in total, but we will highlight specific core parts of the model to show the main ideas.

Our approach to quality modeling includes *high-level* and *specific* models. The aim of the high-level model is to define a basic set of facts, attributes, and activities that are independent of specific processes and domains. It is simultaneously abstract and general enough to be reusable in various companies and for various products. In order to fit to specific projects and situations the high-level models are refined and tailored into specific models.

## Goals

In accordance with existing standards , we see four basic principles needed for defining usability:

-   *Efficiency.* The utilization of resources.

-   *Effectiveness.* The sharing of successful tasks.

-   *Satisfaction.* The enjoyment of product use.

-   *Safety.* The assurance of non-harmful behavior.

Frøkjær, Hertzum, and Hornbæk  support the importance of these aspects: "Unless domain specific studies suggest otherwise, effectiveness, efficiency, and satisfaction should be considered independent aspects of usability and all be included in usability testing." However, we do not use these principles directly for analysis, but rather to define the usability goals of the system. The goals are split into several attributes of the activities inside the model. For example, the effectiveness of the user interface depends on the probability of error for all activities of usage. Therefore, all impacts on the attribute *probability of error* of activities are impacts on the effectiveness and efficiency. We describe more examples below after first presenting the most important facts, activity trees, and attributes.

## The Activity Subtree "Interacting with the Product"

The activity tree in the usability model has the root node *use* that denotes any kind of usage of the software-based system under consideration. It has two children, namely *execution of secondary tasks* and *interacting with the product*. The former stands for all additional tasks a user has that are not directly related to the software product. The latter is more interesting in our context because it describes the interaction with the software itself. We provide a more detailed explanation of this subtree in the following.

### Activities.

The activity *interacting with the product* is further decomposed, based on the seven stages of action from Norman  that we arranged in a tree structure (Fig. ). We believe that this decomposition is the key for a better understanding of the relationships in usability engineering. Different system properties can have very different influences on different aspects of the use of the system. Only if these are clearly separated will we be able to derive well-founded analyses. The three activities, *forming the goal*, *executing*, and *evaluating*, comprise the first layer of decomposition. The first activity is the mental activity of deciding which goal the user wants to achieve. The second activity refers to the actual action of planning and realizing the task. Finally, the third activity stands for the gathering of information about the world's state and understanding the outcome.

The *executing* node has again three children: First, the user forms his intention to do a specific action. Secondly, the action is specified, i. e. it is determined what is to be done. Thirdly, the action is executed. The *evaluating* node is decomposed into three mental activities: The user perceives the state of the world that exists after executing the action. This observation is then interpreted by the user and, based on this, the outcome of the performed action is evaluated. Scholars often use and adapt this model of action. For example, Sutcliffe  linked error types to the different stages of action and Andre et al.  developed the User Action Framework based on this model.

### Attributes.

To be able to define the relation of the facts and activities to the general usability goals defined above, such as *efficiency* or *effectiveness*, we need to describe additional properties of the activities. This is done by a simple set of attributes that is associated with the activities:

-   *Frequency.* The number of occurrences of a task.

-   *Duration.* The amount of time a task requires.

-   *Physical stress.* The amount of physical requirements necessary to perform a task.

-   *Cognitive load.* The amount of mental requirements necessary to perform a task.

-   *Probability of error.* The distribution of successful and erroneous performances of a task.

As discussed in Sec. , these activity attributes can be used to analyze the usability goals defined during requirements engineering. We already argued that the effectiveness of a user interface is actually determined by the probability of error of the user tasks. In our model, we can explicitly model which facts and situations have an impact on that. The efficiency sets the frequency of an activity into relation to a type of resources: time (duration), physical stress, or cognitive load. We can explicitly model the impacts on the efficiency of these resources. Further attributes can be used to assess other goals.

## The Fact Subtree "Logical User Interface"

The fact tree in the usability model contains several areas that need to be considered in usability engineering, such as the physical user interface or the usage context. By means of the *user* component, important properties of the user can be described. Together with the *application* it forms the context of use. The *physical output devices* and the *physical input devices* are assumed to be part of the physical user interface. However, we concentrate on a part we consider very important: the *logical user interface*. The decomposition follows mainly the logical architecture of a user interface as shown in Fig. .

### Facts.

The logical user interface contains *input channels*, *output channels*, and *dialogue management*. In addition to the architecture, we also add data that is sent via the channels explicitly: *input data* and *output data*. The architecture in Fig.  also contains a specialization of input data, *application-initiated messages*. These messages, which are sent by the *application*, report interrupts of the environment or the application itself to the *dialogue management* outside the normal response to inputs.

### Attributes.

The attributes play an important role in the quality model because they are the properties of the facts that can actually be assessed manually or automatically. It is interesting to note that it is a rather small set of attributes that is capable of describing the important properties of the facts. These attributes are also one main building block that can be reused in company- or domain-specific usability models. Moreover, we observe that the attributes used in the usability model differ only slightly from the ones contained in the maintainability model of . Hence, there seems to be a common basic set of those attributes that is sufficient – in combination with facts – for quality modeling.

-   *Existence*. The most basic attribute that we use is whether a fact exists or not. The pure existence of a fact can have a positive or negative impact on some activities.

-   *Relevance*. When a fact is relevant, it means that it is appropriate and important in the context in which it is described.

-   *Unambiguousness.* An unambiguous fact is precise and clear. This is often important for information or user interface elements that need to be clearly interpreted.

-   *Simplicity*. For various facts it is important that in some contexts they are simple. This often means something similar to small and straightforward.

-   *Conformity*. There are two kinds of conformity: conformity to existing standards and guidelines, and conformity to the expectations of the user. In both cases the fact conforms to something else, i. e. it respects and follows the rules or models that exist.

-   *Consistency*. There are also two kinds of consistency: internal consistency and external consistency. The internal consistency means that the entire product follows the same rules and logic. The external consistency aims at correspondence with external facts, such as analogies, or a common understanding of things. In both cases it describes a kind of homogeneous behavior.

-   *Controllability*. A controllable fact is a fact which relates to behavior that can be strongly influenced by the actions of the user. The user can control its behavior.

-   *Customizability*. A customizable fact is similar to a controllable fact in the sense that the user can change it. However, a customizable fact can be preset and fixed to the needs and preferences of the user.

-   *Guardedness*. In contrast to customizability and controllability, a guarded fact cannot be adjusted by the user. This is a desirable property for some critical parts of the system.

-   *Adaptability*. An adaptive fact is able to adjust to the user's needs or to its context dependent on the context information. The main difference to customizability is that an adaptive fact functions without the explicit input of the user.

## Examples

The entire model is composed of the activities with attributes, the facts with the corresponding attributes and the impacts between attributed facts and attributed activities. The model with all these details is too large to be described in detail, but we present some interesting examples: triplets of an attributed fact, an attributed activity and a corresponding impact. These examples aim to demonstrate the structuring that can be achieved by using the quality meta-model as described in Sec. .

#### Consistent Dialogue Management.

A central component in the logical user interface concept proposed in Sec.  is the *dialogue management*. It controls the dynamic exchange of information between the product and the user. In the activities tree, the important activity is carried out by the user by interpreting the information given by the user interface. One attribute of the dialogue management that has an impact on the interpretation is its *internal consistency*. This means that its usage concepts are similar in the entire dialogue management component. The corresponding impact description:

\[Dialogue Management$\,\vert\,$`<!-- -->`{=html}INTERNAL CONSISTENCY\] $\stackrel{-}{\longrightarrow}$ \[Interpretation$\,\vert$ PROB. OF ERROR\]

Obviously, this is still too abstract to be easily assessed. This is the point where company-specific usability models come in. This general relationship needs to be refined for the specific context. For example, menus in a graphical user interface should always open the same way.

#### Guarded Physical Interface.

The usability model does not only contain the logical user interface concept, but also the physical user interface. The *physical interface* refers to all the hardware parts that the user interacts with in order to communicate with the software-based system. One important attribute of such a physical interface is *guardedness*. This means that the parts of the interface must be guarded against unintentional activation. Hence, the guardedness of a physical interface has a positive impact on the *executing* activity:

\[Physical Interface$\,\vert\,$`<!-- -->`{=html}GUARDEDNESS\] $\stackrel{-}{\longrightarrow}$ \[Executing$\,\vert$ PROBABILITY OF ERROR\]

A physical interface that is not often guarded is the touchpad of a notebook computer. Due to its nearness to the location of the hands while typing, the cursor might move unintentionally. Therefore, a usability model of a notebook computer should contain the triplet that describes the impact of whether the touchpad is guarded against unintentional operation or not.

# Case Study: Modeling the ISO 15005

To evaluate our usability modeling approach we refine the high-level model described in Sec.  into a specific model based on the ISO 15005 . This standard describes ergonomic principles for the design of *transport information and control systems* (TICS). Examples for TICS are driver information systems (e. g. navigation systems) and driver assistance systems (e. g. cruise control). In particular, principles related to dialogues are provided, since the design of TICS must take into consideration that a TICS is used in addition to the driving activity itself.

The standard describes three main *principles* which are further subdivided into eight *sub-principles*. Each sub-principle is motivated and consists of a number of *requirements* and/or *recommendations*. For each requirement or recommendation a number of examples are given.

For example, the main principle *suitability for use while driving* is decomposed among others into the sub-principle *simplicity*, i. e. the need to limit the amount of information to the task-dependent minimum. This sub-principle consists, among others, of the recommendation to optimize the driver's mental and physical effort. All in all the standard consists of 13 requirements, 16 recommendations, and 80 examples.

## Approach

We follow two goals when applying our method to the standard: First, we want to prove that our high-level usability model can be refined to model such principles. Secondly, we want to discover inconsistencies, ill-structuredness, and implicitness of important information.

Our approach models every element of the standard (e. g. high-level principles, requirements, etc.) by refinement of the high-level model. For this, the meta-model elements (e. g. facts, attributes, impacts, etc.) are used. We develop the specific model by means of the tool described in Sec. . The final specific model consists of 41 facts, 12 activities, 15 attributes, 48 attributed facts, and 51 impacts.

## Examples

To illustrate how the elements of the standard are represented in our specific model, we present the following examples.

#### Representation of Output Data.

An element in the logical user interface concept proposed in Sec.  is the *output data*, i. e. the information sent to the driver. A central aspect is the representation of the data. One attribute of the representation that has an impact on the interpretation of the state of the system is its *unambiguousness*, i. e. that the representation is precise and clear. This is especially important so that the driver can identify the exact priority of the data. For example, warning messages are represented in a way that they are clearly distinguishable from status messages.

\[Output Data$\,\vert\,$`<!-- -->`{=html}UNAMBIGUOUSNESS\] $\stackrel{-}{\longrightarrow}$ \[Interpretation$\,\vert$ PROBABILITY OF ERROR\]

Another attribute of the representation that has an impact on the interpretation is the *internal consistency*. If the representations of the output data follow the same rules and logic, it is easier for the driver to create a mental model of the system. The ease of creating a mental model has a strong impact on the ease of interpreting the state of the system:

\[Output Data$\,\vert\,$`<!-- -->`{=html}INTERNAL CONSISTENCY\] $\stackrel{-}{\longrightarrow}$ \[Interpretation$\,\vert$ DURATION\]

One attribute of the representation that has an impact on the perception is *simplicity*. It is important for the representation to be simple, since this makes it easier for the driver to perceive the information:

\[Output Data$\,\vert\,$`<!-- -->`{=html}SIMPLICITY\] $\stackrel{-}{\longrightarrow}$ \[Perception$\,\vert$ COGNITIVE LOAD\]

#### Guarded Feature.

A TICS consists of several features which must not be used while driving the vehicle. This is determined by the manufacturer as well as by regulations. One important attribute of such features is its *guardedness*. This means that the feature is inoperable while the vehicle is moving. This protects the driver from becoming distracted while using the feature. The guardedness of certain features has a positive impact on the *driving* activity:

\[Television$\,\vert\,$`<!-- -->`{=html}GUARDEDNESS\] $\stackrel{-}{\longrightarrow}$ \[Driving$\,\vert$ PROBABILITY OF ERROR\]

## Observations & Improvements

As a result of the meta-model-based analysis, we found the following inconsistencies and omissions:

#### Inconsistent Main Principles.

One of the three main principles, namely *suitability for the driver*, does not describe any activity. The other two principles use the activities to define the high-level usability goals of the system. For example, one important high-level goal is that the TICS dialogues do not interfere with the driving activity. Hence, we suggest that every main principle should describe an activity and the high-level goals of usability should be defined by means of the attributes of the user's activities.

#### Mixed Sub-Principles.

The aspects described by the sub-principles are mixed: Three sub-principles describe activities without impacts, three describe facts without impacts, and the remaining two describe impacts of attributes on activities. This mix-up of the aspects described by the sub-principles must be resolved.

We believe that in order to make a design decision it is crucial for the software engineer to know which high-level goals will be influenced by it. Sub-principles which only describe attributes of system entities do not contribute toward design decisions. The same holds true for sub-principles which only describe activities, since they are not related to system entities. For this reason we suggest that all sub-principles that only describe activities should be situated at the main principle level, while those sub-principles that describe software entities should be situated at the requirement level.

#### Requirements with Implicit Impacts.

9 out of 13 requirements do not explicitly describe impacts on activities. Requirements serve to define the properties which the system entities should fulfill. If a requirement does not explicitly describe its impacts on activities, the impact could be misunderstood by the software engineer. Hence, we suggest that requirements should be described by attributed facts and their impacts on activities.

#### Incomplete Examples.

14 out of 80 examples only describe facts and their attributes, leaving the impacts and activities implicit. To provide complete examples we suggest that the examples should be described with explicit impacts and activities.

# Discussion

The usability model acts as a central knowledge base for the usability-related relationships in the product and process. It documents in a structured manner how the properties of the system, team, and organization influence different usage activities. Therefore, it is a well-suited basis for quality assurance (QA). It can be used in several ways for constructive as well as analytical QA. Some of these have been shown to be useful in an industrial context w.r.t. maintainability models.

#### Constructive QA.

The knowledge documented in the quality model aids all developers and designers in acquiring a common understanding of the domain, techniques, and influences. This common understanding helps to avoid misunderstandings, and improvements to the quality model become part of a continuous learning process for all developers. For example, by describing the properties of the system artifacts, a glossary or terminology is built and can be easily generated into a document. This glossary is a living artifact of the development process, not only because it is a materiality itself, but also because it is inside and part of a structured model. Hence, by learning and improving the way developers work, it is possible to avoid the introduction of usability defects into the product.

#### Analytical QA.

The identified relationships in the usability model can also be used for analytical QA. With our quality model we aim to break down the properties and attributes to a level where we can measure them and, therefore, are easily able to give concrete instructions in analytical QA. In particular, we are able to generate guidelines and checklists for reviews from the model. The properties and attributes are there and subsets can easily be selected and exported in different formats so that developers and reviewers always have the appropriate guidelines at hand. Moreover, we annotate the attributed properties in the model, whether they are automatically, semi-automatically, or only manually assessable. Hence, we can identify quality aspects that can be analyzed automatically straightforwardly. Thus, we are able to use all potential benefits of automation.

#### Analyses and Predictions.

Finally, more general analysis and predictions are possible based on the quality model. One reason to organize the properties and activities in a tree structure is to be able to aggregate analysis to higher levels. This is important to get concise information about the quality of the system. To be able to do this, the impacts of properties on activities must be quantified. For example, the usability model is a suitable basis for cost/benefit analysis because the identified relationships can be quantified and set into relation to costs similar to the model in . In summary, we are able to aid analytical QA in several ways by utilizing the knowledge coded into the model.

# Conclusion

Usability is a key criteria in the quality of software systems, especially for its user. It can be decisive for its success on the market. However, the notion of usability and its measurement and analysis are still not fully understood. Although there have been interesting advances by consolidated models, e.g. , these models suffer from various shortcomings, such as inconsistencies in the dimensions used. An approach based on an explicit meta-model has proven to be useful for the quality attribute *maintainability*. Hence, we propose a comprehensive usability model that is based on the same meta-model.

Using the meta-model and constructing such a usability model allows us to describe completely the usability of a system by its facts and their relationship with (or impact on) the activities of the user. We support the consistent and unambiguous compilation of the usability knowledge available. The general model still needs to be refined for specific contexts that cannot be included in a general model. By utilizing a specific usability model, we have several benefits, such as the ability to generate guidelines and glossaries or to derive analyses and predictions.

The usefulness of this approach is demonstrated by a case study in which an ISO standard is modeled and several omissions are identified. For example, the standard contains three sub-principles which describe activities, but no impacts on them, as well as nine requirements that have no described impacts. This hampers the justification of the guideline: A rule that is not explicitly justified will not be followed.

For future work we plan to improve further the general usability model and to carry out more case studies in order to validate further the findings of our current research. Furthermore, other quality attributes, e. g. *reliability*, will also be modeled by means of the meta-model to investigate whether this approach works for all attributes. If this be the case, the different models can be combined, since they are all based on a common meta-model.

Springer-Verlag. The final publication is available at
http://link.springer.com/chapter/10.1007/978-3-540-92698-6_7.

[^1]: <http://www.eclipse.org>