abstract: During the rapid development cycle for Internet products (websites and mobile apps), new features are developed and rolled out to users constantly. Features with code defects or design flaws can cause outages and significant degradation of user experience. The traditional method of code review and change management can be time-consuming and error-prone. In order to make the feature rollout process safe and fast, this paper proposes a methodology for rolling out features in an automated way using an adaptive experimental design. Under this framework, a feature is gradually ramped up from a small proportion of users to a larger population based on real-time evaluation of the performance of important metrics. If there are any regression detected during the ramp-up step, the ramp-up process stops and the feature developer is alerted. There are two main algorithm components powering this framework: 1) a continuous monitoring algorithm - using a variant of the sequential probability ratio test (SPRT) to monitor the feature performance metrics and alert feature developers when a metric degradation is detected, 2) an automated ramp-up algorithm - deciding when and how to ramp up to the next stage with larger sample size. This paper presents one monitoring algorithm and three ramping up algorithms including time-based, power-based, and risk-based (a Bayesian approach) schedules. These algorithms are evaluated and compared on both simulated data and real data. There are three benefits provided by this framework for feature rollout: 1) for defective features, it can detect the regression early and reduce negative effect, 2) for healthy features, it rolls out the feature quickly, 3) it reduces the need for manual intervention via the automation of the feature rollout process.
author:
bibliography: ./IEEEabrv.bib; ./IEEEexample.bib
title: Safely and Quickly Deploying New Features with a Staged Rollout Framework Using Sequential Test and Adaptive Experimental Design

# Introduction

During web and mobile software product development cycles, innovations and improvements are continuously made to the products by developing and rolling out new features. Integration of features to the mobile app or digital website requires changes to the code base on the client or server sides. In a complex world of micro-services and intra-service dependencies, it's very hard to ensure reliability in a manual way. While new features are designed and tested cautiously before being released to end users, it's inevitable that a small fraction of these features will have bugs in the code or flaws in their design. Such problematic features will degrade the user experience (e.g. causing application crash, slowing down the performance, draining mobile battery, blocking users from requesting a trip, etc.) when they are released.

Improper ways to release a new feature include (1) making the feature available to all users at once; (2) releasing a feature without a proper monitoring and alerting system in place. Due to the uncertainty of impact embedded in the new features, releasing the feature through an unguided process can be risky:

-   The negative impact may go unnoticed for a long time resulting in a bad customer experience.

-   The problematic feature may impact a large number of users, even if it is reverted timely.

-   When a metric degradation is detected, it is hard to attribute the degradation to a specific feature and fix a bug.

Online controlled experimentation, also known as A/B testing, is one way to evaluate a feature's impact before formal release . In a standard A/B test, users are split into treatment group (with the new feature turned on) and control group (with the new feature turned off), and the analysis is performed on the user engagement and business metrics when the experiment concludes. While experimentation can help detect potential flaws in the feature before its formal release, it is worth noting that such feature release process can still be risky post experimentation. If the experiment was run on certain population (let's say in couple of cities) and then rolled out to a population that were not tested against (rolled out to all cities) then the post-experiment release can be risky since it may affect an under-represented population in the experiment. Thus, even with the presence of an A/B test, a proper rollout process is needed for the post experimentation feature rollout.

A reliable feature release process can systematically reduce the negative impact of problematic feature updates. A reliable feature release process has following desirable properties: 1) early detection of regression, 2) reverting feature rollout if regression is detected, 3) reducing the impact scope in terms of user size, 4) rolling out flawless features at a fast speed, and 5) requiring minimum human attention and intervention in the process.

Based on the criteria mentioned above, this paper proposes a framework for rolling out features autonomously using an adaptive experimental design. The rollout starts with a small proportion of users getting exposed to this feature, gradually ramps up to a larger user population, and eventually rolls out to the entire target population. This rollout framework is composed of two main algorithmic components making the process both intelligent and autonomous. First, a continuous monitoring algorithm based on sequential test that detects regressions early so that the team can be alerted and the feature can be reverted in a timely manner. Second, an adaptive ramp-up algorithm determining when and how much to ramp up to based on required sample size to get enough power at the end stage. This proposed framework satisfies the five properties for a safe feature rollout process. It enables engineers to quickly evaluate and roll out new features with a controlled amount of risk, without too much manual work and intervention during the process. As a result, this framework powers the development and rollout of mobile and web software products.

The contribution of this paper includes:

-   Proposing a staged rollout framework for releasing new features autonomously by using a continuous monitoring algorithm and a ramp-up algorithm.

-   Introducing a scalable nonparametric variance estimator to be used with the sequential test to adjust the sample correlation when the observation is not independent.

-   Providing formulas for estimating power and required sample size for the sequential test.

-   Proposing and compared three ramp-up algorithms: time-based, power-based, and risk-based.

-   Evaluating the empirical performance of the staged rollout framework through examples.

This paper is organized as follows: Section  introduces the background and terminology of this work, as well as its relation to standard A/B testing; Section   presents the concept of the staged rollout framework for feature release; Section   describes the monitoring algorithm: sequential probability ratio test with nonparametric variance estimation; Section   introduces three ramp-up algorithms: the time-based ramp-up, the power-based ramp-up based on statistical power of sequential test, and the risk-based ramp-up algorithm based on Bayesian methods; Section   evaluates how this rollout framework works in practice through both real data and synthetic data examples; and Section   summarize the work and discuss practical considerations.

# Background

## Terminology

In this section, the terminology is defined and introduced for this paper. Some of the terminologies are commonly known in the software development world, and some of them have specific meanings and scopes in this paper.

-   **Feature** A new feature is defined as any change to the product that involves a code or configuaration change. The change can happen either in the client codebase, or in the backend codebase on the server. A feature can be as large as an app redesign, and as small as a few lines of code or configuaration change that are not visible to users.

-   **Feature Flag** Feature flags are a gating system to control code flow in the product. A feature flag can turn on and off a block of code (as a feature). In other words, a feature flag can be used to control if a user can see a new feature or not. For example, turning on a feature is controlled by setting the feature flag value to True, while a feature can be turned off by setting the value to False.

-   **Rollout** The process of exposing a new feature to the user population is defined as a rollout. The rollout starts with the state of no users getting exposed to the new feature and ends with the feature turned on for all users.

-   **Staged Rollout** The staged rollout framework proposed in this paper breaks the feature rollout process into different stages, each stage with a different proportion of the user population getting exposed to the feature. In this framework, a feature is gradually rolled out to the user population from smaller sample sizes to larger sample sizes.

-   **Revert** Reverting a feature means turning off the feature for all users. When the new feature is turned off, a default experience (often the legacy version of the feature) is served to the user.

-   **Ramp-up** Ramping up a feature means turning on the feature for more users. In the staged rollout framework, each subsequent stage has a larger sample size (in proportion) than the previous stage, thus moving forward one stage is equivalent to ramping up the feature to more users.

-   **Regression** A regression is a situation where a feature stops functioning as intended or performs in a suboptimal manner. A regression can cause user experience degradation. A regression can be caused by a new feature that contains a code bug or design flaw.

-   **Outage** An outage, also known as downtime, is a serious regression situation that the core function or the entire app stops functioning. A problematic feature can cause an outage.

## Relation to Standard A/B Testing

```latex
\begin{table*}
\caption {Difference between Staged Rollout and Standard A/B Testing}
\label{tab:difference_experiment}
{\small
\begin{tabular}{ | l | l | l | }
\hline
& {\bf Staged Rollout} & {\bf Standard A/B Testing} \\
\hline
Main Purpose & Automated Feature Release Process & Feature Evaluation Process \\
\hline
Question Trying to Answer  & If the feature is causing a regression & If the feature is successful \\
\hline
Experiment Design and Configuration  & Multiple Adaptive Stages  & One Fixed Stage \\
\hline
Metrics  & Core user engagement and software performance  & Feature related user engagement \\
\hline
Statistical Test  & Sequential Test  & Fixed Horizon Test \\
\hline
Analysis Frequency  & Continuous Monitoring  & One-Time Fixed Horizon \\
\hline
\end{tabular}
}
\end{table*}
```

There are similarities between the staged rollouts and standard A/B testing. In both cases, there is a randomized control group to compare with the treatment group. A large portion of the infrastructure can also be shared between these two frameworks. However, there are major differences between them two, and the standard A/B testing cannot solve the problem that the staged rollouts are focused on. See Table  for a list of difference between staged rollouts and standard A/B testing. In practice, the staged rollout framework can be used by itself to roll out a feature, but it can also be used within an A/B testing to monitor and ramp up the experiment.

# Staged Rollout Framework Overview

The design for the staged rollout framework comes with two main components.

First, the feature rollout process is staged such that the feature is gradually ramped up. The initial stage comes with a relatively small sample size (e.g. $1\%$ users in treatment), while the last stage consists of a large sample size reflecting the final rollout goal, usually $100\%$ of users in the treatment. The stages are designed to be increasing in sample size. The number of stages and the sample size at each stage can be determined by either manual inputs or algorithms (to be introduced in following sections).

Second, the feature rollout process is monitored and controlled by statistical algorithms to evaluate the feature impact and infer the optimal rollout actions: revert, ramp up, or stay on the current stage. There are two main statistical algorithms supporting the rollout decision-making: a continuous monitoring algorithm and an adaptive ramp-up algorithm. The monitoring algorithm continuously monitors the metrics for the rollout via a sequential test (Section ). If there is no significant difference detected in the metrics between treatment and control, the rollout process will continue. On the other hand, if there is a significant difference detected, an alert with the metric signal will be sent to the feature developers and the rollout can be reverted by the developer or the system. In addition to the monitoring algorithm for deciding when to revert a rollout, there are three methods to decide when to ramp up to the next stage introduced in Section . The time-base ramp-up schedule is the most basic but transparent approach. The power-based rampup aims to achieve sufficient statistical power at a pre-set sensitivity level (minimum detectable difference) within a given time frame. The risk-based ramp-up algorithm sets the rollout speed and scope according to the signals collected and risk tolerance based on a Bayesian framework.

Table shows an example scenario for a staged rollout. Each staged rollout is carried out for one feature, such that if a regression is detected, the feature development team can associate the impact back to the feature. But multiple features can be rolled out simultaneously in parallel, each in a separate rollout experiment, using the multi-layering system . Upon setting up the rollout, the feature engineer specifies the target population for the rollout (e.g. all active users in US using the new app version 2.0 on Android), which is defined as all users. In this example, the feature is gradually rolled out through five stages, and each stage takes one day. The control group users get the default experience with the feature turned off, and the treatment group users get the new experience with the feature turned on. The monitoring will be performed on the metric data collected from these two groups. The untreated group get the same experience as the control group, but the data are not used for analysis, and these users are treated as outside of the experiment. The reason for keeping equal sample sizes between the control group and the treatment group is to avoid Simpson's Paradox . During the rollout process, key user engagement and app health metrics are monitored by the sequential test to be introduced in the following section. The metrics can be defined on different analytics unit levels: trip, session, or user. And the metrics can be proportional metrics, continuous metric, and ratio metrics. For simplicity, the session level proportion metrics are used as examples.

```latex
\begin{table*}
\centering
\caption {Example Staged Rollout Template}
\label{tab:staged_rollout_template}
{\small
\begin{tabular}{ | p{4cm} | p{1.2cm} | p{1.2cm} | p{1.2cm} | p{1.2cm} | p{1.2cm} | }
% * <firstname.lastname@example.com> 2018-07-31T01:35:40.204Z:
%
% > { | p{4cm} | p{1.2cm} | p{1.2cm} | p{1.2cm} | p{1.2cm} | p{1.2cm} | }
%
% ^.
\hline
Feature Flag & \multicolumn{5}{|l|}{Feature X} \\
\hline
Target Population & \multicolumn{5}{|l|}{Users in US using App Version 2.0 on Android} \\
\hline
Stages & 1 & 2 & 3 & 4 & 5 \\
\hline
Time & Day 1 & Day 2 & Day 3 & Day 4 & Day 5 \\
\hline
Control Sample Size & $1\%$ & $5\%$ & $20\%$ & $50\%$ & $0\%$ \\
\hline
Treatment Sample Size & $1\%$ & $5\%$ & $20\%$ & $50\%$ & $100\%$ \\
\hline
Untreated Sample Size & $98\%$ & $90\%$ & $60\%$ & $0\%$ & $0\%$ \\
\hline
Metrics  & \multicolumn{5}{|l|}{$\%$ Sessions with Login Success, $\%$ Sessions with Orders,} \\
& \multicolumn{5}{|l|}{$\%$ Sessions with App Crash, etc.} \\
\hline
\end{tabular}
}
\end{table*}
```

# Monitoring Algorithm - Sequential Test

In this section, a variant of the sequential test is introduced, called a mixture sequential probability ratio test (mSPRT), used as the core monitoring algorithm. In addition, a nonparametric variance estimation method (delete-a-group jackknife) is introduced to correct the correlation for the sequential algorithm. The statistical power and sample size estimation is derived for the sequential test as well.

## Mixture Sequential Probability Ratio Test (mSPRT)

Sequential probability ratio test is widely used in clinical research, where scientists often allow sample size dependent decisions to be made based on likelihood ratio of two hypothesis. The mSPRT introduced in and applies to A/B testing to enable multiple tests without inflating the false positive rate (FPR). In an A/B testing setting, assume the control variables $\{X^{(i)}_{ctrl} \}_{i=1}^{n_{ctrl}}$ are independent random variables from a distribution with a density function $f(x|\mu_{ctrl}, \sigma_{ctrl})$ where $\mu_{ctrl}$ and $\sigma_{ctrl}$ represent mean and standard deviation. Similarly, the distribution density function for the treatment variables $\{X^{(i)}_{trt} \}_{i=1}^{n_{trt}}$ is $f(x|\mu_{trt}, \sigma_{trt})$. The hypothesis to be tested is on the difference in distribution mean: $$\begin{aligned}
H_{0}: & \delta := \mu_{trt} - \mu_{ctrl} = \delta_{0} \\
H_{1}: & \delta := \mu_{trt} - \mu_{ctrl} \neq \delta_{0}
\end{aligned}$$ where $\delta$ represents the difference in mean between treatment and control, and $\delta_0$ is the difference value under null hypothesis ($\delta_0=0$ for testing if two groups have the same metric mean). The test statistic used in mSPRT is a likelihood ratio integrated over a prior distribution of $\delta$ values under the alternative hypothesis. Denote the prior density function as $h(\delta)$, and for simplicity, a normal prior is chosen $h(\delta) \sim N(\delta_{0}, \tau)$ in this paper. It can be proven that the integrated likelihood ratio statistic is a Martingale under alternative hypothesis. The $(1-\alpha)$ confidence interval for $\delta$ is derived by as $$\begin{aligned}
\bar{x}_{trt} - \bar{x}_{ctrl} \pm \sqrt{
\frac{V(V + \tau)}{\tau}(-2log(\alpha) - log(\frac{V}{V+\tau})}
\end{aligned}$$ with $\bar{x}_{trt}$ and $\bar{x}_{ctrl}$ as sample means and V as the variance of sample mean difference (e.g. $V = \frac{v_{ctrl}}{n_{ctrl}} + \frac{v_{trt}}{n_{trt}}$ where $v_{ctrl}$ and $v_{trt}$ are the sample variance estimates for control and treatment).

## Variance Estimation

One assumption made by the sequential testing is the independence of each observation. However, this assumption often does not hold in practice. For example, if the click through rate is the metric of interest, it is improper to assume each impression is independent, since the same user can use the product multiple times on different days, and multiple impressions and clicks can be generated by the same user. Such observations are correlated since they are generated by the same user. Violating the independence assumption in the sequential test can produce an inflated false positive rate. Embedding a variance estimation with correlation correction is one way to generalize the mSPRT to correlated data.

Several previous papers discuss the adjustment of metric variance in A/B testing. The delta method and Bootstrap method are two variance estimation approaches that can be applied to correct the variance without the assumption of independence. These two methods work well in theory, however, they require storing raw data (e.g. all mobile event level data) in order to perform analysis. Therefore, they are hard to scale when the data storage is limited. In addition, processing the raw event level data adds computation latency which is a concern for monitoring all feature rollouts in real time. Therefore, a fast and scalable variance method is preferred. In our system, a version of the delete-a-group jackknife variance estimation method is implemented, which meets such requirements.

To implement the delete-a-group jackknife method, the users (user ID or device ID, which is the experiment unit) are split into $R$ partitions with equal probability using a hash function within each experiment group. The hash function (, ) takes the user ID as input and outputs an integer as the partition ID $\in \{1,2,...,R \}$. Note that the hash function should be different (by choosing a different random seed or algorithm) from the one used for experiment randomization on splitting users into control and treatment groups, so that the user partition is independent to the experiment randomization.

The delete-a-group jackknife variance estimator can be expressed as $$\begin{aligned}
Var(\bar{X}) = \frac{R-1}{R} \sum_{r=1}^R (\bar{X}_{(r)} - \bar{X})^{2}
\end{aligned}$$

with $$\begin{aligned}
\bar{X}_{(r)} =  \frac{1}{\sum_{i = 1}^n I(hash(uuid_i) \neq r)} \sum_{\{i: hash(uuid_i) \neq r\}} X_i
\end{aligned}$$ where $\bar{X}_{(r)}$ is the metric mean for all users except partition $r$, $I(\cdot)$ is an indicator function with value $1$ if the argument condition is true and value $0$ otherwise, $hash(\cdot)$ is a hash function, and $uuid_i$ is the user ID for the $i$th observation.

The delete-a-group jackknife variance estimation is scalable since only the partition level metrics are needed and stored for calculation, instead of the raw event-level data. Taking the CTR metric as an example, at each stage, the system will calculate the total number of impressions and clicks by partition, and then the cumulative (from the initial stage up to the current stage) CTR (as $\bar{X}_{(r)}$) can be calculated by dividing the sum of clicks by the sum of impressions across stages except partition $r$. The overall cumulative CTR (as $\bar{X}$) can be calculated by dividing the sum of clicks by the sum of impressions, where both sums are across stages and partitions.

The empirical performance of different variance estimation methods are compared with real data reflecting the correlation structure in practice. Four methods are compared: sequential test (assuming observation independence), sequential test with bootstrap, sequential test with delta method, and sequential test with (delete-a-group) jackknife (with $R=10$). A sample dataset is prepared for three metrics in one-week time period. The false positive rate in Figure is calculated by simulating an A/A test scenario: randomly assign the sample data into control and treatment groups by hashing the user ID. The power in Figure is calculated by simulating an A/B test scenario: using the same experiment assignment logic as an A/A test, but artificially adding a $5\%$ metric increase in the treatment group. The data is fed into the algorithm at hourly level, and a positive result is defined if the test returns at least one significance (at $95 \%$ level) out of the $24 \times 7$ tests. The results show when the independence assumption does not hold, it leads to an inflated false positive rate for the original sequential test. All the three improved variance estimation methods successfully control the false positive rate below $5 \%$, while having a similar level of power. The jackknife method is preferred considering the scalability, and the performance can be improved by increasing the number of partitions $R$. By default, the jackknife method is applied for all examples related to real data in this paper.

## Power and Sample Size Estimation

The statistical power of the the sequential test is needed for evaluating when it is ready to fully ramp up and conclude the rollout. At the beginning of the rollout, the rollout developer will set a sensitivity level for metric degradation detection. For example, the developer can set the system to raise an alert if the true metric difference is greater than $2\%$. In this paper, this minimum detectable difference is defined as the MDE (minimum detectable effect). Then the criteria for the feature to be formally rolled out is to achieve enough statistical power to detect the MDE. If the power is achieved, and there is no regression detected, then system will believe this feature is not causing a regression greater than the MDE with high confidence. In order to make the power-based criteria work, an estimation of power and sample size is needed during the rollout process.

To derive the power and sample size estimation formula, define the stopping time $N_{\delta}$ as the smallest sample size that the sequential test becomes significant given the true difference as $\delta$. Note $N_{\delta}$ is a random variable. The power of the sequential test for a finite sample size can be expressed as the probability of the sequential process becoming significant before the given sample size: $$\begin{aligned}
Pr(N_{\delta} < N_{{\delta}_{1-\beta}} | {\delta}) = 1 - \beta
\end{aligned}$$ where $1 - \beta$ indicates the power and $N_{{\delta}_{1 - \beta}}$ denotes the sample size threshold for achieving the power $1 - \beta$ given $\delta$.

Following the intuition that $N_{{\delta}_{1 - \beta}}$ is the $1 - \beta$ percentile of the distribution for $N_{\delta}$, it can be approximated by a linear combination of the distribution mean and standard deviation: $$\begin{aligned}
N_{{\delta}_{1 - \beta}} \approx E[N_{\delta}] + f(\beta) * \sqrt{(Var(N_{\delta}))}
\end{aligned}$$

A previous Monte Carlo study found that the variance of the sample size needed to detect a given MDE for sequential test is approximately proportional to the square of the average sample size: $Var(N_{\delta}) \propto (E[N_{\delta}])^2$.

In addition, the average sample size can be approximated by $E(N_{\delta}) \approx \frac{v_{x} + v_{y}}{{\delta}^{2}}\{log(-2 log \alpha) - 2 log \alpha \}$ by taking $\tau = {\delta}^{2}$ (see Appendix).

To estimate the coefficient $f(\beta)$, an empirical power curve is fitted in a simulation study that yields the following approximation for the sample size required to achieve a given statistical power: $$\begin{aligned}
N_{{\delta}_{1-\beta}} &\approx & E(N_{\delta}) + f(\beta) E(N_{\delta}) \\
&\approx& (0.35 - 0.79 log\beta)(\frac{v_{x} + v_{y}}{{\delta}^{2}}) \\
&& \times (log(-2 log \alpha) - 2 log \alpha)
\end{aligned}$$

Given this formula, the power can be estimated by solving the formula for $\beta$ for a given sample size $N_{{\delta}_{1-\beta}}$. This power estimation works well empirically for sample size larger than $500$. Figure  shows the estimated power compared with the actual power.

# Ramp-up Algorithms

During the rollout, when the monitoring algorithm raises an alert, the rollout will be paused or reverted for investigation. However, if the monitoring algorithm does not raise an alert, it does not mean the feature is guaranteed to be flawless, but it could be the case that the sample size is not sufficient for the monitoring algorithm to catch the regression at certain levels of sensitivity. With the uncertainty in the feature performance, rolling out too fast could expose more than necessary users to the feature. Therefore, a separate ramp-up algorithm is needed to decide how to increase the rollout percentage from one stage to the next stage. Next, three approaches for ramping up the rollout are introduced.

## Time-based Ramp-up

The time-based ramp-up schedule requires engineers to decide the stages and corresponding sample size percentage before the rollout begins. In addition, the time between each step is specified. After the rollout starts, if there is no alert made by the monitoring algorithm for the given time length, the rollout will ramp up to the next stage. In the example shown in Table , if the rollout is set to be ramping up every day, then it will be fully rolled out in five days if there is regression captured. The advantage of the time-based ramp up is to give the developer more control on the rollout. However, since this ramp-up schedule does not adapt to the information collected during the rollout, it may suffer in 1) ramping up too quickly when the risk and uncertainty is high or 2) ramping up too slow when the data shows high level of certainty.

## Power-based Ramp-up

The power-based ramp-up schedule aims to achieve sufficient statistical power for detecting the pre-defined MDE $\delta_{MDE}$ within a given time frame. For example, if the feature developer wants to roll out a feature with $80\%$ power to detect potential $2\%$ metric difference within one week, then the power-based ramp-up schedule will adjust the sample size adaptively to achieve the rollout goal. At each stage, if the observed difference $\hat{\delta}$ is less than pre-defined MDE $\delta_{MDE}$, then the sample size required for the MDE is calculated. Otherwise, if the observed difference is bigger than MDE, then the sample size required for detecting the current difference is calculated. This way, the system makes sure the ramp-up does not put too many users under risk if the observed signal is negative. In addition, a maximum ramp-up proportion threshold can be put for each stage, so the ramp-up speed is not too aggressive. In sum, the sample size recommended by the power-based algorithm is $$min(N_{\delta_{MDE,1-\beta}}, N_{\hat{\delta}_{1-\beta}}, N_{stage\_limit})$$ where $N_{stage\_limit}$ is the maximum sample size allowed at a given stage, $N_{\delta_{1-\beta}}$ is the sample size required for the MDE $\delta$, and $N_{\hat{\delta}_{1-\beta}}$ is the sample size corresponding to the observed difference $\hat{\delta}$. The recommended sample size can then be transformed into sample size percentage by taking account of the predicted total sample size for the next stage.

## Risk-based Ramp-up

Risk-based ramp-up algorithm proceeds the rollout at the maximum allowed speed on a tolerable risk level. For example, if the risk criteria is set as "avoiding loosing trips from more than 1000 users". Then at any given stage, the system can calculate what the maximum number of extra users that could be exposed given the current experiment signals.

To state this idea in formal formulation, the potential risk of uncertain metric degradation is quantified and controlled by a probability threshold. Without loss of generality, assume the metric of interest is a proportion metric, e.g. $\%$ users with an order (the continuous metric scenario can be derived in a similar approach). The negative impact of the feature can be quantified as number of treatment users not making an order because of the feature: $N_{trt}(\mu_{trt} - \mu_{ctrl}) = N_{trt} \delta$. The risk tolerance during the rollout can be defined as: $$\begin{aligned}
\label{risk_formula}
Pr(N_{trt} \delta \leq -C | D) \leq R
\end{aligned}$$ where

-   $C$ is a positive constant, as the pre-specified tolerable cost threshold. One choice can be $C =
    \delta_{MDE} * N_{\delta_{MDE,1-\beta}}$, that allows enough cost budget to yield sufficient statistical power.

-   $R$ is the pre-specified risk probability threshold. $R$ is a parameter to trade off the rollout safety and speed.

-   $D$ indicates data, which means the probability specified is based on a posterior distribution given the data.

Note that the formula above is an example of one-sided test assuming only potential decrease in the metric is a risk to be controlled. This framework can be easily generalized to one-sided increase risk and two-sided risk specification. Without loss of generality, the one-sided decrease risk is used in the following derivation for simplicity.

In order to solve the inequality (), the posterior distribution $p(\delta | D)$ can be derived as follows. For simplicity, assuming control and treatment have equal sample size and equal variance. By the central limit theorem, assume the sample mean follows a normal distribution $\bar{x}_{ctrl} \sim  N(\mu_{ctrl}, \sigma^2/n)$ and $\bar{x}_{trt} \sim  N(\mu_{trt}, \sigma^2/n)$, where $\mu_{ctrl}$ and $\mu_{trt}$ are the distribution means, $\sigma^2$ is the common variance, and $n$ is the sample size. In addition, assume the prior distributions for the mean parameter $\mu_{ctrl}$ and $\mu_{trt}$ are $q(\mu_{ctrl}) \sim  N(\mu_0, \sigma_0^2)$ and $q(\mu_{trt}) \sim  N(\mu_0+\delta_0, \sigma_0^2)$, where $\mu_0$, $\sigma_0^2$, and $\delta_0$ are prior distribution parameters, indicating prior control mean, prior common variance, and prior difference parameter. Under these assumptions, the posterior distribution can be derived as a normal distribution (see ): $$\begin{aligned}
p(\delta | D) \sim N(m_{\delta}, s^2_{\delta})
\end{aligned}$$ where $$\begin{aligned}
m_{\delta} &=& \frac{1/\sigma_0^2}{n/\sigma^2 + 1/\sigma_0^2} \delta_0 + \frac{n/\sigma^2}{n/\sigma^2 + 1/\sigma_0^2} (\bar{x}_{trt} - \bar{x}_{ctrl}) \\
s^2_{\delta} &=& \frac{2 \sigma^2 \sigma_0^2}{\sigma^2 + n\sigma_0^2}
\end{aligned}$$ Note that the only unknown parameter in this posterior distribution is the metric variance $\sigma^2$. In practice, this parameter can be estimated and substituted by the pooled sample variance.

At stage $t$, the observed cumulative sample size for treatment is $\sum_{s=1}^t n_{trt, s}$. The risk-based ramp-up algorithm aims to decide an appropriate sample size for next stage $n^*_{trt, t+1}$ that controls the potential risk. The risk inequality () can be written as $$\begin{aligned}
& Pr(N_{trt} \delta \leq -C | D) \\
= & Pr((\sum_{s=1}^t n_{trt, s} + n^*_{trt, t+1})  \delta \leq -C | D) \\
= & Pr(\delta \leq \frac{-C}{(\sum_{s=1}^t n_{trt, s} + n^*_{trt, t+1})} |D) \\
\leq & R
\end{aligned}$$

The left side of the inequality becomes the cumulative distribution function of the posterior distribution of $\delta$. The the maximum tolerable sample size $n^*_{trt, t+1}$ can be derived as:

$$\begin{aligned}
\frac{1}{s_{\delta}} \cdot (\frac{-C}{(\sum_{s=1}^t n_{trt, s} + n^*_{trt, t+1}) } - m_{\delta}) = \Phi^{-1}(R) \\
n^*_{trt, t+1} =  \max(\frac{- C}{s_{\delta} \Phi^{-1}(R) + m_{\delta}} -  \sum_{s=1}^t n_{trt, s}, 0)
\end{aligned}$$ The corresponding treatment rollout percentage for the next stage $p_{t+1}$ can be calculated by dividing the treatment rollout sample size $n^*_{trt, t+1}$ by a predicted total population size for the next stage $\hat{n}_{total, t+1}$ as: $$p_{t+1} = \max( \min ( p_t, \frac{n^*_{trt, t+1}}{\hat{n}_{total, t+1}}), 0.5)$$

The $\min$ and $\max$ conditions are added to ensure the rollout percentage is monotonic increasing (unless reverting to $0$), and the maximum rollout percentage that can be achieved by the ramp-up algorithm is $50\%$. The final decision from ramping up from $50\%$ to $100\%$ (in one stage) is determined by power calculation, which is beyond the control of the ramp-up algorithm. The predicted total population size for the next stage $\hat{n}_{total, t+1}$ can be estimated by a generic time series model. The discussion of this prediction model is beyond the scope of this paper.

# Examples

```latex
\begin{table*}
\centering
\caption {Empirical Performance of Ramp-up Algorithms based on Synthetic Data under A/A and A/B Tests (NA is shown when the observed cases are too few to draw a conclusion.)}
\label{tab:staged_rollout_performance_simulated_data}
{\small
\begin{tabular}{ | p{2.5cm} | p{1.5cm} | p{1.5cm} | p{1.5cm} |p{1.5cm} | p{1.5cm} | p{1.5cm} | p{1.5cm} | p{1.5cm} |}
\hline
Metrics & Time-based A/A (Low Speed)& Time-based A/B (Low Speed)& Time-based A/A (High peed)& Time-based A/B (High Speed)& Power-based A/A & Power-based A/B & Risk-based A/A & Risk-based A/B \\
\hline
Positive Rate (FPR for AA, Power for AB) & 0.04& 0.985 & 0.015 & 0.985 & 0.015 & 0.975 & 0.01 & 0.98 \\
\hline
Average Time before Detection (h) & NA & 63 & NA & 55 & NA & 57  & NA & 52 \\
\hline
Average Time before Fully Rollout (h) & 61 & NA & 57 & NA & 52 & NA & 47 & NA \\
\hline
Weighted Average Rollout Pct before Detection & NA & $1\%$,$5\%$, $17\%$,$14\%$ & NA & $1\%$,$10\%$, $20\%$,$16\%$ & NA & $1\%$,$13\%$, $17\%$,$9\%$, $4\%$,$1\%$ & NA & $1\%$,$18.3\%$, $18.5\%$,$10\%$, $3\%$,$1\%$ \\
\hline
Weighted Average Rollout Pct before Fully Rollout & $1\%$,$5\%$, $31\%$,$87\%$ & NA & $1\%$,$10\%$, $39\%$,$91\%$ & NA & $1\%$,$17\%$, $25\%$,$65\%$, $94\%$,$99\%$ & NA & $1\%$,$17\%$, $34\%$,$73\%$ & NA \\
\hline
Average Sample Size Used before Detection & NA & 12085 & NA & 11803 & NA & 11528 & NA & 11857 \\
\hline
Average Sample Size Used before Fully Rollout & 9591 & NA & 10186 & NA & 9876 & NA & 9300 & NA \\
\hline
Avg Total Loss & 21 & 136 & 18 & 137 & 20 & 132 & 18 & 136 \\
\hline
$\%$ of Tests Exceeding the Loss Tolerance $C$ & $0\%$ & $5\%$ & $0\%$  & $9\%$ & $0\%$ & $7\%$ & $0.05\%$ & $10.05\%$ \\
\hline
\end{tabular}
}
\end{table*}
```
```latex
\begin{table*}
\centering
\caption {Empirical Performance of Ramp-up Algorithms based on Real Data under A/A and A/B Tests (NA is shown when the observed cases are too few to draw a conclusion.)}
\label{tab:staged_rollout_performance_real_data}
{\small
\begin{tabular}{ | p{2.5cm} | p{1.5cm} | p{1.5cm} | p{1.5cm} |p{1.5cm} | p{1.5cm} | p{1.5cm} | p{1.5cm} | p{1.5cm} |}
\hline
Metrics & Time-based A/A (Low Speed)& Time-based A/B (Low Speed)& Time-based A/A (High peed)& Time-based A/B (High Speed)& Power-based A/A & Power-based A/B & Risk-based A/A & Risk-based A/B \\
\hline
Positive Rate (FPR for AA, Power for AB) & 0.01 &0.97 & 0.01 & 0.99 & 0.015 & 0.99 & 0.01 & 0.99 \\
\hline
Average Time before Detection (h) & NA &58 & NA &52 & NA &54 & NA &44 \\
\hline
Average Time before Fully Rollout (h) &67 &NA & 63 & NA& 60 &NA & 52 & NA \\
\hline
Weighted Average Rollout Pct before Detection &NA &$1\%$,$5\%$, $20\%$,$25\%$ & NA &$1\%$,$10\%$, $20\%$,$25\%$ & NA & $1\%$,$12\%$, $15\%$,$12\%$, $7\%$,$3\%$, $1\%$  & NA & $1\%$,$18\%$, $19\%$,$7\%$, $1\%$\\
\hline
Weighted Average Rollout Pct before Fully Rollout &$1\%$,$5\%$, $24\%$,$77\%$ &NA &$1\%$,$10\%$, $25\%$,$86\%$ &NA & $1\%$,$16\%$, $20\%$,$56\%$, $87\%$,$97\%$ &  NA & $1\%$,$16\%$, $42\%$,$55\%$, $98\%$ &NA \\
\hline
Average Sample Size Used before Detection &NA &11126 &NA &10981 & NA &9282 & NA & 9438 \\
\hline
Average Sample Size Used before Fully Rollout &13281 &NA &13104 &NA & 13420 &NA & 12420 & NA \\
\hline
Avg Total Loss &30 &135 &27 & 141 & 29 &124 & 20 & 127 \\
\hline
$\%$ of Tests Exceeding the Loss Tolerance $C$ &$0\%$ & $12\%$ & $0\%$ & $22\%$ & $0\%$ & $14\%$ & $0\%$ & $12\%$ \\
\hline
%\hline
\end{tabular}
}
\end{table*}
```

This section evaluates the empirical performance of staged rollout framework with the three ramp-up algorithms on both real data and synthetic data examples.

The real data comes from a feature rollout experiment, which does not cause a regression. One of the key session level metric monitored is a binary metric with mean $0.7$. However, the session level metric can not be regarded as i.i.d. sample from Bernoulli distribution, since the same user can have multiple sessions. The data is collected hourly, and there are $7$ days' data in total. On the other hand, the synthetic data is generated as i.i.d. sample from a Bernoulli distribution with mean $0.7$. During the rollout, the sequential test runs every hour and the rollout is reverted when a significant metric degradation is detected. The criteria for rolling out to all users is to achieve a MDE of $0.05$ at power of $0.9$. If no regression is detected, the system ramps up the rollout to the next stage on the next day. Although the ramp-up frequency is fixed (daily), the sample size for the next stage is determined by the three ramp-up algorithms.

Under this setting, the three ramp-up algorithms are evaluated through both A/A and A/B tests, each with $200$ replications. Each trial of A/A test is generated by randomly splitting the data into treatment and control groups by hashing user ID. The success of an A/A test can be defined as the feature getting fully rolled out in a fast speed. Each trial of A/B test is first generated by the same random split as A/A test, but then an artificial metric difference is introduced to the treatment group. For each metric value in the treatment group, a relative difference is drawn from gamma distribution with shape $= 6.25$ and scale $= 0.008$ (indicating mean as $5\%$ and standard deviation as $2\%$), that is applied to create the artificial difference. The success of an A/B test can be defined as detecting the difference with relatively small sample size.

The Time-based ramp-up algorithm is tested at two speed levels with rollout percentages: ($1\%$, $5\%$, $20\%$, $50\%$) and ($1\%$, $10\%$, $20\%$, $50\%$). The Power-based ramp-up algorithm sets the target MDE as $5\%$. For the risk-based ramp-up algorithms, we set prior parameters as $\delta_0 = 0.05$, prior variance $\sigma_0^2 = 0.0004$, and risk probability threshold $R = 0.1$. Note that a non-zero prior mean is set in order to be conservative about the feature performance in the beginning. In this example, we are controlling the risk in the increasing direction.

Table and Table present the results of different rollout approaches. Overall, all the three algorithms achieve false positive rate below $0.05$ for A/A tests, and power above $0.9$ for A/B tests. The Time-based algorithm, as a simple benchmark approach, seems to take longer time and more samples to make a final correct decision. The Power-based ramp up algorithm gives relatively stable performance. Since this algorithm is set up based on the sequential test used for monitoring, it often uses the smallest sample size to detect the regression. This is reasonable since the power-based algorithm estimates the number of sample size needed for the regression detection, which prevents over-exposing users. The Risk-based algorithm on average achieves the earliest final detection in both A/A and A/B test. It also controls the risk to be around the desired level $R = 10\%$ ($10.05\%$ in simulated data example and $12\%$ in real data example). In contrast, both time-based and power-based ramp-up algorithms do not exhibit consistent risk control performance.

Note that there are parameters that can be tuned for each of the three algorithms, especially the risk-based algorithm have multiple parameters. The results only display the outputs based on specific parameters. The performance order can change with different parameter settings and preference over safety and speed. In practice, such parameters can be determined or tuned through simulation before the actual rollout begins.

# Conclusion

In this paper, a staged rollout framework is presented to automatically release a feature with special attention to optimizing the safety and speed. This framework can be used broadly for code changes to product, both with and without an A/B experiment. In our experiments, the variance corrected sequential test with sample size and power calculation accommodates the practical needs as a continuous monitoring solution. The three ramp-up algorithms represent different perspectives of trade-offs made between safety and speed.

The empirical evaluation of the system shows the sequential monitoring system can control false positive rate effectively while deliver a reasonable power. All three ramp-up algorithms, when configured properly, can make reasonable trade-offs between safety and speed with emphasis on different aspects. This framework has been tested and proven useful in practice for regression detection and ramp-up management to assist developers with rolling out both safely and quickly.

For implementing the ramp-up algorithms in production, a practical recommendation is to start with time-based algorithm, and then power-based algorithm or risk-based algorithm. In practice, the time-based ramp-up algorithm is easy to implement and it also gives more control and transparency to the feature development team. Also compared with a direct rollout, the time-based algorithm can provide sufficient safety for most business case scenarios. If lack of statistical power is a concern, then the power-based algorithm can be further used. Otherwise, if the risk during the rollout period needs to be further controlled, the risk-based algorithm can be utilized.

While the examples of this paper mainly focus on one metric case for illustration, the staged rollout framework can adapt to the case with multiple metrics. The monitoring algorithm can monitor multiple metrics in parallel, and the significance level can be adjusted to control for family-wise false positives. While a large set of metrics can be put under the monitoring system, a smaller set of key metrics can be selected to be used in the ramp-up algorithm. The time-based ramp-up can be extended to multi-metric case naturally. The power-based ramp-up can take the maximum over the recommended sample sizes across different metrics to achieve the power. The risk-based ramp-up can take the minimum over the recommended sample sizes across different metrics to control risk.

Another practical consideration in the rollout system is to monitor the data quality of the rollout. Poor data quality (caused by logging system, outliers, etc.) can lead to inflation of false positives or false negatives . Data quality check and outlier removal is useful in practice for the rollout system. When a red flag is raised on data quality, the rollout platform team and the feature development team need to engage and investigate the root cause. Detailed discussion on the data quality monitoring and diagnosis is out of the scope of this paper.

The power-based ramp-up algorithm is built on the frequentist inference, and the risk-based ramp-up algorithm is based on Bayesian framework. One direction of future development is using reinforcement learning to guide the rollout process as a Markov Decision Process. Instead of focusing on statistical power and potential risk, the reinforcement learning algorithm will make the trade-off between the value of rolling out the new feature versus the cost associated with the rollout. It will eventually optimize the final reward given the value and cost specification of the feature. This algorithm is currently in development.

# Appendix

## Prior Distribution for mSPRT

In Section , the mSPRT test statistic is presented with a prior distribution $h(\delta) \sim N(0, \tau)$. Here we describe a practical choice for parameter $\tau$. Although choice of $\tau$ value does not affect the martingale and the statistical test property in theory, it affects the empirical convergence speed (i.e. sample size needed to achieve significance) in practice. Pollak and Siegmund derived the average sample size needed to detect the real difference $\delta$ under sequential analysis. In our context, the following formula provides a reasonable approximation to the average sample size: $$\begin{aligned}
E[N_{\delta}] \approx \frac{v_{ctrl} + v_{trt}}{{\delta}^{2}}\{log[\frac{-2\tau e^{{\delta}^{2}/{\tau}} log(\alpha)}{{\delta}^{2}\alpha^{2}}]-1\}.
\end{aligned}$$

Taking derivatives yields $\tau = {\delta}^{2}$ to achieve the smallest sample size. However, as the real difference $\delta$ is unknown to experimenters, we want to find a substitute for $\delta$ to provide comparable type I and type II error. Compared with fixed-horizon t test, where we have the sample size $n \propto \frac{v_{ctrl} + v_{trt}}{{\delta}^{2}}$. We want to use $\tau = {\delta}^{2} = A \frac{v_{ctrl} + v_{trt}}{n}.$ Below is a simulation illustrating the choice of $\tau$ to type I and type II error.
A sample data is generated from binomial distribution with p varying from 0.1 to 0.9 and relative difference equal to 0 or 0.05. Along the sequence, 200 checks are conducted in equal interval with each checkpoint using all previous observations. Treatment and control group have equal sample size per each check. A positive result is defined as the test reports at least one significance (at $95 \%$ level) out of the 200 tests, which indicates false positive rate in Figure and power in Figure

From above simulation, we see that using $\bar{\delta}^{2}$ i.e. the square of Observed Difference and using $\tau = (z_{1-\alpha / 2} + z_{1-\beta / 2})^{2} \frac{v_{ctrl} + v_{trt}}{n}$ has similar performance in both empirical power and false positive rate. This is because in fixed-horizon test, the average sample size under Type I error $= \alpha$ and Type II error $=\beta$ has the following relationship $z_{1-\alpha / 2} + z_{1-\alpha / 2} = \frac{\delta}{\sqrt{(v_{ctrl} + v_{trt})/n}}$. In Figure , the empirical power under three parameter choices are plotted against the empirical power under $\tau = {\delta}^{2}$. When the empirical power under $\tau = {\delta}^{2}$ is small, the other three choices all yield relatively bigger power which is because of an overestimate of $\tau$ due to small sample size. This discrepancy dies down as the empirical power of $\tau = {\delta}^{2}$ increases. To balance out the type I and the type II error, we chose $A = z_{1-\alpha/2}^{2}$, yielding $\tau = {\delta}^{2} = z_{1-\alpha / 2}^{2} \frac{v_{ctrl} + v_{trt}}{n}$ in practice.

# Acknowledgment

We would like to extend our thanks to Akash Parikh, Donald Stayner, and Anando Sen for insightful discussion on defining the problem and forming the solution, to Sisil Mehta and Tim Knapik for thoughtful design and implementation of the algorithms in production. Special thanks to Professor Peter Dayan on discussion and formulation of Reinforcement Learning algorithm as future development. We would also like to thank Olivia Liao for her early research and empirical work of applying sequential test on the experimentation platform.


abstract: Convolutional Neural Networks (CNNs) are build specifically for computer vision tasks for which it is known that the input data is a hierarchical structure based on locally correlated elements. The question that naturally arises is what happens with the performance of CNNs if one of the basic properties of the data is removed, e.g. what happens if the image pixels are randomly permuted? Intuitively one expects that the convolutional network performs poorly in these circumstances in contrast to a multilayer perceptron (MLPs) whose classification accuracy should not be affected by the pixel randomization. This work shows that by randomizing image pixels the hierarchical structure of the data is destroyed and long range correlations are introduced which standard CNNs are not able to capture. We show that their classification accuracy is heavily dependent on the class similarities as well as the pixel randomization process. We also indicate that dilated convolutions are able to recover some of the pixel correlations and improve the performance.
author: Cristian Ivan
Romanian Institute of Science and Technology
Cireşilor 29, 400487 Cluj-Napoca, Romania
`email@example.com`
title: Convolutional Neural Networks on Randomized Data

# Introduction

Convolutional Neural Networks are inspired by the visual system of living organisms and are built to exploit the 2D structure of natural images . The receptive field offered by the convolution kernels greatly reduces the number of trainable parameters and increases the performance of these networks as compared to fully connected feed forward networks. CNNs are used not only for visual tasks but also on other kind of data where local correlations are still present.

One question that can be asked is what is the performance of CNNs when trained on images where the individual pixels are randomly permuted? Since the local spatial structure of an image is destroyed one would expect that a convolutional network is not able to find representative features of the data and the accuracy should be very low.

For this study two types of feed-forward networks are used for image classification tasks. CNNs and MLPs are trained on natural images and their pixel-wise permutations. The hyper-parameters of the networks are kept the same throughout the performed experiments and are trained for the same number of epochs. The MLP is used as a baseline and sanity check for the analysis and, due to the known limitations it has when trained on complex image databases, its performance is not intended to be used as a direct comparison to the CNN performance.

The chosen architecture of the CNN follows closely the VGG16 network and the MLP consists of the last fully connected layers of the CNN. The only difference between the CNN used in this paper and the VGGNet is the total number of parameters since they are trained on much smaller images, 28 and 32 pixels per side. Both networks in this study use *adam* optimizer with a learning rate of 0.0001 and a decay of $10^{-6}$. The training procedure does not use any form of data augmentation.

We perform three types of experiments where we compare the classification accuracy of a CNN trained on natural images with the accuracy of a CNN with the same architecture trained on images whose contents are randomized based on three different procedures.

In the first experiment the order of the image pixels is fully shuffled. The classification accuracy of the CNN is investigated for increasingly complex datasets: MNIST , Fashion-MNIST and CIFAR10 . In the second and third experiment we develop two different parametrized methods for controlling the image randomization and investigate the CNN classification accuracy on only the CIFAR10 dataset.

# Pixel-wise permutations

If we consider the pixels of an $(n \times n)$ image in row-major order as the set $\left\lbrace 1,2,3,4,5, \cdots, n^{2} \right\rbrace$ then a pixel-wise permutation can be expressed in Cauchy's two-line notation as: $$\label{permutation}
\sigma= \Bigl(\begin{matrix}
1 & 2 & 3 & 4       &5  &  \cdots &     n^{2}\\
4 & 5 & 1 & n^{2}   &3  &  \cdots &     2
\end{matrix}\Bigr)$$`<!-- -->`{=html}

where the second line represents the new arrangement of the original pixels in the permuted image. Therefore pixel 1 from the original image is moved to position 4 in the permuted image, pixel 2 is moved to position 5 etc.

## MNIST

A sample of both natural and pixel-wise permuted images is shown in Figure : the left panel shows handwritten digits from the MNIST dataset and the right panel shows a pixel permutation of those images. The same permutation is applied on all train and test images.

The test data accuracy of the networks can be seen in Figure . The CNN trained on natural images reaches an accuracy of $99.5\%$ while the one trained on permuted images shows a delayed learning curve as well as a consistently lower performance.

The performance of the MLP trained on both natural and pixel-wise permuted images is almost identical throughout the entire training phase. A more interesting observation is that the MLP, which consists of only the last fully connected layers of the CNN, has higher performance on permuted images than the CNN at every point during the training phase.

## Fashion-MNIST

The Fashion-MNIST dataset has a higher complexity than MNIST and poses a greater difficulty for the networks. The left panel of Figure shows a sample of the greater variability of clothing images and their pixel-wise permutations.

Figure shows a lower performance, as compared to MNIST, on natural and pixel-wise randomized images for both types of networks. The peak performance of the CNN trained on the natural images is $94.3\%$, decreasing to $89.6\%$ when trained on the randomized data set. The same behaviour is observed as in the previous experiment: the test accuracy of the CNN trained on permuted images is consistently below the MLP.

## CIFAR10

The CIFAR10 dataset is far more complex than the previous two: the images are in 3 color channels, the objects are not centered, they are less similar to each other and the background is not uniform. Figure shows the natural and pixel-wise randomization images, for the latter the identical randomization procedure being applied to all channels. The various labels are listed in table .

Training a CNN on this database reveals an even lower classification accuracy when using natural images, slightly below $90\%$, and a dramatic performance decrease when training on pixel-wise permuted images, reaching only about $57\%$, while the MLP is invariant under this type of transformation.

Figure shows the evolution of the accuracy on the test images. There is again the trend of the accuracy of the CNN trained on permuted images to stay consistently below the baseline performance of the MLP. Table summarizes the peak accuracies of the networks trained on all three datasets.

|             |   CNN   |          |   MLP   |          |
|:-----------:|:-------:|:--------:|:-------:|:--------:|
| **Images**  | natural | permuted | natural | permuted |
|  **MNIST**  |  99.5%  |  98.2%   |  98.7%  |  98.6%   |
| **Fashion** |  94.3%  |  89.6%   |  91.0%  |  90.9%   |
| **CIFAR10** |  88.9%  |  57.3%   |  59.3%  |  59.3%   |

Accuracy of a CNN and MLP trained on natural images and their permutations.

# Image patch permutations

To further investigate the behaviour of the network on data randomization a parametrized method is developed in order to gain a better control on the randomization process. The images are sliced in square patches which are then shuffled in the same manner as described by equation , where the numbers, instead of image pixels, denote the image patches. The parameter that controls the randomization is the size of the patch: a size of 1 is equivalent to a full pixel-wise permutation and a size of 32 is equivalent with the natural image. Figure shows a few examples of permutations with intermediate patch sizes.

Alternatively one can consider the randomization parameter the number of slices the image is cut into along an axis which results in a convenient series of powers of 2. The classification accuracy of the CNN trained with this parametrization is displayed in Figure . It shows how strongly the CNN performance is influenced by the size of the patches used for the randomization. Even when cutting the images in 4 slices per side the CNN looses in accuracy considerably.

# Local permutations

In this experiment the image pixels are randomized inside a restricted neighborhood. The algorithm is as follows: the image is scanned in a row-major order and each pixel is swapped with a randomly chosen pixel inside a neighborhood of $D$ pixels.

A zero distance is equivalent to no permutation and a 32 distance to a full pixel-wise permutation. There are no restrictions on the number of times a pixel can be moved. Hence there is a non-zero chance that a pixel might migrate a distance longer than $D$. This effect is rather small, as can be seen in Figure where the top-right image shows a local permutation with distance 1.

Figure shows the performance of the CNN as a function of several distances, ranging from 0 up to the size of the whole image. The larger the randomization distance is the less features the $[3\times3]$ kernels are able to capture and the accuracy quickly decreases, reaching the same level as a CNN trained on fully pixel-wise randomized images.

It is interesting to note how the CNN classification accuracy decreases when trained on separate image channels. Humans are able to recognize objects in images being either in color or gray-scale, with shape being the crucial factor in the classification process. The CNN correlates not only local pixels in terms of shape but also local pixels in terms of colors. Training on natural images and single channels results in a $2\%$ performance decrease as compared to training on all channels. When the local pixel correlations are destroyed the network performance degrades considerably more when trained on just single color channels. Comparing the color curves in Figure with the black curve indicates that the network relies also on color correlations when doing classification, since the same randomization procedure is applied on all channels.

The color curves also show a slight data bias due to the growing discrepancy between the classification accuracy on the three different channels. Training on just the red channel results in the worst performance for all randomization distances indicating that there might be less information in this channel.

# Discussion

Figures , and show common and consistent behaviors: the performance of the CNN drops when switching from training on natural to permuted images and always stays below the accuracy of the MLP throughout the whole training phase. The accuracy decrease cannot be attributed to a particular configuration of the network's initial random weights because repeated trainings display the same effect.

In Figures and we have shown how the performance of the CNN changes as the image pixels are gradually randomized. The stronger the randomization is the lower the accuracy of the network becomes. By applying the same permutation to all examples the local patterns that repeat within the same image (e.g. edges at various inclinations) are destroyed but the intra-class similarities and inter-class differences are kept. The accuracy of the CNN decreases because the convolution kernels can not find hierarchical features in the randomized images but it does not decrease to the level of random guessing since the separation between classes still remains. It is unclear how to quantify the accuracy decrease the randomization induces.

The convolutional network has a built-in infinitely strong prior which constrains the values of some parameters to zero making it highly sensitive to the spatial structure of the data . The more the local pixel correlation is removed the lower the classification accuracy becomes. The performance of a CNN has at least two independent components:

1.  like any other feed forward ANN, the network finds intra-class similarities and inter-class difference via gradient descent.

2.  the kernels learn local patterns occurring repeatedly within different regions of the **same image**, within different examples of the **same class** and within different examples of **different classes**.

Natural images are structures where local correlations are important. This is the reason why edges, corners, Gabor-like features etc. area learned by the CNN's first few layers when trained on natural images . Combining these basic building blocks in complex hierarchical structures results in large varieties of images and objects. At a more profound level it is speculated that this is the reason why deep learning works so well in practice and that the very structure of the universe is basically a large hierarchy of simple elements . If pixels are moved in random positions across the image domain their initial local correlation is destroyed an it becomes a long range correlation. If there is a correlation among pixels from more distant locations a kernel spread on a wider area of the image is more likely to capture them than a local $[3\times3]$ kernel.

Based on this idea we experimented with a modified version of the CNN where we replaced the whole stack of many convolution layers with two identical dilated convolutional layers: 64 filters with $[4\times4]$ kernels, dilation rate of 4 and a stride of 1. Figure shows a $62\%$ accuracy when training on pixel-wise permuted images indicating that some of the long range correlations are captured by the dilated kernels. There is a problematic aspect of this approach in that it does not permit arbitrary number of such layers due to the inevitable image size reduction of this convolution operation. We have also tested a single convolutional layer with an $[8\times8]$ kernel with a stride of 4 but it did not exceed $60\%$ accuracy. Stacking many fully connected layers together did not surpass the classification accuracy of the dilated convolutional network indicating that deep MLPs are not a solution for this type of images.

# Data properties

In this section we will present some basic properties of the data and show how strong the correlation between training on natural images vs. pixel-wise permutations is. This, in turn, indicates the underlying structure of the data even when individual image pixels are randomly permuted inside the images.

## Fashion-MNIST

The Fashion-MNIST dataset is more complex than MNIST, as can be seen from the classification accuracy in Table . But the greater difficulty of this dataset comes not from the objects alone, but rather from the strong similarities between differently labeled classes. The left panel of Figure shows how similar many examples of **shirt** look like **T-shirt** or **coat**. The average Fashion-MNIST images are shown in the top row of Figure and the standard deviations in the bottom row.

| **0** | T-shirt/top | **5** |   Sandal   |
|:-----:|:-----------:|:-----:|:----------:|
| **1** |   Trouser   | **6** |   Shirt    |
| **2** |  Pullover   | **7** |  Sneaker   |
| **3** |    Dress    | **8** |    Bag     |
| **4** |    Coat     | **9** | Ankle boot |

Fashion-MNIST labels

The confusion matrix in Figure illustrates how often label 6 (**shirt**) is misclassified as **T-shirt**, **coat**, **pullover** and **dress**. If it were not for these strong similarities between labels 0, 2, 4 and 6 the overall performance of the CNN would be higher, most of the other categories having a classification accuracy significantly more than 90%. Other similar correlations can be seen in the non-zero entries in the $7^{th}$ column, which indicate that the network identified shoe-like features in **sandal**, **sneaker** and **ankle boots**.

## CIFAR10

CIFAR10 has considerably more variation than Fashion-MNIST and the mean and standard deviation figures are not relevant for this kind of analysis as the distributions are much more uniform. The image features and class commonalities are difficult to see by the naked eye and more sophisticated statistical tools are needed to shed light on the intra-class and inter-class correlations. However, by investigating the confusion matrix one can identify the correlations the networks learn.

Figure shows the confusion matrices of the CNN trained on natural (top-panel) and pixel-wise permutations, respectively (bottom-panel). For training on natural images the top two highest accurate predictions are for the **ship** and **automobile**. The largest classification error is made in the case of the **cat** which is $9\%$ of the time misclassified as a **dog**. Notice that the confusion matrices are only approximately symmetric and the reverse misclassifications are slightly different, e.g. **dog** confused as a **cat** is about $9.8\%$.

| **0** |  airplane  | **5** |  dog  |
|:-----:|:----------:|:-----:|:-----:|
| **1** | automobile | **6** | frog  |
| **2** |    bird    | **7** | horse |
| **3** |    cat     | **8** | ship  |
| **4** |    deer    | **9** | truck |

CIFAR10 labels

Although overall the classification accuracy drops from $89\%$ to $57\%$ when switching from training on natural images to permuted images the correlations made by the network in one case are very similar to the other case. One can observe the same central pattern in both figures. The top two classification accuracies are the same - **ship** and **automobile** and the **cat-dog** misclassification remains still the highest.

Figure shows the confusion matrices of the MLP trained on natural and pixel-wise permuted images. Unlike the CNN, the overall accuracy difference is very small, $0.14\%$, but some of the individual accuracies show a relatively high variation, the largest deviation being for **bird** and **frog** with $-6.9\%$ and $+4\%$, respectively. A similarly strong **cat-dog** confusion is done also for this network.

We can obtain a quantification of the similarities between the two CNNs and MLPs by calculating the Pearson correlation coefficient between the prediction a network does for a particular category when trained on natural images and the prediction it makes for the same category when training on permuted images. In other words for the CNN we correlate the upper columns of Figure with the lower columns from the same figure. The same calculations are performed for the MLP confusion matrices in Figure . Thus we obtain 10 correlation coefficients which are summarized in table together with the average correlation coefficient.

| **Class**  |  **CNN**  |  **MLP**  |
|:----------:|:---------:|:---------:|
|  airplane  |   0.951   |   0.862   |
| automobile |   0.974   |   0.995   |
|    bird    |   0.745   |   0.974   |
|    cat     |   0.963   |   0.960   |
|    deer    |   0.645   |   0.902   |
|    dog     |   0.982   |   0.983   |
|    frog    |   0.753   |   0.916   |
|   horse    |   0.907   |   0.926   |
|    ship    |   0.857   |   0.967   |
|   truck    |   0.925   |   0.992   |
|  **Mean**  | **0.870** | **0.947** |

Correlations between the predictions of networks trained on natural images and pixel-wise permutations of CIFAR10 images.

# Conclusions and further studies

This paper presents the limitations of convolutional neural networks when trained on images where individual pixel positions have been randomly permuted. We show a comparison of classification accuracies between CNNs trained on natural images and pixel-wise permutations together with the performance of an MLP as baseline. The absolute value of the accuracies are not relevant to the study, just the relative performances.

We have shown that the use of standard convolutional networks is inappropriate for cases where image pixels are permuted inside the image domain. We create long range correlations which can be better captured by kernels covering larger image areas than standard localized kernels. We have shown that by applying the same permutation to all images from the dataset there is still an underlying structure which can be discovered by neural networks. This suggests the possibility for further model improvements. It is important to design networks with architectures that can be invariant or at least less sensitive to data permutations or other types of data encryption. There are studies which show that CNNs are still capable of classifying encrypted images, although in that particular case the transformation is a homomorphic encryption which preserves more of the data structure than the pixel randomizations do. This further raises the question whether it is possible to train a network on encrypted data but then be able to reconstruct the initial data once a few examples of human interpretable data become available.

Many types of analyses, where data is not necessary locally correlated, would benefit from such empowered architectures. For example, high energy physics experiments require the analysis of large data sets from particle collisions where the data appears on an event-by-event basis as random tracks in the detectors. However there are very strong underlying correlations since the subatomic processes obey the laws of physics. Often, , CNNs are used for the analysis of features identified by physicists through standard methodologies. Networks which would perform well on seemingly random data would be of great use for this kind of studies.

Other domains could also greatly benefit from more powerful networks designed specifically for capturing long range correlations, situations which can easily arise in the experimental physical sciences.


abstract: Celiac disease prevalence and diagnosis have increased substantially in recent years. The current gold standard for celiac disease confirmation is visual examination of duodenal mucosal biopsies. An accurate computer-aided biopsy analysis system using deep learning can help pathologists diagnose celiac disease more efficiently. In this study, we trained a deep learning model to detect celiac disease on duodenal biopsy images. Our model uses a state-of-the-art residual convolutional neural network to evaluate patches of duodenal tissue and then aggregates those predictions for whole-slide classification. We tested the model on an independent set of 212 images and evaluated its classification results against reference standards established by pathologists. Our model identified celiac disease, normal tissue, and nonspecific duodenitis with accuracies of 95.3%, 91.0%, and 89.2%, respectively. The area under the receiver operating characteristic curve was greater than 0.95 for all classes. We have developed an automated biopsy analysis system that achieves high performance in detecting celiac disease on biopsy slides. Our system can highlight areas of interest and provide preliminary classification of duodenal biopsies prior to review by pathologists. This technology has great potential for improving the accuracy and efficiency of celiac disease diagnosis.
address: $^{1}$Department of Biomedical Data Science, Dartmouth College
  $^{2}$Department of Computer Science, Dartmouth College
  $^{3}$Department of Pathology and Laboratory Medicine, Dartmouth-Hitchcock Medical Center
  $^{4}$Department of Epidemiology, Dartmouth College
title: Automated detection of celiac disease on duodenal biopsy slides:
  a deep learning approach

# Introduction

Celiac disease (CD), an autoimmune disorder triggered from the consumption of gluten, affects as much as one percent of the population worldwide \[1, 2\]. Patients who are diagnosed with CD undergo treatment in the form of a lifelong gluten-free diet, which requires substantial patient education, motivation, and follow-up \[3\]. Recent studies have found that the prevalence of CD has increased dramatically in the United States and Europe, and that undiagnosed CD was associated with a nearly four-fold increase in risk of death \[4-6\]. In fact, CD remains undiagnosed in the majority of affected people, highlighting the need for more frequent and accurate methods for its detection \[7-9\].

Celiac disease diagnosis involves serological testing of celiac-specific antibodies, followed by microscopic examination of duodenal biopsies, which are considered the gold standard in diagnostic confirmation of CD \[10, 11\]. Typically, four to six duodenal samples are taken from the patient by an endoscopic procedure, and these samples are visually examined by a pathologist. A confirmatory diagnosis requires detection of histological changes associated with the disease, which are classified according to guidelines from either Marsh \[12\], Marsh modified (Oberhuber) \[13\], or Corazza \[14\]. Endoscopic findings that indicate CD include scalloped folds with or without mosaic pattern mucosa, reduction in the number of folds, and nodular mucosa \[15\]. The spectrum of histologic changes in CD ranges from only increasing intraepithelial lymphocytes with preserved villous architecture to mild villous blunting to complete villous atrophy \[16\]. Studies have shown that the histological diagnosis of biopsies are subject to a significant degree of inter-observer variability \[17-20\]. One potential method for improving the accuracy of CD detection on duodenal biopsies is to apply automated image analysis to aid pathologists. Because the prevalence of CD is increasing, active case-finding is currently being used to screen more patients \[21, 22\]. An automated biopsy analysis system could help pathologists by filtering and pre-populating scans, improving efficiency and turnaround-time.

Recently, a subfield in artificial intelligence known as deep learning has produced a set of image analysis techniques that automatically extract relevant features, transforming the field of computer vision \[23\]. Deep neural networks use a data-driven approach to learn multi-level representations of data, allowing for comprehensive image analysis and classification \[24\]. These techniques are being increasingly applied to medical imaging to assist radiologists and pathologists \[25\]. In gastroenterology, previous studies have already used deep neural networks to classify colorectal polyps on biopsy and colonoscopy images \[26-28\], intraductal papillary mucinous neoplasms in MRI images \[29\], and diabetic retinopathy in retinal fundus photographs \[30\]. For CD in particular, large video datasets captured during endoscopies have facilitated quantitative analysis with deep learning \[31, 32\]. However, endoscopic classification is for the most part not used for confirming the diagnosis of CD. In this study, we developed a deep learning model that detects CD from duodenal biopsy images, the gold standard for diagnosis. We evaluated our model on an independent test set of 212 whole-slide images.

# METHODS

## Data Collection

To train and evaluate our model for celiac disease detection, we collected whole-slide images from all patients who underwent duodenal biopsies from 2016 to 2018 at the Dartmouth-Hitchcock Medical Center (DHMC), a tertiary academic care center in Lebanon, NH. These slides contain hematoxylin-eosin stained formalin-fixed paraffin-embedded (FFPE) tissue specimens and were scanned by a Leica Aperio whole-slide scanner at 20x magnification by the Department of Pathology and Laboratory Medicine at DHMC. In total, we collected 1230 slides from 1048 patients. We randomly partitioned 1018 of these whole-slide images from 681 patients for model training and 212 whole-slide images from 163 patients as an independent test set for final evaluation of our model. There was no patient overlap for the slides in the training and test sets.

## Slide Annotation

All whole-slide images used in our study were diagnosed by attending pathologists on gastrointestinal pathology service at the time as either normal, celiac disease, or nonspecific duodenitis. Normal duodenal biopsies show preserved villous architecture with no mucosal injury or acute or chronic inflammation. Celiac disease biopsies show a spectrum of histologic changes as described in Marsh classification \[12\], including partial to total villous atrophy with intraepithelial lymphocytosis, chronic inflammation and crypt regenerative hyperplasia. Nonspecific duodenitis includes histologic changes including peptic duodenitis, drug induced injury, and various other differential diagnoses of villous atrophy and acute and chronic inflammation. These labels were parsed from the medical record database and assigned as reference standard for slides used during model training. The training slides were then further split into a training set of 790 images and a development set of 228 images. Training set slides were used for training our neural network, while development set images were used for hyperparameter tuning. For the independent test set of 212 images, however, all labels were separately reviewed and confirmed by two gastrointestinal pathologists. Disagreements between original labels and new labels were reviewed by a senior gastrointestinal pathologist, who determined final classifications. The class distributions and roles in the data flow for our training, development, and test set are shown in Figure 1.

## Model Development

In recent years, research in deep learning has demonstrated successful application of convolutional neural networks for image classification, including medical image analysis. In our study, we used the deep residual network (ResNet) \[33\], a neural network architecture built from residual blocks. ResNet significantly outperforms early deep learning models such as AlexNet \[34\] and VGG \[35\], and achieved state-of-the-art performance on the ImageNet and COCO image recognition benchmarks \[36, 37\]. We implemented ResNet to take in square patches as inputs and output a prediction probability for each of the three classes: normal, celiac disease, and nonspecific duodenitis.

For model training, we used a sliding window method on each high-resolution whole-slide image to generate small patches of size 224 by 224 pixels. Since some classes had more whole-slide images than others, we generated patches with different overlapping areas for each class. When inputting a patch into the model for training, we normalized the RGB color channels to the mean and standard deviation of the entire training set to neutralize differences in color among slides. Then, we performed color jittering on the brightness, contrast, saturation, and hue of each patch. Finally, we randomly rotated and flipped the images across the horizontal and vertical axes. In total, we generated 80,000 patches for each of our three classes, which were then uniquely augmented during each epoch in training.

In terms of model parameters, we initialized ResNet-50, the fastest ResNet with three-layer residual blocks, with weights from the He initialization \[38\]. We trained our ResNet model by optimizing on a multi-class cross-entropy loss function for forty epochs on the augmented training set, starting with an initial learning rate of 0.0001 and decaying by a factor of 0.85 every epoch. We used the Adam optimizer \[39\] and weight decay regularization (L2 penalty) \[40\] of 0.0001. Total training time was twelve hours on a Titan Xp graphics processing unit (GPU).

## Whole-Slide Inference

In whole-slide inference, we aimed to classify each whole-slide image as either normal, celiac disease, or nonspecific duodenitis. The model is trained to classify small patches rather than entire slides, so we again used the sliding window algorithm to break down each whole slide into a collection of patches, each overlapping by one-third area. Next, we applied our trained ResNet model to classify each patch, and we filtered out noise using thresholding to discard predictions of low confidence. Given the distribution of patch predictions, we used the following heuristic to determine the whole-slide class: if more than $\gamma$ patches were classified as nonspecific duodenitis, then the whole slide was classified as nonspecific duodenitis. Otherwise, the most commonly predicted class was chosen as the whole-slide prediction. Thresholds for filtering noise, as well as $\gamma$, were optimized by performing a grid search over the development set. This allowed for accurate classification of slides with a significant amount of nonspecific duodenitis that was not covering the majority of the specimen area. Figure 2 depicts the whole-slide inference process. Inference time for a single whole-slide image was about fifteen seconds on a single Titan Xp GPU.

## Evaluation and Visualization

For final evaluation, we applied our model to the independent test set of 212 whole-slide images. We compared the predictions of our model with reference standards established by pathologists, and measured accuracy, precision, recall, and F1 score for each class. We calculated confidence intervals for all performance metrics using the Clopper and Pearson method \[41\]. In addition, we plotted Receiver Operating Characteristics (ROC) curves and calculated area under the curve (AUC) for each class.

Furthermore, we visualized our model's predictions at both the whole-slide and patch level. At the whole-slide level, we overlaid color-coded dots on patches for which the model predicted a particular pattern. This helps pathologists quickly identify regions of the slide containing abnormal tissue. At the patch level, we used the class activation mapping (CAM) method \[42\] to generate a pixel-level heat map that highlights the most informative regions of the image relevant to the predicted class. This demystified our classification method for each patch by revealing the most significant histologic features on the patch for each class for our model.

# RESULTS

For model selection, we validated our neural network model on the development set of 228 images. We found the optimal thresholds for filtering out noise at the patch-level to be 0.7 for the normal class, 0.8 for the celiac disease class, and 0.85 for nonspecific duodenitis. For selection of the $\gamma$ threshold for percent area needed to classify a whole-slide as nonspecific duodenitis, we considered our gastrointestinal pathologist's subjective examination of our model's predictions in addition to a grid search to arrive at $\gamma$ = 0.25. After threshold optimization, our best model applied to the development set achieved an accuracy of 95.6% for normal, 98.7% for celiac disease, and 94.3% for nonspecific duodenitis after threshold optimization.

Performance of our model on the independent test is shown in Table 1, which includes accuracy, precision, recall, and F1 score with 95% confidence intervals. Notably, our model detects the presence of CD with an accuracy of 95.3% and an F1 score of 93.5%. Table 2 shows the confusion matrix for predicted labels versus reference labels. ROC curves and AUC for each class are shown in Figure 3. AUC was greater than 0.95 for all classes. Figure 4 depicts whole-slide visualizations of twelve biopsy samples using dots to indicate predicted patch labels. Finally, CAM visualizations of individual patches are shown in Figure 5 to highlight relevant features used in our model's classification process. A subjective qualitative investigation of these visualizations by a gastrointestinal pathologist confirmed that the predictions of our model are generally on target.

```latex
\begin{table*}[t]\centering
\begin{tabular}{lcccc}
\hline
& Accuracy (\%) &   Precision (\%)  & Recall (\%) & F1 Score (\%) \\ \hline
Normal (n = 71) &   91.0 (87.2-94.9)&   83.3 (75.1-91.6)&   91.5 (85.1-98.0) &  87.2 (79.9-95.2)\\
Celiac Disease (n = 74) &95.3 (92.4-98.1)&  90.0 (83.4-96.6)&   97.3 (93.6-99.9)&   93.5 (87.8-99.3) \\
Nonspecific Duodenitis (n = 67) &89.2 (85.0-93.3)&  90.7 (83.0-98.5)&   73.1 (62.5-83.7)&   81.0 (71.5-90.5) \\ \hline
Average&    87.7 (83.3-92.2)&   88.0 (80.5-95.4)&   87.3 (79.5-95.2)&   87.2 (79.4-95.1)\\ \hline
\end{tabular}
\caption{Performance of our final model for celiac disease detection on 212 duodenal biopsy whole-slide images in our test set. 95\% confidence intervals are shown in parentheses.}
\end{table*}
```

| Prediction     | Reference |                |            |
|:---------------|:---------:|:--------------:|:----------:|
|                |  Normal   | Celiac Disease | Duodenitis |
| Normal         |    65     |       2        |     11     |
| Celiac Disease |     1     |       72       |     7      |
| Duodenitis     |     5     |       0        |     49     |

Confusion matrix of our final model for celiac disease detection on 212 duodenal biopsy whole-slide images in our test set.

# DISCUSSION

Duodenal biopsies are the gold standard for confirming the diagnosis of CD. The prevalence of CD has increased dramatically in recent years, and active case-finding calls for more serological tests and duodenal biopsies. Detection of CD on these biopsies could potentially be enhanced and facilitated by automated image processing. In this study, we presented a deep learning model that classifies duodenal tissue and highlights the associated features and regions of interest. Previous work has used deep learning to detect CD from endoscopic images \[31, 32\]. While acknowledging the substantive work of these investigators, endoscopies are for the most part not used for confirming CD diagnoses. Our model not only detects CD on duodenal biopsies, but it also visualizes regions of normal tissue, celiac disease, and nonspecific duodenitis to aid review by pathologists. Of note, we are not aware of any other existing system for CD detection on biopsy images.

Our model achieved high performance for detection of celiac disease. On the independent test set of 212 images, our model detected CD with a considerable F1 score of 93.5% and AUC of 0.993. Since our model made predictions at the patch level and then aggregated them for whole-slide inference, it was relatively unaffected by noise and achieved high accuracy. For normal and nonspecific duodenitis, F1 scores were 87.2% and 81.0% respectively. Identification of nonspecific duodenitis was more challenging because this class's tissue often also contains some normal tissue fragments, which complicate the analysis. Sixteen slides were misclassified between normal and nonspecific duodenitis, and pathologist evaluation of these errors revealed errors related to tissue orientation, fixation artifact, and patchy histologic changes. In addition, seven slides of nonspecific duodenitis were identified as celiac disease due to focal increase of intraepithelial lymphocytes and partial villous atrophy. Performance measures for the nonspecific duodenitis class were the lowest across the board. There are several reasons for this. One could be that nonspecific duodenitis had the lowest number of training samples, comprising only 64 images in the training set compared to 620 and 106 for normal tissue and celiac disease respectively. Another could be that this category comprises several disease entities including peptic chronic duodenitis, active duodenitis, and other nonspecific reactive changes, making it more challenging to detect since there was a wider range of histologic attributes to learn. Finally, slides labeled as nonspecific duodenitis often contained some portions of normal tissue, and since we extracted patch labels based on whole-slide labels during the training process, it is likely that some mislabeled data was used in model training and made it harder to detect this class in our approach.

In terms of visualization, the class activation mapping results of our model's selected areas of attention indicate that our model has learned the correct histologic features for each class. Classification of normal tissue tended to be holistic, with attention to almost all tissue area including normal villous architecture. For celiac disease, the model correctly identified villous atrophy, intraepithelial lymphocytosis, and chronic inflammation in the lamina propria. In the case of nonspecific duodenitis, the model identified villous thickening, Brunner's gland hyperplasia, foveolar metaplasia, and chronic inflammation.

Our results indicate that deep neural networks have substantial potential to aid gastrointestinal pathologists in diagnosing CD. For application in a clinical setting, our model could be integrated into existing laboratory information management systems to pre-populate patch predictions on slides and provide preliminary diagnoses prior to review by pathologists. In addition, a visualization of the slide evaluated by our model at the piecewise level could highlight precise tissue area containing abnormal or sprue patterns, allowing pathologists to quickly examine regions of interest. As CD prevalence has increased dramatically in recent years, more serological screenings and duodenal biopsies are being done for patients at risk \[21, 22\]. With biopsies as the gold standard for diagnosis, our work aims to provide pathologists with a tool for more accurate and efficient detection of CD.

The model presented in this paper is rooted in solid deep learning methodology and achieves commendable performance, but there are several limitations of our study. One limitation is that all biopsy slides were collected from a single medical center and scanned with the same equipment, so our data may not be representative of the entire range of histologic patterns in patients worldwide. Although our whole-slide scans are high resolution and we were able to extract a large number of patches for training with the sliding window method, our dataset is still small in comparison to conventional datasets in deep learning, which contain more than ten thousand unique samples per class \[43, 44\] and more than a million unique images in total \[45, 46\]. Overfitting is unlikely because we generated a large number of small patches for training and conducted final evaluation on an independent test set, but it is still a possibility. Collecting more data in collaboration with another medical center in future work would allow us to train a more generalizable neural network and could also improve our model's performance in classifying nonspecific duodenitis.

Moving forward, more work will be done to further the capabilities of our model and evaluate its use a clinical setting. Collecting an annotated dataset with specific histopathological classifications of celiac disease and labeled bounding boxes around lesions would allow our model to classify and locate specific lesion types, providing pathologists with more comprehensive slide analysis, particularly for the nonspecific duodenitis class. Furthermore, once more data is collected, we can predict slide level results by using patch predictions to train a traditional machine learning classifier such as a support vector machine or random forest, which may yield better results than our current thresholding method. In terms of clinical application, we plan on validating our model on a larger test set from multiple institutions and deploying a trial implementation of our model into laboratory information management systems at the Dartmouth-Hitchcock Medical Center to measure its ability to improve CD detection accuracy and efficiency. However, widespread clinical implementation of such artificial intelligence tools will require major future steps, which our group will be undertaking. Any deep learning models for computer-aided pathology must be thoroughly validated through clinical trials and be proven to enhance outcomes. Such model must also not impact the established workflow of pathologists or slow down the speed of existing programs. Most importantly, deep learning models must be accurate and gain the confidence of physicians, patients, and the medical community. In its current state, artificial intelligence has the ability to analyze images and make preliminary classifications, but much more work must be done before patients and physicians will be able to trust computers to make medical decisions. We believe the work presented in this paper is a preliminary step in this direction.

In summary, we have demonstrated that deep learning can achieve high accuracy in detecting CD in duodenal biopsies. Our model uses a state-of-the-art residual neural network architecture for whole-slide classification and achieved exemplary results on an independent test set of 212 whole-slide images. As CD prevalence and screening increases, we expect our model could assist pathologists in more accurate and efficient evaluation of duodenal biopsy slides.

# ACKNOWLEDGEMENTS

We thank Matthew Suriawinata for assistance with slide scanning and Sophie Montgomery and Lamar Moss for their feedback on the manuscript. This research was supported in part by a National Institutes of Health grant, P20GM104416.

# REFERENCES