abstract: In computer experiments, a mathematical model implemented on a computer is used to represent complex physical phenomena. These models, known as computer simulators, enable experimental study of a virtual representation of the complex phenomena. Simulators can be thought of as complex functions that take many inputs and provide an output. Often these simulators are themselves expensive to compute, and may be approximated by "surrogate models" such as statistical regression models. In this paper we consider a new kind of surrogate model, a Bayesian ensemble of trees (Chipman et al. 2010), with the specific goal of learning enough about the simulator that a particular feature of the simulator can be estimated. We focus on identifying the simulator's global minimum. Utilizing the Bayesian version of the Expected Improvement criterion (Jones et al. 1998), we show that this ensemble is particularly effective when the simulator is ill-behaved, exhibiting nonstationarity or abrupt changes in the response. A number of illustrations of the approach are given, including a tidal power application.
  KEY WORDS: $$ Additive regression trees; Global optimization; Expected improvement; Nonstationary simulators; Tidal power model.
author: Hugh Chipman, Pritam Ranjan and Weiwei Wang
Department of Mathematics and Statistics, Acadia University, NS, Canada
(firstname.lastname@example.com, email@example.com, firstname.lastname@example.com)
title: Sequential Design for Computer Experiments with a Flexible Bayesian Additive Model

# Introduction

Many phenomena, such as tidal flow, nuclear reactions, climate behaviour and universe expansion are sufficiently large and complex that direct scientific experiments are either impossible or impractical, due to time and cost constraints. As a result, mathematical models are often used to build a realistic representation of these phenomena, enabling experimentation. For example, with a model of tidal flow, researchers can study the effect of placing underwater electrical turbines, using the mathematical model to "observe" how tidal flow would change, and predict how much electrical power might be generated.

These mathematical models, often known as computer simulators, typically take a number of inputs and when they are run, generate an output. Running computer simulators can be time-consuming when output values are desired for a large number of different input values. In such situations, a second level of approximation, in which a "surrogate model" is used to approximate the input-output relationship of the simulator, is often used. These surrogate models are flexible regression models, taking an input vector $x$ and predicting an output $y$.

As with most statistical models, a surrogate model is estimated with training data, consisting of $n$ observations $(x_1,y_1), \ldots,
(x_n,y_n)$. Unlike many real-world phenomena, a computer simulator is often deterministic. That is, every time the simulator is run with input $x$, the same numeric value of output $y$ will result. Surrogate models that capture such behaviour, interpolating exactly the training data, are popular tools in such computer experiments.

In this paper we consider computer experiments with the very specific goal of optimization. We wish to find the values of input $x$ which minimize (or maximize) the output $y$. This sort of approach is taken in Jones, Schonlau and Welch (1998), where the training data are collected sequentially, allowing refinement of the surrogate model and increasingly accurate prediction of the location of the global minimizer.

By far the most popular statistical surrogate model is the Gaussian Process (GP) model (for an overview, see Santner, Williams and Notz (2003) and Rasmussen and Williams (2006)). The GP assumes a multivariate normal distribution for a vector of responses at corresponding input locations. The covariance of the responses is taken to depend on the distances between the input locations. Closely related to spatial models (e.g., kriging), the GP model allows interpolation of a deterministic function, and can predict quite effectively in the input space.

The GP model makes rather strong assumptions, however. The assumption that covariance depends only on distance between points and not their location leads to a stationary model, in which the smoothness of the response function is the same over the entire input space. Limitations such as this have led to extensions. For example, Gramacy and Lee (2008) develop a treed Gaussian Process (TGP) model, in which a tree recursively partitions the input space into rectangular regions, and within each region, a stationary GP model is fit.

In this paper we consider a more radical departure from the GP model. We build upon a Bayesian *ensemble of trees* model (Chipman, George and McCulloch 2010, here onwards referred to as CGM), which combines the outputs from a large number of regression trees via a summation. This gives a flexible model capable of capturing nonstationarity and complex relationships, and if present, additive structure. It is also able to emulate both deterministic and stochastic (noisy) simulators. By utilizing Markov chain Monte Carlo (MCMC) for estimation, full inference for the model is available. This in turn enables computation of merit based criteria, for instance, an Expected Improvement (EI) criterion (Jones et al. 1998), that can guide the sequential design of experiments for identification of a global minimum (our focus in this article) or other features of interest.

The remainder of the paper is organized as follows. In Section 2, we introduce Bayesian Additive Regression Trees (BART), the ensemble that will be used as the surrogate model. After discussing adaptation of prior distributions for BART to computer experiments, we briefly outline how TGP may be used as another surrogate model. The two models are illustrated with a tidal power generation example. In Section 3, the Expected Improvement criterion of Jones et al. (1998) is reviewed and adapted to the BART model. A number of illustrations are given in Section 4, with test functions playing the role of computer models, and up to four-dimensional inputs considered. Section 5 concludes with a discussion of future directions for research.

# Statistical Surrogate Models

In this section, we present a brief review of the BART model (see CGM for more details). We also discuss the Treed Gaussian Process (TGP) model of Gramacy & Lee (2008), which will be compared to BART in Section .

## The BART model

Assume the simulator takes a $d$-dimensional input vector $x=(x_1, \ldots, x_d)$ and has real-valued output $y(x)$. The BART model represents the output as a sum of $m$ adaptively chosen functions and an independent normal error, $$\label{eq:bart}
y(x) = \sum_{j=1}^m g(x; T_j,M_j)\,  + \,
\epsilon = h(x) + \epsilon ,
 \qquad \epsilon \sim N(0,\sigma^2).$$ The function $g(x;T,M)$ produces an output when provided with input vector $x$ and parameters $T$ and $M$. It is the shorthand notation for a "regression tree" model. The predictions for a particular value of $x$ are generated by following the sequence of decision rules in tree $T$ until arriving at a terminal node $b$, at which point an associated scalar prediction $\mu_b$ is returned. Decision rules in $T$ branch on a single element of the vector $x$, say $x_1$, yeilding rules such as $x_1 < 2.4$. Different rules in each tree may use different variables. For a tree $T$ with $B$ terminal nodes (i.e., partitioning the input space into $B$ rectangular regions), let $M=(\mu_1,\ldots, \mu_B)$ denote the collection of terminal node predictions.

Thus, viewed as a function of $x$, tree model $g$ produces a piecewise-constant output. By combining together an "ensemble" of $m$ such tree models in (), a flexible modelling framework is created. For instance, if each individual $T_j$ uses partitions on only a single input variable, then the BART model becomes an additive model. BART is however capable of incorporating higher-dimensional interactions, by adaptively choosing the structure and individual rules of the $T_j$'s. Furthermore, many individual trees ($T_j$) may place split points in the same area, allowing the predicted function to change rapidly nearby, effectively capturing nonstationary behaviour such as abrupt changes in the response.

Viewing this as a statistical model, we have parameters $\Theta
=(T_1,\ldots T_m, M_1, \ldots, M_m, \sigma)$. CGM take quite large values of $m$ (say, 50 to 200), and estimate () in a Bayesian framework using MCMC. With a sufficiently large number of terms in the ensemble, the interpretation of individual trees becomes irrelevant, and the most useful interpretation of BART is to view it as a way of placing a prior on $E(Y|X)$, that is a prior on functions. Of course, all regression models, including GPs, place a prior on functions. What is interesting is the sort of functions that receive prior mass.

The BART model places a prior on functions that differs from GPs in several important ways. First, as mentioned above, the BART model is biased towards additive and low-dimensional functions (i.e. $g(x;T_j,M_j)$ in () with small trees $T_j$, that are functions of one or a few variables). Second, the BART model does not assume continuity of the response, thus making it appropriate when there are abrupt changes or non-stationarity in the response. In contrast, the explicit specification of a spatial correlation structure in a GP model implies continuity and, for many common correlation functions, stationarity (i.e., a constant amount of "wiggle") of the response.

The Bayesian framework equips the BART model with a full suite of inferential tools. Most importantly, uncertainty in predicting $h(x)$ due to all parameters is available. As will be demonstrated in Section 3, this is a key ingredient in sequential design.

## Prior specification of BART for computer experiments

CGM demonstrate effective performance of BART in a wide range of simulated and real-data examples. In all examples, there was considerable noise, that is $\sigma \gg 0$. Often in computer experiments, the simulator is deterministic ($\sigma =0$) or has small errors ($\sigma \approx 0$). This property underlies some important changes to CGM's default choice of prior distributions. This section briefly outlines those choices, with emphasis on modifications for surrogate modelling of the deterministic simulator output.

We follow the same general formulation of prior distributions as in CGM, with different parameter choices. CGM simplify prior specification by assuming that, a priori, i) $T_1, \ldots, T_m$ are i.i.d., ii) all elements of $M_1, \ldots, M_m$ (i.e., the $\mu_{i,b}$, node $b$ of tree $i$) are i.i.d. given all $T$'s, and $\sigma$ is independent of all $T$'s and $M$'s.

A special prior for a generic output $\mu$ of a tree is used. In a computer experiment, with $\sigma \approx 0$, we believe $h(x) \approx y(x)$, that is, the overall output is quite close to the observed response. Thus we choose a larger prior variance for individual tree output $\mu_{b,i}$ than the CGM default. This allows the outputs more flexibility, giving $h(x) \approx y(x)$ at the training points. This is accomplished as follows: After scaling data values of $Y$ to the interval (-0.5, 0.5), CGM recommend a default prior $\mu\sim
N(0,\sigma^2_{\mu}) = N(0,1/(4k^2m))$ with $k=2$. Since the prediction $h(x)$ in () is a sum of $m$ distinct $\mu$'s, the default prior means that $h(x) \sim N(0,1/(4k^2))$. There is thus a prior probability of approximately 95% that $h(x)$ falls within $0 \pm 2\sigma_{\mu} = 0 \pm 1/k$, or (-0.5, 0.5) for $k=2$. We recommend relaxing this prior to $k=1$. Of note is the fact CGM's default prior specification applies considerable shrinkage to individual output $\mu_{i,b}$. Choosing a smaller value of $k$ (e.g., $k=1$) increases the prior variance of output $h(x)$, applying less shrinkage (or smoothness) of the response.

The assumption that $\sigma \approx 0$ requires modification to the prior on $\sigma$. Essentially, we place most of the prior mass near 0. This is accomplished with the same inverted-chi-squared prior for $\sigma^2$ as in CGM, using their recommended value of 3 degrees of freedom, and anchoring the 90th percentile of the $\sigma$ prior at `0.20`$\times$`sd(y)`, where `sd(y)` is the sample standard deviation of the training $y$ values. The only difference from the CGM default is the value used to anchor the 90th percentile (i.e., `0.20`$\times$`sd(y)`). This prior specification allows for some noise in the response values. This strategy facilitates MCMC mixing for BART, and can also be considered as having a similar motivation to Gramacy and Lee (2012) who advocate the inclusion of a residual error when approximating deterministic functions with GPs, for numeric stability and predictive accuracy.

Although not part of the prior specification, several other operating parameters of BART are chosen as follows: The number of trees in the ensemble is chosen as $m=100$. Individual trees are allowed to split on a fine grid of 1000 cutpoints along each axis. The MCMC algorithm uses 6000 iterations, discarding the first 2000 (burn in) and keeping every 20th thereafter, for a sample of 200 posterior draws. Larger posteriors samples might be desirable, but with the quick mixing behaviour of BART observed by CGM, a sample of this size will be sufficient for the sequential design step covered in Section .

The remaining elements of the prior specification involve the tree $T$. We use defaults from CGM, with a prior on tree $T$ that puts probability mass of 0.05, 0.55, 0.28, 0.09 and 0.03 on trees with $1, 2, 3, 4$, and $\geq 5$ terminal nodes. While favouring small trees, this prior does not rule out the possibility of very large trees.

## Treed Gaussian processes and Gaussian process models

Later in the paper we shall make comparisons between BART and another tree-based model, the treed Gaussian Process (TGP) model of Gramacy and Lee (2008). We briefly review the model and prior specification here. The TGP model has form similar to (), but with $m=1$, i.e. a single tree. The replacement of the ensemble by a single tree is offset by an enrichment of the terminal node model. Instead of a constant mean in each terminal node, Gramacy and Lee assume that conditional on the tree structure, the response $y(x)$ follows a GP model with var$(y(x)) = \sigma^2_h + \sigma^2$ and cov$(y(x), y(x')) = \sigma^2_hK(x-x')$, where $K(x-x')$ is the spatial correlation structure, and $\sigma^2_h$ and $\sigma^2$ are the process and noise variance respectively. By allowing a distinct GP model for each terminal node, TGP can accommodate nonstationarity in the response.

Since the TGP model was developed for emulation of computer simulators, less modification is needed of the prior settings. In all our experiments we used default parameters in the R implementation (the `tgp` package), except for `BTE = c(2000,6000,20), nug.p = c(1,10,1,10^5)`. The BTE parameter indicates that the predictive samples are saved every E $(20)$ MCMC rounds starting at round B $(2000)$, and stopping at T $(6000)$. That is, TGP predictions were also based on 200 posterior draws, the same as for BART. The `nug.p` parameter specifies a mixture of exponential priors (with expected values $10^{-1}$ and $10^{-5}$) on the "nugget" ($\omega$), where $\sigma^2 =
\sigma^2_h\cdot \omega$.

Gaussian process (GP) models are also popular for sequential optimization. Our comparisons also include GPs, using the implementation from the ` tgp` package in R. All settings for the GP model are as above for TGP, except of course without the trees.

## Small illustration: 1D tidal power

To give a taste of these two models, we briefly describe a tidal power modelling application. A full description of the application is given in Wang (2010) and Ranjan et. al. (2011). The basic problem is to predict the amount of kinetic energy (which in turn will be used for producing electricity) that could be extracted from tidal flow through the Minas Passage in Nova Scotia, Canada. This 7-km wide passage has some of the highest tides in the world. According to Karsten et al. (2008), an individual turbine can generate up to 1 MW of power, and approximately 2.5 GW of power can be harnessed from the tidal kinetic energy by placing large collections of turbines in the Minas Passage. Karsten et al. suggest that a "fence" of turbines across the passage is optimal for extracting the maximum possible energy at the minimum cost.

Wang considered the emulation of a simplified tidal flow model (simulator), in which fences of turbines are placed in the Minas Passage. The only parameter to control in this simulator is the location of the fence along the passage. That is, our "$x$", when scaled to the interval $(0,1)$, represents the fence position along the passage. Variation in the shape of the shoreline leads to considerable difference in the flow at different locations. A second very significant influence on flow is the placement of other turbine fences. In the area surrounding a previously placed turbine fence, the flow can be expected to be much lower.

A tidal model can provide output of the power (MW) as a function of the fence location. Thus we have a one-dimensional optimization problem. The true power function (multiplied by $-1$ so our goal is minimization) is displayed in each of four panels of Figure . Note that in this simplified model the power function can be efficiently evaluated at many $x$, making the study of emulators more of an academic exercise. For this particular power function, the effect of a pre-existing turbine fence at roughly $x=0.83$ is evident, as the power dips (or negative power spikes) at this location.

The power function is the same in all panels. Each panel displays the same training set, and the corresponding predictions (posterior mean at a given $x$) for BART and TGP. The power function displays considerable nonstationarity (smooth, gradual variation for $0 \leq x \leq
0.65$ and rapid fluctuations for $0.65 \leq x \leq 1$). BART deals more consistently with this nonstationarity, closely tracking the observed values. TGP is more likely to declare that this variation is noise, and the resultant prediction is attenuated, relative to the true function. Although TGP is able to partition the space into different regions and fit a GP model in each area, in this case the TGP model is unable to consistently capture nonstationarity.

# Sequential Design with BART and Expected Improvement

As mentioned in the introduction, we consider a specific kind of sequential design, in which the goal is to find the optimum of a computer simulator. In the remainder of the paper, we shall assume that the goal is minimization, since it is trivial to switch between minimization and maximization.

We follow the approach of Jones et al. (1998). Let $f_{\min}=\min\{y_{1},\ldots y_{n_0}\}$ be the current best (i.e., smallest) function value among the $n_0$ points sampled thus far. Jones et al. (1998) define the improvement at a point $x$ as $I(x)=\max\{f_{\min}-y(x), 0\}$, which is positive if the (unobserved) response $y(x)$ at location $x$ is less than the current best function value, and $0$ otherwise. They follow the maximum likelihood approach and take the expectation of this improvement function with respect to $y(x) \sim N(\hat{y}(x), s^2(x))$, where $\hat{y}(x)$ and $s^2(x)$ are the BLUP and the mean square error of response $y$ at $x$. The GP assumption leads to a closed-form expression, $$\label{eq:JonesEI}
EI(x) = E[I(x)] = (f_{\min}-\hat{y})
 \Phi\left(\frac{f_{\min}-\hat{y}}{s}\right) + s
\phi\left(\frac{f_{\min}-\hat{y}}{s}\right),$$ where $\Phi$ and $\phi$ are the standard normal CDF and density. The first term () captures "local search" that seeks to improve the estimate of the minimum near a currently identified minimum. The second term captures "global search" which places points in regions where there is sufficient uncertainty that the minimum response could be nearby.

There are a variety of approaches for evaluating () or similar expressions. All involve some form of integration over the predictive distribution for the unobserved response at a new input $x$. In this paper, we take a fully Bayesian approach, calculating the expected improvement by taking the expectation of $I(x)$ with respect to the posterior for all parameters, using the MCMC samples. This accounts for both predictive uncertainty and parameter uncertainty. We employ a similar approach for TGP and GP. Although we do not use (), the ideas of local and global search are still useful concepts to bear in mind when considering how EI chooses runs in a sequential design.

The details of the computation of EI are as follows: For BART, we calculate EI directly from the MCMC output. For every posterior draw $\Theta^{(i)} = (T_1^{(i)}, \ldots, T_m^{(i)}, M_1^{(i)},
\ldots, M_m^{(i)}, \sigma^{(i)})$, it is straightforward to get the posterior realization of $y(x)$ and hence $I(x)$. Taking a sample average of $I(x)$ values over the $N$ MCMC draws yields a MCMC approximation to the Expected Improvement $$\begin{aligned}
\nonumber EI(x) &=& \frac{1}{N}\sum_{i=1}^N I(x; \Theta^{(i)} )\\
 & =&  \frac{1}{N}\sum_{i=1}^N \max\{f_{\min}-y(x;\Theta^{(i)}),0\},\label{eq:correctEI}
\end{aligned}$$ where $y(x;\Theta^{(i)}) = h(x;\Theta^{(i)})$. This expression for EI incorporates uncertainty in all BART parameters. We use the same MCMC-based strategy for calculating EI with TGP and GP models.

In order to select a new design point, EI must be maximized over all possible $x$. The current implementation of BART simultaneously runs MCMC and generates posterior draws for $h(x)$ at a specified set of $x$ points. Thus, we use space-filling designs (random latin hypercube designs) to generate a large candidate set, evaluate () at all points in the candidate set, and select the point with the largest value of EI as the next input at which to obtain simulator output $y(x)$. To make comparisons with TGP, we employ the same strategy.

As will be illustrated in the higher dimensional examples in the next section, this "candidate set" strategy for locating the maximizer of EI becomes computationally challenging as the dimensionality of the input space grows.

The sequential design algorithm can thus be summarized as follows:

1.  Obtain an initial design $X_{n_0}$ with $n_0$ points, and evaluate the simulator at these points, yielding corresponding simulator outputs $Y_{n_0}$.

2.  Set iteration $i=0$

3.  Select a candidate set $X^{\mbox{\em cand}}$ for prediction and evaluation of EI.

4.  Fit the model using $X_{n_0+i}, Y_{n_0+i}$.

5.  Calculate EI for each point $x \in X^{\mbox{\em cand}}$, identifying the maximizer $x^*$ of EI.

6.  Evaluate the simulator at $x^*$, augment $X_{n_0+i}$ and $Y_{n_0+i}$ with $x^*$ and $y(x^*)$, and set $i=i+1$.

7.  Unless a stopping condition is met, go to Step 3.

The initial design in Step 1 is a maximin LHD, with $n_0-2$ runs. The two other runs are $x=(0,0,...,0)$ and $x=(1,1,...,1)$, which are chosen to aid BART in assessing uncertainty near the boundaries of the input space. In Step 3, a new set of candidate points is generated with a random LHD at every iteration of the algorithm. In Step 4, the "model" is BART, TGP or GP. In Step 5 EI is calculated using MCMC integration. The "stopping condition" in Step 7 could either be a convergence criterion or tied to a budget. In the remainder of the article, we shall assume a fixed budget of a total of $n =
n_{new} + n_0$ evaluations of the simulator.

# Experimental Results

In this section, we demonstrate the performance of our proposed approach using a few simulated examples and the tidal power application discussed in Section . First a simple one-dimensional example is used to illustrate the sequential design approach with BART as the emulator. Then, we demonstrate the proposed approach on more complex simulators with one or more inputs. In all the examples presented here, our objective is to find the global minimum.

The average performance of BART in the sequential design framework is compared with TGP, a popular tree based surrogate for emulating non-stationary computer simulators. We also make compraisons with a standard GP model. The performance of the sequential techniques is also compared with non-sequential "one-shot" space-filling designs. Under the one-shot design setup, the simulator outputs based on an $(n_0 + k)$-point random maximin LHD was used to estimate the global minimum, where $0\le k \le n_{new}$. Although a series of different LHDs of increasing size is not a viable sequential design strategy, it does provide a basis for comparison. All results presented in this section are based on 100 realizations (i.e., 100 different starting designs based on random maximin LHDs, each followed by sequential design generation). For convenience, the EI-based sequential design approach using BART with () is referred to as BART-EI, and TGP-EI (GP-EI) refers to the sequential procedure if TGP (GP, respectively) is used instead of BART.

Implementations of BART-EI, TGP-EI and GP-EI follow the algorithm outlined at the end of Section 3. For all examples presented here, BART runs follow the prior parameters and operational settings described in Sections , whereas TGP parameter settings are discussed in . The GP settings are the same as TGP, but with the constraint that there be a single node, giving the same GP over the entire input space. Both TGP and GP also use MCMC integration to evaluate EI.

For all examples, the following details apply. We start with fitting a surrogate to the simulator data on a $n_0$-point random maximin Latin hypercube design (LHD). These designs are generated using the R function `maximinLHS` (Carnell, 2009) which takes random starting points and hence the output designs are random. Jones et al. (1998) and Loeppky et al. (2009) suggest using approximately $n_0=10d$ points as a reasonable rule-of-thumb for an initial design. However, the optimal choice of $n_0$ varies with the complexity of the computer simulator. The remaining $n_{new} = n-n_0$ points are sequentially added one at-a-time by optimizing the EI criterion and refitting the surrogate (BART, TGP or GP). The size of the candidate sets $X^{\mbox{\em cand}}$ for evaluation of EI at unsampled input locations varies with the complexity of the simulators.

We use the running best estimate of the global minimum (i.e., the smallest $y$ value observed thus far) as our performance measure. An alternative considered, but not reported here, is the distance between the input locations of the true and estimated global minimum. This may not be useful if the simulator has multiple global minima, which is the case in Example 3.
**Example 1.** Suppose the simulator outputs are generated using a the simple one-dimensional test function $$y(x) = \frac{\sin(10 \pi x)}{ 2 x} + (x-1)^4, \quad x \in [0.5, 2.5].
\label{eq:testfn_1d}$$ The inputs are scaled to $[0, 1]$ for implementation. Gramacy and Lee (2012) used this example to demonstrate the benefits of including a nugget in a GP model. Figure  illustrates the sequential BART-EI methodology for $n_0=10$ and $n_{new} = 15$ runs.

Although BART does not accurately capture all high-frequency oscillations of the function, it appears to effectively guide sequential design for the minimum, utilizing a combination of local and global search. Figure  shows that the first follow-up trial is in the vicinity of the current best estimate of the global minimum (i.e., supporting the local search). Figure  shows that the next four follow-up points support exploration and the points are away from the running global minimum. Most additional points are near the running global minimum (see panel (c)), although some points such as 11, 12, 14 and 15 are placed in regions of high uncertainty (i.e., global search). The emulator constructed using BART exhibits "football shaped" uncertainty bounds as typically shown by GP models for deterministic simulators. The variance of the predicted response at the training points is not zero, however, due to the inclusion of a noise term in the BART model.

To compare GP, TGP and BART, we repeated the experiment 100 times. Following the $n_0=10d$ rule of thumb, we started with $n_0=10$ points and added $n_{new}=40$ runs sequentially one at-a-time. We used 1000-point random LHDs as candidate sets ($X^{\mbox{\em cand}}$) at which EI was evaluated to choose the next run. Figure  presents the performance comparison between BART-EI, TGP-EI, GP-EI and the one-shot design method. Performance is summarized using mean and median (over 100 realizations) of the running best estimates of the global minimum for each method.

As expected, the sequential design approaches outperform the one-shot design scheme. BART-EI slightly outperforms TGP-EI and is clearly superior to GP-EI. The mean value of the running best estimate is typically larger than the median because the best estimate is bounded below by the global minimum of y(x). Thus, the horzontal lines just below -0.85 indicate that the global minimum has been found in over half of the 100 replicates. The more gradual approach of the mean lines indicates that in some cases, the global optimum has not yet been found.
**Example 2.** We revisit the tidal power example, where our objective is to maximize the power function (or equivalently, minimize the negative power function). Recall from Figure  that the negative power function is very spiky near the global minimum and there is another spike with the function value close to the global minimum. This makes the optimization problem tricky especially if the design points are not densely sampled in both the spikes. Thus, we started the sequential algorithms with a slightly larger initial design, $n_0=15$, and added $n_{new}=45$ follow-up points one at-a-time. The EI criterion was calculated on a 1000-point random LHD. Figure  summarizes the mean and median (over 100 realizations) of the running best estimates of the global minimum.

As in Example 1, the sequential design approaches outperform the naive one-shot approach. But, the performance comparison of BART-EI and TGP-EI shows interesting results. The mean curve for BART-EI is converging at a slower rate than TGP-EI or GP-EI, but in terms of median performance, BART dominates. It turns out that BART-EI can sometimes get stuck in the wrong spike if there are not enough design points in the regions of interest (see Figure ).

**Example 3.** Let $x=(x_1, x_2) \in [0, 1]^2$, and the simulator outputs be generated from a two-dimensional additive function given by (Rönkkönen et al. 2008) $$%y(x) = \frac{1}{4}\sum_{i=1}^2 [\cos((G_i - 1)2\pi w_i) + \alpha \cdot \cos((G_i-1)2\pi L_i w_i)],
y(x) = \frac{1}{4}\sum_{i=1}^2 [\cos(4\pi w_i) + \alpha \cdot \cos(8\pi w_i)],
\label{eq:testfn_2d}$$ where $\alpha = 0.8$ and $w_i = \sum_{j=0}^{n_i} {n_i \choose j}P_{i,j}(1-x_i)^{n_i-j}x_i^j$ for $i=1,...,d$, $n_1=n_2=4$ and $P_1 = (0,0.1,0.2,0.5,1), P_2 = (0,0.5,0.8,0.9,1)$. The contour plot in Figure () shows that the test function has 16 global minima with $y_{min} \approx -0.478$ (shown in red diamonds).

Since the function has multiple global minima, one might suspect that the estimation of the global minimum would be relatively easier. But the local versus global search trade-off feature of the EI criterion forces the follow-up trials to jump from the neighbourhood of one global minimum to another. That is, instead of precisely estimating one global minimum, the EI criterion tries to minimize the prediction uncertainty near several global minima. Consequently, the sequential algorithms require a large number of follow-up trials to accurately estimate the global minimum. Following the $n_0=10d$ rule, we started the implementation with $n_0=20$ points chosen using random maximin LHD and sequentially added $n_{new}=40$ follow-up points using the EI criterion which was evaluated at 5000-point random LHD over $[0, 1]^2$. Figure  presents the mean and median (over 100 simulations) of the running best estimate of the global minimum.

It is clear from Figure  that BART-EI performs significantly better than TGP-EI and GP-EI. The fact that the spikes are aligned to the axes should help both BART and TGP. TGP should benefit more from the smoothness of the response function, while BART may be better suited to identify the additive structure. The slower convergence of TGP-EI can perhaps be attributed to inconsistent prediction of TGP.

In this example a larger candidate set $X^{\mbox{\em cand}}$ was used to evaluate EI and select the next run. Recall from Section  that the EI criterion is evaluated only at these cadidate points, and thus their distribution and density play a role in estimating the global minimum. With a two dimensional input space, we elected to use more candidate points.
**Example 4.** Suppose the computer simulator outputs are generated using the four-dimensional test function given by $$y(x) = \sum_{i=1}^4 -\sin(x_{i}) - 2 \exp\left(-30x_{i}^2\right), \quad x_i \in [-2,2],
\label{eq:testfn_4d}$$ where the input variables, $x = (x_1, x_2, x_3, x_4)$, are scaled to $[0,
1]^4$. This test function is based on a one-dimensional function from DiMatteo, Genovese and Kass (2001), and has a unique global minimum with $y_{min} = -8$, but the global minimum is in a narrow spike. Figure  shows the one-dimensional function. The detection of this spike in the four dimensional $[0, 1]^4$ space would require at least a few design points in $[0.4, 0.6]^4$, that is in $0.16\%$ of the total volume, otherwise the surrogate models can get misled by the overall shape (i.e, excluding the spike).

A space-filling LHD-based initial design would have to be very large to guarantee a few points in the spiky region $[0.4, 0.6]^4$. On the other hand, if we ignore this spike, the test function or simulator is relatively simple to emulate. We start the sequential procedure with only $n_0=30$ points (smaller than the recommended $n_0=10d$ rule) and leave the discovery of the spiky region to the follow-up runs. We sequentially added $n_{new}=120$ follow-up points. To ensure the evaluation of the EI criterion in the spiky region, we considered dense candidate sets $X^{\mbox{\em cand}}$ with 20,000 and 50,000 points. Figure  summarizes the running global minimum.

BART-EI very clearly outperforms the other methods, with the median curves indicating that the global optimum is attained or nearly attained with an additional 50 points, in over 50% of the 100 replicates. Evidently, BART is much better suited to exploit the additive structure in (). There is a small but non-negligible benefit to evaluating EI criterion for BART with a larger (50K) candidate set.

# Discussion

The various examples in Section 4 demonstrate that BART can be an effective engine for sequential design and optimization. In situations where a GP model or perhaps localized GP models are appropriate surrogates, optimization with BART is still competitive. When there are nonstationarities, abrupt changes or additive structure, we see that BART can deliver the optimum with fewer runs of the simulator.

As in any sequential design procedure, the size and configuration of points in the initial design can impact the performance of the sequential design. Jones et al. (1998) and Loeppky et al. (2009) suggest using approximately $n_0=10d$ points as a reasonable rule-of-thumb for an initial design. However, the optimal choice of $n_0$ depends on the complexity of the simulator. Certainly if a very small number of runs are used in the initial design, the algorithm based on BART (or TGP) may take longer to find the optimum. On the other hand, one may not want use a very large initial design and put several points in the unimportant regions of the input space.

The reader may notice that the BART model is not continuous, but rather piecewise constant. However, by utilizing many trees in the sum ($m=100$ in our examples) and also employing posterior averaging over these trees, the BART model builds a surrogate that can have many small steps, providing an effective approximation to a continuous function. In our experience, the BART model seems effective at deciding where to probe the simulator (i.e. carry out additional runs), even in situations where it may not provide as "nice" a fit as a GP model. Of course, if a smooth prediction is desired (and possibly a better extrapolation to the minimum, if there is smoothness nearby) one may train a GP or TGP model on the set of all simulator runs, once the sequential design procedure has terminated. If, on the other hand, there is a possibility of nonstationarity, abrurpt changes or even discontinuities in the response function, then BART seems more likely to be an effective engine for sequential design and optimization.

As observed in Section 3, to evaluate EI (), we must specify (or design) a candidate set of points at which to generate predictions from the BART model. Since this set can be large (e.g., 20,000 to 50,000 in the 4-dimensional example), we used a random LHD. A maximin LHD (or other space-filling design) with thousands of points could be computationally challenging to identify.

Also related to the evaluation of EI, for the sake of comparison we used a "batch predict" approach to both BART and TGP. The current implementation of BART is not equipped with a "predict" function than allows predictions for new inputs. Instead the predictions are generated during the MCMC estimation of the model. To make comparisons with TGP, we thus used the same "batch predict" strategy. We are currently exploring alternate ways to evaluate EI and find the most promising points to add during the sequential design.

**ACKNOWLEDGMENTS**

We thank Richard Karsten for providing the tidal power model. The work of Wang was supported by Acadia Honours Student Research Award, and the work of Chipman and Ranjan were supported by Discovery grants from the Natural Sciences and Engineering Research Council of Canada. Computing resources were provided by the Acadia Centre for Mathematical Modelling and Computation.

**REFERENCES**


author:
bibliography: bibliography.bib
title: Lessons Learned from Real-World Experiments with
  DyRET: the Dynamic Robot for Embodied Testing

# INTRODUCTION

Robots are used in more and more complex environments, and are expected to be able to adapt to changes and unknown situations. The easiest and quickest way to adapt is to change the control system of the robot, but for increasingly complex environments one should also change the body of the robot – its morphology – to better fit the task at hand .

The theory of *Embodied Cognition* states that control is not the only source of cognition, and the body, environment, interaction between these and the mind all contribute as cognitive resources . Taking advantage of these concepts could lead to improved adaptivity, robustness, and versatility , however, executing these concepts on real-world robots puts additional requirements on the hardware and has several challenges when compared to learning just control .

In contrast to the majority of work in *Evolutionary Robotics*, Eiben argues for real-world experiments in his "Grand Challenges for Evolutionary Robotics" . This requires robust hardware platforms that are capable of repeated experiments which should at the same time be flexible when unforeseen demands arise.

In this paper, we introduce our unique robot platform with self-adaptive morphology. We discuss the challenges we have faced when designing it, and the lessons learned from real-world testing and learning.

# THE 'DyRET' ROBOT

Our robot, DyRET (Dynamic Robot for Embodied Testing), was developed to be a platform for experiments on self-adaptive morphologies and embodied cognition , shown in Fig. . It is a fully certified open source hardware project, and documentation, code and design files are freely available online . Since it is intended for use with machine learning techniques it is designed to be robust, withstanding falls from unstable gaits . It can actively reconfigure its morphology by changing the lengths of its femur and tibia, which can be used to mechanically gear the motors, and allow the robot to change the trade-off between movement speed and force surplus continuously .

# EXPERIENCES AND CHALLENGES

In this section, we present some key lessons we have learned when working with DyRET. We have tried to summarize the lessons, followed by more detailed explanations.

Legged robots are very complex systems, and anticipating all demands and challenges early in the design process is impossible. Techniques from *rapid prototyping* allowed us to quickly get physical prototypes of the robot, which allowed us to see and fix challenges that would be difficult to find without having physical proof-of-concept models of the system available. An important part of this, is to use already existing *Commercial-off-the-shelf* (COTS) components where available. This allows us to capitalize on the work of others, and also makes it easier for others to build or utilize lessons learned from our designs. *Desig for manufacturability* is another important concept, and promotes adapting the design to manufacturing considerations during the initial design process, where they can be solved much more easily than during operation. Making a robot that is easy and cheap to build can be important, but our experience is that maintainability is even more important, especially when using machine learning that puts huge strains on the physical robot.

Designing parts for legged robots is always a trade-off between strength and weight, and mechanical failures during prototyping is guaranteed. Strengthening the part that broke can be a quick fix, but our experience is that this often moves the problem. Both high persistent forces and sudden shock travel through the mechanical design, and lead to failure in the next weakest link of the chain. Reducing stress concentrations locally in a particular part can sometimes be successful in allowing the robot to withstand a similar situation again, however, excessive force can often lead to cascading failures throughout the system. Having a clear strategy for when and what to do when mechanical failures happen is important, and early on deciding on a balance between quick spot repairs and laborious systematic analyses of failures. Once an experiment is underway, replacing parts with similar parts might be the only option without skewing the results, so extra efforts on failure identification during the prototyping phase might be worth the effort. Larger cracks in the material are often easy to identify, but deflection during operation, small fractures, or material creep can be harder to detect.

Learning legged locomotion, or a gait, is a difficult challenge. To optimize the gait, the movement of the legs is parameterized through a gait controller. A lot of a priori knowledge can be embedded into the controller, and result in few parameters that are easy to optimize. Less prior knowledge requires more of the optimization algorithm, resulting in an increased number of evaluations. The more knowledge that is embedded, the less room there is for a varied range of behaviors, which might be needed to adapt to new or changing tasks, environments or the robot itself . Finding the right complexity balance can be very challenging, especially in real-world learning where the number of evaluations are limited. We have successfully used a gait controller with dynamic complexity , but similar performance can be achieved by focusing on complexity early on in the process. Another option is using different controllers for different environments or tasks , for instance a complex controller when optimizing the gait in a simulator with cheap evaluations, and a less complex controller in the real world.

One of the key insights we have experienced after several papers worth of real-world experiments on DyRET is how components change characteristics during the course of experiments. Because of this gradual change it is important to store as much information as possible so that automatic procedures can be applied to detect differences during and after experiments. A big difference between simulation and real-world experiments is that a real-world experiment can never be perfectly replicated. The change in characteristics should also guide the experiment design in the real world. Because components are expected to change it is important to evenly test different solution so as to not bias the experiment towards a specific solution. A concrete example is the reduction in performance of our joints as the motors heat up. If the solutions are always tested in the same order, this might affect the results, and give spurious effects that cloud the results.

Evaluating solutions on a physical robot system can take several seconds to minutes, depending on gait complexity and experiment design. Evaluating on physics simulations or with simplified models, often done in software, can give a speedup of several orders of magnitude. This often makes simulation a flexible and easier starting point. However, our experience with DyRET indicates that going from a real-world robot to simulation can yield more realistic simulation results which in turn translates to more sensible real-world gaits after software optimization. Not basing a virtual robot on a physical prototype makes it easier to make choices resulting in solutions that turn out to be infeasible in the real world.  .

# CONCLUSION

In this short abstract we have presented lessons learned through several years of experimentation on the DyRET platform, which have resulted in 13 scientific publications. From initial design considerations to the challenges, such as the trade-off between simulated experiments and real world learning, which have guided the choices we have taken. We hope to encourage more researchers within the robotics community to try real-world experiments and by sharing knowledge usually not found in publications we believe getting started with hardware could be easier.


abstract: Ways to give medium- and short-term predictions of solar flares are proposed according to the statistical analysis of events during solar cycle 23. On one hand, the time distribution of both C and M class flares shows two main periods of 13.2 and 26.4 months in this cycle by wavelet analysis. On the other hand, active regions of specific magnetic configurations and their evolutions give high productivity of C class flares but relatively low productivity of energetic (M and X class) flares. Furthermore, by considering the measurable kinetic features of active regions, i.e., the rotation of the sunspots, some active regions of specified types are observed to have high energetic flare productivity, above $66\%$. The periodicity of the activity revealed can be used for medium-term C and M class flare forecasting and the high productivity of active regions forms the basis for short-term prediction of individual energetic flares.
author: Z. Q. Qu, C. L. Xu , X. L. Yan, Z.K.Xue and Z.N.Qu
title: Prediction of Solar Flares from a Statistical Analysis of Events during Solar Cycle 23

# Introduction

Flare prediction is an emergent but very difficult task for solar physicists. Not only because the mechanisms involved in flare occurrence are complicated (Tandberg-Hansen and Emislie, 1988), but also the theories and observations are not very operationally related. It is now well known that the short-term prediction of individual flares should be done before or during the phase of magnetic energy storage resulting in the energy release later. Therefore, searching for sufficient or sufficient-like pre-flare conditions is more helpful than searching for necessary ones in this aspect. Statistical analysis is still a good tool to find such conditions, and these sufficient or sufficient-like conditions must correspond to the emergence of some determinant features of active regions. In this paper, we try to find such conditions by the statistics of events during solar cycle 23.

Concerning the search for the pre-flare conditions, several ways were proposed by investigators in the literature. For instance, the most popular way is to find the correlations between sunspot magnetic features and major flares(e.g. Greatrix 1963; Mayfield and Lawrence, 1985; Atac 1987; Sammis et al., 2000; Ternullo et al., 2006). Based on this kind of correlation and solar cycle data, Qahwaji and Colak (2007) established an automatic short-term prediction system with an ability of machine-learning. On the other hand, some statistical correlation between flare productivity and more detailed magnetic features, like maximum horizontal gradient of photospheric vector magnetic fields, length of neutral lines, and number of singular points, as well as magnetic shear, has been done by Cui and Wang (2007), and Cui et al.(2007). They found the correlation can be fitted by Boltzmann sigmoid functions and proposed that this relationship can be used for modeling flare forecasting.

At the present stage, we are far from forecasting all the flares or even only significant flares individually. However, it is very useful to give medium- and short-term predictions of some individual energetic(M and X class) flares. This is the main purpose of this paper. Gallagher, Moon and Wang(2002) based on the flare productivity of active regions, proposed some kind of flare forecasting, giving even 10 variables including McIntosh classification, sunspot group area, the number of sunspots, the average hardness indexes, etc. in their model. Like Jakimiec and Bartkowiak (1994), other similar features were included(Zhu and Wang, 2003). However, the short-term prediction of a great number of individual flares is far from satisfactory (Zhu and Wang, 2003). Therefore, searching the determinant variables which promote the prediction is crucial. Another way was proposed by Wheatland(2001), that the rate of flare production in single active regions from 1981 to 1999 can be obtained by studying the waiting-time distribution(WTD) for individual regions and the relation reflecting WTD, say, described by Poisson process can be used for the prediction. Evidently these two methods are complementary and can be used for further investigations in this field.

# Periodicity of Activity and High Flare Productivity Found during Solar Cycle 23

Solar cycle 23 provides some clues. Gaining some characteristics of this cycle can help us make the predictions for the next cycle. All of the data for the events used for our analysis stem from the website http://www.solarmonitor.org/. When counting flares, we should be careful to get rid of recounted flares recorded on the site in neighboring days. The time span covers the whole of solar cycle 23, i.e., from Feb. 1996 to Jan.2008. The total number of flares recorded by the website is 21,620, while 19,577 flares were classified as C class, 1,887 as M class and 156 as X class according to soft X ray flux classification. There were 11,138 (10,516 C class, 587 M class and 35 X class) flares not related to numerated active regions. They are also included for the statistical analysis. In order to have an overview of these recorded flares, Fig.1 outlines the number distribution of the flares of different soft X ray flux. One can see that the M1 class of flares stands out as an exception and this may become a feature of the activity of the cycle . We use the exponential function $$N(x)=a+be^{-c log_{10}x}$$ to describe the number distribution (solid line), where $x$ means the soft X ray flux irradiated by flares. The coefficients ($a=76,
b=2.0\times 10^{-5}, c=4.07$) can become other characteristic quantities describing the activity of this cycle. In the fitting, we do not include either the faintest flares (C1.0) recorded, because the number of these flares are likely underestimated, or the highest energetic flares above X10.0 because they are very rare.

In the upper panel of Fig.2, the number evolution of events with solar flare classification is depicted. One can see that the pattern takes on periodicity except for X class flares, whose number evolution is not plotted individually. This stimulates us to use the wavelet analysis with the Marlet function as the wavelet function to find the periods. In the lower panel, the corresponding power spectra are shown. It is obvious that all the curves have two prominent peaks at the 13.2 and 26.4 months for both C class and M class flare occurrence, and finally for all the recorded flares. Furthermore, the phases are also very close to each other. Such periodicity can be used for medium-term prediction of the next cycle, although the periods cannot be expected to be same. Finally, we may assign these periods as features of this cycle.

It is generally accepted that special magnetic configurations and their evolution induce flare eruptions. At the present stage, the three dimensional classification is not at hand for all the numerated active regions and the two dimensional Hale and McIntosh classifications are more frequently used. The lower panel of Fig.3 presents the numbers of the active regions belonging to particular magnetic configurations (Hale classification) without any configuration variation within one day. A similar diagram also appeared in the work by Ternullo et al.(2006) but only into June 2003. In the following, the flare productivity is defined as the ratio of the number of active regions hosting flares of specific class to the total number of active regions of the same specified type. The configuration $\beta$ is dominant in this cycle. However, the flare production does not favor of this configuration or another simple one, $\alpha$. High flare productivity occurs in other configurations, i.e. $\beta\delta$ ($64\%$, 25 regions hosting flares in 39 regions owned this configuration), $\beta\gamma$ (53$\%$, 547 in 1033), $\beta\gamma\delta$ ($75\%$, 300 in 402), $\gamma$ (100$\%$, 1 in 1) and $\gamma\delta$ (83$\%$, 5 in 6) (upper panel). For these regions containing the listed configurations, the M class flare productivity reached $28\%$(11 in 39) in the active regions containing configurations of $\beta\delta$ producing 18 M class flares; $10\%$( 102 in 1033) in regions of $\beta\gamma$ hosting 139 M class flares; 34$\%$(138 in 402) in regions of $\beta\gamma\delta$ inducing 256 M flares, and for rare configurations, 100$\%$(1 in 1) productivity in the region of $\gamma$ and 50$\%$(3 in 6) in regions of $\gamma\delta$. Finally for X class flare production, the highest efficiency took place in active regions containing configurations of $\beta\gamma\delta$ hosting 46 X class flares by 38 of 402 regions, with productivity of only 9$\%$; and 8$\%$ in those of $\beta\delta$ producing 4 X class flares. This statistics reveals the conclusion that energetic flares are not only relatively favored regions classified $\beta\gamma\delta$(Sammis et al., 2000) but also more configurations relatively. But the absolute productivity is not so high for those regions hosting more than 10 energetic flares. A similar conclusion has been drawn by Ternullo et al. (2006) but with a shorter time span in the cycle.

On the other hand, detectable variations of configuration can reveal another way to get the flare production efficiency.

In the top panel of Fig.4, we plot the productivity of 13 magnetic configuration evolution patterns among 31 patterns during the cycle. These patterns are selected by the criterion that the number of active regions that underwent one of these evolutions is more than 9. In the bottom panel is illustrated the number of corresponding active regions. It should be noted that some active regions could experience several evolution patterns. The symbol $'\diamondsuit'$ indicates the productivity including all flares above class C1.0, while the symbol $'\triangle'$ stands for the productivity for M class flares and symbol $'*'$ for X class flares. The highest productivity (75$\%$) of flares of all classes involves the evolution from $\beta\gamma$ to $\alpha$, though the most frequent configuration change pattern was from $\alpha$ to $\beta$. In fact, there are 6 important evolution patterns that show very high productivity (above 55$\%$). They are $\beta\delta\rightarrow
\beta$, $\beta\gamma\rightarrow \alpha$, $\beta\gamma\delta\rightarrow \beta$, $\beta\gamma \rightarrow
\beta$, $\beta\gamma \rightarrow \beta\gamma\delta$ and $\beta\gamma\delta \rightarrow \beta\gamma$. On the other hand, the M or X class flares were not highly favored by any those 13 kinds of active regions. The highest M class flare productivity ($33\%$) occurred in the regions experiencing evolution from $\beta\gamma$ to $\alpha$, and only 10 M class flares were produced in 4 regions. It was also this evolution pattern which contained the highest X class flare rate of production among the 13 evolution patterns. 2 among 12 of such regions produced 5 X class flares with a productivity of 17$\%$. The relatively high M class productivity occurs in the evolution patterns from $\beta\delta\gamma$ configuration to other configurations. For instance, 33$\%$(1 in 3) in the regions of configurations from $\beta\delta\gamma$ to $\alpha$, 30$\%$(16 in 53) to $\beta$, 33$\%$(3 in 9) to $\beta\delta$, 50$\%$(1 in 2) to $\beta\gamma$. And finally 30$\%$(36 in 118) from $\beta\delta\gamma$ to $\beta\gamma$.

For very rare evolution patterns, like that from $\gamma$ to $\beta\gamma\delta$, 100$\%$ M class productivity has been reached but only one active region once experienced such evolution. However, the very low X class flare production rate was evidenced for the most frequently evolution patterns. The rate of production is only 12.5$\%$ (1 in 8 regions) obtained by the evolution from $\beta\delta$ to $\beta\gamma\delta$ and 11$\%$(1 in 9 regions) from $\beta\delta$ to $\beta\gamma$.

# One Way to Give Short-term Prediction of Solar M and X Class Flares

In the above analysis, we see the statistical correlation of the Hale classification and its evolution to flare productivity obtained from solar cycle 23. Evidently, some specific configurations and evolutions of active regions have very high flare production efficiencies. However, for M and especially X class flares (which are significant for the prediction), we did not see enough high productivity (say, above 50$\%$), with enough flares, by observing the magnetic features of active regions. Naturally, one always expects to forecast more individual energetic flares. Therefore, to get a valuable and accurate warning, one should search more determinant features for those active regions hosting energetic flares.

One way is to consider the real time measurable kinetic features of sunspots among which two features, to our knowledge, are good choices, i.e., the emergence of new flux (Li et al., 2007) and rotation of the sunspots (Yan et al., 2007) . Here, we focus on the latter, and try to analyze the energetic flare productivity of active regions having sunspot rotations of different types, along with their magnetic configurations.

During solar cycle 23, 182 rotating sunspots from December 1996 to December 2007 were found (Yan et al., 2008a). Among them, 35 X class flares were related to 17 active regions with their sunspots rotated before flare occurrence. The rotation causes the energy accumulation of the flare (see Yan et al., 2007). By tracing the rotation and the following flare occurrence, we give the statistical analysis of the correlation between flares and rotations. In our recent research (Yan et al., 2008b), sunspots with rotations are classified into several types related to their magnetic configurations and the productivity was listed.

Figure 5 illustrates the time difference between the recognizable start time of rotation and the onset time of X class flares. It should be pointed out that the start time of sunspot rotation depends on the data sources (Yan et al., 2008b), typically the error is about 2 hours. One can see from the figure that the smallest time difference is a little longer than 2 hours while the greatest difference is less than 150 hours (a little longer than 6 days), which happened for type Vb. On the other hand, some of these regions produced more than one flare, and the time difference between successive flares within the same region can also be seen from the figure. The definition of these rotation types related to the magnetic configuration can be found in Yan et al.(2008b).

From the statistics, the energetic M and X class flare productivity can reach 100$\%$ in active regions of special types. For instance, in those active regions with rotating sunspots and magnetic configurations of type Vb (NOAA10930, NOAA09236, the only two regions found belonging to the type in the cycle), there were 5 X class flares occurring in the two regions. For M class flares, 100$\%$ production efficiency was met by types IVc, IVd and Vb. 5 M class flares occurred in NOAA09070 of type IVc, 1 M class flare happened in NOAA10464 of type IVd, 9 M flares took place in NOAA10930 and NOAA09236 of type Vb. Therefore, active regions of type Vb are very productive with respect to energetic flares! This type of active region was defined as follow: in these regions there is one sunspot that not only spins but also rotates around another sunspot with opposite polarity(Yan et al.,2008b).

Other types of rotation patterns which exist in the active regions can also induce high production rates of M and X class flares. For instance, for M class flare, types IIa, IIIa, IIIb, and VIb can have more than 66$\%$ productivity. In detail, 6 in 8 active regions of type IIa manufactured 42 M class flares, with productivity of $75\%$; 5 in 6 active regions of type IIIa gave birth to 31 M class flares and 5 in 6 regions of type IIIb produced 42 M class flares with the same productivity of 83$\%$; 2 in 3 regions of type VIb induced 8 M class flares, with productivity of $67\%$.

Therefore, we conclude that, if the measurable kinetic features of sunspots in active regions are taken into account, one may obtain a very high reliability for energetic flare prediction.

# Conclusions

Evidently, the most difficult mission for solar physicists is to forecast flares far from sunspot-inhabited regions. However, in active regions, one can find some clues to issue even short-term predictions of individual flares as we provided in the above section. At the present stage, researchers cannot give exact short-term predictions for all individual solar flares. Therefore, one way is to issue predictions with high reliability.

In this paper, we find C and M class flare occurrence periods of 13.2 months occurring twice during solar cycle 23 and we believe that such activity periodicity may exist in the past cycles or the coming cycles. This can be used for medium-term forecasting. Additionally, some special magnetic configurations and their evolution patterns can also help us present short-term prediction. They are listed as follows:

1\) Magnetic configurations. Usually, the more complicated are the configurations of the active regions, the higher the productivity they have. For all kinds of flare classes, configuration $\beta\gamma\delta$ has the highest rate of production, followed by $\beta\gamma$;

2\) Magnetic configuration evolution patterns. The flares are favored by evolution patterns from configuration $\beta\gamma\delta$ to other ones.

However, as pointed out above, one cannot get high reliability for predicting energetic flares by only observing the two dimensional magnetic configurations and their evolutions. But we also showed that it is possible to make short-term predictions with high reliability of more individual energetic flares if the real time measurable kinetic features of active regions, like sunspot rotation, are taken into account. Especially, we listed the sunspot rotation patterns of active regions related to magnetic configurations that showed high productivity of energetic flares as follows:

1\) High X class productivity. Active regions having sunspot rotation type Vb(100$\%$);

2\) High M class productivity. Active regions having sunspot rotation types IVc, IVd, and Vb have 100$\%$ productivity and active regions of types IIa, IIIa, IIIb, and VIb have production rates above 66$\%$.

The features of active regions having 100$\%$ M or X class flare productivity are candidates for the sufficient conditions we are searching for. Other features with high productivity, e.g., above 66$\%$, are also very useful. Of course, more real time measurable features can be included, for instance, the emergence of new flux into the active regions (Li et al., 2007). Therefore, it is reasonable to expect that more and more energetic flares can be predicted. On the other hand, one can also consider the WTD for these specific individual regions, like the work done by Wheatland(2001). It is evident that combining these two methods will promote short-term prediction for more and more individual flares.

Finally, it should be pointed out that the productivity of the above specified types of active regions may experience fluctuations from one cycle to another. However, our work here provides some clues or methods to promote medium- and short-term flare prediction.