Building Robust Stochastic Configuration Networks with Kernel Density Estimation

This paper aims at developing robust data modelling techniques using stochastic configuration networks (SCNs), where a weighted least squares method with the well-known kernel density estimation (KDE) is used in the design of SCNs. The alternating optimization (AO) technique is applied for iteratively building a robust SCN model that can reduce some negative impacts, caused by corrupted data or outliers, in learning process. Simulation studies are carried out on a function approximation and four benchmark datasets, also a case study on industrial application is reported. Comparisons against other robust modelling techniques, including the probabilistic robust learning algorithm for neural networks with random weights (PRNNRW) and an Improved RVFL, demonstrate that our proposed robust stochastic configuration algorithm with KDE (RSC-KED) perform favourably.


Introduction
Feed-forward neural networks have been popular and widely applied in many domain applications due to its universal approximation capability. Basically, a large amount of learning algorithms are developed on the basis of the minimum of a mean squared error (MSE) criterion, which indeed is sufficient and used in uncertain data modelling problems with assumption on noise fitting a Gaussian function. However, it is generally known that, in many real applications data are not only noisy but may inevitably contain outliers [1]. In fact, the outliers can cause a great threat to the training process of neural networks as just one notorious outlier could severely distort the training, which consequently make the resulting learner model based on MSE useless.
Over the last decades, a lot of robust techniques have been established in the field of robust statistics [1][2][3], including M-estimates (maximum likelihood estimates), L-estimates (linear combination of order statistics), R-estimates (estimates based on rank transformations), and LMedS estimates (least median square). In essence, a majority of these efforts lay great emphasize on using new cost function in training the neural network model, where the conventional back-propagation (BP) algorithm is applied in tuning the model parameters, such as hidden parameters and output weights. For example in [4], M-estimator and Hampels hyperbolic tangent estimates were employed in the objective function, aiming to alleviating the impacts of outliers. Liano [5] made use of the mean log squared error (MLSE) criterion as a robust cost function with assumption that the errors follow Cauchy distribution. In [6], the author proposed a robust neural network algorithm based on a M-estimator cost function with consideration of the random sample consensus (RANSAC) framework that has become the standard method of dealing with outliers in the computer vision and image processing community [7][8][9]. However, robust neural network methods based on BP algorithm still suffer from some inherent drawbacks, including the ambiguity in selecting the initialization weights and the problem of a local minimum and slow convergence speed. On the other hand, although some methods based on SVM are also obtained and successfully verified for function approximation problems with data corrupted with outliers, [10][11][12], its effectiveness and efficiency may meet challenge when dealing with training data with high level outliers or other large scale robust modeling problems. That means more flexible learning tool together with some advanced learning algorithms are desirable.
With the increasing demands of dealing with large scale data processing, randomized methodology becomes a pathway to access a fast and effective solution in computing exercises [13,14]. The use of randomness in neural networks can be traced back to later 80's [15] and some milestones on this topic can be read in our survey paper [16]. There is few work addressing robust modeling problem based on Random Vector Functional-link (RVFL) networks [17][18][19]. In [21], the authors employed a hybrid regularization loss function with assumption of the sparsity among outliers, and proposed a probabilistic robust learning algorithm for neural networks with random weights (PRNNRW), where some parameters must be set properly and this is quite difficult to be done in practice. In [22], the authors suggested an improved version of the original RVFL networks (Imporoved RVFL), where a weighted cost function is used to improve the model's robustness by differentiating each sample's contribution for the whole objective function. Specifically, kernel density estimation (KDE) method, which acts as a non-parametric way to estimate the probability density function of a random variable by computing a smooth density estimation from data samples by placing on a each sample point a function representing its contribution to the density, is used to renew the associated penalty weights. However, these two algorithms, i.e., PRNNRW and Improved RVFL do not clarify the random selection strategy of the hidden parameters, which indeed could not fully guarantee the effectiveness of the robust algorithms.
Our recent work reported in [23] proposed a new framework of randomized learner model, termed as stochastic configuration networks (SCNs), which has been proved as universal approximators. The main contribution of the work in [23] lies in the supervisory mechanism for randomly assigning the hidden parameters throughout the training process, which can guarantee the universal approximation property of the randomized learner model. It should be pointed out that our proposed supervisory mechanism in [23] is original and unique in literature up to date. This paper is built on [23] and attempts to develop a robust version of SCNs, aiming to copy with modeling problems with data corrupted by outliers. Based on the construction process of SCN, we employ a weighted least squares objective function that consequently leads to a weighted configuration criterion (compared with the original SCN) for randomly assigning the hidden parameters. At the meantime, the penalty weights representing the degree of contribution of individual data samples to the objective function will be updated by using a well-defined kernel density estimation (KDE) method. In this paper, alternating optimization (AO) technique is employed to renew the penalty weights and the SCN model. Finally, the performance of our proposed RSC-KDE algorithm is extensively evaluated on synthetic and real data contaminated by artificial outliers, including some benchmark datasets from the UCI repository of standard machine learning datasets [24] and a dataset from engineering application [22]. Comparative studies indicate that RSC-KDE outperforms other existing randomized neural networks in terms of effectiveness and robustness.
The rest of paper is organized as follows: Section II briefly reviews stochastic configuration networks and the kernel density estimation (KDE) method. Section III details our proposed RSC-KDE algorithm. Section IV reports experimental results with comparisons and discussions. Section V concludes this paper with some remarks.

Related Work
This section briefly reviews SCN framework, of which the highlight is the configuration of hidden parameters under a supervisor mechanism. In general, SCN performs in two interrelated phases, i.e., input parameter (inner weights and biases) configuration and output parameters (output weights) determination. Details of the SCN framework can be read in [23]. Also, as a non-parametric way to estimate the probability density function of a random variable, the kernel density estimation (KDE) method is introduced here.

Revisit of Stochastic Configuration Networks
Suppose that we consider the approximation problem of unknown function f : The constructive process of SCN starts with a small size of network then adds hidden nodes and output weights incrementally (automatically) until an acceptable approximation performance is achieved. That does not require any prior knowledge about sufficient complexity of the network for the objective tasks. For instance, given a target function f : R d → R m , assume a SLFN with L − 1 hidden nodes (L = 1, 2, . . .) have already been constructed, that is, Basically, SCN can provide an effective solution for how to add β L , g L (w L and b L ) leading to f L = f L−1 + β L g L until the residual error e L = f − f L is suitable for the given task, that is, e L is smaller than an expected specific tolerance ǫ. The following Theorem 1 restates the universal approximation property of SCN, corresponding to Theorem 7 in [23].
Theorem 1. Suppose that span(Γ) is dense in L 2 space and ∀g ∈ Γ, 0 < g < b g for some b g ∈ R + . Given 0 < r < 1 and a nonnegative real number sequence {µ L } with lim L→+∞ µ L = 0 and µ L ≤ (1 − r). For L = 1, 2, . . ., denoted by If the random basis function g L is generated to satisfy the following inequality: and the output weights are constructively evaluated by

Kernel Density Estimation
Basically, the kernel density estimator computes a smooth density estimation from data samples by placing on a each sample point a function representing its contribution to the density. The distribution is obtained by summing all these contributions. Readers may refer to [25] and [26] for further details on the kernel density estimation method.
Using KDE method, the underlying probability density function of a random variable η can be estimated as where K is a "kernel function" (typically a Gaussian) centered at the data points, η k , k = 1, 2, . . . , N , and δ i are weighting coefficients (typically uniform weights are used, i.e., δ k = 1/N ).

Robust Stochastic Configuration Networks
In some industrial process, the collected samples are always contaminated by the outliers caused by the failure of the measuring or transmission devices or unusual disturbances. That means a robust neural network learner that could fit a functional model to data corrupted with outliers is of high importance in practical applications, especially on some industrial system design. This section contributes to the development of robust stochastic configuration networks (RSCNs) by using kernel density estimation (KDE). To begin with, let us shed some light on the technique of weighted least squares (WLS), which has been received considerable amount of attention in robust modeling. In general, a weighted least squares (WLS) learning strategy when training a neural network model can be formulated as follows. For a target function f : R d → R m , given a training dataset with inputs . . , N , a robust neural network model as an approximator of f can be obtained by taking consideration of the weighted least squares problem, i.e., where θ i ≥ 0 is the penalty weight representing the contribution of the corresponding sample to the objective function. g is the activation function and L is the number of hidden neurons. For j = 1, 2, . . . , L, w j , b j are the input weights and biases, respectively. β j represent the output weights.
Generally, the penalty weights θ i (i = 1, 2, . . . , N ) can be determined according to the reliability of the sample x i . It is obvious that, high reliability means the sample should be a normal data that correctly represents the process behavior, whereas low reliability indicates that the sample might be a suspected outlier or noise embodying worthless or even wrong information. Decreasing and enlarging the weight of low and high reliability samples, respectively, can eliminate and even remove the impact of the outliers. As such, the built robust model will possess extremely small sensitivity to outlying data.
It is easy to combine the new objective (6) with our SCN framework shown in Theorem 1. Before introducing WLS into the procedures based on SCN, some specified notations are needed. In practice, as the exact functional form of e L−1 is unavailable, we use its consistent estimation on According to (6), a weighted form of e L−1 (X) should be concerned, aiming to implanting the penalty weights during the constructive process of SCN. Here we can denotẽ where Given a set of penalty weights θ 1 , θ 2 , . . . , θ N , we denotẽ According to the configuration mechanism in Theorem 1 and taking into account the contribution of each sample to the global objective function, the hidden parameters (w and b) should be selected whenξ L ≥ 0.
Now it is obvious that the remaining question is how to assign penalty weights θ 1 , θ 2 , . . . , θ N along with the processes of SCN. In fact, if the probability density function of the residuals can be obtained or estimated, the reliabilities of the samples will be determined. Inspired by [25], we attempt to incorporate KDE method (as reviewed in Section 2) into our SCN framework, aiming to estimate the reliability of the sample and then assign θ i in terms of the estimated reliability of the sample. Based on (5), the probability density function of the residuals e L can be calculated as (here e L is regarded as a random variable) is the estimation window width which exhibits a strong influence on the resulting estimate,σ is the standard deviation of the residual; K is the Gaussian function defined as: With the above equations, the probability of each residual e L (x i ) (i = 1, 2, . . . , N ) can be achieved as Φ(e L (x i )). Also, it is clear that the larger the probability, the higher the reliability. Concretely, the penalty weight θ i thus can be assigned as (i = 1, 2, . . . , N ) Once the set of penalty weights θ i is assigned and fixed, the output weights β 1 , β 2 , . . . , β L can be determined by solving the followed minimization problem where It is easy to obtain the solution of (10) below So far, we can generally describe the whole learning process in building a robust SCN model, where alternating optimization (AO) technique works effectively in finding the most appropriate values of penalty weights. To begin with, we assign equal penalty weight for each sample (i.e., θ i = 1) and conduct the procedures of SCN to obtain a learner with L hidden neurons, of which the hidden parameters w and b are configured based on (9), and β is determined by (11). Then the penalty weights are updated by new residuals caused by the resultant learner. Finally, a RSCN is established by repeating this procedure until some user-defined accuracy or stopping criterion is reached. It should be clarified that, the renew process of penalty weights corresponds to the whole process similar to the original SC algorithm. That means at each step of iteration, the penalty weights remain unchanged throughout the process of building the neural network architecture using SCN framework. Once a temp learner is designed, the penalty weights will be updated and the new setting will be employed in the new round for constructing an universal approximator by SCN principles.
Based on the idea of alternating optimization (AO), the penalty weight θ i and the output weight β can be calculated iteratively by and where ν denotes the νth iteration and L (x i ) to represent the changing (with L increase) residual vector throughout the constructing process in which θ (ν) i are utilized as the present penalty weights.
Different from our previously proposed SCN framework that all samples will contribute equally to the value of the objective function, this newly designed method based on KDE treats individual samples differently and give more emphasis on the entry with higher reliability, which is equivalent to, lager value of penalty weights. This means that if the output y j is occluded or corrupted, the input entry corresponding to outliers will provide small contributions to the cost function. As a result, the noise can be handled uniformly and smoothly. Now, our proposed robust stochastic configuration algorithm, namely RSC-KDE, can be summarized as followed:
Save w L and b L in W , ξ L in Ω, respectively; 9.
Else go back to Step 4 10.
If W is not empty 13. find Break (go to Step 18); 15.

Performance Evaluation
In this section, the performance of our proposed RSC-KDE algorithm is demonstrated by several regression problems. First, it is evaluated on a synthetic 1-D regression problem, of which in particular the visualization of the 1-D function approximation result is provided. Then we test the proposed algorithm on some benchmark problems from the UCI repository of standard machine learning datasets. In addition, both the effectiveness and advantages of our proposed algorithm are assessed in the problem of particle size estimation of grinding processes, aiming to illustrate its practical application in the field of engineering. For each regression problem, the performance of our proposed algorithm is compared with three randomized algorithms, including RVFL, improved RVFL presented in [22], and the probabilistic learning algorithm PRNNRW proposed in [21]. It should also be mentioned that all algorithms under comparison are analyzed under several scenarios and levels of outlying data.
As commonly used in data analysis literature, the root mean squared error (RMSE) is calculated to measure the performance of function approximation on various datasets. For each regression problem, the RMSE values of the trained model are reported on the corresponding outlier-free test datasets that are different from the data used for training, in order to reflect the generalization ability of each algorithm. Both the average value and standard deviation of RMSE are reported in all our experiments, as for the consideration of the impact caused by randomness. In addition, we conduct a robustness analysis on this task about the setting of ν and L max so as to offer a basic reference for the assignment of their values for this specific problem.
It should be noted that, in our simulations, all input and output attributes were normalized into [0,1] before artificially adding certain level of outliers. All the results reported in the experimental study are based on 100 independent trials, where the average value together with standard deviation are concerned. For both regression tasks, the maximum times of random configuration T max is set as 100. For the special parameters involving in the learning process, such as the regularization factor η, the maximum number of hidden nodes L max , the expected error tolerance ǫ and the index r, we will specify their corresponding setting later in the analysis of the simulations.

1-D Function Approximation
The four algorithms are examined to approximate the 1-D function This testing function firstly appeared in [20] and later was used in [23]. Herein, the training set has 600 points randomly generated by sampling the independent variable in the range [0, 1] and using the above equation to calculate the dependent variable. The following scenario is employed to introduce different levels of outliers in this dataset. A variable percentage ξ of the data points is selected at random and substituted with background noise, of which the values are uniformly distributed on the range [-0.2, 0.8]. In particular, Fig. 1 illustrates the training dataset used in our experiments at ξ = 25% along with the demonstration of the target function. Overall, the outlier percentage ζ is changed from 0 to 30% and each of the four training algorithms under comparison is employed at each outlier percentage. Finally, the RMSE is calculated from the resultant network model. This procedure is repeated 50 times, and the average obtained RMSE scores are plotted versus ξ in Fig.  2. The resultant models (the approximated functions) at ξ = 25% are graphed in Fig. 3.  It should be noted that, apart from our proposed algorithm, we try to use different choices of the random selection range of hidden parameters, i.e., λ = 1, 10, 30, 50, 100, 150, respectively, in the experiments to demonstrate the significant impact of the setting of the range selection range on the random model's performance. Then for each λ, different neural network architectures (L = 40, 60, 100, 120, 150, 200) are used in RVFL, Improved RVFL, and PRNNRW in order to find the most favorable performance for each combination of λ and L. It is clear that from Fig. 2 that our proposed RSC-KDE algorithm contributes to the best performance among all these randomized algorithms. In particular, for λ = 1, it is found that the approximation performance of all those three algorithms (RVFL, Improved RVFL and PRNNRW) could not be accepted. That means once the random selection range of hidden parameter is assigned without any reference and consideration   of training data information, the resultant neural model may not have any universal approximation capability and generalization ability. Additionally and importantly, for this specific test function, λ = 1 could not lead to an accepted learner no matter what the number of hidden neurons is and irrespective of any level of outliers. It is also clear in Fig.3 that our proposed algorithm has indeed shown the most robustness among the four algorithms at ζ = 25%, in which λ = 1 for the other three algorithms RVFL, Improved RVFL and PRNNRW. On the other hand, the four algorithms start with very different performance even when the outlier percentage is rather low, in which it can be easily seen that our proposed algorithm could produce the best model in this regression problem even if the dataset is corrupted with low level outliers or without outliers. That could strongly support the advantage of the configuration mechanism of our proposed algorithm. Table I shows that the (cases with comparatively low outlier percentage are not listed here) test performance in scenarios as λ = 1, 30, 50, 100, respectively. Even at the extreme outlier percentage of ζ = 30%, it offers a preferable robust model with RMSE of 0.0138 ± 0.0027 that is far better than the other three algorithms. For example, Improved RVFL leads to the most optimal result among them with RMSE of 0.0255 ± 0.0036 when λ = 50.

Benchmark Datasets
We also evaluate the four algorithms on some real high-dimensional datasets that are publicly available from the UCI repository of standard machine learning datasets and the KEEL-dataset repository [24]. The specifications of the four datasets considered in this paper are summarized in Table II. The four algorithms are used to find the functional relationship of the predicted attribute (output variable) of each dataset as a function of the remaining attributes (input variable). To begin with, both the input and output attributes are normalized into [0,1] and then we randomly select 75% of the whole dataset as the training data while the left acts as the test data. Similar to the 1-D function approximation problem, we artificially introduce different levels of outliers into the training datasets. In detail, for each normalized training dataset, a variable percentage ζ of the data points is selected at random and the associated output values are substituted with background noise uniformly distributed on the range [-0.5,0.5]. Indeed, the resultant output values of each corrupted training dataset are distributed in [-0.5,1.5] while the corresponding test dataset is free from outliers or noise (here we take no account of the possible existing noise or outliers involved in the original data).
For each dataset, we evaluate the comparative algorithms (RVFL, Improved RVFL and PRNNRW) with different parameter settings of λ and L, aiming to find their most appropriate combination. In particular, for each algorithm, we tried five scenarios with λ = 0.1, 0.5, 1, 3, 5, respectively, then different choices of the neural network architect (L = [30, 50, 100, 150, 200]) are attempted for each fixed λ. The experimental procedure is repeated 50 times for each λ and L, by which we calculate the average value and standard deviation of RMSE for each problem at a different level of outliers. In Fig, 4, we only provide the comparison result for the case of λ = 1 as it is already sufficient to indicate the influence of λ in building a universal approximtor, as also addressed in the last 1-D function approximation problem. It is clear in Fig. 4 that our proposed algorithm outperforms the other three randomized algorithms at each outlier percentage.
Clearly from Table III, our proposed algorithm exhibits the best accuracy for all these four datasets at each outlier percentage. It should be noted that, the reported values of the three algorithms indeed are the best results that are chosen among all the cases of the combination of λ and L. It is obvious that, among these three algorithms: RVFL, Improved RVFL, PRNNRW, the performance of Improved RVFL is the the most closest to our proposed RSC-KDE algorithm to some extent. Although the KDE method is utilized in both Improve RVFL and RSC-KDE, our stochastic configuration scheme in the learning process greatly enhances the system's effectiveness as its generalization capability evaluated on each problem has been improved a lot.
It should be mentioned that, in Fig. 4 for λ = 1, the algorithm PRNNRW exhibits the worst accuracy on stock, laser, concrete datasets, although the changing trend along with different outlier percentage stay stable. In Table III, its results obtained by the 'best' combination of λ (with values 0.1, 0.5, 1, 3, 5) and L (30, 50, 100, 150, 200) are much better than the performance shown in Fig. 4. This phenomenon illustrates the ineffectiveness and unreasonableness when the hidden parameters are randomly assigned in [-1,1]. Apart from [-1,1], much more appropriate random selection range exists, that is the reason why we prefer more choices of λ in the experiments and try to find the influences. Indeed, this parameter setting should be related to the training data information as well as other user-defined parameters that play an important role in training a good learner, like the regularization factors considered in PRNNRW.

Case Study: Particle Size Estimation of the Grinding Process
In order to test the effectiveness and advantages of our proposed algorithm in real applications especially in the industrial community, we address a practical industrial automation problem, that is particle size estimation of grinding process, to evaluate the four algorithms and finally demonstrate our RSC-KDE algorithm's merits on this engineering application [22]. In detail, the problem formulation as well as the specifications of our experimental setup are presented later. Some robustness analyses are conducted to see the impact of learning parameters on the robust SCN model.

Process description
Basically, the whole grinding process can be briefly reviewed as follows. To begin with, coarse fresh ore O F is fed continuously into the ball mill by the conveyor. Meanwhile, a certain amount of mill water Q F is fed by means of a pipe to maintain the proper pulp density. The knocking and tumbling action of the steel balls within the mill crushes the coarse ore to a finer size. After grinding, the mixed ore pulp including both coarser and finer particles is continuously discharged from the mill into the spiral classifier for classification. The ore pulp is diluted by dilution water Q D to improve the classification condition. Being in the control of spiral classifier, the pulp is separated into two streams namely overflow and underflow pulp. The underflow pulp with coarser particles is recycled back to the mill for regrinding, whilst the overflow pulp with finer particles, as product, is transported to the subsequent procedure (Fig. 5)

Experimental Setup
For the particle size estimation of the grinding process, the task of the data-driven compensation model is to realize the nonlinear mapping of process variables(i.e.,fresh ore feed rate O F , mill water flow rate Q F , and dilution water flow rate Q D ) to the unmodelled dynamic △r, namely, {△r} ← O F , Q F , Q D Consequently, the input vector of the improved RVFLN is denoted as x = [O F , Q F , Q D ] T , and the model output y refers to the estimation of the unmodelled dynamic △r. By combining the estimation of the unmodelled dynamic △r with the mechanism model outputr, the estimated value of the PSr is generated byr =r + △r. For each λ, we report the best results obtained by the 'most appropriate' setting of L (only for RVFL, Improved RVFL and PRNNRW), as well as the results of our proposed RSC-KDE algorithm in Table IV. For λ = 1, the test results of the four algorithms are shown in Fig. 8, where both the real and estimation values of the normalized particle size (of the whole test samples) are plotted. It can be clearly seen that RSC-KDE algorithm outperforms the other three randomized algorithms with better robustness in this case study.

Results and Discussion
Similar to the reported results in the previous subsection, apart from our proposed algorithm, we evaluate those three algorithms on different combination of λ and L. The averaged RMSE together with the standard deviation are plotted in Fig. 7, where for each λ the results of RVFL, Improved RVFL and PRNNRW are the most satisfied ones after trying different neural network architect (L = 30, 50, 100, 150, 200). It can be easily found that our proposed algorithm outperforms the other three methods in most cases. For λ = 0.1, the performance of PRNNRW is far away from the other examined algorithms, which indicates the fact that the resultant model by PRNNRW with λ = 1 does not have any generalization capability. Although the Improved RVFLN exhibits very close performance when the outlier percentage is relatively low, RSC-KDE algorithm has indeed shown the most robustness even at high outlier contamination rate. When λ = 0.5 and λ = 1, the performance of PRNNRW has been improved a lot compared with the case of λ = 0.1 but are still unaccepted for this problem. For both RVFL and Improved RVFLN, there seems no big difference between the results obtained by λ = 0.5 and λ = 1 respectively. Fig. 7 (d) demonstrates a preferable performance of PRNNRW where its test result at a rather high outlier percentage (for example ζ = 20%, 25%, 30%) is slightly better than RSC-KDE. However, our RSC-KDE algorithm still outperforms all those three algorithms at lower outlier percentage, which makes it more applicable in the practical particle size estimation problem of the grinding processes.  Figure 6: Test results of the four algorithms at ζ = 30% on particle size estimation problem

Robustness Analysis
Here we attempt to study the robustness of our proposed algorithm with different neural network architecture (L) and various setting of AO (alternating optimization) times (ν). The test performance with different combination of L and ν are reported in Table V, VI, VII, respectively corresponding to the case of ζ = 0%, ζ = 10%, and ζ = 30%. In Table V, it can be found that there is no big difference among the performance of different ν values when L is fixed. On the other hand, L = 50, , 60, 80 all can suit for this special case. However, in Table VI for ζ = 10%, the most appropriate setting of the architecture is L = 20 while the number of AO can be selected among ν = 3, 5, 8, 10, 12. Moreover, it seems not difficult to deduce that the model's performance with L = 20 is not only preferable but also can stay at a stable level when the number of AO is set equal or larger than 3. Similarly, in Table VII for ζ = 30%, the most appropriate setting of the architecture is L = 10 while the number of AO can be selected among ν = 5, 8, 10, 12, in which we can conclude that the resultant model with L = 10 can perform robustly with number of AO chosen as ν ≥ 5. Overall, the resultant model with fixed setting of the number of hidden neurons can contribute to a stable performance when the number of AO is set larger than 3. In comparison, the model's generalization capability (indicated by test performance) can be significantly influenced by the setting of L especially for cases then the training data are contaminated with high level outliers. Accordingly, some heuristic methods determining the neural network architecture deserve further study and more empirical studies on robustness analysis are needed.

Conclusions
This paper contributes a development of randomized method for robust data modelling, which is an important research topic in many domain applications. Stochastic configuration networks, which was recently proposed and reported in [23], are employed to build robust predictive models against  This work sets a basis for future researches on building robust learner models with random parameters. Extensions of the present algorithm to online version is being expected. In addition, the proposed algorithm in this paper can be easily extended to distributed learning framework for dealing with large scale robust modelling problems.