Abstract
ABSTRACT Constructing an interval model for nonlinear distributed parameter systems (DPSs) is challenging due to strong nonlinearity, spatiotemporal nature, and influence of noise. Although many methods have been used to construct the interval model, they are only effective in the modeling of lumped parameter systems, due to their inability to handle spatial information. In this paper, an interval modeling approach is proposed for strongly nonlinear DPS under noisy conditions. The spatiotemporal dataset is first divided into several subsets, and each subset is represented by the spatiotemporal least-squares support vector machine sub-model. Using these sub-models, a distribution modeling method is then developed to construct the mean and variance models of DPS. The confidence intervals are further derived based on these mean and variance models. The effectiveness of the proposed method is demonstrated using experiments on a practical curing thermal process and a long, thin rod in a reactor.
Abstract
INDEX TERMS Distributed parameter systems, noise, spatiotemporal LS-SVM model, mean, variance, confidence interval.
I. INTRODUCTION
Many industrial processes are inherently distributed in space and time [1] - [3] , [10] - [12] , such as distillation, a continuous stirring reaction and the processes involved in heat exchange used in the chemical industry, and the integrated circuit (IC) cure/reflow process used in the electronics packaging industry. Each of these processes is complex and nonlinear distributed parameter system (DPS) and is disturbed by all sorts of noise. Most of these processes require an interval model for uniformity measurement or process monitoring. For example, in the electronics packaging industry, the temperature uniformity has a significantly influences on the packaging quality [4] during the IC cure/reflow process. If the fluctuation of the temperature in the cure/reflow oven exceeds a reasonable range, this may result in the cracking and bubbling of the cured products [5] . However, constructing an interval model for DPS is challenging due to strong nonlinearity, spatiotemporal nature and noise influence.
I. INTRODUCTION
Generally, distributed parameter processes are described by using partial differential equations (PDEs) [6] - [9] , in which a variety of parameters, including both the input and output, can vary both temporally and spatially [6] , [13] . Many data-driven methods have been developed to model DPS. The spatial basis function in these methods was either determined beforehand, which usually using the Green's function [14] and the finite element basis function [15] , or was determined from a set of snapshots using the single value decomposition (SVD) method [16] , [17] or the Karkunen-Loève (KL) method [18] , [19] . The temporal dynamic model was then constructed by using Galerkin function [20] , neural network [21] , Volterra [22] , Hammersteinm [23] , Wiener [24] , autoregressive exogenous (ARX) model [25] and fuzzy model [26] . Recently, the spatiotemporal LS-SVM method [5] , [27] , [30] and the spatiotemporal extreme learning machine (ELM) method [28] , [29] were developed to model the strongly nonlinear DPS. Although the existing DPS modeling approaches have had a great deal of successful applications, they paid no attention to the interval modeling of the strongly nonlinear DPS. Also, they less considered the influence of noise to the DPS model. To our best knowledge, no method is found to address the interval modeling of DPS under noise.
I. INTRODUCTION
Many studies have contributed to the construction of the interval model for the lumped parameter systems. In the early stages of neural network (NN) modeling studies, only an average estimate of the reliability of the neural model was provided using the mean square model error on a test set [31] . The reliability of the neural model has been improved in the past decades by using the bootstrap methods [32] . Additionally, confidence intervals were derived using the least squares estimation based on the linearized NN model [33] . In order to address the construction of the interval estimation, the Bayesian approach, the Markov chain Monte Carlo method [34] , [35] , the leave-one-out cross-validation estimator [36] , [37] and the resampling method [38] , [41] were also used. Recently, both SVM [38] , [39] and ELM [28] , [40] were developed for interval modeling. Although these approaches have had a large number of successful applications, they are ineffective for the interval modeling of DPS due to their inability to handle space information. Thus, the development of an effective interval modeling method is necessary for the strongly nonlinear DPS with the presence of noise.
I. INTRODUCTION
In this paper, an interval modeling approach is proposed to model strongly nonlinear DPS under noise. In order to estimate the distribution of data with noise, the spatiotemporal datasets are divided into several subsets, and the spatiotemporal LS-SVM is used to represent each subset. The mean and variance models of DPS are then constructed based on the output of sub models. Furthermore, the confidence intervals are derived by using these mean and variance models. Actual thermal experiments were used to demonstrate the effectiveness of this modeling method.
II. PROBLEM DESCRIPTION
Many industrial processes are distributed parameter systems and disturbed by noise, which can be described by the following partial differential equation:
II. PROBLEM DESCRIPTION
The boundaries and the initial conditions can be described as follows:
II. PROBLEM DESCRIPTION
These processes represent a kind of practical industrial processes. Due to the physical realizability of these processes, their solution is existence. Generally, these processes require an interval model for uniformity measurement or process monitoring; however, achieving the interval model of this type of processes is difficult due to the following reasons:
II. PROBLEM DESCRIPTION
Time/space coupled system, the strong nonlinearity of the spatiotemporal dynamics, and unknown and boundary conditions. Influence of all sort of noise.
II. PROBLEM DESCRIPTION
Currently, almost all of the interval modeling methods are intended for the lumped parameter systems and are ineffective for DPS with the presence of noise because they neglect the influence of the spatial dynamics. Thus, a new approach needs to be developed to construct the interval model of this type of processes under noise.
III. CONFIDENCE INTERVAL PREDICTION FOR DISTRIBUTED PARAMETER PROCESSES
In order to determine the confidence intervals caused by both the spatial distribution and noise, an interval modeling approach is developed for the strongly nonlinear DPS with the presence of noise, as shown in Figure 1 . First, the influence of noise can be estimated by resampling the system into several sub spatiotemporal datasets. Then, the spatiotemporal LS-SVM is used to reconstruct the spatiotemporal dynamics of each sub spatiotemporal dataset. Using this spatiotemporal LS-SVM model, the information from unsampled time/space points may be obtained. Additionally, a distribution modeling method is developed, upon which the mean and variance models of DPS are derived. Finally, the confidence intervals are constructed by using the derived mean and variance models. In summary, the main ideas of the proposed method are listed as follows:
III. CONFIDENCE INTERVAL PREDICTION FOR DISTRIBUTED PARAMETER PROCESSES
Resampling-in order to estimate the influence of noise. The datasets are resampled into several sub datasets. Process estimation-in order to model spatiotemporal dynamics. The spatiotemporal LS-SVM model is used to represent the spatiotemporal process of the data subset. By this model, dynamics of any space position can be estimated. Distribution modeling-in order to construct the mean and variance models. The mean and variance models of the DPS are constructed based on all of the spatiotemporal LS-SVM model outputs. Confidence interval construction. Using the estimated mean and variance models, the confidence intervals for the original system is constructed.
A. DATA DIVISION AND SPATIOTEMPORAL LS-SVM MODEL
Collecting a set of spatiotemporal data from the DPS as the
A. DATA DIVISION AND SPATIOTEMPORAL LS-SVM MODEL
, where x j is the space location of the j-th sensor and t k is the k-th time, and s and L are the number of all sensors and the length of the sampling time, respectively. The sensors are divided into many sub groups according to the resampling theory [41] . As shown in Figure  2 , all of the sensors for training are randomly partitioned into I groups, where each group is defined as a subset that contains n sensors (n < s). This resampling technique, used to construct local models, can uncover more information from data, thus, it is effective to be used for interval modeling even in the presence of noise.
A. DATA DIVISION AND SPATIOTEMPORAL LS-SVM MODEL
The spatiotemporal LS-SVM [5] , [30] has the ability to model DPS since it considers both the nonlinear temporal dynamics and the nonlinear spatial properties of DPS. Thus, it is employed to construct the sub-model by using the data from the sensors in each subset. For the ith local subset, the following spatiotemporal LS-SVM model is developed to represent its model:
A. DATA DIVISION AND SPATIOTEMPORAL LS-SVM MODEL
Here, y i (x j , t k ) is the output of the spatial location x j at time t k of the ith local subset, φ i (·) is a spatial-related nonlinear projection function that is used to represent the nonlinear relationship between the spatial locations, w i (t k ) is the weight matrix, and b i (t k ) is the bias term. According to the SVM theory, the solution to Eq. (4) can be determined by solving the following optimization problem:
A. DATA DIVISION AND SPATIOTEMPORAL LS-SVM MODEL
Here, γ i is the regularization factor for the tradeoff between approximation accuracy and generalization, and e i (x j , t k ) is the modeling error. Solving Eq. (5) using the Lagrange multiplier α i j (t k ), the resulting LS-SVM model becomes:
A. DATA DIVISION AND SPATIOTEMPORAL LS-SVM MODEL
Here, α i j (t k ) and b i (t k ) can be solved using the Lagrange method, and
A. DATA DIVISION AND SPATIOTEMPORAL LS-SVM MODEL
is a kernel function, and the radial basic function (RBF) kernel is generally chosen as the space kernel function. This model (6) can estimate the temperature of the un-sampled space points or the untrained time points.
B. DISTRIBUTION MODELING AND CONFIDENCE INTERVAL CONSTRUCTION
Furthermore, a distribution modeling method is developed to construct the confidence intervals, as shown in Figure 3 . The main ideas are given as follows:
B. DISTRIBUTION MODELING AND CONFIDENCE INTERVAL CONSTRUCTION
Collecting all of the outputs of the spatiotemporal LS-SVM sub-models in order to calculate the mean and variance. At time t k , each spatiotemporal LS-SVM submodel built may predict the temperature values at all J spatial positions and all of the outputs for the submodels at J spatial positions will construct an array Y k (I × J ), where its elements are y i (x j , t k ) and i and j are the ith sub-model and the jth spatial position, respectively. Then, the mean and the variance of Y k (I × J ) are calculated. Collecting all of the mean and variance data for the whole time, the mean and variance model is then constructed and, using these models, the confidence intervals model is also constructed.
B. DISTRIBUTION MODELING AND CONFIDENCE INTERVAL CONSTRUCTION
The details of this approach are presented as follows. 
1) MEAN ESTIMATION
The mean at time t k is calculated as follows:
1) MEAN ESTIMATION
In order to estimate the mean at the untrained or the unsampled time points, the mean model must be established. Here, the following LS-SVM [43] is used to construct the mean model:
1) MEAN ESTIMATION
Where
1) MEAN ESTIMATION
is the mean at time t k , ω and c are the weight vector and the bias term, and ϕ is the unknown mapping function. The solution to Eq. (8) can be determined using the least square method with the temporal Lagrange multiplier β. The resulting LS-SVM mean model becomes:
1) MEAN ESTIMATION
WhereK (, ) is the RBF kernel function.
2) VARIANCE ESTIMATION
According to the definition of the variance, the variance at time t k is calculated as follows:
2) VARIANCE ESTIMATION
In order to estimate the variance at the untrained or the unsampled time points, the following LS-SVM model is developed to represent the variance model:
2) VARIANCE ESTIMATION
Here, and υ are the weight vector and the bias term, ς is the unknown mapping function, ψ is the unknown mapping function, and the model input isz
2) VARIANCE ESTIMATION
The solution of Eq. (11) can be determined using the least square method with the temporal Lagrange multiplier θ. Then, the resulting LS-SVM variance model becomes:
2) VARIANCE ESTIMATION
WhereK (, ) is the RBF kernel function.
3) CONFIDENCE INTERVAL CONSTRUCTION
Then, the derived mean model (9) and the variance model (12) are used to construct confidence intervals. According to the central limit theorem [42] , there has the following asymptotical relation:
3) CONFIDENCE INTERVAL CONSTRUCTION
Here, D −→ denotes the convergence in the distribution, N (0, 1) is the normal distribution with the mean equal to zero and the variance equal to 1. Thus, giving a confidence level equal to 100(1 − α)%, the confidence intervals of y(x, t) can be estimated as follows:
3) CONFIDENCE INTERVAL CONSTRUCTION
Where α is the confidence degree and z 1−α/2 is the critical value of the standard Gaussian distribution, which depends on the desired confidence level 100(1 − α)%. The higher the confidence level is, the greater the corresponding confidence intervals will be. Therefore, this model (14) may be used for uniformity measurement or the process monitoring of DPS with the presence of noise.
IV. EXPERIMENTAL VALIDATION
In this section, the proposed modeling method is validated by using two practical distributed parameter processes. These modeling processes are conducted on the MATLAB simulation software, which is commonly used in data analysis and modeling. The relative error is defined as follows:
A. CASE 1: SNAP CURING PROCESS
The curing thermal process in the snap oven [5] , [27] , [30] , as shown in Figure 4 , is used to validate the proposed modeling method. This curing system consists of a computer with dSPACE 1102 controller, PCLD-789 signal condition board, PCL-855 relay board, and a snap oven. The software for the snap oven control includes Matlab and dSPACE. Chips placed on the lead-frame cure best at a set temperature profile.
A. CASE 1: SNAP CURING PROCESS
In the thermal process, we adjust heat in the oven by using four heaters (h1-h4). The temperature distribution inside the chamber is required for the fundamental analysis and better curing quality. For modeling this system, the J-type thermocouples (Range: -40 • C ∼375 • C, the maximal measurement error: 1.5 • C) placed on the lead-frame are used to measure temperature, as shown in Figure 4 (b). For this interval modeling, random input signal was used to power the heaters in order to excite the thermal process. The input signal of the heater h2 is shown in Figure 5 . Since the temperature in the oven changes slowly, a sampling interval t = 10 second is set. In the experiment, according to the practical experience and prior process knowledge, random noise with the zero mean and the variance equal to 0.05 is added into experimental data from sensors. The sensors (s1-s16) are used for training and the space points (c1-c9) are used for testing, as shown in Figure 4 . According to the resampling theory [41] , the training sensors (s1-s16) were randomly partitioned into one hundred groups and each group included ten sensors. Then, the sensor data for each group was used to train the spatiotemporal LS-SVM sub-model. After all of the sub-models were built, the mean and the variance models were then constructed using the collected mean and variance data. Then, the interval model was constructed by using the mean and variance models.
A. CASE 1: SNAP CURING PROCESS
Then, the mean and variance models are verified using experiments. The mean of the real output and the model output for training and prediction are shown in Figures 6 and 7 . The variance of the real output and the model output for training and prediction are shown in Figures 8 and 9 . The relative error traces are also shown in Figures 6∼9. From these Figures, it is obvious that the mean and variance models fit and predict the practical mean and variance well. Thus, this modeling method is effective for modeling DPS with the presence of noise.
A. CASE 1: SNAP CURING PROCESS
The interval models are also validated with experiments. According to the user's requirement, a 95% confidence level was used to derive the confidence interval models. Data from all of the training sensors and nine test points (c1-c9) were used to check the interval models. As shown in Figure 10 , the real output from all of the training sensors are represented by black dotted lines and the output of the test points are represented by mauve dotted lines, and the output of the interval models are represented by red and blue lines. It is clear from this figure that the interval models are effective at modeling DPS with the presence of noise, because the real output from all of the training sensors and the output of test points are almost within the interval model, as well these interval models are able to successfully track the dynamics of the DPS under noisy conditions. Then, three common interval modeling methods, a LS-SVM interval modeling method [28] , an extreme learning machine (ELM) interval modeling method [40] and a neural network (NN) interval modeling method [33] , are employed for comparison. If the data for any sensor at any time falls outside of the interval models, the temperature of this sensor is determined to be outside of the interval models. The comparison results are shown in Table 1 . From this Table, the practical confidence level of the proposed method is closer to the 95% desirable confidence level as compared to other methods. Thus, the proposed method is superior to these existing methods, because it considers the nonlinear spatiotemporal dynamics and can obtain the interval model even in the presence of noise, but the other common interval modeling methods are only for time dynamics without consideration of space dynamics.
B. A CATALYTIC ROD
A long thin rod in a reactor that is typically found in the transport-reaction process in the chemical industry [6] , [25] , as shown in Figure 11 , is then used to verify the proposed model.
B. A CATALYTIC ROD
During the transport-reaction process, the reactant A is fed into the inlet of the rod reactor, and then a zero-th order exothermic catalytic reaction takes place in the rod, which transforms the conditions of reactant A into product B. Finally, the product and the surplus come out at the outlet. Since the reaction is exothermic, a cooling medium is in contact with the rod for cooling.
B. A CATALYTIC ROD
Under the assumption of a constant density, heat capacity, and a constant conductivity of the rod, as well as a constant temperature at both sides of the rod and an excess of reactant in the furnace, the following partial differential equation is used to describe the spatiotemporal evolution of the rod:
B. A CATALYTIC ROD
It is subject to the Dirichlet boundary and the following initial conditions:
B. A CATALYTIC ROD
where y(x, t), β T , γ , β u , b(x), u(t) and d(x, t) represent the temperature in the reactor, the heat of the reaction, the activation energy, the heat transfer coefficient, the actuator distribution, the manipulated input and the random process noise, respectively. The thermal process parameters are set as β T = 50, β u = 2, and γ = 4. Here, u(t) = [u 1 (t), . . . , u 4 
(t)]
T is the manipulated input with the spatial distribution function:
(t)]
and H (·) is the standard Heaviside function. In this modeling process, the manipulated input u i (t) = 1.1+5 sin(t/10+i/10) (i = 1, . . . , 4) is used to excite the system. The sampling interval t is 0.01 and 501 data points were collected and the temperature was measured using twenty-seven sensors placed on the rod. In this modeling, according to the practical experience and prior process knowledge, random noise with the zero mean and the variance equal to 0.0085 is added into data produced from Eq. (15) . An interval model should be constructed for the entire process in order to measure and predict the temperature uniformity. In this experiment, twenty-seven sensors were used to train the interval model and nine test points were selected to test this built model. For this interval modeling, all of the training sensors were randomly partitioned into one hundred groups and each group had seventeen sensors. Then, the sensor data from each group is used to train the spatiotemporal LS-SVM sub-model. After all of the sub-models are built, the mean and variance models are then constructed using the collected mean and variance data. Then, the interval model is constructed using the mean and variance models.
(t)]
Then, the mean and variance models are verified using experiments. The mean of the real output and the model output for training and prediction are shown in Figures 12 and 13 . The variance of the real output and the model output for training and prediction are shown in Figures 14 and 15 . From these Figures, it is clear that the mean and variance models fit and predict the practical mean and variance well. Thus, this modeling method is effective for modeling DPS under noisy condition.
(t)]
Furthermore, the interval models were validated using experiments. According to the experimental requirement, an 84% confidence level was used to derive the confidence interval models. Data from all of the training sensors and nine test points were used to check the interval models. The real output from all of the training sensors are represented by black dotted lines, the output of the testing points are represented by mauve dotted lines, and the output of the interval models are represented by red and blue lines, as shown in Figure 16 . It is clear from this Figure that the interval models are effective for modeling DPS with noise, because the real output from all of the training sensors and the output of test points are almost within the interval model, as well these interval models are able to successfully track the dynamics of the DPS under noisy condition.
(t)]
Then, three common interval modeling methods, a LS-SVM interval modeling method [28] , an extreme learning machine (ELM) interval modeling method [40] and a neural network (NN) interval modeling method [33] , are employed for comparison. If the data for any sensor at any time falls outside of the interval models, the temperature of this sensor is determined to be outside of the interval models. The comparison results are shown in Table 2 . From this Table, the practical confidence level of the proposed method is closer to the 85% desirable confidence level as compared to these existing methods. Thus, the proposed method is superior to the existing method, because it considers the nonlinear spatiotemporal dynamics and can obtain the interval model even in the presence of noise, but the other common interval modeling methods are only for time dynamics without consideration of space dynamics.
V. CONCLUSION
An interval modeling approach is proposed for the modeling of a complex DPS under noisy condition. The results indicate VOLUME 6, 2018 that the distribution modeling method is able to accurately represent the mean and variance of the DPS. The derived confidence interval models could also effectively track the dynamics of the DPS under noisy condition. Ultimately, this will benefit the uniformity measurement and/or process monitoring. Furthermore, the effectiveness of the proposed method was evaluated using both the experiment and the simulation. These results demonstrate that the developed interval modeling method can effectively estimate the dynamic behavior of a DPS under noisy condition. In future, we will contribute to the interval modeling of a time-varying DPS under all sorts of noise.

{'paper_id': '50778812', 'title': 'Construction of Confidence Intervals for Distributed Parameter Processes Under Noise', 's2_url': 'https://api.semanticscholar.org/CorpusID:50778812', 'source': 'S2ORC'}