IEEE TRANSACTIONS, IN REVIEW

1

Modeling The Stable Operating Envelope For
Partially Stable Combustion Engines Using Class
Imbalance Learning

arXiv:1306.5702v1 [] 24 Jun 2013

Vijay Manikandan Janakiraman, XuanLong Nguyen, Jeff Sterniak, and Dennis Assanis

Abstract—Advanced combustion technologies such as homogeneous charge compression ignition (HCCI) engines have a
narrow stable operating region defined by complex control
strategies such as exhaust gas recirculation (EGR) and variable
valve timing among others. For such systems, it is important
to identify the operating envelope or the boundary of stable
operation for diagnostics and control purposes. Obtaining a
good model of the operating envelope using physics becomes
intractable owing to engine transient effects. In this paper, a
machine learning based approach is employed to identify the
stable operating boundary of HCCI combustion directly from
experimental data. Owing to imbalance in class proportions in
the data, two approaches are considered. A re-sampling (undersampling, over-sampling) based approach is used to develop
models using existing algorithms while a cost-sensitive approach
is used to modify the learning algorithm without modifying
the data set. Support vector machines and recently developed
extreme learning machines are used for model development and
results compared against linear classification methods show that
cost-sensitive versions of ELM and SVM algorithms are well
suited to model the HCCI operating envelope. The prediction
results indicate that the models have the potential to be used
for predicting HCCI instability based on sensor measurement
history.
Index Terms—HCCI engine, Stable Boundary Learning, Costsensitive Classification, Class Imbalance, Extreme Learning Machine, Support Vector Machine, Misfire Prediction.

I. I NTRODUCTION
HCCI engines have been studied in the last decade owing to
their ability to reduce emissions and fuel consumption significantly compared to traditional spark ignition and compression
ignition engines [1], [2], [3]. The highly efficient operation
of HCCI is achieved using advanced control strategies such
as exhaust gas recirculation (EGR) [4], variable valve timings
(VVT) [5], intake charge heating [6] among others. However,
complex manipulation of the system results in a highly nonlinear behavior [7] and narrow region of stable operation [8],
[9].
In order to develop controllers and operate the system in a
stable manner, it is imperative that the operating envelope of
the system be determined. In general, the operating envelope
can be defined as a region in the input space (of permissible
Vijay Manikandan Janakiraman is with the Department of Mechanical
Engineering, University of Michigan, Ann Arbor, MI, USA e-mail: vijai@umich.edu.
XuanLong Nguyen is with the Department of Statistics, University of
Michigan, Ann Arbor, MI, USA.
Jeff Sterniak is with Robert Bosch LLC, Farmington Hills, MI, USA.
Dennis Assanis is with the Stony Brook University, NY, USA.

values of system actuators for different thermal conditions of
the engine) that results in a stable operation of the engine.
Knowledge of the operating envelope is crucial for designing
efficient controllers for the following reasons. The developer
can get insights on the actuator extremes (for example, minimum and maximum fuel injection rates at a given speed
and load condition) of the engine especially during transients
[4]. Such information can be used to enforce constraints on
the control variables for desired engine operation. Also, the
operating envelope model can act as a filter to perform system
identification by eliminating excitations that might lead the
system to be unstable. Further, the model can be used to alarm
the onboard diagnostics if the engine is about to misfire [10]
owing to changes in system or operating conditions.
HCCI engines are very complex systems involving chemical
kinetics and thermal dynamics which requires high-fidelity
modeling using numerical simulations for capturing accurate combustion behavior [11], [12], [13], [14]. Such an
approach is computationally expensive particularly when there
are several influencing variables. The situation is worsened
by the transient effects, i.e., the variables along with its time
history affects the system behavior, a characteristic of dynamic
systems. The operating envelope that depends on the system
variables along with its time history [4] can be considered
a dynamic system, and capturing this time varying behavior
using conventional methods becomes intractable. Hence an
approach using machine learning is considered in this paper
where time series data from the sensors can be used to model
the operating envelope of the HCCI engine.
The problem of identifying the operating envelope using
experimental data reduces to a binary classification problem.
Experimental data from the engine sensors can be labeled as
belonging to stable or unstable operation and a decision boundary can be modeled using existing classification algorithms. A
support vector machine algorithm was used to identify the
operating envelope of a GDI engine [15] but the boundary
was assumed to be a static system and time history was not
considered resulting in a simple binary classification problem.
However, for HCCI engines whose combustion behavior is
influenced by EGR from previous cycles, the importance of
considering the time history of measurements becomes important [4]. Designing a classifier based on dynamic HCCI data
is one of the objectives of this paper. Also, the experimental
data consists of a large set of stable class data with limited
unstable class data, as misfiring the engine is undesirable for
the emission control hardware, a regular classification might

IEEE TRANSACTIONS, IN REVIEW

2

not be an appropriate solution since the decision boundary
would be biased to one class of data, resulting in over-fitting.
Therefore, the other objective of this paper is to perform
imbalanced class learning on the HCCI engine data. The
following two approaches have been compared
1) Heuristic re-sampling of data: apply preprocessing methods such as under-sampling and over-sampling of data
to get a balanced data set.
2) Cost-sensitive approach: modify the objective function
of the learning system to weigh the minority class data
more heavily.
Three classification models including support vector machines
(SVM), extreme learning machines (ELM) and logistic regression (LR) have been applied for the HCCI boundary learning
problem. The models are compared for generalization accuracy, memory requirement in terms of number of parameters
used to store the model and potential for online learning.
The paper is organized as follows. A brief background on
the classification algorithms along with cost-sensitive modifications is given in section II. The HCCI engine experiments
and data processing are briefed in section III with the envelope modeling and prediction results discussed in section IV
followed by conclusions in section V.
II. C LASSIFICATION A LGORITHMS

Consider the data set {(x1 , y1 ), ..., (xN , yN )} ∈ X , Y
where X denotes the space of the input features (let X = Rn )
while Y takes values in {-1,+1} and N denotes the number
of observations. The goal of the classification algorithm is
to model the underlying boundary separating the data by
minimizing a risk function R(w) with respect to the model
parameters w.
R(w) =

N
1
1 X
L(yi − yˆi (x|w)) + wT w
N i=1
2

(1)

Here, R(w) has two components - the empirical risk minimizing the training error and the structural risk minimizing the
model parameters, L represents a loss function and ŷ(x|w, b)
represents the model prediction, whose structures are given
by the learning algorithm (see following subsections). The
algorithms considered in this study are logistic regression,
support vector machines and extreme learning machines. Each
of the algorithm is unique in its formulation, loss function
used, convergence rates, computation demand, prediction accuracy and potential for online learning. However, the main
criteria used for evaluation in this study are prediction accuracy, number of parameters used for modeling and potential
for online implementation for the HCCI engine system. The
HCCI classification problem involves identifying the boundary
separating the input space that result in a stable or unstable
operation. Also, when the engine misfires, the excitation
command is changed to attempt a stable operation [16]. This
results in a class imbalance learning problem as the number
of unstable class data is significantly smaller than the number
of stable class data.

A. Class Imbalance Learning
Class imbalance learning (CIL) is encountered during situations when the number of instances of one class is very
different from the number of instances in others. In a binary
classification problem, the class where the number of observations are large in number is referred to as the majority class
(labeled +1) while the other class is referred to as the minority
class (labeled -1). Imbalanced data sets need careful attention, as machine learning (typically an optimization problem)
causes the decision boundary to be favorable to the majority
class data while ignoring the minority class data [17], [18].
Several solutions have been proposed to handle CIL problems including re-sampling the data where the minority class
can be duplicated to be in proportion with the majority class
(referred to as over-sampling) or some majority class data is
removed to match proportions with the minority class (referred
to as under-sampling). Although both sampling methods aim to
artificially obtain a balanced data set, under-sampling is prone
to loss of majority class information while over-sampling is
prone to over-fitting [17], [18]. Algorithm level modifications
are also common which include cost-sensitive learning that
weights the minority class data more than the majority class
data in the optimization objective function. Other methods
such as adjusting the decision threshold, one-class learning
etc. are available in literature, but the focus of this paper is
the comparison of under-sampling, over-sampling and costsensitive methods.
B. Logistic Regression
Logistic regression (LR) is a classical linear classifier that
proves to be effective especially for large data set problems
owing to its computational efficiency. LR makes use of a
logistic function given by equation (2) which confines the
output of the function to lie between 0 and 1. Unlike the linear
regression model which solves a least squares problem with
a squared loss function, LR solves a nonlinear optimization
problem using a logistic loss function (see Figure 11 in
appendix A). The logistic loss function is particularly attractive
for classification because the algorithm does not penalize the
correctly classified points (at large positive margin in Figure
11) as much as the squared loss improving convergence.
ψ(x) =

1
1 + e−x

(2)

The conditional probability of estimating y from x can be
expressed in terms of the model parameters β = [β0 β1 ]T as
P (Y = y|X = x) =

1
1+

T
e−y(β1 x+β0 )

(3)

where X and Y represent the input and output random
variables. The goal of logistic regression is to determine β such
that P (Y |X, β) is maximized using the following optimization
problem (see appendix B)
β ∗ = arg min
β

N
X
i=1



T
log 1 + e−y(β1 x+β0 )

(4)

IEEE TRANSACTIONS, IN REVIEW

3

The equation (4) is nonlinear in β and can be solved by simple
iterative methods [19]. The LR decision hypothesis is given
by
f (x) = sgn(β1T x + β0 )
(5)
where

(
sgn(x) =

1
−1

x>0
x ≤ 0.

(6)

C. Support Vector Machines
Support Vector Machines (SVM) involves determining the
boundary that maximizes the margin between the data based
on a hinge loss Lhinge (w, b) = max(0, 1 − yf (x)), where
yf (x) gives the margin [20]. This translates to finding the
optimal model parameters (w∗ , b∗ ) by solving the following
optimization problem
N

min

w,b,ζi

(
subjected to

X
1 T
w w+C
ζi
2
i=1

(7)

yi [hw, φ(xi )i + b] ≥ 1 − ζi
ζi ≥ 0

(8)

for i = 1, .., N . Here ζi represents the slack variable for data
observation i, C represents the cost penalty hyper-parameter.
The slack variables ζi are required in order to allow for
misclassifications in a noisy overlapping binary data set that
cannot be completely separated by a decision boundary. The
input vectors x are mapped onto a higher dimensional space
using the function φ. By making this transformation, the
nonlinear data is aligned linearly in the high dimensional space
where SVM finds a maximum margin separating hyperplane.
The transformation is performed implicitly using a kernel
matrix K(xi , xj ) = [k(xi , xj )]i,j where k(xi , xj ) could be
any function satisfying Mercer’s condition [20]. The gaussian
kernel function (equation (9)) is used in this paper. More
details on SVM formulation can be found in [20].
2

k(xi , xj ) = e−σkxi −xj k , σ > 0

(9)

The convex constrained optimization problem in equation
(7) is in the primal form, and the variables w, b and ζi
are referred to as primal variables. The primal problem is
converted to a dual formulation in equation (10) and solved
for the dual variables αi .


N X
N
N

 1X
X
max −
yi yj αi αj K(xi , xj ) +
αi
(10)
αi  2

i=1 j=1

i=1

(P
N
subjected to

i=1 αi yi = 0
0 ≤ αi ≤ C

for i = 1, .., N . The SVM hypothesis is given by
!
N
X
f (x) = sgn
αi yi K(xi , x) + b

(11)

(12)

i=1

The above formulation is not designed for an imbalanced data
set where the majority class data outnumbers the minority class
data. A cost-sensitive version of the SVM algorithm is used

in such cases, where the cost penalty parameter C in equation
(7) is modified to weigh more to the penalties of the minority
class data compared to the majority class data [21], [18]. All
implementations of SVM are done using LibSVM [21]. The
cost modification can be performed as follows
(
C
majority class data
(13)
Ci =
C(r × f ) minority class data
where r represents the ratio of number of majority class data
to the number of minority class data and f represents a scaling
factor to be tuned for a given data set.
D. Extreme Learning Machines
Extreme Learning Machine is an emerging learning
paradigm for multi-class classification and regression problems
[22], [23]. The highlight of ELM is that the training speed
is extremely fast (or computationally inexpensive). The key
enabler for ELM’s training speed is the random assignment
of input layer parameters which do not require adaptation to
the data. In such a setup, the output layer parameters can be
determined analytically using a least squares approach. Some
of the attractive features of ELM [22] include the universal
approximation capability of ELM, the convex optimization
problem of ELM resulting in the smallest training error
without getting trapped in local minima, closed form solution
of ELM eliminating iterative training and better generalization
capability of ELM. ELM training involves solving the following optimization problem

min kHW − Y k2 + λkW k2
(14)
W

H T = ψ(WrT x(k) + br ) ∈ Rnh ×1

(15)

where λ represents the regularization coefficient, Y represents
the vector of outputs or targets, ψ represents the hidden
layer activation function (a sigmoidal function takes the same
structure as (2)) and Wr ∈ Rn×nh , W ∈ Rnh ×1 represents
the input and output layer parameters respectively. Here, nh
represents the number of hidden neurons of the ELM model,
H represents the hidden layer output matrix. The matrix Wr
consists of randomly assigned elements that maps the input
vector to a high dimensional feature space while br ∈ Rnh is
a bias component assigned in a random manner similar to Wr .
The number of hidden neurons determines the dimension of
the transformed feature space. The elements can be assigned
based on any continuous random distribution [23] and remains
fixed during the learning process. Hence the training reduces
to a single step calculation given by equation (16). The ELM
decision hypothesis can be expressed as in equation (17). It
should be noted that the hidden layer and the corresponding
activation functions give a nonlinear mapping of the data,
which if eliminated, becomes a linear least squares (Linear
LS) model and is considered as one of the baseline models in
this study.
−1 T
H Y
(16)
W ∗ = H T H + λI

f (x) = sgn W T [ψ(WrT x + br )]
(17)

IEEE TRANSACTIONS, IN REVIEW

The above ELM formulation is not designed to handle
imbalanced or skewed data sets. As a modification to weigh
the minority class data more, a simple weighting method can
be incorporated in the ELM objective function (14) as

min (HW − Y )T Γ(HW − Y ) + λW T W
(18)
W


γ1 0 . . 0
 0 γ2 . . 0 

Γ=
 .
. . . 0 
0 0 . . γN
(
1
majority class data
(19)
γi =
r × f minority class data
where Γ represents the weight matrix, r represents the ratio of
number of majority class data to number minority class data
and f represents a scaling factor to be tuned for a given data
set. This results in the training step given by equation (20)
and hypothesis given by equation (21).
−1 T
W ∗ = H T ΓH + λI
H ΓY
(20)

f (x) = sgn W T [ψ(WrT x + br )]
(21)
III. HCCI ENGINE AND DATA PROCESSING
For the purpose of identifying the stable operating envelope
of HCCI engine, transient experiments are performed by exciting the engine and recording time sequences of engine variables. In this section, the HCCI engine system and experiments
performed are briefly explained followed by a methodology of
labeling the data suitable for classification.
A. HCCI System and Experimentation
The concerned system of interest is a gasoline HCCI engine
with a variable valve timing system. The engine specifications
are listed in Table [24]. A schematic of the experimental setup
and instrumentation is shown in Fig. 1. HCCI is achieved
by auto-ignition of the gas mixture in the cylinder. The fuel
is injected early and given sufficient time to mix with air
forming a homogeneous mixture. A large fraction of exhaust
gas from the previous combustion cycle is retained to elevate
the temperature and hence the reaction rates of the fuel and
air mixture. The variable valve timing capability of the engine
enables trapping suitable quantities of exhaust gas in the
cylinder.
The engine can be controlled using precalculated inputs
such as injected fuel mass (FM in mg/cyc), crank angle at
intake valve opening (IVO), crank angle at exhaust valve closing (EVC), crank angle at start of fuel injection (SOI). Other
important physical variables that influence the performance of
HCCI combustion include intake manifold temperature Tin ,
intake manifold pressure Pin , mass flow rate of air at intake
ṁin , exhaust gas temperature Tex , exhaust manifold pressure
Pex , coolant temperature Tc , fuel to air ratio (FA) etc. The
engine performance metrics are given by combustion phasing
indicated by the crank angle at 50% mass fraction burned
(CA50), combustion work output indicated by net mean effective pressure (NMEP). Both CA50 and NMEP are determined

4

TABLE I: Specifications of the experimental HCCI engine
Engine Type
Fuel
Displacement
Bore/Stroke
Compression Ratio
Injection Type

Valvetrain

HCCI strategy

4-stroke In-line
Gasoline
2.0 L
86/86 mm
11.25:1
Direct Injection
Variable Valve Timing with
hydraulic cam phaser having
119 degree constant duration
defined at 0.25mm lift, 3.5mm peak
lift and 50 degree crank angle
phasing authority
Exhaust recompression
using negative valve overlap

from the high speed in-cylinder pressure measurements. The
above variables at present time instant k along with their time
histories are considered as inputs to the model (see section IV,
equation (23)). For further reading on HCCI combustion and
related variables, please refer [25].
As mentioned in section I, the goal of this paper is to
identify the HCCI operating boundary in transient operation.
This requires an appropriate experiment design to obtain
transient data from the engine. A set of transient experiments
is conducted at a constant speed of 2500 RPM and naturally
aspirated conditions by varying FM, IVO, EVC and SOI in
a uniformly random manner. The experiments are conducted
and data recorded using specialized engine rapid prototyping
hardware. An amplitude modulated pseudo-random binary
sequence (A-PRBS) has been used to design the excitation
signals. A-PRBS enables exciting the engine at different
amplitudes and frequencies suitable for the identification problem considered in this work. The data is sampled using the
AVL Indiset acquisition system where in-cylinder pressure
is sensed every crank angle while NMEP, CA50 and Rmax
are determined on a per-combustion cycle basis. More details
on HCCI combustion and experiments can be found in [16].
The data is pre-processed and labeled to identify stable and
unstable observations as explained in section III-C.
B. HCCI Instabilities
A subset of the data collected from the engine is shown
in Figure 2 where it can be observed that for some combinations of the inputs (left figures), the HCCI engine misfires
(seen in the right figures where IMEP drops below 0 bar).
HCCI operation is limited by several phenomena that lead to
undesirable engine behavior. As described in [26], the HCCI
operating range is conceptually constrained to a small region
of permissible unburned (pre-combustion) and burned (postcombustion) charge temperature states. As previously noted,
sufficiently high unburned gas temperatures are required to
achieve ignition in the HCCI operating range without which
complete misfire will occur. If the resulting combustion cannot
achieve sufficiently high burned gas temperatures, commonly
occurring in conditions with low fuel to diluent ratios or late
combustion phasing, various degrees of quenching can occur
resulting in reduced work output and increased hydrocarbon

IEEE TRANSACTIONS, IN REVIEW

5

Air in
9

7

VVT

3

15

8

Turbocharger
13

6

11

Intake Manifold

Throttle

1

10

12

Exhaust out

Exhaust Manifold

5

2
4

Piston

1.

Coolant
thermocouple

2.

Exhaust manifold
thermocouple

Crank shaft
3.

Exhaust manifold
pressure transducer

4.

Post-turbine
thermocouple

14

5.
6.
7.

Lambda sensor
Air mass flow
sensor
Spark plug

0𝐶
0

𝐶

𝑚𝑚𝑚𝑚
0

𝐶

−

𝑘𝑘/ℎ
−

8.

Fuel injector

9.

Cam phaser

10,
11.

Cylinder 1 Runner
T and P

12.

Intake manifold
thermocouple

13.

Intake manifold
pressure transducer

14.

RPM sensor

15.

In-cylinder pressure
transducer

𝑚𝑚𝑚𝑚
−

0𝐶 ,

𝑚
𝑚𝑚𝑚𝑚
0

𝐶

𝑚𝑚𝑚𝑚
𝑅𝑅𝑅
𝑏𝑏𝑏

IMEP (bar)

11
10
9
8
5400

5600

5800

6000

120
100
80
5400

5600

5800

6000

−80
−90

R

−110
5200

5400

5600

5800

0

6000

5400

5600

5800

6000

5400

5600

5800

6000

5400

5600

5800

6000

5400

5600

5800

6000

60
40
20
0
−20

max

−100

2

80

(bar/deg)

5200

4

5200

CA50 (deg aTDC)

7
5200

5200

10
5
0
5200
2

360
340

λ (−)

SOI (deg aETDC)

EVC (deg aCTDC)

IVO (deg aCTDC)

Fuel Mass (mg/cyc)

Fig. 1: A schematic of the HCCI engine setup and instrumentation (only relevant instrumentation is shown).

320

1.5

300
280
5200

1
5400

5600

5800

6000

5200

Fig. 2: A-PRBS inputs and outputs showing misfire regions.

and carbon monoxide emissions. Under some conditions, this
may lead to high cyclic variation due to the positive feedback
loop existing through the trapped residual gas [27], [28].
Operation with high burned gas temperature, although stable
and commonly reached at higher fueling rates where the fuel
to diluent ratio is also high, yields high heat release and thus
pressure rise rates that may pose challenges for engine noise
and durability constraints. A discussion of the temperatures at
which these phenomena occur may be found in [26].
In this paper, the considered instabilities include those
modes with high cyclic variability and those with complete

misfire characterized by zero work output that can be readily
identified through the two aforementioned cylinder pressurebased combustion features. The other phenomena could be
included with the availability of additional sensing capability
or analysis methods, e.g. fast response Flame Ionization Detection exhaust sampling equipment and detailed combustion
noise analysis. Finally, it must be noted that control of these
burned and unburned gas states, and therefore the potential
for undesirable combustion cycles, in a recompression HCCI
engine is very much a function of the engine control variables.
For instance, the EVC timing will determine the trapped
residual mass that will be present in the upcoming cycle,
while the IVO affects both the mass of incoming air and
the state of the charge during the compression stroke leading
up to the autoignition. The combination of IVO and EVC
(see Fig. 3) define a negative valve overlap (NVO) period
where exhaust gas from the previous cycle is trapped and
compressed. A larger NVO period would necessarily yield a
higher trapped residual mass that would tend to increase the
charge temperature and advance CA50. Likewise, the timing
and mass of the fuel injection event can significantly impact
the charge temperature by changing the thermodynamic properties and air-fuel ratio of the charge present during NVO. The
relatively high temperatures present during NVO can even lead
to reactions of the fuel that will impact the temperature and
chemical composition of the charge. Successful combustion of
charge with a higher FM will tend to yield higher residual gas
temperatures, thereby advancing CA50 in the following cycle.
Likewise an earlier SOI in NVO will tend to increase charge
temperatures and reduce the ignition delay of the charge,
thereby advancing CA50. As such, an improper combination
of control inputs (IVO, EVC, FM and SOI) in HCCI engines

IEEE TRANSACTIONS, IN REVIEW

cTDC

6

eTDC

cTDC

eTDC

35
Cycle
(k−1)

3.5

Cycle
(k+1)

Cycle
(k)

20

SOI

15

Data
Record

2.5

3
2.5

2

2

1.5

25
IMEP (bar)
p

In−cylinder Pressure
(bar)

30

1.5

1

1

0.5

0.5

0

0

0.5

-0.5

10

EVC

IVO

Misfire
limit

1

-1

3750

3800

3850

3900

5

3950

4000

4050

4100

4150

4200

𝒌 𝒌+𝟏

4250

Engine cycles

0
25

0

180

360

540
720
Crank Angle (deg)

900

1080

15

20

Fig. 3: HCCI combustion cycle definition showing valve events
(IVO and EVC), injected fuel mass (FM) and start of injection
(SOI). Negative valve overlap can be seen as a smaller peak
at eTDC.

CA50 (deg aTDC)

−5
−180

15

10

10
5

0

5
0

0

𝒌

-5
-10
2400

2450

2500

2550

2600

2650

2700

2750

𝒌+𝒑

Engine cycles

Fig. 4: Illustration showing labeling of unstable observations.
has the potential to shift operation from stable combustion
to combustion with excessive heat release rates, high cyclic
variability or misfires in a single cycle.
IMEP (bar)

The goal of the learning algorithm is to classify the input
space into stable (future HCCI cycles are stable) or unstable
(future HCCI cycles misfire or have high variability) engine
behavior. The input space includes sensor measurements until
the present time instant while the indicator label depends on
the future. For this purpose, the data is labeled as follows. If
either of the following two conditions are met, then the data
at time instant k is labeled to be unstable (see Fig. 4) with a
label value -1.
1) an input (control inputs and past engine measurements
up to an order of Nh ) at cycle k results in an IMEP of
less than 0.1 bar (chosen misfire limit) for any cylinder
at cycle k + 1.
2) an input at cycle k results in a high variance of CA50
(any cylinder) for cycles k + 1 to k + p.
Only the first unstable data is considered in a sequence of
unstable measurements. The labeling of stable data is as follows. A window of Nw = 2p combustion cycles is considered
(see Fig. 5). If the data at cycle k is obtained as a result of
stable operation in the past p cycles as well as results in stable
operation in the next p cycles, it is labeled stable with a label
value of +1. If the time history of Nh is considered, then
the data at cycle k along with the previous Nh samples are
considered as inputs. If Nh is a large value, then the window
length Nw can be increased accordingly. In this study, Nw and
Nh correspond to 10 and 2 respectively.
IV. M ODELING THE S TABLE O PERATING E NVELOPE OF
HCCI E NGINE
The HCCI operating envelope is a function of the engine
control inputs and engine variables such as temperatures and
pressures. Also, the envelope is a dynamic system and hence
a predictive model requires the measurement history up to an
order of Nh . The dynamic classifier model can be given by
(22)

𝒌−𝒑

𝒌

𝒌+𝒑

3200

3300

3400

3500

3600

3700

3800

3900

4000

3200

3300

3400

3500
3600
EEngine
i cycles l

3700

3800

3900

4000

3
2.5
2
1.5

Misfire limit

10
CA50 (deg aTDC)

C. Data Preprocessing and Labeling

ŷk+1 = sgn(f (xk ))

3.5

8
6
4
2
0

Fig. 5: Illustration showing labeling of stable observations.
where ŷk+1 indicates model prediction for the future cycle
k + 1, f can take any structure depending on the learning
algorithm and xk is given by
xk = [IV O, EV C, F M, SOI, Tin , Pin , ṁin ,
Tex , Pex , Tc , F A, IM EP, CA50]T

(23)

at cycle k upto cycle k − Nh + 1.
A. Model Selection
In this section, classification algorithms are developed based
on Linear Regression (LS), Logistic regression (LR), SVM and
ELM models. SVM and ELM models have variants based on
under-sampling, over-sampling or no-sampling (regular) the
data set or the cost-sensitive version. The linear models (LR
and LS) are compared as baselines and have their respective
variants. The engine measurements and their time histories
(defined by xk ) are considered inputs while the stability
labels are considered outputs. The measured variables such
as FM, IVO, EVC, SOI, Tc , Tin , Pin , ṁin , Tex , Pex , NMEP,
CA50 and FA along with 2 cycles of history constitute the
feature vector (input dimension n = 39). The measurement set

IEEE TRANSACTIONS, IN REVIEW

7

consists of about 17000 observations out of which about 6400
observations are sampled as training set while about 10200
observations sampled as testing set. The ratio of number of
majority class data to number minority class data (r) for the
training set is 17.5 and for the testing set is 16.7.
For the class imbalance problem considered here, a typical
error metric like the overall misclassification rate cannot be
used as it would find a classifier that ignores the minor class
inaccuracies. Hence the following accuracy metric for skewed
data sets are considered. Let T P and T N represent the total
number of positive and negative class data classified correctly
by the classifier. If N + and N − represent the total number of
positive and negative class data respectively, the true positive
rate (TPR) and true negative rate (TNR) and the total accuracy
of the classifier can be defined as follows [29]
TP
N+
TN
TNR =
N−
TPR + TNR
Total Accuracy =
(24)
2
Each of the considered models have a set of hyperparameters (cost penalty C and kernel parameter σ for SVM
while regularization coefficient λ and number of hidden
neurons nh for ELM) which needs tuning to suit the data
set. A full grid search cross-validation is employed where
the optimal combination of hyper-parameters are determined
based on observed total accuracy of the classifier. The hyperparameter tuning results are shown in Table II for ELM, Table
III for SVM for the no-sampling and re-sampling cases. It
can be observed that the total accuracy is generally high for
SVM models compared to the ELM models. It can also be
observed that by under-sampling or over-sampling the data,
better accuracies can be achieved compared to the no-sampling
case. Also both sampling methods give similar accuracy levels
for both ELM and SVM models. However, an advantage of
under-sampling can be realized in reduced computation as
training is performed with a smaller subset of the training
data.
The model tuning for cost-sensitive SVM and ELM is
summarized in Table IV. It can be observed that the costsensitive models give the desired result of accurate positive
and negative class predictions without re-sampling the data.
Also, the total accuracy levels are slightly higher compared
to the re-sampling methods. However, it can be observed that
the TPR and TNR are not close to each other for both ELM
and SVM models and the same can be observed for undersampling and over-sampling cases too indicating that it could
be a limitation in the data set to classify both classes to similar
accuracy. By varying the scaling factor f , the boundary can
be perturbed to suit the application which require either high
TPR or high TNR. Sensitivity plots have been shown in Figure
6 and Figure 7 for SVM and ELM models respectively to
observe the variation of total accuracy with the scaling factor.
By understanding the sensitivity of the weight factors, an
optimal weight for the minority class data (combination of
r and f ) can be determined.
TPR

=

1
0.95
0.9
0.85
0.8
0.75
0.7

TPR
TNR
Total
Accuracy

0.65
0.6
0

0.5

1

1.5
2
Scaling factor

2.5

3

3.5

Fig. 6: Sensitivity plot for TPR, TNR and Total Accuracy with
scaling factor f for cost-sensitive SVM.
1

0.9

0.8

0.7

0.6
TPR
TNR
Total
Accuracy

0.5

0.4
0

0.5

1
Scaling factor

1.5

2

Fig. 7: Sensitivity plot for TPR, TNR and Total Accuracy with
scaling factor f for cost-sensitive ELM.

As mentioned earlier, the SVM models have a better total
accuracy compared to the ELM models. One reason could
be that SVM minimizes a hinge loss while ELM minimizes
a squared loss as shown in Figure 11 and hence better
regularization performance for SVM. Another reason could be
that the ELM models are too simple to identify the decision
boundary accurately. A possibility is to add more number of
hidden neurons to the ELM models. Also, the ELM solution
greatly depends on the random initialization of the input layer
parameters (Wr and br ). In an attempt to evaluate more number of hidden neurons and different random initializations of
the input layer parameters, a further experiment was conducted
and results summarized in Table V. It can be observed that as
more number of hidden neurons are added, the total accuracy
improves. Also, different randomization helps in finding a
compact model at a given accuracy level (compare case 2
with case 9 where 400 additional neurons are required for a
negligible improvement). Hence determining an efficient way
of initialization of input layer parameters is required for ELM
models and will be considered in the future.
B. Prediction Results
The models developed using SVM and ELM are compared
against baseline linear models and the results summarized in
Table VI. The case 2 model in Table V is considered as the

IEEE TRANSACTIONS, IN REVIEW

8

TABLE II: Grid search results for ELM model selection for the regular ELM, ELM with under-sampling and ELM with
over-sampling (The models resulting in lowest total accuracy is highlighted in bold).

PP λ 0.01
nh
P
P

PP

10
30
50
70
90

0.995
0.994
0.995
0.996
0.995

Regular ELM

ELM with under-sampling

ELM with over-sampling

TPR

TPR

TPR

0.1

1

10

100

0.01

0.1

1

10

100

0.01

0.1

1

10

100

0.995
0.994
0.995
0.996
0.995

0.995
0.995
0.995
0.996
0.995

0.996
0.995
0.995
0.996
0.995

1.000
0.997
0.998
0.996
0.996

0.909
0.917
0.917
0.936
0.915

0.909
0.916
0.918
0.937
0.918

0.912
0.918
0.925
0.931
0.924

0.906
0.923
0.948
0.936
0.929

0.878
0.896
0.915
0.948
0.927

0.925
0.934
0.930
0.944
0.938

0.925
0.934
0.931
0.945
0.938

0.925
0.934
0.932
0.946
0.939

0.925
0.936
0.935
0.946
0.944

0.919
0.936
0.951
0.946
0.942

TNR

PP λ 0.01
nh
P
P

PP

10
30
50
70
90

0.333
0.387
0.430
0.426
0.433

TNR

0.1

1

10

100

0.01

0.1

1

10

100

0.01

0.1

1

10

100

0.333
0.389
0.430
0.423
0.430

0.327
0.387
0.423
0.413
0.420

0.258
0.366
0.407
0.404
0.405

0.000
0.160
0.294
0.351
0.356

0.732
0.714
0.771
0.771
0.789

0.734
0.719
0.773
0.770
0.794

0.737
0.730
0.752
0.775
0.784

0.771
0.732
0.735
0.748
0.768

0.743
0.732
0.727
0.704
0.755

0.716
0.727
0.773
0.739
0.763

0.716
0.725
0.773
0.739
0.768

0.716
0.729
0.771
0.739
0.773

0.722
0.735
0.755
0.758
0.775

0.757
0.727
0.724
0.740
0.775

Total Accuracy

PP
nh

λ

PP
0.01
P
P
10
30
50
70
90

0.664
0.691
0.712
0.711
0.714

TNR

Total Accuracy

Total Accuracy

0.1

1

10

100

0.01

0.1

1

10

100

0.01

0.1

1

10

100

0.664
0.692
0.712
0.709
0.712

0.661
0.691
0.709
0.705
0.707

0.627
0.680
0.701
0.700
0.700

0.500
0.579
0.646
0.674
0.676

0.821
0.815
0.844
0.853
0.852

0.821
0.818
0.846
0.853
0.856

0.825
0.824
0.838
0.853
0.854

0.839
0.828
0.842
0.842
0.848

0.811
0.814
0.821
0.826
0.841

0.820
0.831
0.852
0.842
0.850

0.820
0.830
0.852
0.842
0.853

0.820
0.832
0.851
0.842
0.856

0.824
0.835
0.845
0.852
0.859

0.838
0.832
0.838
0.843
0.858

TABLE III: Grid search results for SVM model selection for the regular SVM, SVM with under-sampling and SVM with
over-sampling (The models resulting in lowest total accuracy is highlighted in bold).
Regular SVM

SVM with under-sampling

TPR

H σ 0.01
C H
H

H

0.1
1
10
100
500

1.000
0.996
0.996
0.996
0.995

TPR

0.1
1
10
100
500

0.000
0.397
0.423
0.430
0.436

1

10

100

0.01

0.1

1

10

100

0.01

0.1

1

10

100

0.996
0.996
0.995
0.990
0.988

1.000
0.994
0.990
0.987
0.985

1.000
0.998
0.997
0.996
0.996

1.000
1.000
0.999
0.999
0.999

0.928
0.965
0.916
0.924
0.925

0.933
0.906
0.915
0.927
0.917

0.767
0.910
0.909
0.896
0.892

0.120
0.782
0.792
0.793
0.793

0.106
0.588
0.618
0.618
0.618

0.966
0.932
0.933
0.935
0.945

0.923
0.931
0.951
0.966
0.971

0.917
0.962
0.976
0.983
0.983

0.899
0.989
0.996
0.996
0.996

0.993
0.999
0.999
0.999
0.999

TNR

0.1
1
10
100
500

0.500
0.696
0.709
0.713
0.716

TNR

0.1

1

10

100

0.01

0.1

1

10

100

0.01

0.1

1

10

100

0.374
0.423
0.444
0.567
0.627

0.000
0.552
0.645
0.650
0.637

0.000
0.108
0.145
0.165
0.165

0.000
0.054
0.080
0.080
0.080

0.632
0.668
0.802
0.833
0.853

0.735
0.825
0.882
0.886
0.882

0.931
0.923
0.925
0.915
0.910

0.998
0.967
0.958
0.954
0.954

0.998
0.987
0.975
0.975
0.975

0.667
0.792
0.822
0.848
0.833

0.820
0.884
0.817
0.763
0.724

0.913
0.814
0.732
0.642
0.623

0.958
0.221
0.167
0.165
0.165

0.162
0.082
0.080
0.080
0.080

Total Accuracy

HH σ
0.01
C H
H

TPR

0.1

TNR

HH σ
0.01
C H
H

SVM with over-sampling

Total Accuracy

Total Accuracy

0.1

1

10

100

0.01

0.1

1

10

100

0.01

0.1

1

10

100

0.685
0.710
0.720
0.779
0.808

0.500
0.773
0.818
0.819
0.811

0.500
0.553
0.571
0.580
0.580

0.500
0.527
0.539
0.539
0.539

0.780
0.817
0.859
0.879
0.889

0.834
0.866
0.899
0.906
0.899

0.849
0.916
0.917
0.905
0.901

0.559
0.875
0.875
0.874
0.874

0.552
0.787
0.797
0.797
0.797

0.816
0.862
0.878
0.892
0.889

0.871
0.907
0.884
0.865
0.847

0.915
0.888
0.854
0.812
0.803

0.928
0.605
0.581
0.580
0.580

0.577
0.540
0.539
0.539
0.539

IEEE TRANSACTIONS, IN REVIEW

9

TABLE IV: Grid search results for Cost-sensitive SVM and Cost-sensitive ELM models (The models resulting in lowest total
accuracy is highlighted in bold).
Cost-sensitive SVM

Cost-sensitive ELM

TPR

H σ
H
0.001
C H
H

TPR

0.01

0.1

1

10

100

0.1
1
10
100
500

0.000
0.984
0.995
1.000
1.000

0.000
0.972
0.994
0.999
0.997

0.000
0.921
0.993
0.993
0.989

0.000
0.918
0.988
0.987
0.985

0.000
0.884
0.996
0.996
0.996

0.000
0.764
0.999
0.999
0.999

0.1
1
10
100
500

0.001
1.000
0.490
0.399
0.181
0.126

0.01
1.000
0.627
0.443
0.355
0.405

TNR
0.1
1.000
0.806
0.489
0.495
0.554

1
1.000
0.912
0.660
0.647
0.639

10
1.000
0.961
0.154
0.165
0.165

100
1.000
0.972
0.080
0.080
0.080

0.1
1
10
100
500

0.001
0.500
0.737
0.697
0.591
0.563

Total
0.01
0.500
0.800
0.718
0.677
0.701

Accuracy
0.1
1
0.500
0.500
0.863
0.915
0.741
0.824
0.744
0.817
0.771
0.812

10
0.500
0.922
0.575
0.580
0.580

100
0.500
0.868
0.539
0.539
0.539

PP
P

λ

0.1

1

10

100

10
30
50
70
90

0.909
0.924
0.925
0.939
0.932

0.909
0.924
0.925
0.939
0.932

0.909
0.923
0.926
0.939
0.933

0.907
0.927
0.930
0.934
0.936

0.901
0.927
0.936
0.931
0.930

10
30
50
70
90

0.01
0.742
0.737
0.778
0.770
0.786

0.1
0.743
0.737
0.778
0.770
0.786

TNR
1
0.745
0.735
0.775
0.765
0.794

10
0.753
0.740
0.771
0.779
0.784

100
0.779
0.735
0.748
0.768
0.784

10
30
50
70
90

0.01
0.825
0.830
0.852
0.854
0.859

Total Accuracy
0.1
1
10
0.826
0.827
0.830
0.830
0.829
0.833
0.851
0.850
0.851
0.854
0.852
0.857
0.859 0.864
0.860

100
0.840
0.831
0.842
0.850
0.857

nh

0.01
PP
P

TABLE VI: Summary of results for SVM and ELM models for all cases (Regular model, under-sampling, over-sampling and
cost-sensitive). The results of the linear models (logistic regression and linear least squares) are also compared. The value of
hyper-parameters and number of model parameters np are also included for every model
SVM

ELM

Regular

Undersampling

Oversampling

Costsensitive

Regular

Undersampling

Oversampling

Costsensitive

Best
ELM model

TPR
TNR
Total Accuracy

0.987
0.650
0.819

0.909
0.925
0.917

0.899
0.958
0.928

0.907
0.954
0.931

0.995
0.433
0.714

0.918
0.794
0.856

0.944
0.775
0.859

0.933
0.794
0.864

0.921
0.879
0.900

λ
nh
f
C
σ
np

100
1
33696

10
1
16965

0.1
10
236691

2.104
1
10
120900

0.010
90.000
3690

0.100
90.000
3690

10.000
90.000
3690

1.000
90.000
0.909
3690

10.000
600.000
0.769
24600

Logistic Regression

TPR
TNR
Total Accuracy
np

Linear LS

Regular

Undersampling

Oversampling

Regular

Undersampling

Oversampling

Costsensitive

0.995
0.441
0.718
40

0.911
0.791
0.851
40

0.928
0.786
0.857
40

0.996
0.389
0.692
40

0.941
0.704
0.822
40

0.955
0.699
0.827
40

0.875
0.828
0.852
40

IEEE TRANSACTIONS, IN REVIEW

10

TABLE V: Cost-sensitive ELM models with different random
initialization of input layer parameters
#

TPR

TNR

1
2
3
4
5
6
7
8
9
10

0.932
0.921
0.933
0.930
0.934
0.927
0.939
0.927
0.929
0.928

0.871
0.879
0.871
0.863
0.859
0.871
0.864
0.869
0.873
0.866

Total
Accuracy
0.901
0.900
0.902
0.896
0.896
0.899
0.902
0.898
0.901
0.897

λ

nh

f

10.000
10.000
10.000
10.000
10.000
10.000
10.000
10.000
10.000
10.000

800.000
600.000
900.000
600.000
800.000
800.000
800.000
700.000
1000.000
900.000

1.2
1.3
1.3
1.2
1.3
1.2
1.2
1.1
1.3
1.3

best ELM model and included in the summary table. From
the modeling results, it can be observed that both re-sampling
methods (under-sampling and over-sampling) as well as costsensitive classification are suitable for the problem considered
in this work. The nonlinear models result in better accuracies
compared to the linear models indicating that the HCCI boundary is a nonlinear system and that nonlinear classification
methods are necessary. However the cost paid for selecting
a nonlinear model is the additional computation and memory
required and the tradeoff can be evaluated specific to the
application based on the importance of having accurate versus
having low complexity models. For instance, the number of
parameters required to identify the classification boundary for
different models is summarized in Table VI. It is obvious
that the decision boundary identified by linear models (underparameterized) is very simple and does not capture the right
behavior. SVM and ELM models on the other hand requires
a large number of parameters in an attempt to capture more
complex behavior. SVM is a non-parametric model and hence
the number of parameters grows with number of training data.
ELM on the other hand is a parametric model and hence the
number of parameters are fixed and hence requires about 80%
less number of parameters for an inaccuracy of 3%. This is a
major drawback of SVM for applications onboard the engine
ECU which is limited in memory and computation.
For the engine problem considered here, it was observed that
both SVM and ELM are capable of identifying the stable and
unstable boundary of HCCI from experimental data. However,
an important criteria for HCCI engine application is that
the models must be able to be adapted online. As HCCI
combustion is sensitive to other variables like engine speed,
ambient temperature, pressure and humidity levels etc. and
hence experiments and data size might increase rapidly. Also,
performing an experiment that covers the entire region of
operation is infeasible. Hence a system that can learn and adapt
in a sequential manner is of extreme importance. Hence even
though SVM outperformed ELM in terms of accuracy, ELM
requires relatively less number of parameters and is simple and
efficient in implementation for on-line learning and chosen as
suitable for the application in hand.
The most accurate model using ELM and SVM are used
to make predictions on unseen engine inputs and predictions
are summarized in Figure 8 and Figure 9 respectively while
quantitative results are included in Table VI. Color codes are

used to represent the predictions - a red marker on the plots
indicate the model’s prediction being ”unstable” while a green
marker indicates prediction being ”stable”. It can be seen from
the plots that both models predict the HCCI instabilities well.
Dotted lines are shown as references for stable operation (Data
falling outside the dotted lines in CA50 plot or data falling
below the dotted line in NMEP plot indicate instability).
In order to get a closer look, a subsection of the above
figures are plotted in Figure 10a and Figure 10b for ELM and
SVM respectively. It can be observed from Figure 10a that
the input sample (at cycle 34) along with other input measurements is predicted unstable and rightly so, the following cycles
are unstable as indicated by CA50 overshooting 10 degree after
TDC. This is the primary goal of the proposed method that
predicts if the engine operation is stable/unstable at time k +1,
given the measurements up to time k. Hence using the model,
any given input actuator setting and history of measurements
of key combustion variables, it would be possible to predict
if the subsequent set of combustion cycles misfire or not. A
similar plot can be shown for SVM in Figure 10b which has a
slightly different prediction compared to the ELM model. For
instance, SVM predicts most of the region beyond cycle 34
to be unstable but ELM predicts to be stable between cycles
75 and 95. However, these cycles do not fall into completely
stable or completely unstable and hence might be in the fuzzy
region between the two classes. Such predictions are to be
expected from both models as training was not performed
using a comprehensive data set that had a dense distribution
of data in both classes.
V. C ONCLUSION
Complex and highly sensitive systems such as HCCI engines have a narrow region of stable operation and it is
important to gain knowledge about the operating envelope for
diagnostics and controls development. In this paper, a novel
solution using soft computing has been developed that predicts the future combustion events based on past and present
measurements along with excitation inputs. An imbalanced
classification problem has been formulated and solved using
linear and nonlinear methods such as logistic regression, linear
regression, SVM and ELM.
A comparison of data re-sampling methods and costsensitive learning approaches have been performed and results
summarized. Re-sampling methods are found to work well
but cost-sensitive methods have a slightly better accuracy
and avoid artificial modifications in the data distributions.
A modification to the ELM algorithm has been made by
weighting the minority class data more to handle the imbalance
in the data set. The cost-sensitive SVM classifier outperforms
the other algorithms in terms of accuracy but requires a large
fraction of the data to be stored for predictions, typical of
non-parametric methods. ELM (and its variants) results in
an inferior accuracy compared to SVM (and corresponding
variants) but preferred over SVM as the ELM model is much
simpler (less number of parameters) and is more suitable for
online learning for a memory-limited, real-time application
such as the HCCI engine. Future work will involve identifying

11

20

CA50
(deg after TDC)

CA50
(deg after TDC)

IEEE TRANSACTIONS, IN REVIEW

10
0
−10

0

500

1000

1500

20
10
0
−10

2
0
500

1000

1000

1500

0

500

1000

1500

0

500

1000

1500

2

1500

30

30
Fuel input
(mg/cycle)

Fuel input
(mg/cycle)

500

0
0

20
10

0

4
NMEP

NMEP

4

0

500

1000

20
10

1500

Combustion Cycles

Combustion Cycles

(a)

20

CA50
(deg after TDC)

CA50
(deg after TDC)

(a)

10
0
−10

0

200

400

600

800

1000

20
10
0
−10

2
0
200

400

600

800

400

600

800

1000

0

200

400

600

800

1000

0

200

400
600
Combustion Cycles

800

1000

2

1000

30

30
Fuel input
(mg/cycle)

Fuel input
(mg/cycle)

200

0
0

20
10

0

4
NMEP

NMEP

4

0

200

400
600
Combustion Cycles

800

20
10

1000

(b)

20

CA50
(deg after TDC)

CA50
(deg after TDC)

(b)

10
0
−10

0

200

400

600

800

1000

20
10
0
−10

2
0
200

400

600

800

400

600

800

1000

0

200

400

600

800

1000

0

200

400
600
Combustion Cycles

800

1000

2

1000
30
Fuel input
(mg/cycle)

30
Fuel input
(mg/cycle)

200

0
0

20
10

0

4
NMEP

NMEP

4

0

200

400
600
Combustion Cycles

800

1000

20
10

(c)

(c)

Fig. 8: Prediction results of the cost-sensitive ELM showing
CA50, IMEP and one input variable (fueling) for 3 unseen data
sets. The color code indicates model prediction - green (and
red) indicate stable (and unstable) prediction by the model.
The dotted line in the IMEP plot indicates misfire limit, dotted
ellipse in CA50 plot indicates high variability instability mode
while dotted rectangle indicates a wrong predictions by model.

Fig. 9: Prediction results of the cost-sensitive SVM showing
CA50, IMEP and one input variable (fueling) for 3 unseen data
sets. The color code indicates model prediction - green (and
red) indicate stable (and unstable) prediction by the model.
The dotted line in the IMEP plot indicates misfire limit, dotted
ellipse in CA50 plot indicates high variability instability mode
while dotted rectangle indicates a wrong predictions by model.

CA50 (deg after TDC)

IEEE TRANSACTIONS, IN REVIEW

12

9

20
10

0−1 loss
Logistic loss
Squared loss
Hinge loss

8

0
7
−10

0

20

40

60

80

100

120

6

Loss

NMEP

4
2

4

0
Fuel input (mg/cycle)

0

20

40

60

80

100

3

120

30
2
20
10

1

0

20

40

60
Combustion Cycles

80

100

0
−2

120

(a) Cost-sensitive ELM
CA50 (deg after TDC)

5

−1.5

−1

−0.5

0
0.5
Margin y.f(x)

1

1.5

2

Fig. 11: A comparison of the loss functions of the algorithms
used in this paper with the baseline 0-1 error standard.

20
10

A PPENDIX A
L OSS FUNCTIONS

0
−10

0

20

40

60

80

100

120

NMEP

4
2

Fuel input (mg/cycle)

0
0

20

40

60

80

100

120

0

20

40

60
Combustion Cycles

80

100

120

30
20
10

A plot showing the different loss functions used for classification can be shown in Fig. 11. The 0-1 loss is the most
efficient loss function for binary classification but is nondifferentiable and hence not used in developing algorithms.
A logistic loss is given by equation (25) is used in logistic
regression algorithm while a hinge loss (equation (26)) and
squared loss (equation (27)) are implemented in SVM and
ELM algorithms respectively.
Llogistic

(b) Cost-sensitive SVM

=

= max(0, 1 − yf (x))

Lhinge

Fig. 10: A small subset of prediction results of the costsensitive ELM and SVM showing CA50, IMEP and one
input variable to compare predictions in perspective to input
variables. The green points indicate stable operation while red
points indicate unstable operation.

log(1 + e−yf (x) )

Lsquared

=

(1 − yf (x))

2

(25)
(26)
(27)

A PPENDIX B
L OGISTIC R EGRESSION
more unstable modes of HCCI and perform online model
adaptation using ELM’s online learning algorithm to improve
accuracy and explore unexcited engine operation. Further,
application of the developed model towards a closed system
identification and controller development will be explored in
the future.
ACKNOWLEDGMENT
1

This material is based upon work supported by the Department of Energy and performed as a part of the ACCESS
project consortium (Robert Bosch LLC, AVL Inc., Emitec
Inc.) under the direction of PI Hakan Yilmaz, Robert Bosch,
LLC.
1 Disclaimer: This report was prepared as an account of work sponsored by an agency

of the United States Government. Neither the United States Government nor any agency
thereof, nor any of their employees, makes any warranty, express or implied, or assumes
any legal liability or responsibility for the accuracy, completeness, or usefulness of
any information, apparatus, product, or process disclosed, or represents that its use
would not infringe privately owned rights. Reference herein to any specific commercial
product, process, or service by trade name, trademark, manufacturer, or otherwise does
not necessarily constitute or imply its endorsement, recommendation, or favoring by
the United States Government or any agency thereof. The views and opinions of authors
expressed herein do not necessarily state or reflect those of the United States Government
or any agency thereof.

The goal of logistic regression is to determine β such
that P (Y |X, β) is maximized, i.e, to solve the following
optimization problem
β = arg max P (Y |X, β) = arg max ΠN
i=1 P (yi |xi , β) (28)
β

β

Expressing the above in log likelihood form, the optimization
problem becomes
β

=

arg max log{ΠN
i=1 P (yi |xi , β)}

=

arg max

=

=

=

β

β

arg max
β

N
X
i=1
N
X

log P (yi |xi , β)
log

i=1
N
X

arg max −
β

arg min
β

i=1
N
X

1
1+

T
e−y(β1 x+β0 )

T

log 1 + e−y(β1 x+β0 )



T
log 1 + e−y(β1 x+β0 )

i=1

(29)

IEEE TRANSACTIONS, IN REVIEW

R EFERENCES
[1] R. Thring, “Homogeneous-charge compression-ignition engines,” 1989,
SAE paper 892068.
[2] B. J. M. Christensen, P. Einewall, “Homogeneous charge compression
ignition using iso-octane, ethanol and natural gas- a comparison to spark
ignition operation,” in International Fuels & Lubricants Meeting &
Exposition, Tulsa, OK, USA, Oct 1997, SAE paper 972874.
[3] J. M. Y. S. T. Aoyama, Y. Hattori, “An experimental study on premixedcharge compression ignition gasoline engine,” in International Congress
& Exposition, Detroit, MI, USA, Feb 1996, SAE paper 960081.
[4] K. Chang, A. Babajimopoulos, G. A. Lavoie, Z. S. Filipi, and D. N.
Assanis, “Analysis of load and speed transitions in an hcci engine using
1-d cycle simulation and thermal networks.” SAE International, 04
2006.
[5] R. J. P. T. J. Bengtsson, P. Strandh and B. Johansson, “Model predictive
control of homogeneous charge compression ignition (HCCI) engine
dynamics,” in 2006 IEEE International Conference on Control Applications, 2006.
[6] Y. Wang, S. Makkapati, M. Jankovic, M. Zubeck, and D. Lee, “Control
oriented model and dynamometer testing for a single-cylinder, heated-air
hcci engine.” SAE International, 04 2009.
[7] C.-J. Chiang, A. Stefanopoulou, and M. Jankovic, “Nonlinear observerbased control of load transitions in homogeneous charge compression
ignition engines,” Control Systems Technology, IEEE Transactions on,
vol. 15, no. 3, pp. 438–448, 2007.
[8] Y. Urata, M. Awasaka, J. Takanashi, T. Kakinuma, T. Hakozaki, and
A. Umemoto, “A study of gasoline-fuelled HCCI engine equipped with
an electromagnetic valve train,” 06 2004.
[9] R. Scaringe, C. Wildman, and W. K. Cheng, “On the high load limit
of boosted gasoline HCCI engine operating in NVO mode,” SAE Int. J.
Engines, vol. 3, pp. 35–45, 04 2010.
[10] A. Soliman, G. Rizzoni, and V. Krishnaswami, “The effect of engine
misfire on exhaust emission levels in spark ignition engines.” SAE
International, 02 1995.
[11] S.-C. Kong, “A study of natural gas/DME combustion in HCCI engines
using CFD with detailed chemical kinetics,” Fuel, vol. 86, no. 1011, pp.
1483 – 1489, 2007.
[12] Z. Zheng and M. Yao, “Charge stratification to control HCCI: Experiments and CFD modeling with n-heptane as fuel,” Fuel, vol. 88, no. 2,
pp. 354 – 365, 2009.
[13] Z. Wang, S.-J. Shuai, J.-X. Wang, and G.-H. Tian, “A computational
study of direct injection gasoline HCCI engine with secondary injection,”
Fuel, vol. 85, no. 1213, pp. 1831 – 1841, 2006.
[14] M. Yao, Z. Zheng, and H. Liu, “Progress and recent trends in homogeneous charge compression ignition (HCCI) engines,” Progress in Energy
and Combustion Science, vol. 35, no. 5, pp. 398 – 437, 2009.
[15] I. V. Kolmanovsky and E. G. Gilbert, “Support vector machine-based
determination of gasoline direct injected engine admissible operating
envelope.” SAE International, 03 2002.
[16] D. A. V. Janakiraman, X. Nguyen, “A system identification framework
for modeling complex combustion dynamics using support vector machines,” (In press) Lecture Notes in Electrical Engineering - Informatics
in Control, Automation and Robotics, 2012.
[17] H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Trans.
on Knowl. and Data Eng., vol. 21, no. 9, pp. 1263–1284, 2009.
[18] K. Veropoulos, C. Campbell, and N. Cristianini, “Controlling the sensitivity of support vector machines,” in Proceedings of the International
Joint Conference on AI, 1999, pp. 55–60.
[19] A. Dobson, An Introduction to Generalized Linear Models, Second
Edition, ser. Texts in Statistical Science Series. Taylor & Francis,
2010.
[20] V. Vapnik, The Nature of Statitical Learning Theory. Springer-Verlag
GmbH, 1995.
[21] C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector
machines,” ACM Transactions on Intelligent Systems and Technology,
vol. 2, pp. 27:1–27:27, 2011.
[22] G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme learning machine:
Theory and applications,” Neurocomputing, vol. 70, pp. 489–501, 2006.
[23] G.-B. Huang, H. Zhou, X. Ding, and R. Zhang, “Extreme learning
machine for regression and multiclass classification.” IEEE Transactions
on Systems, Man, and Cybernetics, Part B, vol. 42, no. 2, pp. 513–529,
2012.
[24] V. M. Janakiraman, X. Nguyen, and D. Assanis, “Nonlinear identification of a gasoline hcci engine using neural networks coupled with
principal component analysis,” Applied Soft Computing, vol. 13, no. 5,
pp. 2375 – 2389, 2013.

13

[25] D. N. A. J. E. D. J. A. E. P. M. N. Fuquan Zhao, Thomas N. Asmus,
Homogeneous Charge Compression Ignition (HCCI) Engines. SAE
International, March 2003.
[26] G. A. Lavoie, J. Martz, M. Wooldridge, and D. Assanis, “A multimode combustion diagram for spark assisted compression ignition,”
Combustion and Flame, vol. 157, no. 6, pp. 1106 – 1110, 2010.
[27] G. T. Kalghatgi and R. A. Head, “Combustion limits and e?ciency in a
homogeneous charge compression ignition engine,” Int. J. Engine Res,
vol. 7, pp. 215–236, 2006.
[28] M. Shahbakhti and C. R. Koch, “Characterizing the cyclic variability
of ignition timing in a homogenous charge compression ignition engine
fueled with n-heptane/iso-octane blend fuels,” Int. J. Engine Res, vol. 9,
pp. 361–397, 2008.
[29] K.-A. Toh, “Deterministic neural classification,” Neural Comput.,
vol. 20, no. 6, pp. 1565–1595, Jun. 2008.

