Journal XXX manuscript No.
(will be inserted by the editor)

Modeling reverse thinking for machine learning

arXiv:1803.00158v1 [cs.LG] 1 Mar 2018

Huihui Li · Guihua Wen*

Abstract Human inertial thinking schemes can be
formed through learning, which are then applied to
quickly solve similar problems later. However, when
problems are significantly different, inertial thinking
generally presents the solutions that are definitely
imperfect. In such cases, people will apply creative
thinking, such as reverse thinking, to solve problems.
Similarly, machine learning methods also form inertial
thinking schemes through learning the knowledge from
a large amount of data. However, when the testing
data are vastly difference, the formed inertial thinking
schemes will inevitably generate errors. This kind of
inertial thinking is called illusion inertial thinking.
Because all machine learning methods do not consider
illusion inertial thinking, in this paper we propose a new
method that uses reverse thinking to correct illusion
inertial thinking, which increases the generalization
ability of machine learning methods. Experimental
results on benchmark datasets are used to validate the
proposed method.
Keywords Machine learning · Inertial thinking
model · Modeling reverse thinking

1 Introduction
Human learning automatically induces more general
rules from a large number of experiences. These rules
are taken as the elements to form the inertial thinking
schemes that will be used to solve similar problems
later. The reasons that inertial thinking exists for
humans lie in the fact that it can solve a large number
H. Li · G. Wen (Corresponding author)
School of Computer Science and Engineering, South China
University of Technology, China
Tel.: +86-18998384808
E-mail: crghwen@scut.edu.cn

of daily problems when there is insufficient information
and it can solve them quickly [3]. However, the accuracy
of solutions to the problems will generally be reduced
when the problems are different from those for learning.
Like human learning, machine learning automatically learns new concepts from a large number of data.
In the learning process, inertial thinking schemes will
be formed, including illusion inertial thinking, leading
to low generalization ability of the machine learning
methods. According to the law of Hoeffding [5], the
accuracy of machine learning methods depend heavily
on a large number of training samples, and this is
why most advanced machine learning methods require a
large number of training samples [7], e.g., deep learning
methods [8]. However, it is difficult to obtain a large
number of training samples for most real applications,
such as disease diagnosis, where the training samples
are few, easily leading to formation of illusion inertial
thinking in machine learning methods. One way for
humans to solve these kinds of problem is the use
of reverse thinking, which is an effective method of
creative thinking [9]. However, for machine learning
methods, reverse thinking is totally not considered.
Therefore, in this paper we present a new method
that applies reverse thinking to correct illusion inertial
thinking for machine learning methods so that their
generalization ability can be enhanced. This method is
suitable for any machine learning method, as long as
illusion inertial thinking is facilitated.

1.1 Machine learning principles
Machine learning methods are all based on the principle
of inertial thinking. They first learn from the training
data to form the inertial thinking scheme that is then
applied to solve new problems.

2

Huihui Li, Guihua Wen*

Suppose that for the testing sample X, a learning
approach is applied to obtain the probability model P
from the training database, so that
C ∗ = arg max P (Ci = X),
Ci ∈C

(1)

where C = {C1 , · · · , Ci , · · · , Cn } is a set of classes and
P (Ci = X) is the probability that X belongs to class
Ci . For example, if there are two classes C1 , C2 , the
learned probability model will work as follows:
P (C1 = X) > P (C2 = X) =⇒ X ∈ C1 ,

the illusion inertial thinking appears, people can use reverse thinking to deal with it. This process begins with
designing the inertial thinking discrimination model to
judge whether the current testing data is in line with
inertial thinking or illusion inertial thinking. Second,
the reverse thinking model is created to correct illusion
inertial thinking. The overall procedure is shown in
Fig. 1, where classifiers are trained on the training
database to form a suitable inertial thinking model, an
illusion inertial thinking model, and an inertial thinking
discrimination model.

P (C2 = X) > P (C1 = X) =⇒ X ∈ C2
The ability of the learned probability model to
correctly classify the new samples is measured as its
generalization ability. A lower generalization ability
of the model may result from the training samples
being insufficient to correctly fit the data distribution
of the problem. In order to formally illustrate the
generalization ability of the model, all data are taken
as balls of different colors in a jar. It is expected that
the proportion of different color balls in the jar can be
computed through a portion of the balls partially taken
out of the jar. It assumes that the ratio of orange balls
in the jar is µ and the proportion of green balls for
1 − µ. In the balls taken out of the jar, the ratio of
orange balls is ν, and the proportion of green balls is
1 − ν. The relationship between the two cases follows
the Hoeffding inequality [5]:

2.1 Inertial thinking model

P (|µ − ν| > ε) ≤ 2exp(−2ε2 N ).

m(X) =

The suitable inertial thinking model works well on
the simple samples that are easily classified, while
illusion inertial thinking easily occurs when classifying
complicated samples. There are many measures that
can be applied to rank samples by simplicity [10].
Hence, we rank samples in the training data according
to the simplicity of samples using the k-fold crossvalidation method. When samples in a fold are removed
from the training data, the remaining ones are used to
train the classifier, which is then applied to classify all
training data. In such a case, the simplicity of each
sample can be computed.
Definition. Supposing that there are N classification models, the simplicity of sample X is defined by
N

(2)

This law shows that when N is large, the right-hand
side of the inequality will be very small, indicating that
the probability of the difference between µ and ν being
larger than the given ε is very small. In such a case,
the distribution of the overall samples can be inferred
from the distribution of the partially taken samples.
However, in real applications, N is usually difficult to
be sufficiently large, so error is inevitable. The current machine learning methods only consider suitable
inertial thinking, ignoring illusion inertial thinking. In
such a case, when the size of the training samples are
not large enough, the testing samples could have a
high probability of differing from the training samples,
inevitably leading to error. This is why there are many
machine learning methods that perform well on some
experimental data, with even up to 100% accuracy, but
they may behave badly in practice.

1 X
mi (X)
N i=1

1, X is classified correctly;
mi (X) =
0, else.

(3)

The definition of simplicity means that a sample is
simple if it is correctly classified by many models. In
this way, all samples can be ranked according to their
simplicity:
S = {X1 , ..., Xi , ..., Xn |m(Xi ) ≥ m(Xi+1 )}.
Subsequently, a simple training database (SE , LE )
can be constructed, the samples of which are of greater
simplicity than the given threshold θ. Similarly, a
complicated training database (SD , LD ) can also be
constructed:
SE = {Xi |Xi ∈ S ∧ m(Xi ) > θ},
SD = {Xi |Xi ∈ S ∧ m(Xi ) ≤ θ},
S = SE ∪ SD , SE ∩ SD = ∅.

2 Modeling reverse thinking
Generally, inertial thinking is composed of both suitable
inertial thinking and illusion inertial thinking. When

Using these training databases, three models are
trained as follows, where ξ is a classification method
and ϕ is a discrimination method for judging whether
a sample is simple.

Modeling reverse thinking for machine learning

Fig. 1 Training and testing procedures of machine learning methods based on reverse thinking

3

4

Huihui Li, Guihua Wen*

– Inertial thinking model: MI = ξ(S, L).
– Suitable inertial thinking model: MS = ξ(SE , LE ).
– Discrimination model: MD = ϕ(SE ∪ SD , {+, −}) ,
where + denotes samples to be classified easily and
− denotes the samples to be classified difficultly.

2.2 Reverse thinking rules

(4)

In such a case, according to reverse thinking, the
above rule can be reversed to obtain the reverse thinking rule by recomputing the probability:
rr : C2 → C1 .

P (C1 = X|C2 = X) =

(5)

Because the results obtained by illusion inertial
thinking are not absolute, a probabilistic model like the
Bayesian theorem is applied. In conditions of the complete statistical knowledge of a model presented, the
classifier designed according to the Bayesian theorem
can obtain the minimum classification error in terms
of probability, which means that the classifier based on
the reverse thinking rule is optimal.
As the testing sample X has been misclassified as
belonging to the class C2 , according to the Bayesian
theorem its true class is C1 with the probability

P (C2 |C1 )P (C1 = X)
P (C2 = X)

However, the reverse thinking rule may be also applied
to recompute the probability
P (C2 = X|C1 = X) =

Owing to illusion inertial thinking, the learned inertial
thinking model may misclassify the coming samples.
For example, it classifies the test sample X with true
class C1 as belonging to the class C2 ; that is,
r : C1 → C2 .

recompute the probability so as to classify the sample
as belonging to the correct class C1 ; that is,

P (C1 |C2 )P (C2 = X)
.
P (C1 = X)

In such a case,
P (C1 = X|C2 = X)
=
P (C2 = X|C1 = X)
P (C2 |C1 )P (C1 = X)
P (C1 = X)
×
P (C2 = X)
P (C1 |C2 )P (C2 = X)
=

P (C1 = X) 2
P (C2 |C1 )
×(
) .
P (C1 |C2 )
P (C2 = X)
To satisfy X ∈ C1 ,

P (C1 = X) 2
P (C2 |C1 )
×(
) > 1,
P (C1 |C2 )
P (C2 = X)
P (C2 |C1 )
P (C2 = X) 2
>(
) ,
P (C1 |C2 )
P (C1 = X)

(8)

P (C1 |C2 )
P (C2 = X)
P (C2 |C1 )
>
×
.
P (C2 = X)
P (C1 = X) P (C1 = X)
The learning model has classified X as belonging to
C2 – that is, P (C2 = X) > P (C1 = X) – leading to
P (C1 |C2 )
P (C2 |C1 )
>
.
P (C2 = X)
P (C1 = X)

Thus, the reverse thinking rule can enhance the
P (C2 = X|C1 = X)P (C1 = X)
,(6) probability that the classifier will classify X as belongP (C2 = X)
ing to C1 more than it will classify it as belonging to
where P (Ci = X) refers to the probability that X
C2 . This proves that the reverse thinking rule has the
belongs to class Ci , and P (C2 = X|C1 = X) denotes
ability to correct the classification of samples that have
the probability that X with the class C2 is misclassified
been misclassified.
as belonging to C1 . The right-hand side of the above
Since P (C2 = X) > P (C1 = X), their difference is
equation involves the prior probability that is then
less, the inequality (8) is easily satisfied, and then the
applied to compute the posterior probability of the
reverse thinking rule is more useful in correcting the
left-hand side of the equation. Equation (6) is applied
misclassified category. However,
to implement the reverse thinking rule in Eq. (5) by
P (C2 |C1 )
> 1,
recomputing the probability. All posterior probabilities
P (C1 |C2 )
can be computed from the confusion matrix. To avoid
which means that the sample with class C1 is easily misconfusion, Eq. (6) can be rewritten as
classified as belonging to the class C2 in the confusion
P (C2 |C1 )P (C1 = X)
matrix, but not vice versa.
.
(7)
P (C1 = X|C2 = X) =
P (C2 = X)
As the classifier classified the samples depending on
P (C1 ) = X and P (C2 = X), they may be wrong,
Theorem 1. If X ∈ C2 ∧ P (C1 = X) > P (C2 =
so they can be also replaced by the prior probability
X), under certain conditions, P (C2 = X|C1 = X) >
computed in the confusion matrix. Subsequently, we
P (C1 = X|C2 = X), so that X ∈ C1
have another alternative reverse thinking rule, as
Proof. The inertial thinking model classifies the
sample with the true class C1 as belonging to the class
P (C2 |C1 )P (C1 )
.
(9)
P (C1 = X|C2 = X) =
C2 , and the reverse thinking rule can be applied to
P (C2 )
P (C1 = X|C2 = X) =

Modeling reverse thinking for machine learning

5

It is noteworthy that the reverse thinking rule is
only applied to the classification of complicated samples, in which only one reverse direction is considered
to deal with the misclassified samples. Therefore, in
the following context, the necessary condition for the
reverse thinking rule is simplified as follows to design
the new machine learning approach:
P (C1 = X|C2 = X) > P (C1 = X) ⇒ X ∈ C1 ,

(10)

indicating that the reverse thinking rule can enhance
the probability that the classifier correctly classifies the
sample.

2.3 Computing prior probabilities
It is necessary to use the Bayesian theorem to calculate
the probabilities of reverse thinking rules, which can
be calculated by computing the confusion matrix of
the specified classifier on the given training data. For
example, if the training data have 60 samples with
two categories, where each category is composed of 30
samples, the confusion matrix of the given classifier
can be computed through 10-fold cross-validation. As
shown in Table 1, each column in the table represents
the predicted category, and the total number of each
column represents the number of samples predicted for
the category. Each row represents the true category
of the sample, and the total number of samples per
row represents the number of samples in that class. For
example, the number, 23, located at the cross-point of
both the first row and first column indicates that the
samples with the true class C1 are correctly predicted
to be C1 . The number, 5, located at the cross-point
between the first column and second row indicates that
there are five samples with true class C2 , which are
incorrectly predicted to be class C1 .

The reverse thinking for machine learning (RTML) algorithm is described in Table 2, including both training
and testing stages. The inertial thinking model in the
RTML algorithm is obtained by training the classifier
on all the training data. This is because illusion inertial
thinking is formed in the original environment. Second,
for any new testing sample, the discrimination model
is used to decide whether the testing sample is easy to
classify. If the testing sample is easy to classify, it is
classified by using the suitable inertial thinking model
that is trained on the easy training data. If the testing
sample is hard to classify, the inertial thinking model
will be applied, and then the reverse thinking model
is applied to modify its result. The final category is
determined by the modified results.
Table 2 Reverse thinking for machine learning (RTML)
algorithm
Input
Output
Training
1
2
3
4
5
6
Testing
7
8
9
10
11
12
13
14

Table 1 Confusion matrix
Predicted class
⇓
True class⇒

2.4 RTML algorithm

C1
C2

C1

C2

23
5

7
25

15
16

According to the confusion matrix, the prior conditional probabilities that the samples are misclassified
as belonging to each category can be computed by
P (C1 = X|C2 = X) =

23
,
7 + 25

(11)

P (C2 = X|C1 = X) =

25
.
23 + 5

(12)

training data (S, L) and testing sample X
The threshold parameter θ
The class label ω for X
Compute m(Xi ), Xi ∈ S
Build (SE , LE ) ∧ Xi ∈ SE ∧m(Xi ) < θ
Build (SD , LD ) ∧ Xi ∈ S∧m(Xi ) ≥ θ
Train suitable inertial thinking model
MS = ξ(SE , LE )
Train illusion inertial thinking model
MI = ξ(S, L)
Train the discrimination model
MII = ϕ(SE ∪ SD , {+, −})
y ← MII (X)
If y ∈ {+}, ω ← MS (X)
Else /* reverse thinking is required*/
ω ← MI (X)
If ω = C1 and
P (C2 = X|C1 = X) > P (C2 = X)
ω ← C2
End
If ω = C2 and
P (C1 = X|C2 = X) > P (C1 = X)
ω ← C1
End

In this algorithm, only two classes are considered
because most of the datasets in real tasks are binary
class. Second, for binary-class datasets, the use of reverse thinking rules only needs to consider the Bayesian
theorem, without considering the Bayesian network,
so as to simplify the problem. Finally, the multi-class
tasks can be classified through classification methods on
binary-class tasks, such as the support vector machine
(SVM) [6].

6

Huihui Li, Guihua Wen*

The complexity of the RTML algorithm at the
training stage is mainly distributed in computing the
confusion matrix and training three classifiers. Therefore, the complexity of the RTML algorithm is equal to
the maximum complexity of the three classifiers. The
complexity of the RTML algorithm at the testing stage
is also the maximum complexity of the three classifiers
for testing a sample. The calculation of reverse thinking
rules is only related to the number of categories, which
can be regarded as a constant. The RTML algorithm
has an additional parameter, the simplicity threshold,
which determines the boundary between the illusion
inertial thinking model and suitable inertial thinking
model.

1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0.6

0.8

1

(a) SOFTMAX
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4

3 Experimental results

-0.6
-0.8

The effectiveness of the proposed method is demonstrated through experiments on standard datasets. In
principle, the proposed method is applicable to all
existing classification methods, so as to improve their
classification performance. In experiments, a highly
representative machine learning method is selected,
namely SOFTMAX, because it is widely used in deep
learning [11,8], and it is expected that the RTML
algorithm can be applied to deep learning. For the
discrimination model, SVM-KNN (where KNN denotes
k th nearest neighbor) is selected [13], as it inherits the
SVM advantages [6], while be able to elegantly deal
with large datasets.

3.1 Experimental databases
Experimental databases include artificial datasets and
benchmark real datasets.
Using artificial data, we can control the number
of available samples and add noise according to the
experimental purpose. In the experiments, two spirals of
data are selected, which is synthetic, but very difficult
for many machine learning methods. Such data have
been widely used by many classifiers as standard data
[12]. Here, they are applied to observe the effect of
reverse thinking on noisy data of different sizes. One
of the data examples is shown in Fig. 2, where there
are 200 points with some noise disruption. Obviously,
these data are not linearly separable and are nearly
evenly distributed for two classes.
In addition to artificial data, 15 benchmark real
datasets with two classes are selected from LIBSVM [2]
and KEEL [1]; see Table 3. The reason for selecting data
with only two classes is that most of the real datasets
are binary class. Second, for binary-class datasets, the

-1
-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

(b) RTML
Fig. 2 Classification results in which misclassified samples
are indicated in red

use of reverse thinking rules only needs to consider the
Bayesian theorem, without considering the Bayesian
network, so as to simplify the problem. Finally, the
multi-class data can be classified through classification
methods on binary-class data, such as the SVM [6].
Table 3 Illustration of benchmark data used for experiments
No.

data name

classes

features

samples

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

fourclass
waveform
svmguide1
madelon
splice
thyroid
titanic
segment0
vehicle1
glass1
pima
haberman
spambase
mammographic
phoneme

2
2
2
2
2
2
2
2
2
2
2
2
2
2
2

2
21
4
500
60
5
3
19
18
9
8
3
57
5
5

862
5000
3089
2000
1000
215
2201
2308
846
214
768
306
4597
830
5404

3.2 Analysis of RTML algorithm
In order to observe the effectiveness of the RTML
algorithm more clearly, experiments on two spirals of

Modeling reverse thinking for machine learning

3.3 Influence of noisy artificial data on RTML
algorithm
Generally, machine learning methods are easily influenced by noisy data. Some experiments are therefore
conducted to assess the influence of noisy data on the
RTML algorithm, the results of which are shown in
Fig. 5. Here, datasets with 100, 200, and 400 points
are added by noises whose variances are 0.02, 0.04,

titanic

0.776

SOFTMAX
RTML

0.774

Accuracy

0.772

0.77

0.768

0.766

0.764
0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Complexity threshold

(a) titanic
fourclass

0.84
0.82
0.8

SOFTMAX
RTML

0.78

Accuracy

data are conducted, the results of which are presented
in Fig. 2, where the misclassified testing data are
denoted in red. It can be seen that the number of red
points misclassified by SOFTMAX is larger than that
misclassified by the RTML algorithm. Some samples
misclassified by SOFTMAX have been corrected by the
RTML algorithm, showing that the reverse thinking
model is effective. However, when the easy points
are misclassified, the RTML algorithm cannot modify
them. This depends on the ability of the original
classifier, i.e., SOFTMAX. This, in turn, means that
the original classifier should be selected to be the
best as possible. Some misclassified samples cannot be
modified, since in such cases illusion inertial thinking
has not formed.
The RTML algorithm has a simplicity threshold
as a parameter to decide whether the illusion inertia
thinking model can be formed on the database composed of samples with a simplicity larger than the
given threshold. Experiments are conducted on two real
datasets to illustrate the influence of the parameter
on the RTML algorithm. It can be seen from Fig. 3
that the RTML algorithm is sensitive to the threshold.
When the threshold θ > 0.4 in Fig. 3(a), the RTML
algorithm has the worse effect, since in such a case
many samples are mistaken for the easy ones. This
indicates that the threshold parameter values need to
be determined as accurately as possible. However, from
Fig. 3, the parameter varies in terms of a law, indicating
that its optimal value can be determined easily using
the k -fold cross-validation method.
As simplicity of samples plays a very important role
in the RTML algorithm, it needs to be measured as
precisely as possible. In our context, many models have
been applied to define the simplicity. These models are
created using the k -fold cross-validation method on the
training database, with the same classification method.
It can be seen from Fig. 4 that when the number of
models is larger than 35, the accuracy tends to the
optimal one, and the threshold can be easily selected.
This means that the number of models used to measure
the simplicity of the samples should be large as possible
to make the measurement more stable.

7

0.76
0.74
0.72
0.7
0.68
0.66
0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Complexity threshold

(b) RTML
Fig. 3 Impact of the simplicity threshold θ on the performance of RTML algorithm

and 0.06, respectively (as well as a variance of 0.0). In
this way, 12 different datasets are formed to observe
the effects of the noise and the size of the data on
the performance of the RTML algorithm. In order
to observe the sensitivity of the RTML algorithm to
its parameters at the same time, classification results
corresponding to each parameter are computed and
plotted in the figure. It can be seen from Fig. 5 that
the noise has negative effects on the RTML algorithm;
the more intense the noise, the lower the accuracy. This
is because the noise is random, disrupting the formation
of illusion inertial thinking. However, in the case of
noise, the RTML algorithm still obviously increases the
accuracy of SOFTMAX. At the same time, the RTML
algorithm is sensitive to its parameter, but still achieves
remarkable results at many values, indicating that the
choice of parameter is easier. Finally, when the data
size becomes larger, the RTML algorithm increases the
effect more obviously while being less sensitive to its
parameter. This is because the number of misclassified
samples will increase in the larger dataset, leading to
the formation of more stable illusion inertial thinking.
3.4 Effect of real datasets on RTML algorithm
As the artificially constructed examples may not correspond to situations that are likely to occur in practice,
some experiments on real datasets are conducted. One
of the advantages of real data is that they are generated

8

Huihui Li, Guihua Wen*

0.75
0.7

SOFTMAX
RTML

0.75
0.7
0.65

0.65

0.8
SOFTMAX
RTML

0.8
0.75
0.7

0.7
0.65

0.65

0.6

0.6

0.6

0.6

0.55

0.55

0.55

0.6

0.8

1

0

0.4

0.8

1

0

Accuracy

SOFTMAX
RTML

0.7
0.65

0.4

0.6

0.8

1

0

40

0.95
0.9

0.9

0.85

0.85

SOFTMAX
RTML

0.8

0.7
0.65

SOFTMAX
RTML

0.8
0.75
0.7
0.65
0.6

0.6

0.55

0.55

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

45

SOFTMAX
RTML

0.7

0.6

0.4

1

0.65

0.55

Simplicity threshold

0.8

0.8

0.6

0.2

0.6

0.75

0.55
0

0.4

0.95

0.9

0.75

0.2

Simplicity threshold

0.85

0.85
0.8

0.2

Simplicity threshold

35

0.95

0.75

0.6

Simplicity threshold

30

0.9

0.2

Accuracy

0.4

Simplicity threshold

Accuracy

0.2

SOFTMAX
RTML

0.75

0.55
0

Accuracy

0.85

0.85

Accuracy

Accuracy

SOFTMAX
RTML

25

0.9

0.9

0.8

0.8

20

0.95

0.85

0.85

Accuracy

15

0.9

0.9

Accuracy

10

0.95

0

Simplicity threshold

0.2

0.4

0.6

0.8

1

0

Simplicity threshold

0.2

0.4

0.6

0.8

1

Simplicity threshold

Fig. 4 Experimental results of RTML algorithm with different numbers of models

0.7

0.6
0.2

0.4

0.6

0.8

1

1

0.6

0.8

1

0.7

SOFTMAX
RTML

0.2

0.4

0.6

0.8

1

1

0.2

0.4

0.6

0.8

0.7

0.6

0.6
0.5
0.2

0.4

0.6

0.8

Simplicity threshold

1

0.4

0.6

0.8

0.2

0.4

0.6

0.8

Simplicity threshold

1

0.8

1

SOFTMAX
RTML

Accuracy

0

0.2

0.4

0.6

0.8

1

Simplicity threshold
(0,06 400)

0.75
0.7

0.7
SOFTMAX
RTML

0.65

0.65

SOFTMAX
RTML

0.6

0.55
0

0.6

0.8

1

0.6

0.5
0

0.4

Simplicity threshold
(0,06 200)

0.6
0.2

Simplicity threshold
(0,04 400)

0.8

SOFTMAX
RTML

0.7

0.2

0.7

0.75

Accuracy

Accuracy

SOFTMAX
RTML

0

1

SOFTMAX
RTML

0

0.8

0.8

1

0.9

0.8

1

Simplicity threshold
(0,02 400)

0.9

0.9

0.8

0.6
0

Simplicity threshold
(0,00 400)

0.6

0.7

0.6
0

0.4

0.9

0.8
0.7

0.6

0.2

Simplicity threshold
(0,04 200)

1

Accuracy

SOFTMAX
RTML

SOFTMAX
RTML

0.6
0

0.9

0.8

Accuracy

Accuracy

0.4

Simplicity threshold
(0,02 200)

1

0.9

0.2

0.8
0.7

0.6
0

Simplicity threshold
(0,00 200)

SOFTMAX
RTML

0.7

0.6
0

0.9

0.8

Accuracy

0.7

SOFTMAX
RTML

(0,06 100)

1

0.9

0.8

Accuracy

SOFTMAX
RTML

(0,04 100)

1

0.9

0.8

Accuracy

Accuracy

0.9

Accuracy

(0,02 100)

1

Accuracy

(0,00 100)

1

0.55
0

0.2

0.4

0.6

0.8

Simplicity threshold

1

0

0.2

0.4

0.6

0.8

1

Simplicity threshold

Fig. 5 Experimental results of RTML algorithm with different numbers of models

without any knowledge of the classification procedures
that will be used for testing. Here, 15 real datasets,
shown in Table 4, are applied to conduct experiments
using the 10-fold cross-validation method to determine
the training data and testing data, and then the average
classification accuracy can be obtained. In each set of
training data, the 10-fold cross-validation method is
used to select the optimal parameters for the given
classifier.

It can be easily seen from Table 4 that the RTML
algorithm is much effective for most datasets. The
RTML algorithm’s performance was especially prominent in the fourclass, svmguide1, waveform, glass1,
and phoneme data, outperforming SOFTMAX in terms
of accuracy by 30%, 10%, 10%, 10%, and 14%, respectively, indicating that illusion inertial thinking
definitely exists, so that the reverse thinking model
can effectively correct these illusions. This can be
confirmed from the confusion matrices. For exam-

Modeling reverse thinking for machine learning

9

Table 4 Experimental results on real datasets for which SOFTMAX is learning method
No.

Data name

SOFTMAX
(all data)

SOFTMAX
(simple data)

RTML algorithm

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

fourclass
waveform
svmguide1
madelon
splice
thyroid
titanic
segment0
vehicle1
glass1
pima
haberman
spambase
mammographic
phoneme

0.6893±0.0316
0.7742±0.0218
0.8460±0.0140
0.5365±0.0142
0.8405±0.0235
0.8985±0.0671
0.7733±0.0179
0.9970±0.0036
0.8062±0.0466
0.6397±0.0842
0.6849±0.0355
0.7448±0.0425
0.9180± 0.0143
0.7901±0.0509
0.7541±0.0186

0.6893±0.0316
0.7754±0.0231
0.8447±0.0112
0.5685±0.0376
0.8405±0.0216
0.9212±0.0534
0.7833±0.0108
0.9970±0.0036
0.8003±0.0477
0.6111±0.0516
0.6875±0.0300
0.7414±0.0425
0.9213±0.0115
0.7937±0.0491
0.7648±0.0174

0.9837±0.0103
0.8710±0.0236
0.9314±0.0175
0.488±0.0430 0
0.8368±0.0219
0.9396±0.0577
0.7815±0.0125
0.9965±0.0034
0.7920±0.0573
0.7185±0.0522
0.7148±0.0351
0.7148±0.0787
0.9245±0.0096
0.7913±0.0502
0.8960±0.0154

ple, CF(svmguide1) is the confusion matrix of the
svmguide1 data performed by SOFTMAX, where the
significant illusion inertial thinking is established, as it
often classifies the samples with class 1 as belonging to
class 2, but notvice versa:


0.6513 0.3487
CF (svmguide1) =
,
0.0517 0.9483


0.7951 0.2049
CF (f ourclass) =
.
0.3773 0.6227
However, the RTML algorithm fails in some data,
such as madelon and splice, performing poorly on these
two data. The reason is that there the illusion inertial
thinking is not obvious. It can be seen from the confusion matrix, such as CF(madelon), that the illusion
inertial thinking schemes between the two classes are
very similar. The thinking falls into chaos and cannot
form stable illusion inertial thinking in one direction,
leading to the failure of the reverse thinking model. In
this case, good measures should be applied to maintain
suitable inertial thinking and enhance illusion inertial
thinking, so as to improve the overall performance. This
is because strengthening the inertial thinking may also
significantly strengthen the illusion inertial thinking.
Both are consistent and not in conflict with each other:


0.8354 0.1646
CF (splice) =
, CF (madelon) =
0.1481 0.8519


0.7546 0.2454
.
0.2516 0.7484
Additionally, on some datasets, such as madelon
and mammographic, SOFTMAX performs better by
taking easy samples as the training database rather
than by taking all the samples as the training database.
This indicates that it is reasonable to rank samples in
training database by their simplicity.

4 Conclusions
Similar to human learning, machine learning can easily
form illusion inertial thinking, but all machine learning
methods do not consider it. In this paper, we use reverse
thinking to overcome illusion inertial thinking, so as to
improve the generalization ability of machine learning
methods. The experimental results indicate that our
method is very effective. The proposed method is
universal, applicable to any machine learning methods,
especially to those performing badly on some data,
such as unbalanced data, for which machine learning
methods can easily form the illusion inertial thinking.
In planned future work, we will select some machine
learning methods to combine with the RTML algorithm to solve concrete tasks like emotion recognition,
because the training data for emotion recognition are
generally unbalanced, easily inducing machine learning
methods to form illusion inertial thinking. Second, the
proposed method uses the Bayesian theorem, but not
the Bayesian network. As a matter of fact, reverse
and inertial thinking between categories are linked, and
the dialectical unity between them will be investigated
under the framework of the Bayesian network in the
future [4].

Funding
This study was supported by the China National Science Foundation (60973083/61273363), Science and Technology Planning Project of Guangdong Province (2014A010103009/2015A020217002),
and Guangzhou Science and Technology Planning
Project(201504291154480).

10

Conflicts of interest
The authors declare that they have no conflicts of
interest.

Ethical approval
This paper does not contain any studies with human or
animals participants.

References
1. Alcal-Fdez J, Fernndez A, Luengo J, Derrac J, Garca
S, Snchez L, Herrera F (2011) Keel data-mining software
tool: Data set repository, integration of algorithms and
experimental analysis framework. J Mult-Valued Logic Soft
Comput17:255–287.
2. Chang CC, Lin CJ (2011) Libsvm: a library for support
vector machines. ACM Transac Intell Syst Technol (TISJ)
2(27):1–27.
3. Corcoran K, Hundhammer T, Mussweiler T (2009) A tool
for thought! When comparative thinking reduces stereotyping effects. J Exp Soc Psychol 45(4):1008–1011.
4. DeMotta Y, Chen ho Chao M, Kramer T (2016) The effect
of dialectical thinking on the integration of contradictory
information. J Consum Psychol 26(1):40–52.
5. Hoeffding W (1963) Probability inequalities for sums of
bounded random variables. J Am Stat Assoc 58(301):13–
30.
6. Ji Y, Chen Y, Fu H, Yang G (2017) An EnKF-based
scheme to optimize hyper-parameters and features for SVM
classifier. Pattern Recognit 62:202–213.
7. Lake BM, Salakhutdinov R, Tenenbaum JB (2015)
Human-level concept learning through probabilistic program induction. Science 350(6266):1332–1338.
8. LeCun Y, Bengio Y, Hinton G (2015). Deep learning.
Nature 521(7553):436–444.
9. Sawaguchi M (2015) Research on the efficacy of creative
risk management approach based on reverse thinking.
Proced Eng 131:577–589.
10. Smith MR, Tony Martinez, Giraud-Carrier C (2014) An
instance level analysis of data complexity. Mach Learn
95(2):225–256.
11. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan
D, Goodfellow I, Fergus R (2014) Intriguing properties
of neural networks. ICLR. http://arxiv.org/abs/1312.6199.
Accessed dd mm yyyy.
12. Wen G, Wei J, Wang J, Zhou T, Chen L (2013) Cognitive
gravitation model for classification on small noisy data.
Neurcomputing 118:245–252.
13. Zhang H, Berg AC, Maire M, Malik J (2006) SVMKNN: Discriminative nearest neighbor classification for
visual category recognition. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, 17–22 June 2006, pp 2126–2136.
http://dx.doi.org/10.1109/CVPR.2006.301

Huihui Li, Guihua Wen*

