Feature Selection for Microarray Gene Expression Data
using Simulated Annealing guided by the Multivariate Joint Entropy
Félix Fernando González-Navarro∗, Lluı́s A. Belanche-Muñoz†

arXiv:1302.1733v1 [q-bio.QM] 7 Feb 2013

∗ Instituto

de Ingenierı́a
Universidad Autónoma de Baja California, Mexicali, México
fernando.gonzalez@uabc.edu.mx
† Dept.

de Llenguatges i Sistemes Informàtics
Universitat Politècnica de Catalunya, Barcelona, Spain
belanche@lsi.upc.edu

Abstract—In this work a new way to calculate the multivariate joint entropy is presented. This measure is the basis for a
fast information-theoretic based evaluation of gene relevance in
a Microarray Gene Expression data context. Its low complexity
is based on the reuse of previous computations to calculate
current feature relevance.
The µ-TAFS algorithm –named as such to differentiate
it from previous TAFS algorithms– implements a simulated
annealing technique specially designed for feature subset
selection. The algorithm is applied to the maximization of
gene subset relevance in several public-domain microarray
data sets. The experimental results show a notoriously high
classification performance and low size subsets formed by
biologically meaningful genes.
Keywords-Feature Selection; Microarray Gene Expression
Data; Multivariate Joint Entropy; Simulated Annealing.

I. I NTRODUCTION
In cancer diagnosis, classification of the different tumor
types is of great importance. An accurate prediction of
different tumor types provides better treatment and toxicity
minimization on patients. Traditional methods of tackling
this situation are primarily based on morphological characteristics of tumorous tissue [1]. These conventional methods
are reported to have several diagnosis limitations. In order
to analyze the problem of cancer classification using gene
expression data, more systematic approaches have been
developed [2].
Pioneering work in cancer classification by gene expression using DNA microarray showed the possibility to
help the diagnosis by means of Machine Learning or more
generally Data Mining methods [3], which are now extensively used for this task [4]. However, in this setting
gene expression data analysis entails a heavy computational
consumption of resources, due to the extreme sparseness
compared to standard data sets in classification tasks [5].
Typically, a gene expression data set may consist of
dozens of observations but with thousands or even tens
of thousands of genes. Classifying cancer types using this

very high ratio between number of variables and number of
observations is a delicate process. As a result, dimensionality
reduction and in particular feature subset selection (FSS)
techniques may be very useful. Finding small subsets of
very relevant genes among a huge quantity could derive in
much specific and efficient treatments.
This work addresses the problem of selecting a subset of
features by using the TAFS (Thermodynamic Algorithms for
Feature Selection) family of methods for the FSS problem.
Given a suitable objective function, the algorithm makes uses
of an special-purpose simulated annealing (SA) technique to
find a good subset of features that maximizes the objective
function. A distinctive characteristic of TAFS over other
search algorithms for FSS is its probabilistic capability
to accept momentarily worse solutions, which in the end
may result in better hypotheses. Despite their powerful
optimization capability, SA-based search algorithms usually
lack execution speed, involving long convergence times. In
consequence, they have been generally excluded as an option
in FSS problems, let alone in highly complex domains such
as microarray gene expression data. A few contributions using the classical SA algorithm for FSS are found in prostate
protein mass spectrometry data [6], marketing applications
[7], or parameter optimization in clustering gene expression
analysis [8].
Our answer to these computational problems is twofold.
First, we use a filter objective function for FSS (thus
avoiding the development of a predictive model for every
subset evaluation). Second, the objective function itself
is evaluated very efficiently based in the reutilization of
previous computations.
Specifically, a new way to calculate the multivariate joint
entropy for categorical variables is presented that is both
exact and very efficient. This measure is then used by
a SA-based TAFS algorithm to search for small subsets
of highly relevant genes in five public domain microarray
data sets. Classification experiments yield some of the best

results reported so far for these data sets and offer a drastic
reduction in subset sizes.
The paper is organized as follows: section II briefly
reviews the Simulated Annealing technique; section III
reviews and motivates the previous Thermodynamic Algorithms for feature subset selection; section IV develops the
information-theoretic measure for feature relevance and its
efficient implementation; section V describes the data sets
and the experimental settings; section VI presents the results
and their interpretation. The paper ends with the conclusions
and directions for future work.
II. S IMULATED A NNEALING
Simulated Annealing (SA) is a stochastic technique inspired on statistical mechanics for finding (near) globally
optimal solutions to large optimization problems. SA is a
weak method in that it needs almost no information about
the structure of the search space. The algorithm works by
assuming that some parts of the current solution belong to a
potentially better one, and thus these parts should be retained
by exploring neighbors of the current solution. Assuming
the objective function is to be minimized, then SA would
jump from hill to hill and hence escape or simply avoid
sub-optimal solutions.
When a system S (considered as a set of possible states)
is in thermal equilibrium (at a given temperature T ), the
probability that it is in a certain state s, called PT (s),
depends on T and on the energy E(s) of the state s. This
probability follows a Boltzmann distribution:

PT (s) =



exp − E(s)
kT
Z

, with Z =

X
s∈S



E(s)
exp −
kT

where k is the Boltzmann constant and Z acts as a normalization factor. Metropolis and his co-workers developed a
stochastic relaxation method that works by simulating the
behavior of a system at a given temperature T [9]. Being s
the current state and s′ a neighboring state, the probability
of making a transition from s to s′ is the ratio PT (s → s′ )
between the probability of being in s and the probability of
being in s′ :
PT (s → s′ ) =



PT (s′ )
∆E
= exp −
PT (s)
kT

(1)

where we have defined ∆E = E(s′ ) − E(s). Therefore, the
acceptance or rejection of s′ as the new state depends on the
difference of the energies of both states at temperature T .
If PT (s′ ) ≥ PT (s) then the “move” is always accepted.
It PT (s′ ) < PT (s) then it is accepted with probability
PT (s, s′ ) < 1 (this situation corresponds to a transition to a
higher-energy state).
Note that this probability depends upon the current temperature T and decreases as T does. In the end, there will

be a value of T low enough (the freezing point), wherein
these transitions will be very unlikely and the system will
be considered frozen. In order to maximize the probability
of finding states of minimal energy at every value of T ,
thermal equilibrium must be reached. To do this, according
to Metropolis, an annealing schedule is designed to prevent
the process from getting stuck at a local minimum. The SA
algorithm introduced in [10] consists in using the Metropolis
idea at each temperature T for a finite amount of time. In
this algorithm T is first set at a initially high value, spending
enough time at it so to approximate thermal equilibrium.
Then a small decrement of T is performed and the process
is iterated until the system is considered frozen.
If the cooling schedule is well designed, the final reached
state may be considered a near-optimal solution. However,
the whole process is inherently slow, mainly because of the
thermal equilibrium requirement at every temperature T .
III. T HERMODYNAMIC A LGORITHMS

FOR

FSS

In this section we review TAFS (Thermodynamic Algorithm for Feature Selection) and eTAFS, two algorithms for
FSS that were originally designed for problems of moderate
feature size (up to one hundred) [11]. If we consider FSS
as a search of possible feature subsets of the full feature set
X , then SA acts as a combinatorial optimization process
[12]. In this sense, TAFS and eTAFS find a subset of
features that optimize the value of a given objective function
J : P(X ) → R. From now on, we assume that this function
is to be maximized and that J ≥ 01 .
To this end, a special-purpose forward/backward mechanism is embedded into an SA algorithm, taking advantage
of its most distinctive characteristic, the probabilistic acceptance of worse scenarios over a finite time. This characteristic is enhanced by the notion of an ǫ-improvement: a
feature ǫ-improves a current solution if it has a higher value
of the objective function or a value not worse than ǫ%. This
mechanism is intended to account for noise in the evaluation
of the objective function (caused either by the finiteness of
the data set or introduced by the chosen resampling method).
The pseudo-code of TAFS is depicted in Algorithm 1.
The algorithm consists of two major loops. The outer loop
waits for the inner loop to finish and then updates T
according to the chosen cooling schedule. When this loop
reaches Tmin , the algorithm halts. It keeps track of the best
solution found (which is not necessarily the current one).
The inner loop is the core of the algorithm and is
composed of two interleaved procedures: Forward and Backward, that iterate until an equilibrium point is found. These
procedures work independently of each other, but share
information about the results of their respective search in
the form of the current solution. Within them, FSS takes
1 This is the case for accuracy, mutual information, distances and many
other useful measures.

Algorithm 4: Function >ǫ

Algorithm 1: TAFS algorithm for feature selection

1
2
3
4
5
6
7
8
9
10

input : X : Full Feature set {X1 . . . Xn }
J() : Objective Function
α() : Cooling Schedule
ǫ : Epsilon
T0 : Initial Temperature
Tmin : Final Temperature
Xcur ← ∅ Initial current subset
Jcur ← 0 Initial objective function value
T ← T0 Initial temperature
while T > Tmin do
repeat
Y ← Xcur
F orward (Xcur , Jcur )
Backward (Xcur , Jcur )
until Y = Xcur
T ← α(T )

Algorithm 2: Procedure Forward (Z, JZ are modified)
1
2

input : Z, JZ
repeat
x ← argmax J(Z ∪ {Xi })
Xi ∈X \Z

3
4
5
6
7
8
9
10
11
12

if >ǫ (Z, x, true) then
accept ← true
else
∆J ← J(Z ∪ {x}) − J(Z)
∆J
accept ← rand(0, 1) < e T
if accept then
Z ← Z ∪ {x}
if J(Z) > Jcur then
JZ ← J(Z)
until not accept

Algorithm 3: Procedure Backward (Z, JZ are modified)
1
2

input : Z, JZ
repeat
x ← argmax J(Z \ {Xi })
Xi ∈Z

3
4
5
6
7
8
9
10
11
12

if >ǫ (Z, x, f alse) then
accept ← true
else
∆J ← J(Z \ {x}) − J(Z)
∆J
accept ← rand(0, 1) < e T

1
2
3
4
5
6
7
8
9

A. eTAFS: an Enhanced TAFS Algorithm
A modification to Algorithm 1 aimed at speeding up
relaxation time is presented in this section. The algorithm
–named eTAFS, see Algorithms 5 and 6– is endowed with
a feature search window (of size l) in the backward step, as
follows. In forward steps always the best feature is added
(by looking all possible additions). In backward steps this
search is limited to l tries at random (without replacement).
The value of l is incremented by one at every thermalequilibrium point. This mechanism is an additional source
of non-determinism and a bias towards adding a feature
only when it is the best option available. On the contrary,
to remove one, it suffices that its removal ǫ-improves the
current solution. Another direct consequence is of course
a considerable speed-up of the algorithm. Note that the
design of eTAFS is such that it grows more and more
deterministic, informed and costly as it converges towards
the final configuration.
Algorithm 5: eTAFS algorithm for feature selection

if J(Z) > JZ then
JZ ← J(Z)
1
2
3
4

place and the mechanism to escape from local minima starts
working. These procedures iteratively add or remove features
one at a time in such a way that an ǫ-improvement is
accepted unconditionally, whereas a non ǫ-improvement is
accepted probabilistically. The pseudo-code for Forward and
Backward, and ǫ-improvement is outlined in Algorithms 2,
3 and 4. When Forward and Backward finish their respective
tasks, TAFS checks if the current solution is the same as it

∆x ← J(Z ′ ) − J(Z)
if ∆x > 0 then
return true
else
−∆x
return J
<ǫ
(Z)

was prior to their execution. If this is the case, then we
consider that thermal equilibrium has been reached and T
is adjusted, according to the cooling schedule. If it is not,
another loop of Forward and Backward is carried out.

if accept then
Z ← Z \ {x}

until not accept

input : Z, x, d
output: boolean
if d then
Z ′ ← Z ∪ {x}
else
Z ′ ← Z \ {x}

5
6
7
8
9
10
11
12

input : X : Full Feature set {X1 . . . Xn }
J() : Objective Function
α() : Cooling Schedule
ǫ : Epsilon
T0 Initial Temperature
Tmin Final Temperature
Xcur ← ∅ Initial current subset
Jcur ← 0 Initial objective function value
T ← T0 Initial temperature
l ← 2 Window size (for backward steps)
while T > Tmin do
repeat
Y ← Xcur
F orward (Xcur , Jcur , l)
Backward (Xcur , Jcur , l)
until Y = Xcur
T ← α(T )
l←l+1

Algorithm 6: eTAFS Backward procedure (Z, JZ are
modified). Note that X0 can be efficiently computed
while in the for loop).
1
2
3
4
5
6

I(X; Y |Z) = H(Y |Z) − H(Y |X, Z)

if X0 ∈ A then
accept ← true
else
∆J ← J(Z \ {X0 }) − J(Z)
∆J
accept ← rand(0, 1) < e t

11
12
13

if accept then
Z ← Z \ {X0 }

14
15

if J(Z) > JZ then
JZ ← J(Z)

16
17
18

(6)

The MI has been used with success as for feature selection in machine learning tasks. Currently there is no
agreed-upon definition of the general multivariate mutual
information I(X1 ; . . . ; Xn ). An existent proposal is the
interaction information, described e.g. in [14] which, for the
case of three variables X, Y, Z, is defined as I(X; Y ; Z) =
I(X; Y |Z)−I(X; Y ). The extension to the multivariate case
is in terms of the marginal entropies and is given by:

X∈AB
9

(5)

Conditional MI is expressed in the natural way, by conditioning in (4):

X0 ← argmax{J(Z \ {X})}

10

I(Xi ; Y |X1 , . . . , Xi−1 )

= H(Y ) − H(Y |X1 , . . . , Xn ).

AB ← AB ∪ {x}

8

n
X
i=1

input : Z, JZ , l
A ← ∅; AB ← ∅
repeat
for i := 1 to min(l, |Z|) do
Select x ∈ Z \ AB randomly
if >ǫ (Z, x, f alse) then
A ← A ∪ {x}

7

I(X1 , . . . , Xn ; Y ) =

until not accept

I(X1 ; . . . ; Xn ) = −

X

(−1)n−|τ | H(τ ).

τ ⊆{X1 ,...,Xn }

IV. I NFORMATION T HEORETIC F EATURE R ELEVANCE
A. Entropy definitions
Entropy, a main concept in information theory [13], can
be seen as an average of uncertainty in a random variable.
If X is a discrete random variable with probability mass
function p, its entropy is defined by2 :

This definition is impractical due to its exponential character. In the next section, the objective function J takes the
form of an information-theoretic index of relevance based
on the multivariate joint entropy, which has already been
used elsewhere [15]. One of the contributions of this paper
resides in a fast implementation of the calculation and its
application to microarray gene expression data.
B. Incremental Multivariate Joint Entropy

H(X) = −

X

p(x) log p(x) = −EX [log p(X)]

(2)

x

being E[ ] the expectation operator. If a variable (X) is
known and another one (Y ) is not, the conditional entropy
of Y with respect to X: is the mutual entropy with respect
to the corresponding conditional distribution:
H(Y |X) = −

XX
x

p(x, y) log p(y|x).

(3)

I(X; Y ) = H(Y ) − H(Y |X) = EX,Y [log

p(x, y)
]. (4)
p(x)p(y)

The computation of the MI can be extended from the
bivariate to the multivariate case of a number n ≥ 2 of
variables, against another one:
log are to base 2.

H(X, Y ) ≥ H(X)

(7)

This property says that joint entropy is always at least
equal to the entropies of the original system: adding a new
variable can never reduce the available uncertainty. If we
rewrite (7) as an equation:

y

From these two definitions another concept is build, the
mutual information (MI), which can be interpreted as a
measure of the information that a random variable has or
explains about another one.

2 All

For a random variable X, it is known that the joint entropy
obeys the following property:

H(X, Y ) = H(X) + △X (Y ),

(8)

then △X (Y ) ≥ 0 represents the increment in entropy due
to the addition of the variable Y to the system. In a feature
selection setting, given Z a class variable, τ ⊂ X the current
subset and H(τ ) its joint entropy, if a new feature Xi ∈ X \τ
is considered for possible inclusion in the current subset:
H(Z, τ ∪ {Xi }) = H(Z, τ ) + △Z,τ (Xi )

(9)

It turns out that, to obtain the next calculation, it is
computationally far more advantageous to store H(Z, τ ) and
calculate the quantity △Z,τ (Xi ) than to compute the full

X1

P (X1 )

−P (X1 ) log P (X1 )

0
1

0.538
0.462
H(X1 ) =

0.481
0.515
0.996

X1

X2

P (X1 , X2 )

−P (X1 , X2 ) log P (X1 , X2 )

0
0

0
1

1
1

0
1

0.231
0.308
H(X1 , X2 ) =
0.154
0.308
H(X1 , X2 ) =

0.488
0.523
1.011
0.415
0.523
0.939

Table I
Marginal Entropy Scheme (MES) TABLES FOR ONE VARIABLE ( LEFT ) AND THE ADDITION OF A SECOND VARIABLE ( RIGHT ). P (·) IS THE
PROBABILITY MASS FUNCTION , OBTAINED FROM THE DATA ( ALL ENTROPIES ARE IN BITS ).

joint entropy H(Z, τ ∪{Xi }) directly. In order to obtain this
value, and incremental procedure to calculate multivariate
joint entropy has been developed, as described in the sequel.
The incremental multivariate joint entropy (9) must be
computed at every evaluation step involving a possible
candidate feature Xi to be included in the current subset
τ . Throughout the process, τ is associated with its current
Marginal Entropy Scheme (MES), a table storing the unique
values contained in the data set for its forming features and
its corresponding entropy value. An example of a MES table
for two binary variables {X1 , X2 } is shown in Table I.
At the initial step (τ = ∅) the MES table for the addition
of X1 to ∅ is indicated in the left part of Table I. The two
unique values and their entropies H(X1 = 0) = 0.481
and H(X1 = 1) = 0.515 are calculated. Let us suppose
that a feature X2 is to be evaluated w.r.t the current subset
τ = {X1 }. The MES table with its unique forming patterns
are indicated in the right part of Table I. We can see that
by introducing X2 to the current subset τ , four partitions
are generated for each unique value of X1 : {00, 01, 10, 11}.
In the particular case of X1 = 0, a change in its entropy
contribution is produced by the action of X2 by splitting
it into two entropy values: H(X1 = 0, X2 = 0) = 0.488
and H(X1 = 0, X2 = 1) = 0.523, for a total entropy
of H(X1 = 0, X2 ) = 1.011. The increment in entropy
△τ is obtained as the difference between the current MES
(considering the addition of X2 ) and the previous scheme
(without it) –see Table II.
△τ
△τ (X1 = 0)
△τ (X1 = 1)

△τ

H(X1 , X2 ) −P (X1 ) log P (X1 ) difference
1.011
0.939

COMPUTATIONS FROM THE

0.481
0.515
△τ

0.531
0.424
0.954

Table II
Marginal Entropy Scheme – SEE TABLE I.

Algorithm 7: Incremental Multivariate Joint Entropy

1
2
3
4
5
6
7
8
9
10

11

input : τ : Current subset;
Xi : feature to be added;
Hτ : Current subset joint entropy;
Eτ : Marginal entropies scheme of Hτ ;
D : Data set;
output: τ, Hτ , Eτ
if |τ | = 0 then
τ ← {Xi }
D ← Sort(D)
Hτ ← Joint Entropy of D
Eτ ← M arginalEntropyScheme(D|τ )
else
τ + ← τ ∪ {Xi }
Sort(D|τ + )
+
Eτ + ← M
XarginalEntropyScheme(D|τ )
Eτ − ←
Eτj //j runs through the values of τ
△τ ←

Xj

Eτi + − Eτi −

i

12
13
14

τ ← τ+
Hτ ← Hτ + △ τ
Eτ ← Eτ + // new MES

15

4-5), which will be taken as input to the next computation.
Note that these two lines can be efficiently implemented as
one function and using only one loop-cycle, with complexity
θ(|D|), where |D| is the number of training instances.
In the else part of the if clause, the MES is calculated
with the addition of Xi to the current subset τ (named
Eτ + ). Taking into account that previous MES inherits the
ordering sequence derived from a previous stage (because
of lines 5 and 9), entropies generated by changes in the
MES given by τ ∪ {Xi } are summed (Eτ − ) in groups (line
11) by the newly formed patterns, rendering a one-to-one
correspondence between previous MES and current MES.
Algorithm 8: MarginalEntropyScheme Function

Finally, this last value is applied to eq. (9) to obtain the
joint entropy H(X1 , X2 ) = H(X1 ) + △τ (X2 ) = 0.996 +
0.954 = 1.950. The listings in Algorithms 7 and 8 show
the pseudo-code to compute the procedure explained above.
The notation D|τ stands for the restriction of the dataset D
to the features in τ .
Initial entropy is evaluated in lines 2-5. This first step
calculates starting joint entropy as well as its first MES (lines

2

input : D : Data set;
output: E
foreach unique value v in D do
Υ[v] ← fraction of instances in D with value v

3

E ← H(Υ) //calculate entropy of this distribution

1

Thus, the entropy contribution △τ (Xi ), showing the effect of adding Xi to τ , is computed by the difference in both

MESs (line 11), being finally added to the current entropy
Hτ (line 13). The implementation of lines 10-11 follows the
same consideration as lines 4-5, and hence complexity is in
the same order.
The incremental multivariate joint entropy is used to obtain an index of relevance (acting as the objective function)
of a feature Xi ∈ X to a class Z with respect to a subset
τ ⊂ X \ Xi and is defined by:
J(Xi ; Z|τ ) =

H(Z) + H(τ, Xi ) − H(τ, Z, Xi )
H(Z)

(10)

Note the denominator acts as a normalization factor, such
that J ∈ [0, 1], with J = 1 corresponding to the highest
relevance. The reward of using this objective function by a
TAFS-like algorithm consists in the possibility of testing it
in highly complex domains such as microarray data sets. We
name the combination of eTAFS and the objective function
in eq. (10) as the µ-TAFS algorithm.
V. E XPERIMENTAL W ORK
To compute the necessary entropies described in previous
section, a discretization process is needed. This change of
representation does not often result in a significant loss of
accuracy (sometimes significantly improves it [16], [17]); it
also offers reductions in learning time [18]. In this work, the
CAIM algorithm was selected for two reasons: it is designed
to work with supervised data, and does not require the user
to define a specific number of intervals [19].
A. Data sets
Five public-domain microarray gene expression data sets
are used to test and validate the approach proposed in
this work: Colon Tumor: 62 observations of colon tissue,
of which 40 are tumorous and 22 normal, 2,000 genes
[20]. Leukemia: 72 bone marrow observations and 7,129
probes: 6,817 human genes and 312 control genes [3]. The
goal is to tell acute myeloid leukemia (AML) from acute
lymphoblastic leukemia (ALL). Lung Cancer: distinction between malignant pleural mesothelioma and adenocarcinoma
of lung [21]; 181 observations with 12,533 genes. Prostate
Cancer: used in [22] to analyze differences in pathological
features of prostate cancer and to identify genes that might
anticipate its clinical behavior; 136 observations and 12,600
genes. Breast Cancer: 97 patients with primary invasive
breast carcinoma; 12,600 genes analyzed [23].
B. Settings
Provided that the core nature of the µ-TAFS algorithm,
and even many other algorithms –e.g. Genetic Algorithms,
Neural Networks–, resides in their stochasticity, several runs
have to be performed, in order to better asses the average
behaviour of the methods.
The experimental design to test µ-TAFS algorithm measures performance by carrying out m = 100 different

independent runs. In each run, µ-TAFS is executed on
the corresponding dataset and returns the set of all those
feature subsets reaching the best found performance function (maximum relevance, in this case). To overcome the
existence of many solutions, the subset that offers the lowest
mutual information (MI) among its elements –i.e. the less
redundancy– is taken as the subset delivered in this run.
After completing the m execution runs, the obtained subsets
can be ordered from minimum to maximum MI value.
The µ-TAFS parameters are as follows: ǫ = 0.01, T0 =
0.1 and Tmin = 0.0001. These settings were chosen after
preliminary fine-tuning and are kept constant for all the
problems [11]. The cooling function was chosen to be
geometric α(t) = 0.9 t, following recommendations in the
literature [12].
Data set
Colon Tumor
Leukemia
Lung Cancer
Prostate Cancer
Breast Cancer

Time
6.41
6.51
7.45
98.74
136.93

Jeval
503,901
506,489
560,972
7,119,800
10,943,628

size
6.93 ± 0.06
3.36 ± 0.06
2.58 ± 0.04
9.85 ± 0.05
9.62 ± 0.03

Table III
µ-TAFS RUNNING PERFORMANCE . Time INDICATES THE AVERAGE
RUNNING TIME ( IN MINUTES ) OVER THE 100 EXECUTIONS ; Jeval IS THE
AVERAGE NUMBER OF EVALUATIONS OF J ; size THE AVERAGE SIZE OF
THE FINAL SOLUTIONS AND ITS STANDARD ERROR .

VI. E XPERIMENTAL RESULTS
A. µ-TAFS performance results
The evolution of µ-TAFS from a high temperature state to
a frozen point is depicted in Fig. 1. Highly unstable –i.e. high
temperature condition– readings are observed at the initial
stages in each of the datasets. As soon as the algorithm
becomes more relaxed due to eq. (1), worse solutions are
avoided. The frozen condition is observed at the final stages
of each execution, where J values consecutively reach the
maximum possible value (J = 1) in all cases.
The running performance of µ-TAFS is summarized in
Table III. The results show that µ-TAFS yields subsets of
considerably low size and also low variability. Notorious
readings correspond to Leukemia and Lung Cancer. It is
conjectured that such sizes respond to the nature of the proposed information theoretic model on discretized data sets,
in the sense that only a few genes significantly contribute
to increase the index of relevance given by eq. (10). On the
one hand, working with continuous features, the index would
tend to vary smoothly –i.e. generating small increments; as a
consequence, more features are added-deleted. On the other
hand, discrete features variations are normalized by their
discretization scheme, so small increments in the real-value
are merged into a single discrete value. Therefore, mostly
significant increments are truly reflected in its additiondeletion from the current subset.

Leukemia

1

1

0.8

0.8

0.6

0.6

J

J

Colon Tumor

0.4

0.4

0.2

0.2

0
0

20

40
60
80 100
Index of Temperature

0
0

120

80

Prostate Cancer

1

1

0.8

0.8

0.6

0.6

J

J

Lung Cancer

20
40
60
Index of Temperature

7)3 . The non parametric Wilcoxon signed-rank test4 is used
for the (null) hypothesis that the median of the differences
between the errors of the winner classifiers per data set
and another classifier’s error is zero. The non-parametric
Wilcoxon signed-rank test will be used for the (null) hypothesis that the median of differences between classifiers
accuracies are zero, at the 95% level of significance.
Data set
Colon Tumor
Leukemia
Lung Cancer
Prostate Cancer
Breast Cancer

Classifier

10x10 CV

size

lSVM (C = 21 )
lSVM (C = 2−7 )
LR
NN (6)
rSVM (C = 23 , γ = 2−1 )

89.19±0.38
99.62±0.27
99.89±0.07
95.66±0.21
86.90±0.48

5
3
4
7
6

0.4

0.4

0.2

Table IV
µTAFS: 10 X 10 MEAN CROSS - VALIDATION ACCURACY (10x10 CV )

0.2

COMPLEMENTED WITH ITS STANDARD ERROR FOR THE BEST MODEL IN
EACH DATA SET. T HE Classifier COLUMN INDICATES THE BEST METHOD
ALONG WITH BEST PARAMETERS .

0
0

20
40
60
Index of Temperature

80

0
0 20 40 60 80 100 120 140 160 180 200

Index of Temperature

Breast Cancer
1

J

0.8
0.6
0.4
0.2
0
0 20 40 60 80 100 120 140 160 180 200

Index of Temperature

Figure 1. µ-TAFS search processes. The x-axis is the iteration counter
for the outer loop of the algorithm.

Computational demands when processing the smaller data
sets (as for Colon tumor, with 2,000 features) are kept
by µ-TAFS considerable low (5 to 10 minutes). The two
more complex problems, Prostate and Breast Cancer require approximately 1.5 and 2 hours of total processing
time. Unfortunately, there is scarcely any reporting on time
consumption in recent scientific literature that would enable
us to establish a reasonable comparison.
B. µ-TAFS accuracy results
Eight classifiers were evaluated by means of 10 times
10-fold Cross Validation (10x10 CV), a resampling method
designed to handle small-sized data sets. The chosen classifiers are: the nearest-neighbor technique with Euclidean
metric (kNN) and parameter k (number of neighbors running
from 1 to 15), the Naı̈ve Bayes classifier (NB), a Linear and Quadratic Discriminant classifier (LDC), Logistic
Regression (LR), the Support Vector Machine with linear
and quadratic kernel (lSVM and rSVM) and parameter Cregularization constant (with C = 2k , k running from −7
to 7) and the Support Vector Machine with radial basis
function kernel (rSVM) and parameter C and γ-smoothing
in the kernel function (with γ = 2k , k running from −7 to

The obtained solutions are displayed in Table IV. Among
the eight classifiers used to test the solutions, only the final
model is presented. Lung Cancer, Leukemia and Prostate
Cancer reach remarkably high accuracies, while Colon
Tumor and specially Breast Cancer show lower 10x10
CV readings. In all cases, the subset that delivers this
performance is considerable small, having 7 genes or less
(and only 3 genes in the Leukemia data set). Moreover,
all Wilcoxon test p-values signal significant differences
(p < 0.05) between the best method and all other methods
in the corresponding data set, except for the lSVM vs. LR
in Colon Tumor (p = 0.312).
C. Discussion of the results
It is a common practice to compare to similar works in
the literature. Unfortunately, the methodological steps are
in general very different, especially concerning resampling
techniques, making an accurate comparison a delicate undertaking. Nonetheless, such a comparison is presented in Table
V. Seven references which are illustrative of recent work are
indicated, including previous work from the authors. In this
table the validation method, the best classifier and the best
reported result are detailed (final accuracy and number of
genes involved).
The Colon Tumor data set presents difficulties in classification, never reaching 90%. The solution delivered by
µ-TAFS is comparable with the best known (that of BGS 3
[25]); however, it uses 5 genes against the 9 used by BGS 3 .
3 For the experiments, we use a MATLAB implementation; specifically,
for the SVMs we use the MATLAB interface to LIBSVM [24]. All tests
are run on on a regular x86 workstation.
4 The Wilcoxon signed-rank test is a non-parametric statistical hypothesis
test for the analysis of two related samples, or repeated measurements on
a single sample. It can be used as an alternative to the paired Student’s
t-test when the population cannot be assumed to be normally distributed.
It should therefore be used whenever the distributional assumptions that
underlie the t-test cannot be satisfied.

Colon
Lung
Breast
Prostate
Tumor
Leukemia
Cancer
Cancer
Cancer
89.36
97.89
98.84
83.37
93.43
(9,3NN)
(2,NB)
(4,LR)
(12,lSVM) (3,10NN)
[26](F) 200-B.632
88.75
98.2
−
−
−
(14,lSVM) (23,lSVM)
−
−
−
[27](W) 10x10CV
85.48
93.40
−
−
−
(3,NB)
(2,NB)
−
−
−
[28](W)
100-RS
87.31
−
72.20
−
−
(94,SVM)
−
(23,SVM)
−
−
[29](W)
50-HO
77.00
96.00
99.00
79.00
93.00
(33,rSVM) (30,rSVM) (38,rSVM) (46,rSVM) (47,rSVM)
[30](FW) 10x10CV
−
−
99.40
−
96.30
−
−
(135,5NN)
−
(79,5NN)
[31](F)
10CV
−
98.6
99.45
68.04
91.18
−
(2,SVM) (5,SVM ) (8,SVM)
(6,SVM)
Work
[25](F)

Validation
10x10CV

Table V
B EST RESULTS REPORTED IN THE LITERATURE FOR THE EXPLORED PROBLEMS : (F) INDICATES THAT THE REFERENCED WORK USES A F ILTER -BASED
A LGORITHM , (W) FOR WRAPPER AND (F-W) FOR A COMBINATION OF BOTH SCHEMES ; IN PARENTHESES , THE SIZE OF THE SUBSET ( NUMBER OF
GENES ) AND THE INDUCER OPTIMIZED . A − SIGN INDICATES THAT THE PROBLEM WAS NOT STUDIED BY THE REFERENCE . T HE VALIDATIONS ARE :
10 X 10 CV (10 TIMES 10- FOLD C ROSS VALIDATION ), 10 CV (10- FOLD C ROSS VALIDATION ), 100-RS (100 TIMES R ANDOM S UBSAMPLING ), 50-HO
(50 TIMES H OLDOUT ) AND 200-B.632 (0.632 BOOTSTRAP OF SIZE 200).

The other difficult problem seems to be Breast Cancer.
In this data set, µ-TAFS gives the best result among the
references consulted, using also less genes and in front of
solutions that employ a pure wrapper strategy. For the other
three problems, µ-TAFS is also able to yield better solutions
compared to other approaches, many of them using a much
bigger gene subset.
Expression levels for each model in the five data sets are
given in Fig. 2. It is seen that each model posses genes
that are visually identified as the ones that present irregular
expression levels: Colon Tumor genes M76378 and T51288;
Leukemia genes AFFX-CreX-5 at and L09209; Lung Cancer gene 37157 at; Prostate Cancer genes 38322 at and
37639 at; and Breast Cancer genes Contig14882 RC, Contig53822 RC and Contig57657 RC.
Data set
Colon Tumor
Leukemia
Lung Cancer
Prostate Cancer
Breast Cancer

Gene ID
M76378, H08393, T51849, M19311, T51288
AFFX-CreX-5 at, L09209, X75755
37157 at, 33221 at, 107 at, 40790 at
1230 g at, 38322 at, 37639 at, 32909 at, 660 at
35998 at, 34107 at
AB014543, Contig14882 RC, Contig53822 RC
Contig57657 RC, Contig53713 RC, NM 006191

Colon Tumor:
•

•

•

•

Table VI
G ENES IDENTIFICATION FOR EACH FINAL MODEL .

•

D. Biological evidence in the solution subsets
The genes corresponding to the solutions displayed in
Table IV are detailed in Table VI. In the following, known
biological evidence is presented about the effect of gene expressions in each cancer disease. This evidence is assembled
by examining recent relevant medical literature.

M76378 CSRP1-Cysteine and glycine-rich protein 1.
This gene encodes a member of the cysteine-rich protein (CSRP) family. It may be involved in regulatory
processes important for development and cellular differentiation. Hypomethylation, a decrease in the epigenetic methylation of cytosine and adenosine residues in
DNA, of CSRIP1 and other genes were confirmed in
the cancer cells by bisulfite sequencing [32].
H08393 COL11A2-collagen, type XI, alpha 2 (Homo
sapiens). Two alpha chains of type XI collagen, a minor
fibrillar collagen are encoded by this gene [33]. Upregulation of this gene in the mucosa stromal cells
of both familial adenomatosis polyposis and sporadic
colorectal cancer has been detected [34].
T51849 EPHB1-Tyrosine-protein kinase receptor elk
precursor. EphB1 is a member of receptor tyrosine
kinases of the EphB subfamily and has been positively
identified in the development, progress and prognosis
of colorectal cancers [35].
M19131 CALM2-calmodulin 2 (phosphorylase kinase,
delta). Caml2 plays an important role in intracellular
calcium signaling, which regulates a variety of cellular
processes, such as cell proliferation and gene transcription [36]. Increased expression levels of this gene were
found in anaplastic large cell lymphoma cell lines [37].
T51288 ASS1-argininosuccinate synthase (human).
Arginine, a semi-essential amino acid in humans, is
critical for the growth of human cancers as in primary
ovarian, stomach and colorectal cancer, whose expression levels read high values [38].

Leukemia:
•

AFFX-CREX-5 AT NOT IDENTIFIED.

Colon Tumor

Leukemia
10

10

20

20

30

•

30
40
40

50

50

60

60

70
2

4

1

Lung Cancer

2

3

Prostate Cancer
•
20

50

40
60

100

80
100

150

•

120
2

4

2

4

6

Breast Cancer

It is activated during amino-acid deprivation and has
been associated with neurodegenerative diseases, cancer, pathogen infections and myopathies [44].
33221 at PAXIP1-PAX interacting (with transcriptionactivation domain) protein 1. Member of the paired box
(PAX) gene family, this gene plays a critical role in
maintaining genome stability by protecting cells from
DNA damage [39], [45]. Analysis of pulmonary adenocarcinomas in experiment GDS1650 in [33] records
shows over-expression levels of this gene.
40790 at BHLHE40-basic helix-loop-helix family,
member e40. This gene encodes a basic helix-loophelix protein expressed in various tissues, which is may
be involved in the control of cell differentiation [33].
Experiments suggest that loss of DEC1 expression is
an early event in the development of lung cancer [46]
107 at RAB40A-member RAS oncogene family. This
gene encodes a member of the Rab40 subfamily of Rab
small GTP-binding proteins that contains a C-terminal
suppressors of cytokine signaling box [39]. No medical
evidence was found in literature about its role in cancer.

20

Prostate Cancer:

40

•

60
80

2

4

6

•

Figure 2. Expression levels of winner models as indicated in Table V.
Samples for each data set are distributed as follows: Colon Tumor: Tumor
1-41, Normal 42-62; Leukemia: Tumor 1-48, Normal 49-72; Lung Cancer:
Tumor 1-31, Normal 32-181; Prostate Cancer: Tumor 1-78, Normal 79136; and Breast Cancer: Tumor 1-46, Normal 47-97;
•

L09209 APLP2-amyloid beta (A4) precursor-like protein 2 (Homo sapiens). The function of this gene is
not fully understood, but it conjectured that may play a
role in the regulation of hemostasis [39]. This gene was
reported as over-expressed by other scientific literature
as in [40]
• X95735 at ZYX-ZYXIN. It is involved in the spatial
control of actin assembly and in the communication
between the adhesive membrane and the cell nucleus
[41]. This is a gene found in many cancer classification
studies [3], [42], [43], and is highly correlated with
acute myelogenous leukemia.
Lung Cancer:
• 37957 at ATG4-Autophagy related 4 homolog A. Autophagy is the process by which endogenous proteins
and damaged organelles are destroyed intracellularly.
Autophagy is postulated to be essential for cell homeostasis and cell remodeling during differentiation, metamorphosis, non-apoptotic cell death, and aging [39].
•

•

•

1230 g at MTMR11-myotubularin related protein 11.
Experiments on patients with acute lymphoblastic
leukemia and with Burkitt lymphoma, three putative
oncogenes or tumor suppressor genes were found, one
of them was the MTMR11 [47].
38322 at PAGE4-P antigen family, member 4 (prostate
associated). This gene is strongly expressed in prostate
and prostate cancer; and also expressed in other tissues
as in testis, fallopian tube, uterus, placenta, as well as
in testicular cancer and uterine cancer [39].
37639 at HPN-Hepsin. Hepsin is a cell surface serine
protease and plays an essential role in cell growth and
maintenance of cell morphology and it is highly related
with prostate cancer, benign prostatic hyperplasia [39].
32909 at AQP5-aquaporin 5. Acting as a water channel protein, Aquaporins are a family of small integral
membrane proteins linked to other proteins, whose
role is the generation of saliva, tears and pulmonary
secretions [39]. Experiments with cases of normal and
epithelial ovarian tumors tissues suggest an important
role of this gene in the tumorigenesis of the latter, and
a possible relationship with the ascites formation of
ovarian carcinoma [48].
660 at CYP24A1-cytochrome P450, family 24, subfamily A, polypeptide 1. This gene encodes a member
of the cytochrome P450 superfamily of enzymes. The
cytochrome P450 proteins catalyze many reactions involved in drug metabolism and synthesis of cholesterol,
steroids and other lipids [39]. This gene has been
reported as responsible for degradation of the active
vitamin D metabolite 1,25-dihydroxyvitamin D3 which

is known to be antimitotic in prostate cancer cells [49].
Hypothetical
protein
LOC284244
35998 at
(LOC284244). No evidence found.
• 34107 at PFKFB2-6-phosphofructo-2-kinase/fructose2,6-biphosphatase 2. The protein encoded by this
gene is involved in the synthesis and degradation of
fructose-2,6-bisphosphate, a regulatory molecule that
controls glycolysis in eukaryotes [39]. It has been
suggested that the induction of de novo lipid synthesis
–process that protects cancer cells from free radicals
and chemotherapeutics– by androgen requires the upregulation of HK2 and PFKFB2 [50].
Breast Cancer:
• AB014543 CLUAP1-clusterin associated protein 1
(Homo sapiens). This gene is highly expressed in
osteosarcoma, ovarian, colon, and lung cancers [51].
• Contig57657 RC PAK1-p21 protein (Cdc42/Rac)activated kinase 1 (Homo sapiens). This gene encodes
a family member of serine/threonine p21-activating
kinases, known as PAK proteins, whose role is the
regulation of cell motility and morphology [33]. Pak1
is directly related with the Etk/Bmx protein, acting this
later as a control to the proliferation and tumorigenic
growth of mammary epithelial cancer cells [52].
• NM 006191
PA2G4-Proliferation-associated 2G4,
38kDa (PA2G4). Also known as EBP1, this gene
encodes an RNA-binding protein that is involved in
growth regulation [39]. The EBP1 has been shown to
be a transcriptional corepressor that inhibits the growth
of human breast cancer cell lines [53].
Contig53822 RC,
Con• Contig14882 RC,
tig53713 RC NOT IDENTIFIED.
•

VII. C ONCLUSIONS
A new algorithm for feature selection using Simulated
Annealing guided by the discrete multivariate joint entropy
has been introduced and evaluated. Our experimental results
concern the search for small subsets of highly relevant
genes in five public domain Microarray Gene Expression
data samples. The very promising results indicate that the
algorithm offers a promising and general framework for
feature selection in very high dimensional data sets.
The entropic relevance measure has shown to be a good
candidate as the objective function to be optimized by the
algorithm. The reported classification results are competitive
to current standards in analyzing microarray gene expression
data with a very affordable execution time. This last aspect
should not be overlooked, since database size is constantly
growing and the complexity of optimization scenarios (that
make extensive use of resampling methods) is ever greater.

R EFERENCES
[1] F. Chu and L. Wang. Applications of support vector machines
to cancer classification with microarray data. International
Journal of Neural Systems, 15(6):475–484, 2005.
[2] Y. Lu and J. Han. Cancer classification using gene expression
data. Information Systems, 28:243–268, 2003.
[3] T. Golub, D. Slonim, P. Tamayo, C. Huard, M. Gaasenbeek,
J. Mesirov, H. Coller, M. Loh, J. Downing, M. Caligiuri,
C. Bloomfield, and E. Lander. Molecular classification of
cancer: Class discovery and class prediction by gene expression monitoring. Science, 286(5439):531–537, October 1999.
[4] Kai-Bo Duan, Jagath Rajapakse, Haiying Wang, and Francisco Azuaje. Multiple svm-rfe for gene selection in cancer
classification with expression data. IEEE/ACM Transactions
on Nanobioscience, 4(3):228–234, 2005.
[5] Yuchun Tang, Yan-Qing Zhang, and Zhen Huang. Development of two-stage svm-rfe gene selection strategy for
microarray expression data analysis. IEEE/ACM Transactions
on Computational Biology and Bioinformatics, 4(3):365–381,
July-September 2007.
[6] Y. Li and Y. Liu. A wrapper feature selection method
based on simulated annealing algorithm for prostate protein
mass spectrometry data. In Computational Intelligence in
Bioinformatics and Computational Biology, 2008. CIBCB ’08.
IEEE Symposium on, pages 195 –200, 2008.
[7] Ronen Meiri and Jacob Zahavi. Using simulated annealing to
optimize the feature selection problem in marketing applications. European Journal of Operational Research, 171(3):842
– 858, 2006.
[8] Maurizio Filippone, Francesco Masulli, and Stefano Rovetta.
Unsupervised gene selection and clustering using simulated
annealing. In Isabelle Bloch, Alfredo Petrosino, and Andrea
Tettamanzi, editors, Fuzzy Logic and Applications, volume
3849 of Lecture Notes in Computer Science, pages 229–235.
Springer Berlin / Heidelberg, 2006.
[9] N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and
E. Teller. Equations of state calculations by fast computing
machines. Journal of Chemical Physics, 21, 1953.
[10] S. Kirkpatrick. Optimization by simulated annealing: Quantitative studies. Journal of Statistical Physics, 34, 1984.
[11] F. González and L. Belanche. A thermodynamical search
algorithm for feature subset selection. In Masumi Ishikawa,
Kenji Doya, Hiroyuki Miyamoto, and Takeshi Yamakawa, editors, Neural Information Processing, volume 4984 of Lecture
Notes in Computer Science, pages 683–692. Springer Berlin
/ Heidelberg, 2008.

VIII. ACKNOWLEDGMENTS

[12] Colin R. Reeves. Modern Heuristic Techniques for Combinatorial Problems. McGraw Hill, 1995.

Authors wish to thank to Spanish CICyT Project no. CGL2004-04702C02-02, CONACyT and UABC for supporting this research from its
beginning.

[13] Claude E. Shannon. A mathematical theory of communication. The Bell System Technical Journal., 27:379–423, 1948.

[14] Aleks Jakulin and Ivan Bratko. Quantifying and visualizing
attribute interactions: An aproach based on entropy. In
Proceedings of the International Conference on Machine
Learning, 2004.
[15] Felix F. González and Lluı́s A. Belanche. Using machine
learning techniques to explore 1 h-mrs data of brain tumors. In
Mexican International Conference on Artificial Intelligence,
pages 134–139. IEEE Computer Society, 2009.
[16] Manfred Ng and Laiwan Chan. Informative gene discovery
for cancer classification from microarray expression data. In
IEEE Workshop on Machine Learning for Signal Processing,
pages 393–398. IEEE, 2005.
[17] George Potamias, Lefteris Koumakis, and Vassilis Moustakis.
Gene selection via discretized gene-expression profiles and
greedy feature-elimination. In SETN, pages 256–266, 2004.
[18] J. Catlett. On changing continuous attributes into ordered
discrete attributes. In Proceedings of the European working
session on learning on Machine learning, pages 164–178,
New York, NY, USA, 1991. Springer-Verlag New York, Inc.
[19] Lukasz A. Kurgan and Krzysztof J. Cios. Caim discretization
algorithm. IEEE Trans. on Knowledge and Data Engineering,
16(2):145–153, 2004.
[20] U. Alon, N. Barkai, D. Notterman, K. Gish, S. Ybarra,
D. Mack, and A. Levine. Broad patterns of gene expression
revealed by clustering analysis of tumor and normal colon
tissues probed by oligonucleotide arrays. In Proceedings of
The National Academy of Sciences USA, volume 96, pages
6745-6750. IEEE, 1999.
[21] Gavin J. Gordon, Roderick V. Jensen, Li-Li Hsiao, Steven R.
Gullans, Joshua E. Blumenstock, Sridhar Ramaswamy,
William G. Richards, David J. Sugarbaker, and Raphael
Bueno. Translation of microarray data into clinically relevant
cancer diagnostic tests using gene expression ratios in lung
cancer and mesothelioma. Cancer Research, 62:4963–4967,
September 2002.
[22] D. Singh, P. Febbo, K. Ross, D. Jackson, J. Manola, C. Ladd,
P. Tamayo, A. Renshaw, A. D’Amico, J. Richie, E. Lander,
M. Loda, P. Kantoff, T. Golub, and W. Sellers. Gene
expression correlates of clinical prostate cancer behavior.
Cancer Cell, 1:203–209, March 2002.
[23] L. Vant’Veer, H. Dai, M. Vijver, Y. He, A. Hart, M. Mao,
H. Peterse, K. Kooy, M. Marton, A. Witteveen, G. Schreiber,
R. Kerkhoven, C. Roberts, P. Linsley, R. Bernards, and
S. Friend. Gene expression profiling predicts clinical outcome
of breast cancer. Nature, 1(415):530–536, January 2002.
[24] C. Chang and C. Lin. Libsvm : a library for support vector
machines, 2002. In http://www.csie.ntu.edu.tw/∼cjlin/libsvm/.
[25] Felix F. González and Lluı́s A. Belanche. Parsimonious
selection of useful genes in microarray gene expression data.
In Hamid R. Arabnia and Quoc-Nam Tran, editors, Software
Tools and Algorithms for Biological Systems, volume 696 of
Advances in Experimental Medicine and Biology, pages 45–
55. Springer New York, 2011.

[26] Ruichu Cai, Zhifeng Hao, Xioaowei Yang, and Wen Wen. An
efficient gene selection algorithm based on mutual information. Neurocomputing, 72:991–999, 2009.
[27] Roberto Ruiz, José Riquelme, and Jesús Aguilar. Incremental
wrapper-based gene selection from microarray data for cancer
classification. Pattern Recognition, 39:2383–2392, 2006.
[28] Li Wang, Ji Zhu, and Hui Zou. Hybrid huberized support vector machines for microarray classification and gene selection.
Bioinformatics, 24(3):412–419, 2008.
[29] Hua-Long Bu, Guo-Zheng Li, and Xue-Qiang Zeng. Reducing error of tumor classification by using dimension reduction
with feature selection. In The First International Symposium
on Optimization and Systems Biology (OSB0́7), pages 232–
241, 2007.
[30] Jin-Hyuk Hong and Sung-Bae Cho. Cancer classification
with incremental gene selection based on dna microarray
data. In IEEE Symposium on Computational Intelligence
in Bioinformatics and Computational Biology, pages 70–74.
IEEE, 2008.
[31] R. Hewett and F. Kijsanayothin. Tumor classification ranking
from microarray data. BMC Genomics, 9(2), 2008.
[32] Q. Wang, M. Williamson, S. Bott, N. Brookman-Amissah,
A. Freeman, J. Nariculam1, M. Hubank3, A. Ahmed, and
J. Masters. Hypomethylation of wnt5a, crip1 and s100p in
prostate cancer. Oncogene, 26:65606565, 2007.
[33] NCBI. National Center of Biotechnology Information, 2007.
In http://www.ncbi.nlm.nih.gov/.
[34] K. Bowen, A. Reimers, S. Luman, J. Kronz, W. Fyffe, and
J. Oxford. Immunohistochemical localization of collagen
type xi a1 and a2 chains in human colon tissue. Journal
of Histochemistry and Cytochemistry, 56(3):275–283, 2008.
[35] Z. Sheng, J. Wang, Y. Dong, H. Ma, H. Zhou, H. Sugimura,
G. Lu, and X. Zhou. Ephb1 is underexpressed in poorly
differentiated colorectal cancers. Pathobiology, 75(5):274–
280, 2008.
[36] S. Bhattacharya, C. Bunick, and W. Chazin. Target selectivity
in ef-hand calcium binding proteins. Biochimica et Biophysica Acta (BBA) - Molecular Cell Research, 1742(1-3):69–79,
2004.
[37] R. Renata, L. Visser, J. Van der Leij, G. Harms, T. Blokzijl,
J. Deloulme, P. van der Vlies, W. Kamps, K. Kok, M. Lim,
S. Poppema, and A. van den Berg. High expression of
calcium-binding proteins, s100a10, s100a11 and calm2 in
anaplastic large cell lymphoma. British Journal of Haematology, 131(5):596–608, 2005.
[38] B. Delage, D. Fennell, L. Nicholson, I. McNeish, N. Lemoine,
T. Crook, and P. Szlosarek. Arginine deprivation and argininosuccinate synthetase expression in the treatment of cancer.
International Journal of Cancer, 126(12):2762–2772, 2010.
[39] GenCards.
Weizmann
http://www.genecards.org/.

Institute

of

Science,

2009.

[40] J. Shaik and M. Yeasin. A unified framework for finding
differentially expressed genes from microarray experiments.
BMC Bioinformatics, 8(1), 2007.
[41] GeneAtlas. Université René Descartes - Paris, 2007.
http://www.dsi.univ-paris5.fr/genatlas/.

In

[42] W. Chu, Z. Ghahramani, F. Falciani, and D. Wild. Biomarker
discovery in microarray gene expression data with gaussian
processes. Bioinformatics, 21(16):3385–3393, June 2005.
[43] Sounak Chakraborty. Simultaneous cancer classification and
gene selection with bayesian nearest neighbor method: An
integrated approach. Computational Statistics and Data
Analysis, 53(4):1462–1474, 2009.
[44] R. Scherz-Shouval, E. Shvets, E. Fass, H. Shorer, L. Gil, and
Z. Elazar. Reactive oxygen species are essential for autophagy
and specifically regulate the activity of atg4. The EMBO
Journal, 26:1749–1760, 2007.
[45] I. Munoz and J. Rouse. Control of histone methylation and
genome stability by ptip. EMBO reports, 10, 2009.
[46] A. Giatromanolaki, M. Koukourakis, E. Sivridis, H. Turley,
C. Wykoff, K. Gatter, and A. Harris. Dec1 (stra13) protein
expression relates to hypoxia- inducible factor 1-alpha and
carbonic anhydrase-9 overexpression in non-small cell lung
cancer. The Journal of Pathology, 200(2):222–228, 2003.
[47] R. La Starza, B. Crescenzi, V. Pierini, S. Romoli, P. Gorello,
L. Brandimarte, C. Matteucci, M. Kropp, G. Barba,
M. Martelli, and C. Mecucci. A common 93-kb duplicated
dna sequence at 1q21.2 in acute lymphoblastic leukemia
and burkitt lymphoma. Cancer Genetics and Cytogenetics,
175(1):73–76, 2007.
[48] J. Yang, Y. Shi, Q. Cheng, and L. Deng. Expression and
localization of aquaporin-5 in the epithelial ovarian tumors.
Gynecologic Oncology, 100(2):294–299, 2006.
[49] H. Farhana, K. Wahalab, H. Adlercreutzc, and H. Cross.
Isoflavonoids inhibit catabolism of vitamin d in prostate
cancer cells. Journal of Chromatography B, 777(1-2):261–
268, 2002.
[50] M. Jong-Seok, J. Won-Ji, K. Jin-Hye, K. Hyo-Jeong, Y. MiJin, K. Jae-Woo, P. Sahng Wook Park, and K. Kyung-Sup.
Androgen stimulates glycolysis for de novo lipid synthesis by
increasing the activities of hexokinase 2 and 6-phosphofructo2-kinase/fructose-2,6-bisphosphatase 2 in prostate cancer
cells. Biochemical Journal, 433:225–233, 2011.
[51] H. Ishikura, H. Ikeda, H. Abe, T. Ohkuri, H. Hiraga, K. Isu,
T. Tsukahara, N. Sato, H. Kitamura, N. Iwasaki, N. Takeda,
and A. Minami T. Nishimura. Identification of cluap1 as
a human osteosarcoma tumor-associated antigen recognized
by the humoral immune system. International Journal of
Oncology, 30(2):225–233, 2011.
[52] R. Bagheri-Yarmand, M. Mandal, A. Taludker, R. Wang,
R. Vadlamudi, H. Kung, and R. Kumar. Etk/bmx tyrosine kinase activates pak1 and regulates tumorigenicity of breast cancer cells. Journal of Biological Chemistry, 276(31):29403–
29409, 2001.

[53] D. Akinmade, A. Talukder, Y. Zhang, W. Luo, R. Kumar, and
A. Hamburger. Phosphorylation of the erbb3 binding protein
ebp1 by p21-activated kinase 1 in breast cancer cells. British
Journal of Cancer, 98:11321140, 2008.

