1

Unsupervised Learning for Cell-level Visual
Representation in Histopathology Images with
Generative Adversarial Networks

arXiv:1711.11317v2 [] 11 Jan 2018

Bo Hu] , Ye Tang] , Eric I-Chao Chang, Yubo Fan, Maode Lai and Yan Xu*

Abstract—The visual attributes of cells, such as the nuclear
morphology and chromatin openness, are critical for histopathology image analysis. By learning cell-level visual representation,
we can obtain a rich mix of features that are highly reusable for
various tasks, such as cell-level classification, nuclei segmentation,
and cell counting. In this paper, we propose a unified generative
adversarial networks architecture with a new formulation of loss
to perform robust cell-level visual representation learning in an
unsupervised setting. Our model is not only label-free and easily
trained but also capable of cell-level unsupervised classification
with interpretable visualization, which achieves promising results
in the unsupervised classification of bone marrow cellular components. Based on the proposed cell-level visual representation
learning, we further develop a pipeline that exploits the varieties
of cellular elements to perform histopathology image classification, the advantages of which are demonstrated on bone marrow
datasets.
Keywords—unsupervised learning, representation learning, generative adversarial networks, classification, cell.

I. I NTRODUCTION
ISTOPATHOLOGY images are considered to be the
gold standard in the diagnosis of many diseases [1].
In many situations, the cellular components are an important
determinant. For example, in the biopsy sections of bone marrow, the abnormal cellular constitution indicates the presence
of blood disease [2]. Bone marrow is the key component
of both the hematopoietic system and the lymphatic system

H

This work is supported by Microsoft Research under the eHealth program,
the National Natural Science Foundation in China under Grant 81771910, the
National Science and Technology Major Project of the Ministry of Science
and Technology in China under Grant 2017YFC0110903, the Beijing Natural
Science Foundation in China under Grant 4152033, the Technology and
Innovation Commission of Shenzhen in China under Grant shenfagai2016627, Beijing Young Talent Project in China, the Fundamental Research Funds
for the Central Universities of China under Grant SKLSDE-2017ZX-08 from
the State Key Laboratory of Software Development Environment in Beihang
University in China, the 111 Project in China under Grant B13003. * indicates
corresponding author; ] indicates equal contribution.
Bo Hu, Ye Tang, Yubo Fan and Yan Xu are with the State Key Laboratory of Software Development Environment and the Key Laboratory
of Biomechanics and Mechanobiology of Ministry of Education and Research Institute of Beihang University in Shenzhen and Beijing Advanced
Innovation Centre for Biomedical Engineering, Beihang University, Beijing
100191, China (email: bohu1996@gmail.com; yetang1995@gmail.com; yubofan@buaa.edu.cn; xuyan04@gmail.com).
Maode Lai is with the Department of Pathology, School of Medicine,
Zhejiang University (email: lmd@zju.edu.cn).
Eric I-Chao Chang, and Yan Xu are with Microsoft Research, Beijing
100080, China (email: echang@microsoft.com; v-yanx@microsoft.com).

Fig. 1. Examples of five types of cellular elements in bone marrow: (a)
granulocytes precursors such as myeloblasts, (b) cells with dark, dense, and
close phased nuclei, the candidates of which are most likely lymphocytes and
normoblasts, (c) granulocytes such as neutrophils, (d) monocytes, and (e)
megakaryocytes. Five types of cells can be distinguished by the chromatin
openness, the density of nuclei, and if nuclei show the appearance of being
segmented. Megakaryocytes appear the least often, as well are the most
distinguished due to their massive size.

(a) abnormal

(b) normal

Fig. 2. Examples of bone marrow images sliced from Whole Slide Images
(WSI). Too many myeloblasts in (a) indicate the presence of blood disease.

by producing large amounts of blood cells. The cell lines
undergoing maturation in the marrow mostly include myeloid
cells (granulocytes, monocytes, megakaryocytes, and their
precursors), erythroid cells (normoblasts), and lymphoid cells
(lymphocytes and their precursors). Figure 1 are examples
of five main cellular components in bone marrow. These
components are significant to both the systemic circulation and
the immune system. Several kinds of cancer are characterized

2

by the cellular constitution in bone marrow [2]. For instance,
too many granulocytes precursors such as myeloblasts indicate
the presence of chronic myeloid leukemia. Having large,
abnormal lymphocytes heralds the presence of lymphoma.
Figure 2 shows the difference between normal and abnormal
bone marrow histopathology images from the perspective of
cells.
As described above, cell-level information is irreplaceable
for histopathology image analysis. Cell-level visual attributes
such as the morphological features of nuclei and the openness of chromatin are helpful for various tasks such as celllevel classification and nuclei segmentation. We define celllevel images as the output from nuclei segmentation. Each
cell-level image contains only one cell. We opt to perform
representation learning on these cell-level images, in which the
visual attributes such as the nuclei morphology and chromatin
openness are distinguished. The learned features are further
utilized to assist tasks such as cell counting to highlight the
quantification of certain types of cells.
To achieve this, the main obstacle is the labeling of cells.
There are massive amounts of cells in each histopathology
image, which makes manual labeling ambiguous and laborious.
Therefore, an unsupervised cell-level visual representation
learning method based on unlabeled data is believed to be more
reasonable than fully supervised methods. Unsupervised celllevel visual representation learning is known to be difficult.
First, geometrical and morphological appearances of cells
from the same category can have a distinct diversity due to
factors such as cell cycles. Furthermore, the staining conditions
of histopathology images can be pretty diverse, resulting in
inconsistent color characteristics of nuclei and cytoplasm.
Recently, deep learning has been proven to be powerful
in histopathology image analysis such as classification [3],
[4], segmentation [5], [6], and detection [7], [8]. Generative
Adversarial Networks (GANs) [9] are a class of generative
models that use unlabeled data to perform representation
learning. GAN is capable of transforming noise variables
into visually appealing image samples by learning a model
distribution that imitates the real data distribution. Several
GAN architectures such as Deep Convolutional Generative
Adversarial Nets (DCGAN) [10] have proven their advantages
in various natural images datasets. Recently, Wasserstein-GAN
(WGAN) [11] and WGAN with gradient penalty (WGAN-GP)
[12] have greatly improved the stability of training GAN. More
complex network structures such as residual networks [13] can
now be fused into GAN models.
Meanwhile, Information Maximizing Generative Adversarial Networks (InfoGAN) [14] makes a modification that encourages GAN to learn interpretable and meaningful representations. InfoGAN maximizes the mutual information between
the chosen random variables and the observations to make
variables represent interpretable semantic features. The problem is that InfoGAN utilizes a DCGAN architecture, which
requires meticulous attention towards hyperparameters. For our
problem, it suffers a severe convergence problem.
Inspired by WGAN-GP and InfoGAN, we present an unsupervised representation learning method for cell-level images
using a unified GAN architecture with a new formulation

of loss, which inherits the superiority from both WGANGP and InfoGAN. We observe great improvements followed
by the setting of WGAN-GP. Introducing mutual information
into our formulation, we are capable of learning interpretable
and disentangled cell-level visual representations, as well as
allocate cells into different categories according to their most
significant semantic features. Our method achieves promising
results in the unsupervised classification of bone marrow
cellular components.
Based on the cell-level visual representations, the quantification of each cellular component can be obtained by the
trained model. Followed by this, cell proportions for each
histopathology image can then be calculated to assist imagelevel classification. We further develop a pipeline combining
cell-level unsupervised classification and nuclei segmentation
to conduct image-level classification of histopathology images,
which shows its advantages via experimentations on bone
marrow datasets.
The contributions of this work include the following: (1) We
present an unsupervised framework to perform cell-level visual
representation learning using generative adversarial networks.
(2) A unified GAN architecture with a new formulation of
loss is proposed to generate representations that are both highquality and interpretable, which also endows our model the capability of cell-level unsupervised classification. (3) A pipeline
is developed that exploits the varieties of cell-level elements to
perform image-level classification of histopathology images.
II.

RELATED WORKS

A. Directly Related Works
1) Generative Adversarial Networks: Goodfellow et al. [9]
propose GANs, a class of unsupervised generative models
consisting of a generator neural network and an adversarial
discriminator neural network. While the generator is encouraged to produce synthetic samples, the discriminator learns to
discriminate between generated and real samples. This process
is described as a minimax game. Radford et al. [10] propose
one of the most frequently used GAN architectures DCGAN.
Arjovsky et al. [11] propose WGAN, which modifies the
objective function, securing the training process to be more
stable. For regular GANs, the training process optimizes a
lower bound of the Jensen-Shannon (JS) divergence between
the generator distribution and the real data distribution. WGAN
modifies this by optimizing an approximation of the EarthMover (EM) distance. The only challenge is how to enforce
the Lipschitz constraint on the discriminator. While Arjovsky
et al. [11] use weight-clipping, Gulrajani et al. [12] propose
WGAN-GP, which adds a gradient penalty on the discriminator. For our bone marrow datasets, even if we have tried
multiple hyperparameters, DCGAN still suffers from a severe
convergence difficulty. While DCGAN leads to the failure for
our datasets, WGAN-GP greatly eases this problem.
Chen et al. [14] introduce mutual information into GAN
architecture. Mutual information describes the dependencies
between two separate variables. Maximizing mutual information between the chosen random variables and the generated

3

samples, InfoGAN produces representations that are meaningful and interpretable. To exploit the varieties of cellular
components, the superior ability of InfoGAN in learning
disentangled and discrete representations is what regular GAN
lack.
Therefore, we propose a unified GAN architecture with a
new formulation of loss, which inherits the superiority of
both WGAN-GP and InfoGAN. The outstanding stability of
WGAN-GP eases the difficulty in tuning the complicated
hyperparameters of InfoGAN. Introducing mutual information
into our model, we are capable of learning interpretable celllevel visual representations, as well as allocate cells into
different categories according to their most significant semantic
features.
2) Classification of Blood Disease: Nazlibilek et al. [15]
propose a system to help automatically diagnose acute lymphocytic leukemia. This system consists of several stages: nuclei
segmentation, feature extraction, cell-level classification, and
cell counting. In their future work, they claim that the result
of cell counting can be used for further diagnosis of acute
lymphocytic leukemia.
In our work, we design a similar workflow which consists
of nuclei segmentation, cell-level classification, and imagelevel classification. Our advantages lie in the novelty of an
unsupervised setting and the convincing performance of imagelevel classification based on the calculated cell proportions.
B. Cell-level Representation
The representation of individual cells can be used for a
variety of tasks such as cell classification. Traditional celllevel visual representation for classification tasks can be categorized into four categories [16]: morphological [17], texture
[18], [19], intensity [20], and cytology features [21]. These
traditional methods have been employed in the representation
of white blood cells [22], [23], [24]. However, the features used
above need to be manually designed by experienced experts
according to the characteristics of different types of cells.
While images suffer from a distinct variance, discovering,
characterizing and selecting good handcraft features can be
extremely difficult.
To remedy the limitations of manual features in cell classification, Convolutional Neural Network (CNN) learns higherlevel latent features, whose convolution layer can act as a
feature extractor [25]. Xie et al. [26] propose Deep Embedding
Clustering (DEC) that simultaneously learns feature representations and cluster assignments using deep neural networks.
Variational Autoencoder (VAE) [27] serves as a convincing
unsupervised strategy in cell-level visual representation learning [28], [29]. However, how to use VAE to learn categorical and discrete latent variables is still under investigation.
Dilokthanakul et al. [30] and Jiang et al. [31] design models
combining VAE with Gaussian Mixture Model (GMM). But
they demonstrate their experiment on one-dimensional datasets
such as MNIST. To perform clustering and embedding on a
higher-dimensional dataset, their methods still need a feature
extractor.
GANs such as Categorical GAN [32] can merge categorical
variables into the model with little effort, which makes learned

representations disentangled and interpretable. This ability is
critical in medical image analysis where accountability is
especially needed.
C. Cell-level Histopathology Image Analysis
1) Classification: Cell classification has been performed in
diverse histopathology related works such as breast cancer
[33], acute lymphocytes leukemia [34], [35], and colon cancer
[36].
Based on the result of cell classification, some approaches
have been proposed to determine the presence or location of
cancer [21], [37]. In prostate cancer, Nguyen et al. [21] innovatively employ cell classification for automatic cancer detection
and grading. They distinguish the cancer nuclei and normal
nuclei, which are combined with textural features to classify
the image as normal or cancerous and then detect and grade
the cancer regions. In the diagnosis of Glioma, Hou et al. [37]
apply CNN to the classification of morphological attributes
of nuclei. They also claim that the nuclei classification result
provides clinical information for diagnosing and classifying
glioma into subtypes and grades. Zhang et al. [38] and Shi
et al. [39] use either supervised or semi-supervised hashing
models for cell-level analysis.
All of these works require a large amount of accurately annotated data. Obtaining such annotated data is time-consuming
and labor-intensive while GAN can optimally leverage the
wealth of unlabeled data.
2) Segmentation: Nuclei segmentation is of great importance for cell-level classification. Nuclei segmentation methods
can be roughly categorized as follows: intensity thresholding
[40], [41], morphology operation [42], [43], deformable models [44], watershed transform [45], clustering [46], [47], and
graph-based methods [48], [49]. The methods above have been
broadly applied to the segmentation of white blood cells.
D. Generative Adversarial Networks in Medical Images
Recently, several works involving GAN have gathered great
attention in medical image analysis.
In medical image synthesizing, Nie et al. [50] estimate the
CT image from its corresponding MR image with contextaware GAN. In medical image reconstruction, Li et al. [51]
use GAN to reconstruct medical images with the thinner sliced
thickness from regular thick-slice images. Mahapatra et al.
[52] propose a super resolution method that takes a lowresolution input fundus image to generate a high-resolution
super-resolved image. Wolterink et al. [53] employ GAN to
reduce the noise in low-dose CT images. All these recent works
demonstrate the great potential of GAN in solving complicated
medical problems.
III. M ETHODS
In this section, we first introduce an unsupervised method
for cell-level visual representation learning using GAN. Then
we present the details of how image-level classification is
performed on histopathology images based on cell-level representation.

4

(a) Training process. Random variables are composed of Gaussian variables z
and the discrete variable c. Besides playing the minimax game between the
generator (G) and the discriminator (D) through the EM distance, we also
minimize the negative Log-likelihood between c and the output of the auxiliary
network (Q(c|G(c, z)) to maximize mutual information.

G and D by optimizing an approximation of the Earth-Mover
(EM) distance. Meanwhile, we maximize mutual information between the chosen random variables and the generated
samples using an auxiliary network Q. In the test process,
the generator generates the representations for each category
of cells according to different values of the chosen random
variables. Cell images can be allocated to the corresponding
categories by the auxiliary network Q.
1) Training Process: Given cell-level images sampled from
the real data distribution x ∼ Pr , the first goal is to learn a
generator distribution Pg that matches the real data distribution
Pr .
We first define a random noise variable z. The input noise z
is transformed by the generator into a sample x̃ = G(z), z ∼
p(z). x̃ can be viewed as following the generator distribution
Pg . Inspired by WGAN [11], we optimize networks through
the WGAN objective W (Pr , Pg ):
W (Pr , Pg ) = sup

Ex∼Pr [f (x)] − Ex̃∼Pg [f (x̃)].

(1)

kf kL≤1

W (Pr , Pg ) is an efficient approximation of the EM distance,
which is constructed using the Kantorovich-Rubinstein duality
[11]. The EM distance measures how close the generator
distribution and the data distribution are. To distinguish two
distributions Pg and Pr , the adversarial discriminator network
D is trained to learn the function f that maximizes W (Pr , Pg ).
To make Pg approach Pr , the generator instead is trained to
minimize W (Pr , Pg ). The value function V (D, G) is written
as follows:
(b) Test process. Real samples are classified into five categories by the auxiliary
network Q. At the same time, fake samples are generated by giving noises with
the chosen c for each class. In the example of generated samples (fake), one
row contains five samples from the same category in c, and a column shows
the generated images for 5 possible categories in c with z fixed.

V (D, G) = Ex∼Pr [D(x)] − Ez∼p(z) [D(G(z))].

(2)

This minimax game between the generator and the discriminator is written as:
min max V (D, G).
G D∈D

(3)

Followed by the work of WGAN-GP [12], a gradient penalty
is added on the discriminator to enforce the Lipschitz constraint to make sure that the discriminator lies within the space
of 1-Lipschitz functions D ∈ D. The loss of the discriminator
with a hyperparameter λ1 is written as:
LD = Ez∼p(z) [D(G(z))] − Ex∼Pr [D(x)] + λ1 Ex̂∼Px̂ [||∇x̂ D(x̂)||p − 1]2 ,
(c) Illustration of residual blocks (resblocks) in the architecture. There are three
different types of residual blocks considering whether they include nearestneighbor upsampling or mean pooling for downsampling. Batch normalization
layers are used in our generator to help stabilize training.
Fig. 3. Network architecture of our cell-level visual representation learning:
(a) Training process. (b) Test process. (c) The architecture of residual blocks
(written as resblock in (a) and (b)).

A. Cell-level Visual Representation Learning
Given cell-level images that come from nuclei segmentation as the real data, we define a generator network G, a
discriminator network D, and an auxiliary network Q. The
architecture of these networks are shown in Figure 3. In the
training process, we learn a generator distribution that matches
the real data distribution by playing a minimax game between

(4)
where Px̂ is defined sampling uniformly along straight lines
between pairs of points sampled from the data distribution Pr
and the generator distribution Pg .
In this way, our model is capable of generating visually
appealing cell-level images. But still, it fails to exploit information of categories of cells since the noise variable z doesn’t
correspond to any interpretable feature. Motivated by this, our
second goal is to make the chosen variables represent meaningful and interpretable semantic features of cells. Inspired
by InfoGAN [14], we introduce mutual information into our
model:
I(X; Y ) = H(X) − H(X|Y ) = H(Y ) − H(Y |X).

(5)

I(X; Y ) describes the dependencies between two separate
variables X and Y . It measures the different aspects of

5

Fig. 4. Overview of our pipeline as follows: (a) Nuclei segmentation is performed on histopathology images. (b) Using the trained GAN architecture, Cell-level
clustering is performed using the learned auxiliary network Q. Cell proportions are then calculated for each histopathology image. (c) Image-level prediction is
given based on cell proportions. (d) For visualization, the generator G can generate the interpretable representation for each category of cells by changing the
noises.

the association between two random variables. If the chosen
random variables correspond to certain semantic features,
it’s reasonable to assume that mutual information between
generated samples and random variables should be high.
We define a latent variable c sampled from a fixed noise
distribution p(c). The concatenation of the random noise
variable z and the latent variable c is then transformed by
the generator G into a sample G(z, c). Since we encourage
the latent variable to correspond with meaningful semantic
features, there should be high mutual information between c
and G(z, c). Therefore, the next step is to maximize mutual
information I(c; G(z, c)), which can be written as:
I(c; G(z, c)) = H(c) − H(c|G(z, c)).

(6)

Followed by this, a lower bound LI is given by:
LI (G, Q) = Ez∼p(z),c∼p(c) [log Q(c|G(z, c))] + H(c),

(12)

Figure 5 shows how noises are transformed into interpretable
samples during the training process.

Fig. 5.
Example of how a set of noise vectors are transformed into
interpretable image samples over generator iterations. We use a 5-dimensional
categorical variable c and 32 Gaussian noise variables z as input. Different
rows correspond to different values of z. Different columns correspond to
different values of c. The value of c largely corresponds to cell types.

(7)

where H(c) is the entropy of the variable sampled from a fixed
noise distribution. Maximizing this lower bound, we maximize
mutual information I(c; G(z, c)). The proof can be found in
InfoGAN [14].
Since we introduce the latent variable c into the model, the
value function V (D, G) is replaced by:
V (D, G) ← Ex∼Pr [D(x)] − Ez∼p(z),c∼p(c) [D(G(z, c))]. (8)
As we combine the adversarial process with the process of
maximizing mutual information, this information-regularized
minimax game with a hyperparameter λ2 can be written as
follows:
min max V (D, G) − λ2 LI (G, Q).
(9)
G,Q D∈D

The loss of D can be replaced by:
2

LD ← Ez∼p(z),c∼p(c) [D(G(z, c))] − Ex∼Pr [D(x)] + λ1 Ex̂∼Px̂ [||∇x̂ D(x̂)||p − 1] ,

(10)
Since H(c) can be viewed as a constant, the loss of the
auxiliary network Q can be written as the negative loglikelihood between Q(c|G(c, z)) and the discrete variable c.
The losses of G and Q can be interpreted as below:
LG = −Ez∼p(z),c∼p(c) [D(G(z, c))],

LQ = −λ2 Ez∼p(z),c∼p(c) [log Q(c|G(z, c))].

(11)

2) Test Process: In the training process, a generator distribution is learned to imitate the real data distribution. An
auxiliary distribution is learned to maximize the lower bound.
Especially if c is sampled from a categorical distribution, a
softmax function is applied as the final layer of Q. Under this
circumstance, Q can act as a classifier in the test process, since
the posterior Q(c|x) is discrete. Assuming that each category
in c corresponds to a type of cells, the auxiliary network Q
can divide cell-level images into different categories while the
generator G can generate the interpretable representation for
each category of cells.
B. Image-level Classification
Based on the cell-level visual representation learning, we
propose a pipeline combining nuclei segmentation and celllevel visual representation to highlight the varieties of cellular
elements. Image-level classification is performed using the
calculated cell proportions. The illustration of this pipeline is
shown in Figure 4.
1) Nuclei Segmentation: An unsupervised nuclei segmentation approach is ultilized consisting of four stages: normalization, unsupervised color deconvolution, intensity thresholding

6

Fig. 6. Overview of segementation process: (a) the cropped image, (b)
the normalized image, (c) the separated hematoxylin stain image using color
deconvolution, (d) the binary image generated by intensity thresholding, (e)
the labeled image after postprocessing where different grayscale values stand
for different segmented instances, and (f ) the final segmentation image.

and postprocessing to segment nuclei from the background.
Figure 6 is an overview of our segmentation pipeline.
Color Normalization: We employ Reinhard color normalization [54] to convert the color characteristics of all images
into the desired standard by computing the mean and standard
deviations of a target image in LAB space.
Color Deconvolution: Using the PCA-based ‘Macenko’
method [55], unsupervised color deconvolution is performed
to separate the normalized image into two stains. We project
pixels onto a best-fit plane, wherein it selects the stain vectors
as percentiles in the ‘angle distribution’ of the corresponding
plane. With the correct stain matrix for color deconvolution,
the normalized image can be separated into hematoxylin stain
and eosin stain.
Intensity Thresholding: To sufficiently segment cells, we
apply intensity thresholding in the hematoxylin stain image
where the intensity distribution of cells is consistently distinct
from the background. By converting the hematoxylin stain
image into a binary image with a constant global threshold,
the cells are roughly segmented.
Postprocessing: In image postprocessing, objects with
fewer pixels than the minimum area threshold will be removed
from the binary image. Then we employ the method in [41]
to remove thin protrusions from cells. Furthermore, we use
opening operation to separate a few touched cells.
2) Classification: We utilize the model distribution trained
in our unsupervised representation learning as the cell-level
classifier. Assuming that we use a k-dimensional categorical
variable as the chosen variable in the training process, the real
data (cell-level images) distribution is allocated into k dimensions. In the test process, cell-level images are unsupervised
classified into k corresponding categories.
For each histopathology image, we count the numbers of
cell-level instances in each category as the representation of
its cellular constitution, denoted as {X1 , X2 , X3 , . . . , Xk }. For
cellular element i, the ratio of the number of this cellular
element to the total number of the cellular constitution in this
image is calculated by Pi = PkXi X . We define Pi as the cell
i
i=1
proportion of cellular element i.
Given cell proportions {P1 , P2 , P3 , . . . , Pk } as the feature
vector of histopathology images, we utilize either k-means or
SVM to give image-level predictions.
IV. E XPERIMENTS AND R ESULTS
A. Dataset
All our experiments are conducted on bone marrow
histopathology images stained with hematoxylin and eosin. As

described before, the cellular constitution in bone marrow is a
determinant in diagnoses of blood disease.
Dataset A: Publicly available dataset [56] which consists
of eleven images of healthy bone marrow with a resolution of
1200×1200 pixels. Each image contains around 200 cells. The
whole dataset includes 1995 cell-level images in total. We label
all cell-level images into four categories: 34 neutrophils, 751
myeloblasts, 495 monocytes, and 715 lymphocytes. Images are
carefully labeled by two pathologists. When the two pathologists disagree on a particular image, a senior pathologist makes
a decision over the discord.
Dataset B: Dataset provided by the First Affiliated Hospital
of Zhejiang University. It contains whole slides of bone
marrow from 24 patients with blood diseases. We manually
choose the most representative 29 images with a resolution of
1500 × 800 pixels. Dataset B contains around 12000 cells in
total. For this dataset, we label 600 cell-level images into three
categories for evaluation: 200 myeloblasts, 200 monocytes, and
200 lymphocytes. The labeling process is conducted in the
same manner as Dataset A.
Dataset C: Combination of Datasets A and B, which results
in 29 abnormal and 11 normal histopathology images.
B. Implementation
We implement our experiments on Pytorch and HistomicsTK frameworks. We utilize k-means++ [57] to choose the
initial values when using k-means to perform clustering.
Network Parameters: Our generator G, discriminator D
and auxiliary network Q all have the structures of residual
networks. In the training process, all three networks are
updated by Adam optimizer (α = 0.0001, β1 = 0.5, β2 = 0.9,
lr = 2×10−4 ) [58] with a batch size of 64. All our experiments
use hyperparameters λ1 = 10 and λ2 = 1. For each training
iteration, we update D, G and Q in turn. One training iteration
consists of five discriminator iterations, one generator iteration,
and one auxiliary network iteration. For each training process,
we augment the training set by rotating images with angles
90◦ , 180◦ , 270◦ . We train ten epochs for our model in each
experiment.
Noise Sources: The noise fed into the network is the combination of a 5-dimensional categorical variable and 32 Gaussian
noise variables for the training of Dataset A or Dataset B. We
use the combination of a 5-dimensional categorical variable
and 64 Gaussian noise variables for Dataset C.
Segmentation Parameters: The mean value of the standard
image in three channels is [8.98±0.64, 0.08±0.11, 0.02±0.03]
for color normalization. Vectors for color deconvolution are
picked from 1% to 99% angle distribution while the magnitude
below 16 is excluded from the computation. We use the
threshold value of 120 for intensity thresholding. In the postprocess, objects with pixels smaller than 200 will be removed.
An opening operation with 7 × 7 kernel size is performed to
separate touched cells. When the edge of the bounding box of
a cell-level image is larger than 32 pixels, we rescale the image
to make the larger edge match to 32. Each cell is centered in a
32 × 32 pixel image where blank is filled with [255, 255, 255].

7

C. Cell-level Classification Using Various Features
To demonstrate the quality of our representation learning, we
apply the trained model as a feature extractor. The experiment
is conducted on Dataset A. In this experiment, 1596 cell-level
images are used for training; 399 cell-level images are used
for testing.
Comparison: (1) MF: 188-dimensional manual feature
combined of SIFT [59], LBP [60], and L × a × b color
histogram. (2) DNN: 1024-dimensional feature extracted by a
residual neural network (ResNet-50) [13] trained on Imagenet1k. (3) Our Method: We downsample the features after each
residual block of the discriminator into a 4×4 spatial grid using
max pooling. These features are flattened and concatenated
to form an 8192-dimensional vector. On top of the feature
vectors, an L2-SVM is trained to perform classification.
Evaluation: For each class, we denote the number of true
positives T P , the number of false positives F P and the
number of false negatives F N . The precision, recall and Fscore (F1 ) for each class are defined as follows:
TP
,
TP + FP
TP
recall =
,
TP + FN
2 · precision · recall
F1 =
.
precision + recall
precision =

(13)

The average precision, recall and F-socre are calculated
weighted by support (the number of true instances of each
class).
Results: We randomly choose correctly classified and misclassified samples displayed in Figure 7. The comparison of
results is shown as Table I, which proves the advantages of our
representation learning method. Though the dimensions of the
feature vectors of our method are higher, the clustering ability
of our model ensures further unsupervised applications.

Fig. 7. Visualization of cell-level classification performed on Dataset A: (up)
correctly classified samples and (down) misclassified samples. misclassified
samples can be illegible for pathologists either.

Comparison: (1) MF+k-means: Manual features with kmeans. (2) DNN+k-means: DNN features extracted by ResNet50 trained on Imagenet-1K, on top of which k-means is
performed. (3) HOG+DEC: Deep Embedded Clustering (DEC)
[26] on 2048-dimensional HOG features. (4) Our Method:
Cell images are unsupervised allocated to five clusters by the
auxiliary network Q. We also test models such as Categorical
GAN (CatGAN) [32], InfoGAN (under DCGAN architecture),
and Gaussian Mixture VAE (GMVAE) [30] on our datasets
under different hyperparameters, but find them fail to converge.
Evaluation: We evaluate the performance of clustering using the average F-score, purity, and entropy. For the set of clusters {ω1 , ω2 , . . . , ωK } and the set of classes {c1 , c2 , . . . , cJ },
we assume that each cluster ωk is assigned to only one class
argmaxj (|ωk ∩ cj |). The F-score for class cj is then given by
Equation 13. The average F-score is given calculated by the
number of true instances in each class.
Purity and Entropy are also used as evaluation metrics,
which are written as follows:
1 X
purity =
max |ωk ∩ cj |,
j
N
k
(14)
|ωk |
1 X
|ωk | log
.
entropy = −
N
N
k

Larger purity and smaller entropy indicate better clustering
results.
For nuclei segmentation, we use Intersection over Union
(IoU) and the F-score as evaluation metrics. A segmented
instance (I) is matched with the ground truth (G) only if
they intersect at least 50% (i.e., |I ∩ G| > 0.5G). For each
matched instance and its ground truth, the overlapping pixels
are counted as true positive (T P ). The pixels of instance
remain unmatched are counted as false positive (F P ) while
the pixels of ground truth remaining unmatched are counted
as false negative (F N ). The F-score is then calculated using
Equation 13.
Choosing the Number of Clusters: For our method, it
is easy to change the number of clusters by sampling the
categorical noise from a different dimension. We compare the
results of choosing different numbers of clusters shown in
Table II, which shows there is no distinct difference between
choosing four and five clusters. We choose five clusters (a
5-dimensional categorical random variable) in change for a
slightly better performance.
TABLE II.

TABLE I.

Performance when choosing different numbers of clusters.

Performance of cell-level classification using various features.
Methods
MF
DNN
Our Method

precision
0.821
0.838
0.865

recall
0.803
0.817
0.848

F-score
0.811
0.827
0.857

D. Cell-level Clustering
As the priority of image-level classification of histopathology images, cell-level clustering is performed using the trained
auxiliary network Q. We conduct experiments on the three
datasets described above.

Clusters
F-score

4
0.831

5
0.863

6
0.789

For k-means based methods, the average F-score is approximately the same (±0.02) using either four, five, or six clusters.
Annotations: To evaluate the capability of nuclei segmentation, We randomly choose ten patches from Dataset C with
a resolution of 200 × 200 pixels. The ground truth is carefully labeled by two pathologists. When the two pathologists
disagree on a particular image, a senior pathologist makes a
decision over the discord.

8

Fig. 8. Visualization of clustering. We randomly select 60 samples from each one of five clusters, displayed as (a) to (e). Instances in the same cluster have a
distinct consistency. In (b), cells in marrow with dark, dense, and close phased nuclei tend to be lymphocytes or erythroid precursors. In (c) and (e), cells with
dispersed chromatin are most likely granulocytes precursors such as myeloblasts.

Results: For nuclei segmentation, our method achieves 0.56
mean IoU and 0.70 F-score.
For cell-level clustering, the comparison shown as Table III
shows the superiority of our method. To explicitly reveal
the semantic features our model has captured, we randomly
choose 60 samples from each of the five clusters displayed
in Figure 8, which shows a distinct consistency within each
cluster. Reasonable interpretations can be given. Cells are clustered according to the semantic features such as the chromatin
openness, the darkness and density of nuclei, and if nuclei
show the appearance of being segmented.

TABLE III.
Dataset

A

B

C

Performance of cell-level clustering.

Methods
MF+k-means
DNN+k-means
HOG+DEC
Our Method
MF+k-means
DNN+k-means
HOG+DEC
Our Method
MF+k-means
DNN+k-means
HOG+DEC
Our Method

Purity
0.579
0.667
0.729
0.855
0.392
0.719
0.771
0.874
0.459
0.578
0.667
0.769

Entropy
1.376
1.256
1.086
0.750
1.561
0.844
0.697
0.431
1.533
1.377
1.217
0.977

F-score
0.603
0.677
0.737
0.863
0.409
0.760
0.812
0.841
0.484
0.601
0.682
0.777

9

Especially for Dataset A, Figure 9(a) shows the convergence
of V (D, G) (see Equation (8)) and LQ (see Equation (12)).
V (D, G) is used to evaluate how well the generator distribution matches the real data distribution [12]. LQ approaching
zero indicates that mutual information is maximized [14].
Figure 9(b) shows how the purity of clustering increases in
the training process.

TABLE IV.

Performance of image-level classification.

Methods

precision

recall

F-score

DNN

0.906

0.969

0.933

0.936

0.933

0.931

0.950

0.969

0.950

Our Method
(unsupervised)
Our Method
(with SVM)

Using Principal Components Analysis (PCA) to perform a
dimensionality reduction, cell proportions of each histopathology image are projected onto a two-dimension plane to show
that there is a distinct difference between normal and abnormal
images, shown in Figure 10.
(a)

(b)

Fig. 9. Visualization of cell-level clustering performed on Dataset A: (a)
Training losses converge as the network trains. (b) The purity increases
gradually over generator iterations.

E. Image-level Classification
Classification is performed on Dataset C which includes 29
abnormal and 11 normal images. We randomly split Dataset C
into four folds to conduct 4-fold cross-validation. The scores
are reported averagely.
Comparison: (1) DNN: We use ResNet-50 pre-trained on
Imagenet-1K as a feature extractor. Images are center cropped
to 800×800 pixels, then resized into 224×224 pixels. An L2SVM is trained on the feature vectors. We find this produces
a better result than fine-tuning or directly training a ResNet50 without pre-train. (2) Our method (unsupervised): We first
train our GAN architecture on the training set, then conduct
the cell-level clustering on both the training set and test set
using the trained model. Cluster centers are calculated given
cell proportions of each sample in the training set. The predict
label is given by the closest cluster that each sample in the
test set belongs to. (3) Our method (with SVM): An L2-SVM
instead of k-means is used as the final classifier.
Evaluation: We use the precision, recall and F-score for
evaluation, the details of which have been described in Equation 13. The difference is that the labels are binary in this
experiment.
Results: Following the proposed pipeline, we first train the
GAN architecture on the segmentation output of the training
set. This achieves 0.771 purity viewed as a clustering task,
evaluated by labeled cells of Dataset A.
Given the cell proportions, we achieve 0.931 F-score when
using k-means to perform unsupervised classification, which
is comparable to the DNN method with 0.933 F-score. The
advantage is that our model is interpretable. The proportion of
which category of cells is irregular is recognizable.
Since there are a large number of cell-level images (equal
to about 15000), it is difficult to test our method under
full-supervision with a similar pipeline. We instead train an
L2-SVM on cell proportions, taking image-level labels of
histopathology images as targets. This achieves the highest Fscore of 0.950. The comparison is shown as Table IV.

Fig. 10. Visualization of unsupervised classification using cell proportions.
It can be observed that the points representing normal and abnormal samples
are distinctly distributed in two different clusters.

Discussion: Analyzing the results, we find that the cell
proportions {P1 , P2 , · · · , P5 } can indicate the presence of
blood diseases.
For our experiment, cell-level clustering shows that {P1 ,
P4 } correspond to myeloblasts, {P5 } corresponds to lymphocytes and erythroid precursors, and {P2 , P3 } correspond
to monocytes and glanulocytes. For all normal images, P1
and P4 are relatively lower. This matches the constitution in
normal bone marrow where the lymphocytes, glanulocytes and
erythroid precursors are in the majority when the percentage
of cells with open phased nuclei (such as myeloblasts, under
some circumstances plasma cells) is relatively lower (less than
10%). In Figure 10, abnormal images that are confidently
discriminated are reflected in the numerous presence of the
supposed minority myeloblasts or plasma cells, which in turn
is reflected in the sharp increase of P1 and P4 .
However, there are three abnormal images that are exceptional. To analyze what causes the failure, we display the
example image in Figure 11.

10

a confident diagnosis. To classify cells in a more fine-grained
manner, the peculiar distribution information such as erythroid
cells more likely form clusters (erythroid islands) can be
considered.
VI. ACKNOWLEDGMENT
The authors would like to thank the First Affiliated Hospital
of Zhejiang University for providing data and help.
Fig. 11. Example of the failed samples. Too many erythroid precursors
indicate the presence of blood disease. The overlap of nuclei and the lousy
staining condition add to the difficulties of cell-level classification.

In these images, the irregular proportion of erythroid precursors indicates the presence of blood disease. We find that
our model does not correctly classify these cells. The reason
could be that the staining condition of these cells is not as
good as expected. A typical erythroid precursor should have a
close phased, dark-staining nucleus that appears almost black.
As Figure 12 shows, the color of nuclei segmented from these
images differ from the rest of the dataset. Particularly in these
images, our model is still not robust enough to capture the
most significant semantic variance in an unsupervised setting.
Therefore, acquiring high-quality histopathology images is still
a priority.

R EFERENCES
[1]

[2]

[3]

[4]

[5]

[6]
(a)

(b)

[7]
(c)

(d)
[8]

Fig. 12. Variance of staining conditions. (a) and (b) are erythroid precursors
and myeloblasts randomly chosen from failed images. (c) and (d) are samples
selected from correctly predicted images. Our model mistakes erythroid
precursors for myeloblasts particularly in failed images.

V. C ONCLUSION
In this paper, we introduce a unified GAN architecture with
a new formulation of the loss function into cell-level visual
representation learning of histopathology images. Cell-level
unsupervised classification with interpretable visualization is
performed by maximizing mutual information. Based on this
model, we exploit cell-level information by calculating the
cell proportions of histopathology images. Followed by this,
we propose a novel pipeline combining cell-level visual representation learning and nuclei segmentation to highlight the
varieties of cellular elements, which achieves promising results
when tested on bone marrow datasets.
In future work, some improvements can be made to our
method. To diagnose blood disease, in addition to cell proportions, other information about the patients should be carefully
considered. For example, bone marrow cells of children might
not be as varied as those of adults’. In clinical practice,
doctors need to consolidate more critical information to make

[9]

[10]

[11]
[12]

[13]

[14]

[15]

M. N. Gurcan, L. E. Boucheron, A. Can, A. Madabhushi, N. M. Rajpoot,
and B. Yener, “Histopathological image analysis: A review,” IEEE
Reviews in Biomedical Engineering, vol. 2, no. 1, pp. 147–171, 2009.
J. M. Bennett, D. Catovsky, M. T. Daniel, G. Flandrin, D. A. Galton,
H. R. Gralnick, and C. Sultan, “Proposals for the classification of
the acute leukaemias. french-american-british (fab) co-operative group,”
British Journal of Haematology, vol. 33, no. 4, p. 451458, 1976.
Y. Xu, T. Mo, Q. Feng, P. Zhong, M. Lai, and I. C. Chang, “Deep
learning of feature representation with multiple instance learning for
medical image analysis,” in IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1626–1630, 2014.
Y. Xu, Z. Jia, Y. Ai, F. Zhang, M. Lai, I. Eric, and C. Chang, “Deep
convolutional activation features for large scale brain tumor histopathology image classification and segmentation,” in IEEE International
Conference on Acoustics, Speech and Signal Processing, pp. 947–951,
2015.
J. Xu, X. Luo, G. Wang, H. Gilmore, and A. Madabhushi, “A deep
convolutional neural network for segmenting and classifying epithelial
and stromal regions in histopathological images,” Neurocomputing,
vol. 191, no. 1, pp. 214–223, 2016.
H. Chen, X. Qi, L. Yu, and P.-A. Heng, “Dcan: Deep contour-aware
networks for accurate gland segmentation,” in Proceedings of the IEEE
conference on Computer Vision and Pattern Recognition, pp. 2487–
2496, 2016.
T. Chen and C. Chefdhotel, “Deep learning based automatic immune
cell detection for immunohistochemistry images,” in International
Workshop on Machine Learning in Medical Imaging, pp. 17–24, 2014.
D. C. Cireşan, A. Giusti, L. M. Gambardella, and J. Schmidhuber,
“Mitosis detection in breast cancer histology images with deep neural
networks,” in International Conference on Medical Image Computing
and Computer-assisted Intervention, pp. 411–418, 2013.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley,
S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,”
in Advances in neural information processing systems, pp. 2672–2680,
2014.
A. Radford, L. Metz, and S. Chintala, “Unsupervised representation
learning with deep convolutional generative adversarial networks,”
arXiv preprint arXiv:1511.06434, 2015.
M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein gan,” arXiv
preprint arXiv:1701.07875, 2017.
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and
A. Courville, “Improved training of wasserstein gans,” arXiv preprint
arXiv:1704.00028, 2017.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for
image recognition,” in Proceedings of the IEEE conference on Computer
Vision and Pattern Recognition, pp. 770–778, 2016.
X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and
P. Abbeel, “Infogan: Interpretable representation learning by information maximizing generative adversarial nets,” in Advances in neural
information processing systems, pp. 2172–2180, 2016.
S. Nazlibilek, D. Karacor, T. Ercan, M. H. Sazli, O. Kalender, and
Y. Ege, “Automatic segmentation, counting, size determination and
classification of white blood cells,” Measurement, vol. 55, no. 3, pp. 58–
65, 2014.

11

[16]

[17]

Y. Sun and P. A. Sermon, “Methods for nuclei detection, segmentation,
and classification in digital histopathology: A review–current status and
future potential,” IEEE Reviews in Biomedical Engineering, vol. 7,
no. 1-5, p. 97, 2014.

[35]

J. Zhao, M. Zhang, Z. Zhou, J. Chu, and F. Cao, “Automatic detection and classification of leukocytes using convolutional neural
networks,” Medical & biological engineering & computing, vol. 55,
no. 8, pp. 1287–1301, 2017.

[36]

K. Sirinukunwattana, S. E. Ahmed Raza, Y. W. Tsang, D. R. Snead,
I. A. Cree, and N. M. Rajpoot, “Locality sensitive deep learning for
detection and classification of nuclei in routine colon cancer histology
images,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, p. 1196,
2016.

[37]

L. Hou, K. Singh, D. Samaras, T. M. Kurc, Y. Gao, R. J. Seidman, and
J. H. Saltz, “Automatic histopathology image analysis with cnns,” in
Scientific Data Summit (NYSDS), pp. 1–6, 2016.

[38]

X. Zhang, H. Su, L. Yang, and S. Zhang, “Weighted hashing with
multiple cues for cell-level analysis of histopathological images,” in
International Conference on Information Processing in Medical Imaging, pp. 303–314, Springer, 2015.

[39]

X. Shi, F. Xing, Y. Xie, H. Su, and L. Yang, “Cell encoding for
histopathology image classification,” in International Conference on
Medical Image Computing and Computer-Assisted Intervention, pp. 30–
38, Springer, 2017.

[40]

C. Callau, M. Lejeune, A. Korzynska, M. Garca, G. Bueno, R. Bosch,
J. Jan, G. Orero, T. Salvad, and C. Lpez, “Evaluation of cytokeratin19 in breast cancer tissue samples: a comparison of automatic and
manual evaluations of scanned tissue microarray cylinders.,” Biomedical
Engineering Online, vol. 14, no. S2, p. S2, 2015.

[41]

S. Wienert, D. Heim, S. Kai, A. Stenzinger, M. Beil, P. Hufnagl, M. Dietel, C. Denkert, and F. Klauschen, “Detection and segmentation of
cell nuclei in virtual microscopy images: A minimum-model approach,”
Scientific Reports, vol. 2, no. 7, p. 503, 2012.

[42]

Y. Xu, Z. Jia, L.-B. Wang, Y. Ai, F. Zhang, M. Lai, I. Eric, and C. Chang,
“Large scale tissue histopathology image classification, segmentation,
and visualization via deep convolutional activation features,” BMC
bioinformatics, vol. 18, no. 1, p. 281, 2017.

L. B. Dorini, R. Minetto, and N. J. Leite, “Semiautomatic white blood
cell segmentation based on multiscale analysis.,” IEEE Journal of
Biomedical and Health Informatics, vol. 17, no. 1, p. 250, 2013.

[43]

J. Xie, R. Girshick, and A. Farhadi, “Unsupervised deep embedding for
clustering analysis,” in International Conference on Machine Learning,
pp. 478–487, 2016.

O. Schmitt and M. Hasse, “Morphological multiscale decomposition of
connected regions with emphasis on cell clusters,” Computer Vision &
Image Understanding, vol. 113, no. 2, pp. 188–201, 2009.

[44]

O. Dzyubachyk, W. A. van Cappellen, J. Essers, W. J. Niessen, and
E. Meijering, “Advanced level-set-based cell tracking in time-lapse
fluorescence microscopy.,” IEEE Transactions on Medical Imaging,
vol. 29, no. 3, pp. 852–867, 2010.

[45]

F. Long, H. Peng, X. Liu, S. K. Kim, and E. Myers, “A 3d digital
atlas of c. elegans and its application to single-cell analyses.,” Nature
Methods, vol. 6, no. 9, p. 667, 2009.

M. Muthu, Rama Krishnan, C. Chakraborty, R. R. Paul, and A. K. Ray,
“Hybrid segmentation, characterization and classification of basal cell
nuclei from histopathological images of normal oral mucosa and oral
submucous fibrosis,” Expert Systems with Applications, vol. 39, no. 1,
pp. 1062–1077, 2012.

[18]

X. Xu, F. Lin, C. Ng, and K. P. Leong, “Dual spatial pyramid on rotation
invariant texture feature for hep-2 cell classification,” in International
Joint Conference on Neural Networks, pp. 1–8, 2015.

[19]

J. V. Lorenzo-Ginori, W. Curbelo-Jardines, J. D. Lpez-Cabrera, and
S. B. Huergo-Surez, Cervical Cell Classification Using Features Related
to Morphometry and Texture of Nuclei. Springer Berlin Heidelberg,
2013.

[20]

M. M. Dundar, S. Badve, G. Bilgin, V. Raykar, R. Jain, O. Sertel, and
M. N. Gurcan, “Computerized classification of intraductal breast lesions
using histopathological images.,” IEEE Transactions on Biomedical
Engineering, vol. 58, no. 7, pp. 1977–1984, 2011.

[21]

K. Nguyen, A. K. Jain, and B. Sabata, “Prostate cancer detection: Fusion
of cytological and textural features,” Journal of Pathology Informatics,
vol. 2, no. 1, p. 1, 2011.

[22]

W. L. Tai, R. M. Hu, C. W. H. Han, R. M. Chen, and J. J. P. Tsai,
“Blood cell image classification based on hierarchical svm,” in IEEE
International Symposium on Multimedia, pp. 129–136, 2011.

[23]

L. Putzu, G. Caocci, and C. D. Ruberto, “Leucocyte classification
for leukaemia detection using image processing techniques,” Artificial
Intelligence in Medicine, vol. 62, no. 3, pp. 179–191, 2014.

[24]

[25]

[26]

for early diagnosis of acute lymphoblastic leukemia in blood microscopic images. Springer-Verlag, 2014.

M. C. Su, C. Y. Cheng, and P. C. Wang, “A neural-network-based
approach to white blood cell classification,” The scientific world journal,
vol. 2014, no. 4, p. 796371, 2014.

[27]

D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv
preprint arXiv:1312.6114, 2013.

[28]

J. Xu, L. Xiang, Q. Liu, H. Gilmore, J. Wu, J. Tang, and A. Madabhushi,
“Stacked sparse autoencoder (ssae) for nuclei detection on breast cancer histopathology images.,” IEEE Transactions on Medical Imaging,
vol. 35, no. 1, pp. 119–130, 2016.

[46]

[29]

A. A. Cruzroa, J. E. Arevalo Ovalle, A. Madabhushi, and F. A.
Gonzlez Osorio, “A deep learning architecture for image representation, visual interpretability and automated basal-cell carcinoma cancer
detection.,” in International Conference on Medical Image Computing
and Computer-Assisted Intervention, pp. 403–410, 2013.

S. Hai, F. Xing, J. D. Lee, C. A. Peterson, and Y. Lin, “Automatic
myonuclear detection in isolated single muscle fibers using robust
ellipse fitting and sparse representation,” IEEE/ACM Transactions on
Computational Biology & Bioinformatics, vol. 11, no. 4, pp. 714–726,
2014.

[47]

[30]

N. Dilokthanakul, P. A. Mediano, M. Garnelo, M. C. Lee, H. Salimbeni, K. Arulkumaran, and M. Shanahan, “Deep unsupervised clustering with gaussian mixture variational autoencoders,” arXiv preprint
arXiv:1611.02648, 2016.

G. Bueno, R. Gonzlez, O. Dniz, M. Garcarojo, J. Gonzlezgarca, M. M.
Fernndezcarrobles, N. Vllez, and J. Salido, “A parallel solution for
high resolution histological image analysis.,” Computer Methods &
Programs in Biomedicine, vol. 108, no. 1, pp. 388–401, 2012.

[48]

H. Chang, J. Han, A. Borowsky, L. Loss, J. W. Gray, P. T. Spellman, and
B. Parvin, “Invariant delineation of nuclear architecture in glioblastoma
multiforme for clinical and molecular association,” IEEE Transactions
on Medical Imaging, vol. 32, no. 4, pp. 670–682, 2013.

[49]

S. Arslan, T. Ersahin, R. Cetin-Atalay, and C. Gunduz-Demir, “Attributed relational graphs for cell nucleus segmentation in fluorescence
microscopy images,” IEEE Transactions on Medical Imaging, vol. 32,
no. 6, pp. 1121–1131, 2013.

[50]

D. Nie, R. Trullo, J. Lian, C. Petitjean, S. Ruan, Q. Wang, and D. Shen,
“Medical image synthesis with context-aware generative adversarial
networks,” in International Conference on Medical Image Computing
and Computer-Assisted Intervention, pp. 417–425, 2017.

[31]

Z. Jiang, Y. Zheng, H. Tan, B. Tang, and H. Zhou, “Variational deep
embedding: An unsupervised and generative approach to clustering,” in
International Joint Conference on Artificial Intelligence, 2017.

[32]

J. T. Springenberg, “Unsupervised and semi-supervised learning
with categorical generative adversarial networks,” arXiv preprint
arXiv:1511.06390, 2015.

[33]

C. D. Malon and C. Eric, “Classification of mitotic figures with
convolutional neural networks and seeded blob features,” Journal of
Pathology Informatics, vol. 4, no. 1, p. 9, 2013.

[34]

S. Mohapatra, D. Patra, and S. Satpathy, An ensemble classifier system

12

[51]

[52]

[53]

[54]

[55]

[56]

[57]

[58]
[59]

[60]

Z. Li, Y. Wang, and J. Yu, “Reconstruction of thin-slice medical images
using generative adversarial network,” in International Workshop on
Machine Learning in Medical Imaging, pp. 325–333, Springer, 2017.
D. Mahapatra, B. Bozorgtabar, S. Hewavitharanage, and R. Garnavi,
“Image super resolution using generative adversarial networks and local
saliency maps for retinal image analysis,” in International Conference
on Medical Image Computing and Computer-Assisted Intervention,
pp. 382–390, 2017.
J. M. Wolterink, T. Leiner, M. A. Viergever, and I. Isgum, “Generative adversarial networks for noise reduction in low-dose ct.,” IEEE
Transactions on Medical Imaging, vol. PP, no. 99, p. 1, 2017.
E. Reinhard, M. Ashikhmin, B. Gooch, and P. Shirley, “Color transfer
between images,” IEEE Computer Graphics & Applications, vol. 21,
no. 5, pp. 34–41, 2001.
M. Macenko, M. Niethammer, J. S. Marron, D. Borland, J. T. Woosley,
X. Guan, C. Schmitt, and N. E. Thomas, “A method for normalizing
histology slides for quantitative analysis,” in IEEE International Conference on Symposium on Biomedical Imaging: From Nano To Macro,
pp. 1107–1110, 2009.
P. Kainz, M. Urschler, S. Schulter, P. Wohlhart, and V. Lepetit, “You
should use regression to detect cells,” in International Conference
on Medical Image Computing and Computer Assisted Intervention,
pp. 276–283, 2015.
D. Arthur and S. Vassilvitskii, “k-means++: The advantages of careful
seeding,” in Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms, pp. 1027–1035, Society for Industrial and
Applied Mathematics, 2007.
D. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
arXiv preprint arXiv:1412.6980, 2014.
D. G. Lowe, “Distinctive image features from scale-invariant keypoints,”
International journal of computer vision, vol. 60, no. 2, pp. 91–110,
2004.
T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale
and rotation invariant texture classification with local binary patterns,”
IEEE Transactions on Pattern Analysis and Machine Intelligence,
vol. 24, no. 7, pp. 971–987, 2002.

