Galaxy Morphology Classification with Deep Convolutional Neural Networks
Jia-Ming Dai Jizhou Tong


firstpage–lastpage
Controlling spin-orbit interactions in silicon quantum dots using magnetic field direction
    Andrew Dzurak
    
==========================================================================================





We propose a variant of residual networks (ResNets) for galaxy morphology classification. The variant, together with other popular convolutional neural networks (CNNs), are applied to a sample of 28790 galaxy images from Galaxy Zoo 2 dataset, to classify galaxies into five classes, i.e. completely round smooth, in-between smooth (between completely round and cigar-shaped), cigar-shaped smooth, edge-on and spiral. A variety of metrics, such as accuracy, precision, recall, F1 value and AUC, show that the proposed network achieves the state-of-the-art classification performance among the networks, namely, Dieleman, AlexNet, VGG, Inception and ResNets. The overall classification accuracy of our network on the testing set is 95.2083% and the accuracy of each type is given as: completely round, 96.6785%; in-between, 94.4238%; cigar-shaped, 58.6207%; edge-on, 94.3590% and spiral, 97.6953% respectively. Our model algorithm can be applied to large-scale galaxy classification in forthcoming surveys such as the Large Synoptic Survey Telescope (LSST).





methods: data analysis-techniques: image processing-galaxies: general.








§ INTRODUCTION


Galaxies have various shapes, sizes and colors. To understand how these morphologies of galaxies relate to the physics that create them, galaxies need to be classified. Thus galaxy morphology classification is a key step to the study on galaxy formation and evolution. In 1926, Edwin Hubble first proposed the Hubble Sequence   using visual inspection with fewer than 400 galaxy images (also called Hubble Tuning Fork), classifying galaxies into three basic types: elliptical, spirals and irregular <cit.>. And Hubble Sequence  is still in use today. For a long time, astronomers used the visual inspection to classify galaxies and update Hubble' classification scheme. In recent decades, large scale surveys such as the Sloan Digital Sky Survey (SDSS) have resulted a huge amount of galaxy images. Classifying these huge images by astronomers is not only time consuming but also a impossible mission.

Then Galaxy Zoo project attempted to solve the problem and was launched <cit.>. Galaxy Zoo 1 with a dataset made of a million galaxy images by the Sloan Digital Sky Survey, invited a large number of citizen scientists to provide the basic morphological information and identify if a galaxy was spiral, elliptical, a mergeror star/don't know<cit.>. The project achieved a huge success that the million galaxy images were annotated within several months. And then Galaxy Zoo 2 <cit.>, Galaxy Zoo: Hubble <cit.>, and Galaxy Zoo: CANDELS <cit.> are launched respectively. Unfortunately, this approach still doesn't keep up with the pace of data growth. Astronomers turn their sights to a automatic classification method.

Galaxy morphology classification using machine learning methods has played an important role in the past 20 years. Artificial neural networks, Naive Bayers, decision tree and Locally weighted Regression have been applied in galaxy classification on relatively small datasets in early work <cit.>. <cit.> found that the accuracy dropped from 95.66% to 56.33% classifying galaxies into 2 classes to 5 classes. <cit.> used artificial neural networks to assign galaxies to 3 classes with several input parameters, e.g., colors, shapes, concentration and texture. <cit.> used decision tree and fuzzy logic algorithms to galaxy morphology classification based on the designed photometric parameters and spectra parameters. <cit.> measured galaxy morphological parameters including Concentration, Asymmetry, Smoothness, Gini coefficient, Moment, Entropy and Spirality to automatically classify galaxies employed the Linear Discriminant Analysis (LDA).  Other recent galaxy classification methods <cit.> all need feature extraction, which needs human careful design. It is well known that the performance of classification depends on the choice of data representation, called feature engineering <cit.>. Feature engineering needs domain expertise and is time-consuming.

In the past three years galaxy morphology classification using deep learning algorithms has obtained more attention. Deep learning models are composed of multiple non-linear layers to learn data representation, which allow to be fed with raw data directly and automatically learn the representations of data <cit.>.  After multiple non-linear transforming, the representations of higher layers are abstract and beneficial for discrimination and classification. Deep convolutional neural networks (CNNs) have become the dominant approach in image classification task. With the availability of the large number of Galaxy Zoo labeled dataset, some works have yielded good results. <cit.> for the first time used a 7-layers CNN to galaxy morphology classification which exploits galaxy images translation and rotation invariance. Then, <cit.> used the <cit.> model to classify high redshift galaxies in the 5 Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey (CANDELS).  <cit.> used CNNs to estimate the photometric redshift of galaxies.  <cit.> presented a star-galaxy classification framework similar to VGG  <cit.>. Recently, CNNs have been applied to find strong gravitational lenses in the Kilo Degree Survey <cit.>.  And <cit.> used CNNs to classify radio galaxies into FRI, FRII and Bent-tailed radio galaxies.

In this study, we propose a modified residual network (ResNet) for galaxy morphology classification. We select 28790 galaxy images from Galaxy Zoo 2 dataset and use five forms of data augmentation to enlarge the number of our training samples in data preprocessing to avoid overfitting. The variant we proposed combines the advantages of Dieleman model   <cit.> and residual networks. In addition, We implement several other popular CNNs models, including Dieleman, AlexNet <cit.>, VGG <cit.>, Inception <cit.> and ResNets <cit.> and systematically compare the classification performance of ours with these CNNs model. As expected, we demonstrate that our model achieves a state-of-the-art performance. Furthermore, to understand what the CNNs learn, we visualize the filters weights and feature maps to give a qualitative empirical analysis.

This paper is organized as follows. We introduce the dataset selection in Section <ref>. Section <ref> describes deep learning and convolutional neural networks (CNNs).  Section <ref> contains data preprocessing pipeline, data augmentation, the residual network we have proposed and the training tips. Section <ref>
are the results and analysis of our network and other CNNs models. Finally, we draw conclusions and future work in Section <ref>.




§ DATASET
 

The galaxy images in this study are drawn from Galaxy Zoo-the Galaxy Challenge [https://www.kaggle.com/c/galaxy-zoo-the-galaxy-challenge], which contain 61578 JPG color galaxy images with probabilities that each galaxy is classified into different morphologies. Each image is of 424×424×3 pixels in size taken from the Galaxy Zoo 2 main spectroscopic sample from SDSS DR7 [http://www.sdss.org/]. The morphological classifications vote fractions are modified version of the weighted vote fractions in the Galaxy Zoo 2 project [https://www.galaxyzoo.org/]. The classifications vote fractions have high level of agreement and authority with professional astronomers <cit.>. The data has been used in studies of galaxy formation and evolution <cit.>.




In this study clean samples are selected that match a specific morphology category with their appropriate thresholds <cit.>, which depend on the number of votes for a classification task considered to be sufficient. For example, to select the spiral, cuts are the combination of f_features/disk≥ 0.430 , f_edge-on,no≥ 0.715, f_spiral,yes≥ 0.619. These thresholds are considered conservative to select clean samples in <cit.> . By this means, we assign galaxy images to five classes, i.e. completely round smooth, in-between smooth(between completely round and cigar-shaped), cigar-shaped smooth, edge-on and spiral. In practice, all thresholds are derived from <cit.> except thresholds of smooth galaxy are loosened from 0.8 to 0.5, and full details refer to <cit.>. Table <ref> shows the clean samples selection criterion for every class. The 5 classes galaxies are referred to as 0, 1, 2, 3 and 4, each contains a sample of 8434, 8069, 578, 3903 and 7806 respectively. Figure <ref> shows the galaxy images randomly selected from the dataset and each row represents a class. From top to bottom, their labels are: 0, 1, 2, 3 and 4 respectively.



The dataset reduces to 28790 images after filtering, then is divided into training set and testing set by a ratio of 9:1. Thus there are 25911 images for training set to train our model and remaining 2879 images for testing set to evaluate our model. Training set and testing set have the same distribution. Table <ref> gives the number of galaxy images in each morphological class of training set and testing set and Figure <ref> reproduces the dataset graphically.








§ DEEP CONVOLUTIONAL NEURAL NETWORKS
 

Deep learning models are composed of multiple layers to automatically learn data representations from the raw data, which are capital for classification, localisation, detection, segmentation without feature extraction <cit.>. Deep convolutional neural networks (CNNs) have played an important role in deep learning <cit.>. Convolutional neural networks have become the dominant approach in image classification. In this section, we briefly introduce artificial neural networks (ANN), convolutional neural networks (CNNs), especially residual networks (ResNets).



 §.§ Artificial Neural Networks


Artificial neural networks (ANN) are made up of simple adaptive units  interconnected, which can simulate biological nervous system, interaction in response to real world objects  <cit.>. Figure <ref>  shows a simple feed forward neural network. It is composed of input layer, hidden layer and output layer. Formally, define x_i^l, x_j^l+1 as the i-th neuron of l-th layer, the j-th neuron of (l+1)-th layer, define w_ij^l, b_j^l as weights, bias of the l-th layer, respectively. Then, the outputs of the l-th layer are x_j^l+1:


    x_j^l+1=f(∑_i∈ N^l(w_ij^lx_i^l+b_j^l))


where N^l is the number of l-th layer, f is the activation function. Activation functions have many types, such as the popular rectified linear unit (ReLU)  <cit.>, f=max(0,x) , sigmoid, tanh, Leaky ReLU, ELU and so on.



Then, let  ŷ=( ŷ_̂1̂,ŷ_̂2̂ , ⋯, ŷ_̂k̂, ⋯, ŷ_̂m̂) be the output of network, y=( y_1, y_2, ⋯, y_k, ⋯ y_m ) be the desired output and we can define a cost function ℓ( ŷ,y). In classification task, the cost function can be a cross entropy. Especially, in binary classification, the cross entropy can be defined as


    ℓ(ŷ,y)=-ylogŷ -(1-y)log(1-ŷ).

where y∈{0, 1},  ŷ∈[0, 1]. Then, we need to compute cross entropy of all training data. In order to minimum the cross entropy, we use stochastic gradient descent (SGD) to update the weights and bias until the loss function converge:


    w_n+1^l=w_n^l-η∂ℓ/∂ w_n^l.


    b_n+1^l=b_n^l-η∂ℓ/∂ b_n^l.

Where η is learning rate. Of course, now we generally use mini-batch stochastic gradient descent instead of all data stochastic gradient descent in practice in order to save training time, which can seek for a local optimal solution.



 §.§ Convolutional Neural Networks


Convolutional Neural Networks (called CNNs or Convnets) <cit.> are designed to process multiple arrays data, for example, image data. CNNs have become very successful in practical applications. A classical layer of CNN is made up of three stages. In first stage, the layer performs several convolutions. Then, a non-linear activation function such as ReLU is applied. At last, a pooling function modifies the output the layer <cit.>. CNNs generally contain convolutional layers, pooling layers and fully connected layers.

Convolutional layers. Convolution is a specialized kind of linear operation. Discrete convolution can be viewed as multiplication by a matrix. Convolutional layers can be computed by


    x_j^l=f(Σ_i∈ M_jx_i^l-1× k_ij^l+b_j^l).

Where l is the number of layer, f is activation function usually ReLU,  k represents convolutional kernel, M_j represents the receptive field and b is bias.


Pooling layers (also called subsampling). Pooling can be achieved by taking average (average pooling) or taking maximum ( max pooling) within a rectangular neighborhood. For an image, it can reduce the size of images.

Fully connected layers. Fully connected layers are usually followed by the last pooling layer or the convolutional layer, and every neuron in fully connected layers is connected to all the neurons in the upper layers.

Generally, convolutional networks (CNNs) have sparse connectivity, parameter sharing and equivariant representation, three important ideas.

Deep convolutional networks (CNNs) have brought a series of breakthroughs in image classification. And CNNs are getting deeper and deeper, from 8 layers <cit.>, 16/19 layers <cit.>, 42 layers <cit.>, to 152 layers <cit.>. In order to train deeper networks, some new techniques are adopted, such as ReLU <cit.>, dropout <cit.>, GPUs, data augmentation <cit.>, batch normalization (BN) <cit.> and so on. Now CNNs models have developed several versions, primarily including AlexNet <cit.>, VGG <cit.>, Inception <cit.>, ResNets <cit.> and DenseNet <cit.>.



 §.§ Residual Networks


Deep residual networks (ResNets) are reported in <cit.>, which can deepen the networks up to thousands of layers and achieve state-of-the-art performance. In this section, we give a brief description of ResNets.

<cit.> proposed a deep residual learning framework: let the layers try to learn a residual mapping instead of the directly desired underlying mapping of a few stacked layers. Figure <ref> shows a residual building block. Let the desired underlying mapping be H(x_l), let the stacked nonlinear layers fit mapping of F(x_l)=H(x_l)-x_l. This is residual. The formulation F(x_l)=H(x_l)-x_l can be written to  H(x_l)=F(x_l)+ x_l ,  F(x_l)+ x_l can be realized by feed forward neural networks with short connection  (Figure <ref>), which skips one or more layers and perform identity mapping. At last, their outputs are added to the outputs of the stacked layers. A residual unit can be expressed as follows:

    x_l+1=f(h(x_l)+F(x_l,W_l)).

Where x_l and x_l+1 are input and output of the l-th  unit, and F is a residual function. For example, Figure <ref> has two layers, F=W_2σ(W_1x) in which σ denotes ReLU and the biases are omitted for simplifying notations. h(x_l)=x_l and f is a ReLU function.





There are two kinds of residual building blocks in <cit.> as shown in Figure <ref>. The basic residual unit (Figure <ref>, left) contains two layers, 3× 3,  3× 3 convolutions. In order to decrease the training time and network parameters, a modified residual unit is presented as Figure <ref> (right) shows, which is called a bottleneck  building block. The bottleneck  building block uses 3 layers instead of 2 layers and they are 1 × 1,  3× 3,  1 × 1 convolutions, where the 1 × 1 convolutional layers can reduce and increase dimensions.

In <cit.>, both  h(x_l)  and  f are identity  mapping, where signal could be directly propagated from one unit to other units, in both forward and backward passes. The residual unit can be redefined as:

    x_l+1=x_l+F(x_l,W_l).


And <cit.> also adopted pre-activation, where BN-ReLU-Conv  replaced the traditional Conv-BN-ReLU. This is called ResNet V2, which is much easier to train and has better performance than ResNet V1 <cit.>. Now, ResNets have many versions, like ResNet-50/101/152/200, deeper layers up to 1001 layers.




§ APPROACH
 

In the previous section we introduce the theory of ResNets. In this section, we describe our framework including data preprocessing, data augmentation, scale jittering, network architecture, and implementation details.



 §.§ Preprocessing


From the dataset, images are composed of large fields of view with the galaxy of interest in the center. So it is necessary to crop the image at first step. In practice we crop from the center of image to a range scale S=[170,240] in training set for every image (as explained later). It allows all the main information to be contained in the center of image, also eliminates many noises like other secondary objects and reduces the dimension of images almost a quarter for faster training. A complete preprocessing procedure is illustrated in Figure <ref>.



Then, the image is resized to 80 × 80 × 3 pixels, which is just dimension reduction and easy to compute under limited computing source. Next, a random cropping is carried out, which increases the size of training set by a factor of 256. The size of image drops to 64 × 64 × 3 pixels. Next the image is randomly rotated  with 0^∘, 90^∘,180^∘,270^∘ because of rotation invariant of galaxy images and randomly horizontally flipped. Brightness, contrast, saturation and hue adjustment are applied to the image and the last step is image whitening. Above is the whole preprocessing pipeline in training. After those steps, images(64 × 64 × 3 pixels) will be used as input of networks when training.

At testing time, preprocessing procedure does not include random cropping, rotation, horizontal flipping and optical distortion. After center cropping to a fixed value Q={ 180,200,220,240}(as explained later), the image is resized to 80 × 80 × 3 pixels and then performs center cropping again, the size of the image is 64 × 64 × 3 pixels. And  the last step is still image whitening, images will be used as input of networks when testing.



 §.§ Data augmentation


In order to avoid overfitting, data augmentation is one of the common and effective ways to reduce overfitting. Because of our limited training data, data augmentation can enlarge the number of training images. We use five different forms of data augmentation.

Scale jittering is the first form of data augmentation. In training time, we crop the images to a range scale S=[170,240] , which is called multi-scale training images because of the S random value. Since different images can be cropped to different sizes and even the same images also can be cropped to different sizes at different iterations, it is beneficial to take this into account during training. This  can be seen as training set augmentation by scale jittering.

Random cropping is carried out from 80 × 80 × 3 pixels to 64 × 64 × 3 pixels, which increases the size of training set by a factor of 256. Rotating training images with 0^∘, 90^∘,180^∘,270^∘ can enlarge the size of training set by a factor of 4. A horizontal flipping is a doubling of training images.

The first four forms of data augmentation are affine transformations that means the very little computation and they are completed on the CPU before training on the GPUs. Brightness, contrast, saturation and hue adjustment are the same as <cit.>, which are optical distorting for data augmentation.



 §.§ Scale jittering


Scale jittering is derived from <cit.>, in which images of the input are cropped from multi-scale training images and fixed multi-scale testing images.

Training scale jittering. Let set S be multi-scale training (we also refer to S as the training scale), where each training image is individually rescaled by randomly sampling S from a certain range [S_min, S_max] ( we use S_min=170  and   S_max=240 ). By this means different images can be cropped to different sizes and even the same images also can be cropped to different sizes at different iterations, that greatly enlarge the number of training set and effectively avoid overfitting. This can be seen as training set augmentation by scale jittering.

Testing scale jittering. Let set Q be fixed multi-scale testing (we also refer to Q as the testing scale). In practice, we use Q={ 180,200,220,240} when testing, which makes our models achieve better performance.



 §.§ Network architecture


Our model is a variant of ResNets V2 <cit.>. As Section <ref> describes, deep residual networks (ResNets) always seek for deeper and deeper. So the ResNets look like very thin and height. Recent research work shows that such deep residual networks come cross the risk of diminishing feature reuse, which train very slowly and need too much time <cit.>. We propose a network specially designed for galaxy by trying to decrease the depth and widen residual networks. Our overall architecture of network is depicted in Figure <ref> and Table <ref>.

We adopt full pre-activation residual units as Figure <ref> shows. And a bottleneckbuilding block( Figure <ref>, right) presented in <cit.> is used, namely, a combination of 1 × 1,  3× 3,  1 × 1 convolutions, for example, 1 × 1, m × k  convolution,  3× 3,m × k convolution,  1 × 1,n × k convolution, where m, n denotes the number of channel, k is the widening factor. The full pre-activation includes standard BN-ReLU-Conv. In addition to these, we add a dropout after 3× 3  convolution whereas ResNet V2 <cit.> did not use dropout to prevent coadaptation and overfitting. The residual unit is defined as:


    x_l+1=x_l+W_3σ(W_2σ(W_1σ(x_l))).

Here, x_l and x_l+1 are input and output of the l-th unit,  σ denotes BN and ReLU, W_1, W_2, W_2 represent 3 convolutional kernels, dropout is placed after the W_2 operation  and the biases are omitted for simplifying notations.

Then looking at our network architecture (Figure <ref> and Table <ref>), the size of input of network is 64 × 64 × 3 pixels. Firstly, 64 kernels of size of 6 × 6 × 3 with a stride of 1 are performed, which is derived from <cit.> and proven to be optimal. After the first convolutional layer, a max pooling of size of 2× 2 with a stride of 2 is connected. The size of output of image becomes 32 × 32 × 64.

The output of max pooling is fed to 4 convolutional groups: conv2, conv3, conv4 and conv5, respectively. Each group has 2 residual blocks. For example, in convolutional group 2, there are 2 residual blocks: 1 × 1, 64 × 2 (128 channels) convolution, 3× 3,64 × 2 (128 channels) convolution, 1 × 1,256× 2 (512 channels) convolution; 1 × 1, 64 × 2 (128 channels) convolution, 3× 3,64 × 2 (128 channels) convolution, 1 × 1,256× 2 (512 channels) convolution with a stride of 2, which performs downsampling. Group3, group4 and group 5 are the same, except for the last layer of group 5 does not perform downsampling. Downsampling is performed by the last layers in groups conv2, conv3 and conv4 with a stride of 2.

The dashed shortcuts of Figure <ref> decrease dimensions. The contributions of 1× 1 convolutional layers are reducing dimensions at first and then increasing dimensions, to reduce the parameters of model and speed up training. The last layer is global average-pooling layer with 4 × 4 kernel and the size of output of average pooling is 1 × 1 × 4096. At last is a 5-way fully connected layer with softmax.

Where k is the widening factor, N denotes the number of blocks in group. After hundreds of trying, we  finally use k=2, N=2 in practice. So our network is 26 layers totally including 26.3M parameters. The 26-layers network achieves the best performance on accuracy and other metrics.






From our network architecture, some tips are concluded: the first convolutional layer adopts a relatively large convolution filter of 6 × 6; the convolutional layers mostly have 1 × 1 and 3× 3 convolutions. The advantages of 1 × 1 convolutions have been described. The advantages of small  3× 3 filter have been demonstrated in  <cit.>, which can decrease the number of parameters of model and achieve a better performance. The feature maps in each group are the same except the last layer of each convolutional group(conv2, conv3 and conv4). The feature map size is halved, the number of filters is doubled.







 §.§ Implementation Details


We use mini-batch gradient descent with a batch size of 128 and Nesterov momentum of 0.9. The initial learning rate is set to 0.1, then decreased by a factor of 10 at 30k and 60k iterations, and we stop training after 72k iterations. The weight decay is 0.0001, dropout probability value is 0.8 and the weights are initialized as in <cit.>. We adopt BN before activation and convolution, following <cit.>.

Our implementation is based on Python, Pandas, scikit-learn <cit.>, scikit-image <cit.>, TensorFlow <cit.>. It takes  about 31.5 hours to train a single network  with a  NVIDIA Tesla K80 GPU. Our code is available at <https://github.com/Adaydl/GalaxyClassification>.




§ RESULTS AND DISCUSSION
 

In this section, we describe 7 kinds of classification performance metrics: accuracy, precision, recall, F1, confusion matrix, ROC and AUC. Then we show the results of our model and compare systematically the performance of our model with other popular CNNs models, such as Dieleman, AlexNet, VGG, Inception and ResNets. In the end we visualize the filters and feature maps.



 §.§ Classification Performance Metrics


To assess the performance of our classification models, we present 7 kinds of classification performance metrics: accuracy, precision, recall, F1, confusion matrix, ROC and AUC. They are defined as follow:

Accuracy: ŷ_̂î is the predicted value of the i-th sample and y_i is the corresponding true value, then the fraction of correct predictions over n_samples is defined as


    Accuracy(y_i,ŷ_̂î)=1/n_samples∑_i=0^n_samples-11(ŷ_̂î=y_i).


Precision, Recall & F1 <cit.>: Given the number of true positive (TP), false positive (FP), true negative (TN) and false negative (FN), we define:


    P=TP/TP+FP.



    R=TP/TP+FN.



    F1=2PR/P+R.


Confusion Matrix(CM): An entry CM_ij ( i,j=1,2, ⋯, n_samples) is defined as the number of the true class i ,but predicted to class j.

ROC & AUC: A receiver operating characteristic (ROC) curve plots the true positive rate against the false positive rate for every possible classification threshold. AUC is the area under the receiver operating characteristic (ROC) curve. The closer the AUC is to 1, the better the classification performance.




 §.§ Classification Results and Discussion


In this section, we summary the results of our models on 7 kinds of classification performance metrics and compare the results of our model with other popular CNNs.

Table <ref> shows that precision, recall and F1 of our model for each class on testing set. 0, 1, 2, 3 and 4 represent completely round, in-between, cigar-shaped, edge-on and spiral, respectively. The average precision, recall and F1 of the 5 classes galaxies of our model are 0.9512, 0.9521 and 0.9515. The completely round achieves the best precision of 0.9611. The spiral achieves the best recall of 0.9782 and F1 value of 0.9677. On the whole, the results of the completely round, the in-between, the edge-on and the spiral are extremely excellent, except the cigar-shaped. It happens due to the small number of the cigar-shaped images for training.

The confusion matrix of our model for each class on testing set is shown in Table <ref>. Column represents true label and row represents prediction label. 815 completely round, 762 in-between, 34 cigar-shaped, 368 edge-on and 763 spiral are classified correctly. So the accuracy of the 5 galaxy types are: completely round, 96.6785%; in-between, 94.4238%; cigar-shaped, 58.6207%; edge-on, 94.3590% and spiral, 97.6953% respectively. 29 completely round are incorrectly classified as in-between. It is common sense that completely round and in-between are similar itself and easily misclassified. Note that 4 completely round are misclassified as spiral, perhaps due to faint images photographed from far away distance.  It observes that 12 cigar-shaped are misclassified as edge-on and 18 edge-on are misclassified as cigar-shaped, where the number of misclassifications is greater than others.  We suppose that it happens due to the similarity of cigar-shaped and edge-on, which is so surprising.





Figure <ref> shows that ROC curve of our model for 5 classes galaxies on testing set. Each color represents a class. The closer the true positive rate (TPR) is to 1 and false positive rate (FPR) is to 0, the better the curve predicts, namely, the closer the curve is to the upper left corner, the better it predicts. From Figure <ref>,  ROC curve of each class performs well, the edge-on predicts the best and the cigar-shaped predicts relatively worse, which happens due to the small number of cigar-shaped images. The average AUC of our model is 0.9823, and shows that the overall prediction performance of our model is excellent.




Table <ref> summaries  test accuracy of different methods at multiple test scales. Our results are based on average values of the maximum values of 10-times runs of each test scale. Recent research work shows scale jittering at testing time can obtain a better performance <cit.>. Our model obtain the best results with 94.6875% accuracy. Table <ref> shows that Dieleman model <cit.> works well, obtains 93.8800% accuracy, although it is a only 7-layers CNN. It is easy to understand that it is designed specifically for galaxy images and other networks, such as AlexNet <cit.>, VGG <cit.>, Inception <cit.> and ResNets <cit.>, are designed for ImageNet, but they all have excellent performance because of their good generalization performance. AlexNet is a 8-layers CNN won the first place in the ImageNet LSVRC-2012 in 2012 years, here, it achieves a 91.8230% accuracy due to its used relatively large filter( 11 × 11 convolution). VGG-16 achieves a 93.1336% accuracy, which uses many small 3 × 3 filters. Inception here implemented is Inception V3 including 42 layers with careful designed inception module, and here achieves 94.2014% accuracy. ResNet-50 here implemented is pre-act-ResNets and  obtains a 94.0972% accuracy.



Table <ref> summaries  test accuracy, precision, recall, F1 and AUC of different methods. Our results are based on the maximum values of 10-times runs of each testing scale. We notice the results of accuracy are better than the results indicated in Table <ref>, because they are obtained by picking the maximum values of 10-times runs of each testing scale, instead of the average values of the maximum values of 10-times runs of each testing scale. Our model achieves the best accuracy 95.2083% at single testing scale. Because accuracy has a fatal flaw in multi-class task that it depends on the number of the majorities, so we also adopt average precision, recall, F1 and AUC to measure classification performance. Our model obtains the best average precision 0.9512, the best average recall 0.9521 and the best average F1 0.9515.  Inception achieves the best average AUC 0.9852. On the whole, our model works excellent and achieves state-of-the-art performance.






 §.§ Filters and Feature Maps Visualization


Neural networks are always known as black boxes. We want to visualize what the CNN learn by visualizing filters weights and feature maps and then give a qualitative empirical analysis<cit.>. In order to understand easily, we visualize a simple CNN, 7 layers totally, including 4 convolutional layers (6× 6, 32 filters, 5× 5, 64 filters, 3 × 3, 128 filters, and 3× 3, 128 filters, respectively) and 3 fully connected layers.

Figure <ref> shows that filter weights learned on every convolutional layer. The first layer filters detect the different galaxy edges, corners, etc. from original pixel, then use the edge to detect simple shapes in second layer filters, such as the bar, the elliptical and so on, and then use these shapes to detect more advanced features in high level layer filters. More invariant representations are learned with the increase of layers.  And from Figure <ref>, different filters also learn different color information, mainly red and blue that might correspond to the color of galaxy itself, such as red elliptical galaxy and blue spiral galaxy.



Figure <ref> shows that activations of of each layer on a smooth galaxy (GalaxyID: 909652). In first layer, some feature maps recognize the intermediate core of galaxy, and some recognize the background part. In high layers, feature maps recognize the abstract blobs with the combination of high-level features, e.g., in the fourth convolutional layer.
It is seen that after pooling layers, the differentiability of each feature map is stronger, which is exactly what the classification model expects. These interesting phenomenons also can be found in  Figure <ref> and Figure <ref>.











§ CONCLUSIONS 
 

In this paper, we propose a variant of residual networks (ResNets) for Galaxy morphology classification. We classify 28790 galaxies into 5 classes, namely, completely round smooth, in-between smooth (between completely round and cigar-shaped), cigar-shaped smooth, edge-on and spiral using Galaxy Zoo 2 dataset. In data preprocessing, a complete preprocessing pipeline is presented and five forms of data augmentation are adopted to avoid overfitting, especially scale jittering that extremely enlarges the number of training images.

The advantage of our network is combining Dieleman model with residual networks (ResNets), in which we try to decrease the depth and widen residual network. We use a bottleneckresidual unit with full pre-activation BN-ReLU-Conv. In order to ovid overfitting, we use dropout after 3× 3  convolution. Our network has 26 layers with 26.3M parameters. We make a systematic comparation between our model and other popular convolutional networks (CNNs) in deep learning, such as Dieleman, AlexNet, VGG, Inception and ResNets. Our model achieves the best classification performance, the overall accuracy on testing set is 95.2083% and the accuracy of the 5 galaxy types are: completely round, 96.6785%; in-between, 94.4238%; cigar-shaped, 58.6207%; edge-on, 94.3590% and spiral, 97.6953% respectively. The average precision, recall, F1 and AUC of our model are 0.9512, 0.9521, 0.9515 and 0.9823. From the confusion matrix, we find that 12 cigar-shaped are misclassified as edge-on and 18 edge-on are misclassified as cigar-shaped, where the number of misclassifications is greater than others. We suppose that it happens due to the similarity of cigar-shaped and edge-on, which is so surprising. Dieleman model also works well and the average accuracy is 94.6528% because it is specially designed for galaxy images. Although AlexNet, VGG, Inception and ResNets are designed for ImageNet, they all achieve excellent performance because of their good generalization, whose accuracies are: 92.2569%, 93.6458%, 94.5139% and 94.6875%.

By visualizing filters weights and feature maps, we try to understand what the CNN model learn. For instance, the first layer filters detect the different galaxy edges, corners, etc. from original pixel, then use the edge to detect simple shapes in second layer filters, such as the bar and the elliptical, and next use these shapes to detect more advanced features in high level layer filters. We also find that different filters also learn different color information, mainly red and blue, which might correspond to the color of galaxy itself, such as red elliptical galaxy and blue spiral galaxy. About activations of of each layer on a galaxy image, some feature maps recognize the intermediate core, some recognize the background part in the first layer, feature maps recognize the abstract blobs with the combination of high-level features in higher layers. It also is found that after pooling layers, the differentiability of each feature map is stronger, which is exactly what the classification model expects.

In future large-scale surveys, such as the Dark Energy Survey (DES) and the Large Synoptic Survey Telescope (LSST), will obtain billions of galaxy images and our algorithms can be applied to automatically classify galaxies and achieve sate-of-the-art performance.

In future work, we focus on much more fine-grained galaxy morphology classification. We plan to train our model on bigger and higher quality galaxy dataset. In the end, more advanced algorithms in deep learning will merge with galaxy morphology classification.




§ ACKNOWLEDGEMENTS


We would like to thank the galaxy challenge, Galaxy Zoo, SDSS and Kaggle platform for sharing data. We acknowledge the financial support from the National Earth System Science Data Sharing Infrastructure (<http://spacescience.geodata.cn>). We are supported by  CAS e-Science Funds (Grand XXH13503-04).









mnras
 


















