abstract: Face recognition achieves exceptional success thanks to the emergence of deep learning. However, many contemporary face recognition models still perform relatively poor in processing profile faces compared to frontal faces. A key reason is that the number of frontal and profile training faces are highly imbalanced - there are extensively more frontal training samples compared to profile ones. In addition, it is intrinsically hard to learn a deep representation that is geometrically invariant to large pose variations. In this study, we hypothesize that there is an inherent mapping between frontal and profile faces, and consequently, their discrepancy in the deep representation space can be bridged by an equivariant mapping. To exploit this mapping, we formulate a novel Deep Residual EquivAriant Mapping (DREAM) block, which is capable of adaptively adding residuals to the input deep representation to transform a profile face representation to a canonical pose that simplifies recognition. The DREAM block consistently enhances the performance of profile face recognition for many strong deep networks, including ResNet models, without deliberately augmenting training data of profile faces. The block is easy to use, light-weight, and can be implemented with a negligible computational overhead [^1].
author: Kaidi Cao$^{2}$[^2] Yu Rong$^{1,2}$`<!-- -->`{=html} Cheng Li$^{2}$ Xiaoou Tang$^{1}$ Chen Change Loy$^{1}$
$^{1}$`<!-- -->`{=html}Department of Information Engineering, The Chinese University of Hong Kong
$^{2}$`<!-- -->`{=html}SenseTime Research
`{ry017, ccloy, email@example.com `` {caokaidi, email@example.com`
bibliography: short.bib; dream.bib
title: Pose-Robust Face Recognition via Deep Residual Equivariant Mapping

# Introduction

The emergence of deep learning greatly advances the frontier of face recognition . The main focus tends to center around near-frontal faces while there can be no assurance of view consistency when face recognition is conducted in unconstrained environments. Although human performance only drops slightly from frontal-frontal to frontal-profile face verification, many existing algorithms can suffer a drop of over 10% . Thus, large pose variation remains to be a significant challenge that confronts real-world face recognition.

We provide an example in Figure  to show the failure modes of a state-of-the-art face verification model. We trained the same ResNet-18 model as reported in . This model achieves a high accuracy of 99.3% on the LFW benchmark . Despite the strong model, it tends to falsely match profile faces of different identities yielding a number of false positives. In addition, the model is also likely to miss frontal and profile faces of the same identity leading to false negatives.

Why does face recognition work poorly on profile faces? Modern deep learning is heavily data-driven . The generalization power of deep models is usually proportional to the training data size. Given an uneven distribution of profile and frontal faces in the dataset, deeply learned features tend to bias on distinguishing frontal faces rather than profile faces. When it is infeasible to collect a massive dataset that covers all possible poses with even distribution, researchers have turned to alternative approaches to better handle the recognition of profile faces. A large body of methods normalize images to a single frontal pose before recognition, either through elaborated dense 3D facial landmark detection and warping , or another deep model (or generative adversarial network) specialized in face frontalization . Such methods would add processing burden to the whole system. In addition, face frontalization in the wild, especially with extreme profile faces, is still considered challenging. Often, synthesized 'frontal' faces would contain artifacts caused by occlusions and non-rigid expressions. Another potential solution is divide-and-conquer, , training separate models for learning pose-specific identity features . This strategy tends to increase computational cost due to the use of multiple models.

In this study, we hypothesize that the profile face domain possesses a gradual connection with the frontal face domain in the deep feature space. Figure  illustrates a deep representation embedding of faces belong to the same subject but in different poses. Given an input image of arbitrary pose, we can actually map its feature to the frontal space through a mapping function that adds residual. This observation is closely connected to the notion of *feature equivariance* , which finds the representation of many deep layers depends upon transformations of the input image. Interestingly, such transformations can be learned by a mapping function from data and the function can be subsequently applied to manipulate the representation of an input image to achieve the desired transformation.

Motivated by this observation, we formulate a novel module called *Deep Residual EquivAriant Mapping* (DREAM) block, which can model the transformation between frontal-profile faces in the high-level deep feature space. The block adaptively adds residuals to an input representation to transform a profile face to a canonical pose to simplify recognition. The residuals are generated conditioned on the preceding feature representation via a few additional weight layers. To accommodate input faces of arbitrary pose, a soft gate is introduced to adaptively control the amount of residuals such that more residuals are added to extreme profile faces while keeping the representation unchanged if the input is already in a frontal pose.

Our work is conceptually related to the face frontalization  in that our approach also performs 'frontalization' but not in the image space. We observe from our experiments that transforming profile face features to frontal features could yield better performance than image-level frontalization, which is susceptible to the negative influence of artifacts as a result of image synthesis. To our best knowledge, this study is the first attempt to perform *profile-to-frontal face transformation in the deep feature space*.

The DREAM block is appealing in several aspects:

1.  It is simple to implement. Specifically, the DREAM block is implemented as a simple yet effective gated residual branch. It can be integrated into existing convolutional neural network (CNN) architectures through stitching the block to the base network. It does not alter the original dimensionality of the face representation and can be trained end-to-end with standard back-propagation.

2.  It is light-weight. It adds only a tiny amount of parameters and computation to the base model. For instance, it only adds 0.3% parameters on ResNet-18 and increases its forward time by 1.6%.

3.  The proposed approach helps a base network that already does well in near-frontal face recognition to gain better performance in recognizing faces with extreme pose variation. This is done without elaborated data augmentation and face normalization that is practiced by most existing face recognition studies. For instance, the DREAM block reduces the error of ResNet-18 and ResNet-50 by 16.3% and 23.7% on the CFP benchmark . In addition, it also gains 8.5% improvement on the verification task (TAR@FAR=0.001) and reduces the error rate by 17.5% on the identification task (Rank-1 accuracy) on the IJB-A dataset  for ResNet-18. For ResNet-50, the numbers are 7.0% and 12.6%, respectively.

# Related Work

**Deep Learning for Face Recognition.** Deep learning is the prominent technique for face recognition. Most existing studies deploy CNNs, but with different loss functions, such as contrastive loss , triplet loss , and center loss . Center loss represents the current state-of-the-art approach that learns a center for deep features of each class and simultaneously minimize the distances between the deep features and their corresponding class centers, thus intra-class features variations are reduced and discriminative power of learned features are enhanced. Prior to center loss, Joint Bayesian  is widely used to derive a similarity metric for robust face verification. The aforementioned metric learning methods facilitate more robust verification given faces with arbitrary poses. Nevertheless, as shown in our experiments, we found that the DREAM block performs better, especially on extreme profile faces (such as those in the Celebrities in Frontal-Profile (CFP) dataset ).

**Profile Face Recognition.** It is not new for researchers to take pose variations into consideration when dealing with the face recognition problem. Existing methods address profile face recognition through elaborated dense 3D facial landmark detection and warping , face frontalization , or training separate models for learning pose-specific identity features . There are alternative approaches. For instance, Masi   enhances the performance of CNN through augmenting training and test data with face images differ in 3D shape, expression and pose. Yin   propose a multi-task CNN that exploits side tasks, , pose, to serve as regularizations for learning pose-specific identity features. In contrast to the aforementioned studies that require elaborated data augmentation or multi-task training, our approach is light-weight and easy to implement.

# Deep Residual Equivariant Mapping

We first present a background on feature equivariance , and subsequently use it to motivate the proposed DREAM block. We will then present the block's design and different ways we could employ it to achieve pose-robust face recognition.

## Feature Equivariance

The notion of feature equivariance is presented in . It looks at how a representation changes upon transformations of the input image. An important finding of this paper is that most of the layers in deep neural networks change in an easily predictable manner with the input. And such transformations can be learned empirically from data.

Formally, a convolutional neural network (CNN) can be regarded as a function $\phi$ that maps an image $\mathbf{x} \in \mathcal{X}$ to a vector $\phi(\mathbf{x})\in \mathbb{R}^d$. The representation $\phi$ is said equivariant with a transformation $g$ of the input image if the transformation can be transferred to the representation output . That is, equivariance with $g$ is obtained when there exists a map $M_g : \mathbb{R}^d \rightarrow \mathbb{R}^d$ such that $$\label{eqn:equivariance}
 \forall \mathbf{x} \in \mathcal{X}: \;\;\; \phi(g\mathbf{x}) \approx M_g\phi(\mathbf{x}).$$ Interestingly, by requiring the same mapping $M_g$ to work for any input image, the function would capture intrinsic geometric properties of the representations. In , the authors focus on geometric transformation such as affine warping and flips of images. In our problem context, the transformation $g$ is more challenging as it involves 3D geometric changes from profile to frontal faces.

## Problem Formulation and the DREAM Block

Face recognition relies on robust representation extracted from deep models. For instance, DeepID series  extract features from the penultimate layer (the fully-connected layer before the output layer) as a feature vector and feed it into a classifier. Our goal in this study is to design a light-weight solution to make the representation robust to pose variation.

We begin by introducing the problem formulation. We denote a CNN as a function $\phi$ and the image representation it maps from image $\mathbf{x}$ as $\phi(\mathbf{x})$. We call the network a stem CNN or base network. Let's assume that we are given two types of face images, namely frontal face image, represented as $\mathbf{x}_f$ and profile face image, denoted by $\mathbf{x}_p$. Note that we assume this setting to facilitate our discussion; the proposed method can work with faces of an arbitrary pose.

Motivated by Eq. , we wish to obtain a transformed representation of a profile image $\mathbf{x}_p$ through a mapping function $M_g$, so that $M_g\phi(\mathbf{x}_p) \approx \phi(\mathbf{x}_f)$. To facilitate the incorporation of $M_g\phi(\mathbf{x}_p)$ to a stem CNN, we formulate it as a sum of the original profile feature $\phi(\mathbf{x}_p)$ with residuals given by a residual function $\mathcal{R}(\phi(\mathbf{x}_p))$ weighted by a yaw coefficient $\mathcal{Y}(\mathbf{x}_p)$. That is $$\begin{aligned}
\label{eqn:residual_mapping}
\phi(g\mathbf{x}_p) & = & M_g\phi(\mathbf{x}_p) \\ \nonumber
                    & = & \phi(\mathbf{x}_p) + \mathcal{Y}(\mathbf{x}_p)\mathcal{R}(\phi(\mathbf{x}_p)) \\ \nonumber
                    & \approx & \phi(\mathbf{x}_f).
\end{aligned}$$ By performing this transformation, we wish that the fixed representation $\phi(\mathbf{x}_p) + \mathcal{Y}(\mathbf{x}_p)\mathcal{R}(\phi(\mathbf{x}_p))$ will be mapped to the frontal face space as illustrated in Figure .

Equation  is capable of coping with input images of arbitrary pose, thanks to the yaw coefficient $\mathcal{Y}(\mathbf{x}) \in [0,1]$, which acts as a *soft gate* of the residuals. The role of this soft gate is to provide a higher magnitude of residuals (thus a heavier fix) to a face that deviates more from the frontal pose. Intuitively, $\mathcal{Y}(\mathbf{x})=0$ if the face image is frontal, and its value gradually changes from $0$ to $1$ when the face pose shifts from frontal to a complete profile. The residual's magnitude is thus the largest at the complete profile pose. The soft gate is essential. Without it residuals $\mathcal{R}(\phi(\mathbf{x}))$ will be added blindly to input images of any poses, affecting the final face recognition performance.

The soft gated residual block presented in this study is related to the residual structure that is used to achieve identity mapping in very deep networks. While utilizes the residual block to increase the effective depth of networks. Our attempt of combining a soft control gate can be viewed as a correction mechanism that adopts top-down information (the yaw in our case) to influence the feed-forward process. The role of the proposed control gate is to determine the amount of residuals to be passed to the next level. Besides, it is worth noticing that roll and pitch angles are not considered. The effect of roll will be eliminated by face alignment while face images with large pitch angles are rare and it is possible to address pitch angles by adding another branch in our DREAM block.

**Architecture and Training**. The residual formulation in Eq.  allows us to design a succinct network structure shown in Figure . Specifically, we use a stem CNN, , ResNet-18 or ResNet-50, to extract features from the input face image. To adapt ResNet  for our recognition task, we add a 256-dimensional fully connected layer between the average pooling layer and the original fully connected layer. We call this newly added layer as feature layer. The stem CNN can also be of any of the existing face recognition models . A fully connected layer is then used to extract the initial representation, $\phi(\mathbf{x})$, which is subsequently 'fixed' by the DREAM block. The DREAM block, in our current implementation, consists of two branches:

Residual Branch. The first branch generates the residuals $\mathcal{R}(\phi(\mathbf{x}))$. It has two fully-connected layers with Parametric Rectified Linear Unit (PReLU)  as the activation function. This branch is learnable separately from the stem CNN. Specifically, we train it by minimizing the Euclidean distance between the mapped profile feature and its corresponding frontal feature using stochastic gradient descent. $$\min_{\Theta_R} \mathrm{E} \| \phi(\mathbf{x}) + \mathcal{Y}(\mathbf{x})\mathcal{R}(\phi(\mathbf{x})); \Theta_\mathcal{R} ) - \phi(\mathbf{x}_f ) \|^2_2,$$ where $\Theta_\mathcal{R}$ denotes the parameters of $\mathcal{R}(\cdot)$. We keep the parameters fixed for the $\mathcal{Y}(\cdot)$ branch. During the training process we use dropout strategy  to the last fully connected layer in the branch. In this work, we train this branch on frontal-profile pairs sampled from MS-Celeb-1M dataset.

Soft Gate with Head Rotation Estimator. The second branch produces the soft yaw coefficient $\mathcal{Y}(\mathbf{x})$. This branch assumes an input of 21 facial landmarks following the standard AFLW's protocol . Note that this requirement does not add additional burden to the stem CNN since the face alignment process is a standard preprocessing step of many face recognition pipelines[^3].

Given facial landmarks, the head rotation estimator in the second branch (see Figure ) estimates the head rotation by using the algorithm presented in . Specifically, we adopt the same definition of the face model and head coordinate system as . Slightly different from , we extend their 6-point 3D face model into a 21-point model to achieve a better performance. We then fit the model by estimating the initial solution using the EPnP algorithm , and further refining the pose via non-linear optimization.

The yaw angle obtained from previous steps is then non-linearly mapped to a positive value within the range of $[0,1]$. Specifically, we obtain the training target of yaw coefficient by $\sigma( \frac{4}{\pi}y - 1 )$, where $y$ is the yaw angle of a face image in radian units. The $\sigma$ here is a sigmoid function. Through this mapping the coefficient quickly reaches the value 1 once a face turns more than 45$^\circ$, exerting more residuals for extreme profile faces. Empirically, we found that by adding a monotonous nonlinear mapping to the coefficient improves the effectiveness of residuals to the representation.

The outputs of these two branches are multiplied and added to the initial representation $\phi(\mathbf{x})$. The resulting feature $\phi(\mathbf{x}) + \mathcal{Y}(\mathbf{x})\mathcal{R}(\phi(\mathbf{x}))$ is the final feature output. It is worth noting that pose estimation accuracy has little effect on the final performance. We use the state-of-the-art face alignment method proposed in to obtain the 21 facial landmarks. The method is specially designed to handle large-pose alignment. We have tried adding 20% noise to the estimated yaws in both training and test sets and found that EER in face recognition increases by no more than 2% under multiple settings. This result is still a lot more better than baselines.

## The Usages of DREAM

We describe three ways of utilizing DREAM. Comparative results on all strategies will be provided in the experiment section.

**Stitching**. The most convenient way of deploying the DREAM block is by 'stitching' the block directly to a trained stem CNN. In particular, given a base network, we can just stitch the DREAM block onto the final feature layer of the base network without changing any learned parameters of the original model.

**End-to-end**. The proposed light-weight block can also be trained together with the stem CNN in an end-to-end manner. Given a plain base network, we insert the DREAM block and directly train the new network with random initialization on all of its parameters. If the stem CNN is not plain but trained previously, we can first fine-tune the stem CNN while training the DREAM block end-to-end using an existing face recognition loss (, verification loss, identification loss, or both). We name the strategy as 'end2end'. With this strategy, the performance on profile faces is not guaranteed since the DREAM block may not be able to distinguish frontal and profile cases, since no specific frontal-profile face pairs are used for training the block.

**End-to-end$+$retrain**. We train the stem CNN and DREAM block together then train the DREAM block separately with frontal-profile face pairs.

As a way to demonstrate the effectiveness of DREAM block, we use the GAN model in that could map deep features back to reconstructed images. This GAN model is used to visualize the original and mapped features generated by our model. Some representative results are shown in Fig. . Note that the reconstruction only serves for visualization purposes. The usefulness of DREAM can only be fully validated through examining its performance on the face recognition task. We show strong performance of DREAM in the following section.

# Experiments

We divide our experiments into a few sections. In the first section, we show the effectiveness of our method on the task of frontal to complete profile face verification. In the second section, we compare with state-of-the-art methods on a dataset with full-pose variation. Lastly, we provide a further analysis of the influence of face yaw and present an ablation study.

```latex
\begin{table*}[t]\footnotesize
\begin{center}
\caption{Results on Celebrities in Frontal-Profile (CFP) with Frontal-Profile setting. Equal error rate (EER) is reported. Lower is better. The best result in each row are given in bold.}
\vskip -0.1cm
\resizebox{2.1\columnwidth}{!}{%
\begin{tabular}{ll|c|ccc|ccc}
\hline
 &  &  & \multicolumn{3}{|c|}{Other Strategies} & \multicolumn{3}{|c}{DREAM Variants} \\
Model & Training Data & Na\"{i}ve & CDFE \cite{lin2006inter} & JB \cite{chen2012bayesian} & FF~\cite{tran2017disentangled} & stitching & end2end & end2end+retrain \\
  \hline \hline
%  ResNet-18 & CelebA & 14.60  & 14.26 & 14.26 & 14.04 & 13.91 & 14.09 & \textbf{12.77}  \\
  ResNet-18 & MS-Celeb-1M & 8.40  & 8.30 & 8.37 & 14.40 & 7.71 & 7.63 & \textbf{7.03} \\
  ResNet-50 & MS-Celeb-1M & 7.89 &  7.71   &  8.49    & 14.26   &  7.29    &  6.43    & \textbf{6.02}     \\
  %FaceNet & CASIA-WebFace & 13.06  & 12.73 & 12.60 & 16.69 & \textbf{12.11} & - & -  \\
  Center-Loss & MS-Celeb-1M & 8.54  & 8.49 & 8.29 & 14.53 & 7.82 & 7.81 & \textbf{7.26}  \\ \hline
\end{tabular}}
\label{tab:mainExperiment}
\end{center}
\vspace{-0.25cm}
\vskip -0.4cm
\end{table*}
```

We provide a description of the training set we use before moving to the next section. To train our own stem CNNs (ResNet-18 and ResNet-50) and DREAM block, we employ a subset of MS-Celeb-1M . The original data provides images of 100,000 top celebrities in the form of URL, which, however, are collected from Google without manual cleaning. To facilitate the learning of our model, we clean the data and select a subset to form our training and test partitions. The training partition consists of 696,446 images from 13,385 identities. The testing partition contains 70,730 images from 3,084 celebrities. The training and test partitions have exclusive identities. The test partition is only used in our further analysis and ablation study since it contains more profile faces. All face images in training and testing sets are prepocessed to a size of $224\times224$. We use the method presented in and for face detection and face alignment, respectively. To train our stem CNN, we use an identification loss . As in , face verification is performed by measuring the cosine distance between feature representation of queries.

## Evaluation on CFP with Frontal-Profile Setting

**Test dataset**. We first conduct evaluations on the Celebrities in Frontal-Profile (CFP) dataset , a challenging dataset created to examine the problem of frontal to profile face verification 'in the wild'. The dataset contains 500 celebrities, each of which has ten frontal and four profile face images. We follow the standard 10-fold protocol  in our evaluation. In particular, the whole dataset is divided into 10 folds each containing 350 same and 350 not-same pairs generated from 50 individuals (7 same and 7 not-same pairs for each individual). The same protocol is applied on both the Frontal-Profile and Frontal-Frontal settings.

**Stem CNN**. To show the benefits of DREAM on different base networks, we perform our experiments using the following stem CNN architectures. Following , we employ a ResNet-18 as our base network. We also try ResNet-50. In addition, we experiment with a state-of-the-art network presented in [^4], which stacks 11 residual blocks and trains with Center-Loss.

**Baselines**. We compare our method with three representative baselines that also help to alleviate the gap in frontal-profile face verification:

-   CDFE  - Common Discriminant Feature Extraction is a representative method that is specially tailored to the inter-modality problem. In the algorithm, two transforms are simultaneously learned to map the samples in two modalities respectively to the common feature space.

-   JB  - Joint Bayesian is a widely used metric learning approach in face verification, which also supports the transfer from one domain to another. We train a JB model on the CFP dataset. Each time, we use 9 splits for training and the rest for testing. We repeat this process for 10 times and report the average error. The DREAM block does not need such fine-tuning.

-   FF  - Face Frontalization morphs faces from profile to frontal with a generative adversarial network. This baseline converts all profile face images in the CFP test partition to frontal ones.

In addition, we also evaluate the effectiveness of different training strategies of DREAM as discussed in Section , namely, 'stitching', 'end2end', and 'end2end$+$retrain'.

**Results**. The results are summarized in Table . We report the Equal Error Rate (EER). We have following observations:

) Through the comparisons between Naïve and DREAM variants, we found that all DREAM strategies are capable of reducing the EER of various strong models, in spite of their different designs and training sources. The strategy 'end2end$+$retrain' gives the best performance overall. Through visualizing the deep feature space of 'Naïve' ResNet-18 via t-SNE in Figure , we could observe that features of frontal faces are clearly separated based on identities, but the profile ones are mostly messed up exhibiting severe feature overlapping in the space. In contrast, ResNet-18 with the proposed DREAM block clearly separates the features of different subjects without affected by the pose factor.

) Baselines such as CDFE, JB and FF utilize the same supervision as our approach but they are less effective. The fact that FF performs poorer than DREAM in this case suggests that performing 'frontalization' in the deep feature space rather than the image space turns out to be more fruitful. We conjecture that the artifacts of 'frontalized' faces lead to the inferior results of FF to Naïve baseline. Some frontal images synthesized by FF are given in the supplementary material.

) Our best result, which is obtained by ResNet-18 and ResNet-50 trained with MS-Celeb-1M and 'end2end+retrain' strategy, outperforms many existing methods, including , , and that achieves 8.00, 8.85, and 14.97, respectively.

It is worth pointing out that the impact introduced by the proposed block on the frontal face recognition is minimal. In particular, the block hardly affects the performance of non-profile face recognition since the soft yaw coefficient prevents the block from altering the frontal representation. In particular, for ResNet-18, the EER of 'Naïve' on Frontal-Frontal setting is 2.66, while that of 'stitching', 'end2end'. and 'end2end+retrain' are 2.69, 2.00 and 2.00, respectively.

## Evaluation on IJB-A with Full Pose Variation

```latex
\begin{table*}[t]\footnotesize
\begin{center}
\caption{Comparative performance analysis on IJB-A benchmark. Results reported are the `average$\pm$standard deviation' over the 10 folds specified in the IJB-A protocol. Symbol `-' indicates that the metric is not available for that protocol. Standard deviation is not available for all the methods. f.t. denotes fine tuning a deep network multiple times for each training split.}
\vskip -0.2cm
\resizebox{2.1\columnwidth}{!}{
\begin{tabular}{l|c|c|c|c}
 \hline
 \multirow{1}{*}{Methods $\downarrow$} & \multicolumn{2}{c|}{Verification} & \multicolumn{2}{c}{\multirow{1}{*}{Identification}} \\
 \hline
 \multirow{1}{*}{Metrics $\rightarrow$} & TAR @ FAR=0.01 & TAR @ FAR=0.001 & Rec. Rate @ Rank-1 & Rec. Rate @ Rank-5 \\
 \hline \hline
  \textbf{Our Approach with MS-Celeb-1M subset:} &  & & &  \\
  ResNet-18 (na\"{i}ve) &  0.840$\pm$0.026 & 0.656$\pm$0.040 & 0.897$\pm$0.016 & 0.951$\pm$0.011 \\
  ResNet-18 (end2end$+$retrain) &    0.872$\pm$0.018 & 0.712$\pm$0.035 & 0.915$\pm$0.012  & 0.962$\pm$0.008 \\
  ResNet-50 (na\"{i}ve) &  0.881$\pm$0.018 & 0.714$\pm$0.034 & 0.913$\pm$0.013 & 0.957$\pm$0.010 \\
  ResNet-50 (end2end$+$retrain) &    0.891$\pm$0.016 & 0.764$\pm$0.031 & 0.924$\pm$0.016  & 0.962$\pm$0.010 \\
  \hline
  \textbf{Our Approach with full MS-Celeb-1M:} &  & & &  \\
  ResNet-18 (na\"{i}ve)& 0.934$\pm$0.009 & 0.836$\pm$0.016 & 0.939$\pm$0.012 & 0.960$\pm$0.010 \\
  ResNet-18 (end2end$+$retrain)& \textbf{0.944$\pm$0.009} & 0.868$\pm$0.015 & \textbf{0.946$\pm$0.011}  & \textbf{0.968$\pm$0.010} \\

\hline
  \textbf{Existing Methods:} &  & & &  \\
  Wang \etal \cite{wang2017face} & 0.729$\pm$0.035 & 0.510$\pm$0.061& 0.822$\pm$0.023 & 0.931$\pm$0.014 \\
  Pooling Faces \cite{hassner2016pooling} & 0.819$\pm$ ------ & 0.631$\pm$ ------ & 0.846$\pm$ ------ & 0.933$\pm$ ------ \\
  Deep Multi-Pose \cite{abdalmageed2016face} & 0.787$\pm$ ------ & -- & 0.846$\pm$ ------ & 0.927$\pm$ ------ \\
  Multi-task CNN ~\cite{yin2017multi} & 0.787$\pm$0.043  & -- & 0.858$\pm$0.014 & 0.938$\pm$0.009 \\
  PAMs \cite{masi2016pose} & 0.826$\pm$0.018 & 0.652$\pm$0.037 & 0.840$\pm$0.012 & 0.925$\pm$0.008 \\
  DCNN$_\mathrm{fusion}$ (f.t.) \cite{chen2016unconstrained} &  0.838$\pm$0.042 & -- & 0.903$\pm$0.012  & 0.965$\pm$0.008 \\
  Augmentation$+$Video Pooling$+$Rendered Test \cite{masi2016we} & 0.886$\pm$ 0.017 & 0.725$\pm$ 0.044 & 0.906$\pm$ 0.013 & 0.962$\pm$ 0.007 \\
  CNN$_\mathrm{media}$+TPE (f.t.)~\cite{sankaranarayanan2017triplet} & 0.900$\pm$0.010 & 0.813$\pm$0.020 & 0.932$\pm$0.010 & -- \\
  Template Adaptation (f.t.)~\cite{crosswhite2017template} & 0.939$\pm$0.013 & -- & 0.928$\pm$0.010 & -- \\
  Quality Aware Network (f.t.)~\cite{liu2017quality} & 0.942$\pm$0.015 & \textbf{0.893$\pm$0.039} & -- & -- \\
\hline
 \end{tabular}
 }
 \label{tab:IJBA}
\end{center}
\vskip -0.4cm
\end{table*}
```

**Test dataset**. In previous experiments on CFP we focus on frontal-profile face verification. In this experiment, we further evaluate our method on another challenging benchmark called IARPA Janus Benchmark A (IJB-A)  that covers full pose variation (yaw angles between $-90^\circ$ to $+90^\circ$). The dataset contains 500 subjects with of 5,712 images and 20,414 frames extracted from videos. The faces in the IJB-A dataset contain extreme poses and illuminations, more challenging than LFW . Following the standard protocol in , we evaluate our method on both verification (1:1) and identification tasks (1:N).

**Stem CNN**. We use ResNet-18 and ResNet-50 trained on MS-Celeb-1M as our stem CNN. The DREAM block is deployed using our 'end2end$+$retrain' strategy.

**Results**. Table reports our results on IJB-A. It is noteworthy that we employ a strong baseline: a ResNet-18 trained on the MS-Celeb-1M dataset, which achieved 65.6% True Acceptance Rate (TAR) at False Acceptance Rate (FAR) of 0.001 on the verification task and a Rank-1 recognition accuracy of 89.7% on the identification task, comparable to a state-of-the-art method . By adding the DREAM block, the performance of this strong baseline is improved by **8.5**% on the verification task (TAR@FAR=0.001) and the error rate is reduced by **17.5**% on the identification task (Rank-1 accuracy). For the even stronger ResNet-50, the DREAM block also offers a compelling error reduction of 7.0% and 12.6%, respectively.

The aforementioned results are generated by using a sampled subset of MS-Celeb-1M dataset for training. Once we use the full MS-Celeb-1M dataset, we achieve the state-of-the-art results on the identification task and comparable verification performance against Quality Aware Network  that uses a more elaborated loss (triplet$+$softmax) for training.

There are other strategies that can be used to further improve the performance but we did not attempt each of them. We included these methods in Table  for completeness. Most of the techniques perform excellently when they fine-tune their model multiple times for each training split. Such methods are denoted with 'f.t.' in Table . Our model is trained on MS-Celeb-1M and directly tested on IJB-A without fine-tuning. Data augmentation technique can be useful too. Masi   perform task-specific data augmentation on pose, shape, and expression on CASIA-WebFace. Their method also performs pose synthesis at test time. We believe the same technique can be used to improve the performance of our approach. Sankaranarayanan   deploy a metric learning approach to fine-tune the network on the training splits of IJB-A. Liu   train a network with joint triplet and softmax losses. Our method is only trained using the typical identification loss and without fine-tuning.

## A Further Analysis on Influences of Face Yaw

We conduct a more in-depth analysis on the influence of face yaw to the performance of face verification to better understand the effectiveness of DREAM block on profile face recognition. In the analysis, we assume a query face and a matching face with yaw angle $(y_1,y_2)$, respectively. We then select a threshold of equal error rate. Given image pairs of different $(y_1,y_2)$ and the threshold, we quantify their false positive rate and false negative rate and plot these values as heatmaps in Figure . We compare the performance of two approaches, namely the Naïve ResNet-18 as a baseline, and a ResNet-18 equipped with the proposed DREAM block. It can be observed from Figure (a) that false positives mainly concentrate at the bottom right of the heatmap, , when both query and matching faces are of extreme yaw angles. In comparison to the baseline, our approach yields much fewer false positives. From Figure (b), we observe that a majority of false negatives take place when one of the faces is frontal and the other one is profile. Again, our approach performs superior over the baseline in terms of false negative rates.

## Examining the Architecture of DREAM Block

In this subsection we study the effectiveness of different architectures of the DREAM block. The architecture of DREAM block could differ in two aspects: 1) the location to apply the block, and 2) the design. For the first part, we compare different locations of inserting a DREAM block. 'After pooling layer' means we insert the DREAM block directly after the average pooling layer of ResNet. Recall that we introduced a new fully connected layer between the average pooling layer and the original fully connected layer of ResNet (see Sec. ). 'After feature layer' refers to inserting the DREAM block after this feature layer. For the second part, we try to build the DREAM block with only one fully connected layer or replace one fully connected layer with a 1D convolution layer. We found that the best location to insert DREAM block is at the top of the network, where the feature is deep and feature dimension is compact. We believe the two aspects will benefit the learning of DREAM block. From Table we observe that non-linearity plays a role in DREAM block (the setting that uses 2 fc layers outperforms linear mapping).

```latex
\begin{adjustbox}{width=1\linewidth}
\begin{tabular}{c|c|c}
   \hline
  Position of DREAM &  Components of DREAM & EER on MS-Celeb-1M \\
  \hline\hline
  After pooling layer &  two fc & 9.22   \\
  After feature layer &    one fc     &   8.92      \\
  After feature layer & one fc, one conv  & 9.18      \\
  After feature layer & two fc      & \textbf{8.45}  \\
  \hline
 \end{tabular}
 \end{adjustbox}
```

## An Ablation Study on the Soft Gate

The soft gate produces a yaw coefficient to control the amount of residuals to be added to the initial representation. As described in the methodology section, the coefficient can be linear or nonlinear w.r.t. the degree of face yaw. In this experiment, we examine the effectiveness of different settings of the soft gate. In addition, we also report a baseline in which the soft gate is consistently closed, , $\mathcal{Y}(\mathbf{x})=1$. It is noteworthy that this baseline still keeps the same number of parameters but it degenerates to a conventional residual branch that losses the capability of distinguishing frontal and profile faces.

|                  Gate Setting                   | With DREAM Block |
|:-----------------------------------------------:|:----------------:|
| Consistently Close, $\mathcal{Y}(\mathbf{x})=1$ |       9.31       |
|                     Linear                      |       8.82       |
|                    Nonlinear                    |     **8.45**     |

Comparative analysis of different gate settings. Evaluation is conducted on MS-Celeb-1M with ResNet-18. Results are reported as EER error.

From Table , we observe that the performance of face verification significantly drops if the soft gate is closed consistently. The results suggest the improved performance attained by the DREAM block insertion is not merely due to the additional parameters, but the effective mechanism inherently brought by the residual branch and soft gate. From Table , we also observe that nonlinear mapping yields superior performance over the linear setting. This observation ascertains our design of exerting a higher degree of correction to a face pose larger than 45$^\circ$, as this pose range is harder to be handled by the stem CNN (as observed from Figure ).

# Conclusion

We have presented a Deep Residual EquivAriant Mapping (DREAM) block to improve the performance of face recognition on profile faces. Our method is novel in that we take a radically different approach to handle profile faces. Specifically, we bridge the discrepancy between profile and frontal faces through performing equivariant mapping in the deep feature space. The mapping is achieved through the light-weight DREAM block that is easy to implement. Extensive results on CFP, IJB-A, and MS-Celeb-1M datasets demonstrate the applicability of the block on different types of stem CNNs, including ResNet-18, ResNet-50, and Center-Loss models. Interestingly, we observed that performing frontalization in the feature space is more fruitful than the image space for the task of face verification. It is noteworthy that the proposed block is not limited to face recognition with pose variation, it is suitable for other problems,  cross-age face recognition, of which performance also suffers from uneven distribution of training data.

**Acknowledgement:** This work is supported by SenseTime Group Limited and the General Research Fund sponsored by the Research Grants Council of the Hong Kong SAR (CUHK 14241716, 14224316. 14209217).

[^1]: Codes and models are available at <http://mmlab.ie.cuhk.edu.hk/projects/DREAM/>

[^2]: indicates shared first authorship

[^3]: In many face recognition pipelines, face alignment is used to center the face, rotate the face such that the eyes lie along a horizontal line, and scale the faces such that they are approximately identical in size.

[^4]: We use the codes released by the authors at <https://github.com/ydwen/caffe-face>.


abstract: In this paper, we propose a new deep feature selection method based on deep architecture. Our method uses stacked auto-encoders for feature representation in higher-level abstraction. We developed and applied a novel feature learning approach to a specific precision medicine problem, which focuses on assessing and prioritizing risk factors for hypertension (HTN) in a vulnerable demographic subgroup (African-American). Our approach is to use deep learning to identify significant risk factors affecting left ventricular mass indexed to body surface area (LVMI) as an indicator of heart damage risk. The results show that our feature learning and representation approach leads to better results in comparison with others.
  .
  .
  ***Keywords*—Precision Medicine; Deep Learning; Feature Learning and Representation; Stacked Auto-Encoders; Left Ventricular Mass (LVM); Hypertension (HTN).**
author:
;
title: SAFS: A Deep Feature Selection Approach for Precision Medicine

# Introduction

The explosive increase of Electronic Medical Records (EMR) provides many opportunities to carry out data science research by applying data mining and machine learning tools and techniques. EMR contains massive and a wide range of information on patients concerning different aspects of healthcare, such as patient conditions, diagnostic tests, labs, imaging exams, genomics, proteomics, treatments, outcomes or claims, financial records . Particularly, the extensive and powerful patient-centered data enables data scientists and medical researchers to conduct their research in the field of personalized (precision) medicine. Personalized medicine is defined as : "the use of combined knowledge (genetic or otherwise) about a person to predict disease susceptibility, disease prognosis, or treatment response and thereby improve that person's health." In other words, the goal of precision medicine or personalized healthcare is to provide "the right treatment to the right patient at the right time". Personalized medicine is a multi-disciplinary area that combines data science tools and statistics techniques with medical knowledge to develop tailor-made treatment, prevention and intervention plans for individual patients.

In this study we focus on a vulnerable demographic subgroup (African-American) at high-risk for hypertension (HTN), poor blood pressure control and consequently, adverse pressure-related cardiovascular complications. We use left ventricular mass indexed to body surface area (LVMI) as an indicator of heart damage risk. The ability to reduce LVMI would lead to an improvement in hypertension-related cardiovascular outcomes, which, in turn, would further diminish the excess risk of cardiovascular disease complications in African-Americans.

Based on individual clinical data with many features, our objective is to identify and prioritize personalized features to control and predict the amount of LVMI toward decreasing the risk of heart disease. First, we focus on selecting significant features among many features such as demographic characteristics, previous medical history, patient medical condition, laboratory test result, and Cardiovascular Magnetic Resonance (CMR) results that are used to determine LVMI and its reduction with treatment over time. Second, after features selection, we apply supervised machine learning methods to predict the amount of LVMI as a regression model. Third, this prediction method can be implemented as a decision support system (DSS) to assist medical doctors in controlling the amount of LVMI based on significant risk factors.

Figure illustrates our approach in three consecutive steps; this approach is an integrated feature selection model that applies unsupervised learning for producing higher-level abstraction of input data and then using them as an input of a feature selection method for ranking important features. In the final step, supervised leaning method is implemented for forecasting LVMI and model evaluation. Based on the model evaluation results, steps $2$ and $3$ are applied iteratively to finalize the most important features (risk factors).

In this paper, we develop a new feature selection and representation method using a deep learning algorithm for personalized medicine. In our method, we apply stacked auto-encoders as a deep architecture for feature abstraction at higher levels. To our knowledge, it is one of the first methods that use stacked auto-encoders for feature learning and selection, the LVMI prediction approach developed in this research is also a new model of risk forecasting for cardiovascular and other health conditions.

The rest of this paper is organized as follows. Section II reviews the related works in deep feature selection methods. Section III explains deep learning overview and stacked auto-encoders. Section IV describes the developed feature selection method and prediction approach. Section V reports the personalized data and implementation results and finally section VI discusses about the results and conclusion.

# Related Works

Deep learning, including feature representation and predictive modeling, has been researched and applied in a number of areas, such as computer vision, remote sensing, natural language processing and bioinformatics. The main reasons accounting for the extensive applications are improved prediction accuracy, capability of modeling processes of complex systems, and generating high-level representation of features and higher robustness modeling \[12\]. In this literature review, we focus on papers that develop feature selection models using deep learning algorithms. Among many related applications, few researchers have carried out feature selection using deep learning.

Zhou et al. used deep feature selection methods in natural language processing and sentiment classification. They proposed a novel semi-supervised learning algorithm called active deep network (ADN) by using restricted Boltzmann machines (RBMs). They applied unsupervised learning based on labeled reviews and abundant unlabeled reviews and then used active learning to identify and select reviews that should be labeled as training data. Also in , a semi-supervised learning algorithm based on deep belief network, called DBNFS, was proposed for feature selection. Authors applied the method for solving a sentiment classification problem to reduce high dimension of noisy vocabularies. In their method, several hidden layers have been replaced in DBNs, which are computationally expensive with the filter-based feature selection technique using chi-squared tests.

Deep feature selection was also applied in a Remote Sensing Scene classification/recognition problem. Zou et al. developed a new feature selection method by using deep learning that formulates the feature selection problem as a feature reconstruction problem. Their method is implemented as an iterative algorithm, which is based on DBN in an unsupervised learning way to train the inquired reconstruction weights. Finally features with small reconstruction errors were selected as input features for classification.

In biomedical and biomedicine research, deep learning algorithms and tools have been applied in different areas especially in precision medicine and personalized healthcare by using biological and genomics data .

Li et al. developed a deep feature selection (DFS) model for selecting input features in a deep neural network for multi-class data. They used elastic net to add a sparse one-to-one linear layer between the input layer and the first hidden layer of a multi-layer perception and select most important features according to their weights in the input layer given after training. They then applied their DFS model to solve the problem of enhancer-promoter interaction using genomics data. The method that they developed using elastic net is a new shrinkage approach that is flexible to be applied in different deep learning architectures. In terms of performance, their method did not outperform random forest in the experimental implementation that would be considered as a major weakness.

Another study in the biomedical area used DBN and unsupervised active learning to propose a new multi-level feature selection method for selecting genes/miRNAs based on expression profiles. The strength of the proposed approach is better performance in comparison with several feature selection methods. However, for high dimensional features it does not outperform some other methods such as relief algorithm.

In authors showed how unsupervised feature learning approach can be applied for cancer type classification from gene expression data. They proposed a two-phase model for feature engineering and then classification. In the first phase, Principal Component Analysis (PCA) technique was used for dimension reduction of the feature space and in the second phase, a deep sparse encoding was implemented on new data to obtain high-level abstractions for cancer diagnosis and classification. While their approach used stacked auto-encoders for representation learning, they focused on using deep feature extraction to achieve a better accuracy in cancer type prediction instead of finding significant factors in cancer diagnosis. Also in similar research focused on early detection of lung cancer, authors used deep learning for features extraction and applied a binary decision tree as a classifier to create a computer aided diagnosis (CAD) system that can automatically recognize and analyze lung nodules in CT images. They just implemented an auto-encoder for data representation and probably using stacked auto-encoders leads to higher accuracy.

In sum, feature selection based on deep learning techniques were used in a wide range of areas as reviewed above, which is evident by an increasing amount of related studies that have been developed in recent years (after 2013). Particularly, many studies in biomedical applications used or developed deep learning methods for feature selection or feature extraction to reduce the dimensionality of data with many features that increase the performance of prediction and classification tasks.

Precision medicine has emerged as a data-rich area with millions of patients being prospectively enrolled in the Precision Medicine Initiative (PMI). The data is big and noisy thus demanding for more sophisticated data science approaches. Therefore it can be concluded that there are ample opportunities to develop and apply deep learning for feature selection in the new data-rich areas, such as precision medicine.

From what we discussed, most related research works took the advantages of representation learning with deep architecture. In many of them, DBN was applied for data transforming and there lack comprehensive and systematic comparisons of performance among different deep architecture. In our approach we will use stacked auto-encoders with carefully selected architectures for feature selection that can overcome this problem.

# DEEP LEARNING AND STACKED AUTO-ENCODERS

## Introduction to deep learning and its applications

Deep Learning or Deep Machine Learning is a kind of machine learning algorithms that model input data to higher-level abstraction by using a deep architecture with many hidden layers composed of linear and non-linear transformations .

In a simple definition, deep learning means using a neural network with several layers of nodes between input and output and deep architectures that are composed of multiple levels of non-linear operations, such as neural nets with many hidden layers. There are three different types of deep architectures: 1) Feed Forward architectures (multilayer neural nets, convolutional nets), 2) Feed Back architectures (De-convolutional Nets) and 3) Bi-Directional architectures (Deep Boltzmann Machines, Stacked Auto-Encoders) .

Choice of data representation or feature representation plays a significant role in success of machine learning algorithms . For this reason, many efforts in developing machine learning algorithms focus on designing preprocessing mechanisms and data transformations for representation learning that would enable more efficient machine learning algorithms .

Deep learning applications encompass many domains. The major ones are Speech Recognition, Visual Object Recognition, Object Detection (Face Detection) and Bioinformatics or Biomedicine applications such as drug discovery and genomics . In biomedical and health science, increases in technological development, information systems and research equipment have generated a large amount of data with high features and dimensions. Since deep learning outperformed some other methods such as PCA or singular value decomposition in handling high-dimensional biomedical data, it has strong potential for dimension reduction and feature representation in biomedical and clinical research .

Deep Learning algorithms and tools have not been applied in biomedical and health science areas extensively. Deep supervised, unsupervised, and reinforcement learning methods can be applied in many complex biological and health personalized data and could play an important role in discovering and development of personalized healthcare and medicine .

## Introduction to stacked auto-encoders

Among different kinds of deep architectures mentioned in previous section, four architectures are more popular in biomedical data analysis \[6\]. 1) The Convolutional Neural Network (CNN), which usually consists of one or more convolutional layers with sub-sampling layers and followed by fully connected layers as in a standard deep neural network. 2) Restricted Boltzmann Machine (RBM), which is comprised of one visible layer and one layer of hidden units. 3) Deep Belief Network (DBN), which is a generative graphical model consisted of multiple layers of hidden variables with connections just between layers (not units within layers). 4) Stacked auto-encoders, which is a neural network including multiple layers of sparse auto-encoders \[5\]\[8\].

Training deep neural networks with multiple hidden layers is known to be challenging. Standard approach for learning neural network applies gradient-based optimization with back-propagation by initializing random weights in network concludes poor training results empirically when there are three or more hidden layers in deep network \[4\]. Hinton et al in \[18\] developed a greedy layer-wise unsupervised learning algorithm for training DBN parameters by using a RBM in the top of deep architecture. Bengio et al. \[10\] used greedy layer-wise unsupervised learning to train deep neural network when layer building block is an auto-encoder instead of the RBM. These two greedy layer-wise algorithms are the major algorithms to train a deep architecture based on DBN and Stacked auto-encoders. In this study we use stacked auto-encoders for data representation learning. Stacked auto-encoders shown in Figure is constructed by stacking multiple layers of auto-encoder.

An auto-encoder is trained to reconstruct its own inputs by encoding and decoding processes. Let us define $w^{(k,l)}$, $w^{(k,2)}$, $b^{(k,l)}$, $b^{(k,2)}$ as the parameters of $k^{th}$ auto-encoder for weights in encoding and decoding process respectively. Encoding step of each layer is a forward process and mathematically described as below:

$$\begin{aligned}
\label{eq1}
&a^{(k)}= f(z^{\left( k\right) }),&\\
&z^{(k+1)}= w^{\left( k,1\right) }a^{\left( k\right) }+b^{\left( k,1\right) }.&
\end{aligned}$$

In equation , $f(x)$ is an activation function for transforming data. If $n$ represents the location of middle layer in stacked auto-encoders, the decoding step is to apply the decoding stack of each auto-encoder as following :

$$\begin{aligned}
&a^{(n+k)}= f(z^{\left(n+k\right) }),&\\
&z^{(n+k+1)}= w^{\left(n+k,2\right) }a^{\left(n+k\right) }+b^{\left(n+k,2\right) }.&
\end{aligned}$$

The training algorithm for obtaining parameters of stacked auto-encoders is based on greedy layer-wise strategy . It means that each auto-encoder should be trained by encoding and decoding process one-by-one. By training this deep architecture, $a^{(n)}$ (the middle layer) gives the highest representation of the input . In the simplest case, when an auto-encoder with sigmoid activation function has only one hidden layer and takes input $x$, the output of encoding process will be: $$\begin{aligned}
&z= \mathrm{Sigmoid}_1(wx+b).&
\end{aligned}$$

Therefore $z$ is the vector of transformed input in the middle layer. In the second step (decoding process), $z$ is transformed into the reconstruction $x^{'}$, i.e., $$\begin{aligned}
&x^{'}= \mathrm{Sigmoid}_2(W^{'}z+b^{'}).&
\end{aligned}$$

Finally, auto-encoder is trained by minimizing the reconstruction errors as below: $$\begin{aligned}
&\mathrm{Loss}(x, x^{'})= \Vert x- x^{'}\Vert = &\notag\\
& \Vert x- \mathrm{Sigmoid}_2(W^{'} (\mathrm{Sigmoid}_1(Wx+b))+b^{'}) \Vert.&
\end{aligned}$$

Auto-Encoder with rich representation is an appropriate deep learning architecture to implement on biomedical data . In the next section we will use stacked auto-encoders for feature learning and selection.

# METHODOLOGY

The method developed in this study is an integrated feature selection method with stacked auto-encoders, which is called SAFS. The work flow of our approach is shown in Figure , and it includes three major components as follows:

## Features Partitioning

In the first step, we partition categorical features from continuous features when they co-exist in a data set. Here we focus on representation learning on continuous features since categorical features are often used as dummy variables in linear model based machine learning methods.

## Features Representation by Stacked Auto-Encoders

The second step is unsupervised learning section. Continuous features are represented in higher-level abstraction by deep stacked auto-encoders. As mentioned in Section III stacked auto-encoders is one of the most popular deep architectures for representation learning especially for biomedical data. The deep architecture of stacked auto-encoders considered with $5$ layers as shown in Figure .

In this deep architecture, $N$ is assumed as the number of continuous features and $n$ is a parameter. The middle hidden layer has $N$ nodes, same as input and output layers, and the other two hidden layers have $n$ nodes as a variant. The represented data is given from middle layer and $n$ is determined in an iterative process between unsupervised learning step and supervised learning step.

## Supervised Learning

In the third step, the represented continuous features are combined with categorical features and then supervised learning is applied on new dataset. It starts with feature selection, which can use any feature selection method such as random forests. Selected important features from a main dataset are entered in a supervised classifier for regression or classification and after training step. Model are then evaluated by some specific measures. If the results meet the criteria then selected features considered as significant variables, if not, the model tries different architecture for stacked auto-encoders by changing the number of nodes ($n$) in two hidden layers. This iterative process is continued until the model converges to some stop criteria or fixed number of iterations.

Our presented approach using deep learning for unsupervised non-linear sparse feature learning leads to a construction of higher level features for general feature selection methods and prediction tasks.

# IMPLEMENTATION FOR PERSONALIZED MEDICINE

In this section we implement our methodology and demonstrate its performance using a precision medicine case to find significant risk factors. We demonstrate that SAFS approach selects better features in comparison when we apply feature selection without using represented data.

The data used in this paper are derived from a subgroup of African-Americans with hypertension and poor blood pressure control who have high risk of cardiovascular disease. Data are obtained from patients who were enrolled in the emergency department of Detroit Receiving Hospital. Among many features (more than 700) including demographic characteristics, previous medical history, patient medical condition, laboratory test results, and CMR results to evaluate for LVMI, $172$ remained after a pre-processing step, $106$ of which are continuous variables and $66$ categorical variables for $91$ patients. As mention in Section I, the goal is to find the most important risk factors that affect LVMI.

In this precision medicine case study, we used package H2O in R and applied our approach for $150$ different deep stacked auto-encoders. At first we used random forest method for feature selection and supervised learning. We used Mean Squared Error (MSE) as a measure for regression evaluation. In each iteration, random forest runs for a different number of trees. We compared the average of MSE in different runs based on each deep architecture.

The result shows that different deep architectures, by changing the number of nodes in the hidden layers, demonstrate different performance. The representation learning produces different data; therefore it leads to different performance. According to Figure , the deep stacked auto-encoders with $n=42$ nodes in second and forth hidden layers yields least error (MSE= $63.93$) among all different architectures.

By selecting the deep architecture with $n=42$, we ran SAFS approach and then compared it with the results that were generated by random forest method with un-represented input data based on different number of trees. The performance comparison was shown in Figure . It is clear that our SAFS approach achieves a better accuracy (less error) for a different numbers of trees.

Since the LASSO based approaches are also popular and effective in feature selection from high dimension data, in the second attempt to examine the performance of our approach, we used LASSO for feature selection and prediction in SAFS method, which is implemented in R, by using package "glmnet". The result shows that the best architecture with least MSE is achieved for $n= 74$ nodes in second and forth hidden layers of stacked auto-encoders (MSE = $361.31$). Figure 7 describes the accuracy comparison of SAFS approach and LASSO based on different amount of Lambda ($\lambda$) parameter, where $\lambda$ is a tuning parameter to determine the number of features to be selected. It is obvious that SAFS approach has better performance for a wide range of $\lambda$ values, representing many scenarios of feature selections.

The results concluded from both implementations show that using SAFS approach and representation learning significantly outperform random forest and LASSO methods with unrepresented input data and also using random forest as a non-linear classifier in SAFS approach leads to better performance. The SAFS could be an appropriate choice for the precision medicine problem of identifying which patients at risk should be referred for more extensive testing and, which patients with increased LVMI are more or less likely to benefit from treatments.

According to our results obtained from SAFS approach using random forest, the top $15$ significant risk factors that affect LVMI are mentioned in Table I:

|  Row | Feature Description                          | Weight |
|:----:|:---------------------------------------------|:------:|
|  1   | Troponin Levels                              |  8.85  |
|  2   | Waist Circumference Levels                   |  8.07  |
|  3   | Plasma Aldosterone                           |  7.81  |
|  4   | Average Weight                               |  7.66  |
|  5   | Average Systolic Blood Pressure              |  5.32  |
|  6   | Serum eGFR Levels                            |  5.21  |
|  7   | Total Prescribed Therapeutic Intensity Score |  4.79  |
|  8   | Pulse Wave Velocity                          |  4.66  |
|  9   | Ejection Duration                            |  4.46  |
|  10  | Cornell Product                              |  4.06  |
|  11  | Average Percent Compliance Per Pill Count    |  2.99  |
|  12  | Plasma Renin Levels                          |  2.86  |
|  13  | Transformed Scale Physical Score             |  2.64  |
|  14  | Cholesterol Levels                           |  2.53  |
|  15  | Transformed Scale Mental Score               |  2.43  |

Significant Risk Factors Affecting LVMI

While some of these risk factors represent data specific to the study design, all are plausibly linked with heart risk, representing potential physiological (numbers $1$, $2$, $3$, $4$, $5$, $6$, $8$, $9$, $10$, $12$, and $14$), behavioral (numbers $7$ and $11$), or psychosocial (numbers $13$ and $15$) markers or contributors. In future applications, such an approach to data analysis could be used to make better decisions based on personalized understanding of a specific target subgroup.

# Conclusion

In this paper, a novel feature selection method called SAFS based on deep learning has been developed. The overall proposed approach includes three phases: representation learning, feature selection and supervised learning. For representation learning, we used stacked auto-encoders with $3$ hidden layers in an iterative learning process that chooses the best architecture for feature selection. We applied our model in a specific precision medicine problem with goal of finding the most important risk factors that affect LVMI, an indicator of cardiovascular disease risk, in African-Americans with hypertension. The results show that our method (SAFS) outperforms random forests and LASSO significantly in feature ranking and selection. In future work we will evaluate our model on related precision medicine problems aimed at reducing cardiovascular disparities and apply different deep architectures as well as stacked auto-encoders for representation learning.