Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images
Juil Sock Guillermo Garcia-Hernando Anil Armagan Tae-Kyun Kim
cs.CV






^1Imperial College London   ^2Niantic   ^3KAIST




Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images
    Juil Sock^1, Guillermo Garcia-Hernando^1,2, Anil Armagan^1, Tae-Kyun Kim^1,3
    December 30, 2023. .../latex/6he-sdm-2/6he-sdm-2-3.tex
=============================================================================================================







Most successful approaches to estimate the 6D pose of an object typically train a neural network by supervising the learning with annotated poses in real world images. These annotations are generally expensive to obtain and a common workaround is to generate and train on synthetic scenes, with the drawback of limited generalisation when the model is deployed in the real world.
In this work, a two-stage 6D object pose estimator framework that can be applied on top of existing neural-network-based approaches and that does not require pose annotations on real images is proposed. The first self-supervised stage enforces the pose consistency between rendered predictions and real input images, narrowing the gap between the two domains.
The second stage fine-tunes the previously trained model by enforcing the photometric consistency between pairs of different object views, where one image is warped and aligned to match the view of the other and thus enabling their comparison. 
In the absence of both real image annotations and depth information, applying the proposed framework on top of two recent approaches results in state-of-the-art performance when compared to methods trained only on synthetic data, domain adaptation baselines and a concurrent self-supervised approach on LINEMOD, LINEMOD OCCLUSION and HomebrewedDB datasets.








§ INTRODUCTION

empty

6D object pose estimation <cit.> is an emerging and important topic in the intersection of computer vision, robotics and augmented reality. State-of-the-art approaches have shown to perform reasonably well in real cluttered environments when only utilising colour images <cit.>. An obvious recent trend is the use of deep neural networks (DNNs), which have shown strong performance when annotated training data is available at scale <cit.>. 


To train DNN-based approaches, large quantities of training images can be synthesised where pose labels are automatically generated by rendering 3D object models in the desired poses. However, as we show in our experiments, naively training on synthetic images shows poor generalisation to the real domain <cit.>. Although visually similar, rendered images and real images are different in many ways due to multiple factors from compression effects to lens vignetting. Therefore DNNs often exploit real images with accurate 6D pose labels <cit.> in addition to the synthetic data. Unfortunately, it is cumbersome to obtain real data with high quality 6D pose annotations in large quantities. To overcome this limitation, supervised <cit.> and unsupervised <cit.> domain adaptation as well as self supervised methods <cit.> have been proposed to learn from both real and synthetic domain. However, domain adaptation either requires real pose labels(for supervised) or the performance is sub-optimal(for unsupervised). Furthermore, existing methods often require extra data other colour images to overcome the domain gap.





In this paper, we propose a 6D object pose estimation framework that can be learned without pose labels in the real domain. This framework consists of two main consecutive stages: a self-supervised stage that aims to bridge the synthetic-real gap, and a warp-alignment stage that adjusts the previous prediction by enforcing visual consistency between different object views.

The self-supervised stage uses the object pose consistency as the supervisory signal, inspired by the cycle consistency approach introduced by Zhu et al. <cit.>. It guides the model towards learning to produce consistent pose estimations for both real and synthetic images. This is achieved by generating image-label pairs in the synthetic domain conditioned on the pose predictions in the real domain using a differentiable renderer during the training. 
The proposed approach works with RGB images only, as opposed to previous self-supervised methods that require extra modalities such as depth maps for fitting predictions to data <cit.>. This is desirable as capturing depth information requires special hardware which is not always available, particularly in outdoor applications. 

In the warp-alignment stage, we propose to fine-tune the previously trained model by exploiting the relation between the camera poses and the object poses. Inspired by approaches in monocular depth estimation literature <cit.>, our observation is that the camera poses can be inferred from the predicted object poses to warp the input images into another view, to be perceptually compared and provide a supervisory signal to the pose estimator.

The proposed framework is flexible, and virtually any object pose estimator could benefit. We apply our framework on top of two different state-of-the-art methods, namely BB8 <cit.> and Pix2Pose <cit.>, and observed a boost in performance in the absence of pose annotation on real images. Moreover, we report state-of-the-art performance when compared to recent approaches that are trained on synthetic data, domain adaptation baselines and the aforementioned self-supervised method in the absence of both real pose annotations and depth information on LINEMOD <cit.>, LINEMOD OCCLUSION <cit.> and HomebrewedDB <cit.> datasets.








§ RELATED WORK



Datasets. Annotating geometric labels such as 6D object pose is costly, often involving special hardware <cit.>, and only captures limited views as the space of possible viewpoints is immense. Efforts have been made to create a real RGB dataset with high-quality labels such as Pix3D <cit.>, TLESS <cit.>, YCB <cit.>, Bin-picking <cit.> and HomebrewedDB <cit.>, yet have been limited due to the time and effort required.

Real-synthetic domain gap. Existing methods render synthetic images from a 3D model with perfect labels to train the network in the synthetic domain and deploy them in real-world  <cit.>. However, training purely with synthetic images does not show good generalisation to real images <cit.>. Although visually similar, the rendered and real images are different in many ways due to factors such as compression effects or lens vignetting. To overcome this shortcoming, mainly two different approaches have been proposed for 6D object pose estimation: domain randomisation <cit.> and domain adaptation <cit.>. Domain randomisation hypothesises that enough variability in augmentation for simulated images will generalise to real images <cit.>.

However, the domain randomisation is limited to the type of parameters (e.g. brightness, contrast, etc.) being randomised, selected heuristically. The chosen parameters might not be relevant to the domain gap. Recently, Zakharov  <cit.> proposed a method that learns the optimal weights for augmentation modules which helps achieve maximum domain confusion.

Domain adaptation is categorised into unsupervised <cit.> and supervised <cit.> learning. Supervised domain adaptation uses image-label pairs from both the source and target domain and directly learns the mapping between the two representations <cit.>. These methods show promising results but require labels in the target domain. Recently, unsupervised domain adaptation with Generative Adversarial Networks (GANs) <cit.> has been proposed to generate target domain images without labels. Another example of an unsupervised method is GRL <cit.>, where domain-invariant features are generated to deceive domain classifiers. However, often the performance of these methods is suboptimal as the methods learn to match the distributions of the domains without considering the task at hand <cit.>. Also, such methods are prone to overfitting and degeneration of performance is observed if the samples are out of distribution <cit.>.

6D object pose estimation. Recent 6D object pose estimators utilise only colour images as input and still achieve high accuracy when real data with label is used for training. The most popular approach is to estimate the correspondence between 2D images and 3D object models to obtain both 3D pose and translation via a Perspective-n-Point (PnP) algorithm <cit.>. One branch of such methods <cit.>, including BB8 <cit.>, directly regresses pixel locations of 3D bounding box positions on 2D images where positions of 3D bounding boxes are predetermined. Alternatively, other methods <cit.> including Pix2Pose, estimate normalised object coordinate values for every input image pixel to establish the 2D–3D correspondences. In this work, we chose one representative method from each branch as a baseline for our self-supervised and fine-tuning framework.


Self-supervised pose estimation. Recently, self-supervised methods have shown that a model can be trained to predict hand or object pose by constraining the prediction to be consistent with input data through fitting <cit.>. Although they do not require pose labels, they use an alternative form of ground truth as a weak supervisory signal. <cit.> uses visual and depth alignment to learn 6D pose estimation without labels, however through experiment it has been shown that the accuracy largely relies on the geometric information from the depth map. Supervision from visual alignment does not provide reliable supervision as when the real image and the rendered image of the predicted pose are aligned, and the domain gap between them makes the supervision less meaningful. <cit.> exploits both depth maps and multi-view data to achieve high performance. Self-supervised learning has also been used to learn meaningful feature representations by applying simple geometric augmentation to input images such as rotation <cit.>. In our work, we propose to apply the silhouette of predicted pose to augment the input image, and this technique is shown to be crucial for bridging the synthetic–real domain gap in our experiments.

Another domain where self-supervision is widely used is monocular depth estimation from video <cit.>. The idea is to estimate both the camera viewpoint and depth map for two consecutive frames, which enables the warping of the source frame to the target frame. A photometric loss between the warped frame and the target frame can serve as a proxy loss. Inspired from this idea, we propose an unsupervised learning framework which further enhances the pose estimator trained with our self-supervision. The core observation is that the relative viewpoint between two images can be calculated from the estimated poses. Also, rendering a depth map using the object model with an estimated pose via a differentiable renderer can provide geometric data needed without explicitly predicting pixel-wise depth values.






§ PROPOSED FRAMEWORK



 §.§ Overview

We propose a self-supervised method which results in a domain-invariant 6D pose estimator via pose consistency without pose labels on real RGB images. We assume that the primary differences between the two domains, real and synthetic, are low-level effects such as illumination and noise rather than high-level geometric variations <cit.>. The proposed method adopts learning-by-synthesis in an end-to-end manner, via differentiable PnP and a differentiable renderer. The primary advantage of our proposed framework over the prior arts is that our method gets supervision from the discrepancy between the estimated poses on different domains, rather than getting a training signal from fitting. This eliminates the need for depth maps <cit.> which may not be available in common applications. In the second part of the section, we present a warp-align step that further enhances the accuracy of the pose estimator, again without requiring the real pose label.


 §.§ Self-supervised 6D Object Pose Learning



Let {ℐ^g,y^g,𝐡^g} denote a synthetic generated dataset and {ℐ^r} a real one. ℐ and y represent images and object poses (3D rotation and translation), respectively. 𝐡 is an intermediate representation, which is the output of a 6D object pose estimator with parameters θ and denoted Φ(ℐ;θ). The representation varies depending on the used baseline. For instance, 𝐡 is the projected pixel locations of a 3D bounding box for the baseline BB8 <cit.> and the normalised object coordinates and object mask for Pix2Pose <cit.>. It can be computed as 𝐡^g=π(y^g,ℳ,𝐊), where π(·) is a function that maps the object pose to the intermediate representation for a given object model ℳ, and the camera intrinsic parameters 𝐊. During training (see Fig.<ref>), the baseline pose estimator infers the intermediate representation from a real RGB image, 𝐡_1=Φ(ℐ^r;θ). 𝐡_1 is used to yield ℛ(PnP(𝐡_1,𝐊),ℳ,𝐊)={r_𝐡_1,m_h_1,y^g_𝐡_1} via the PnP algorithm PnP(·) and the renderer ℛ(·). The functional outputs are the new synthetic image r_𝐡_1, its silhouette m_𝐡_1 and the estimated object pose y^g_𝐡_1. 

A masking function M(ℐ^r,r_𝐡_1)=ℐ^r_𝐡_1 outputs the masked real image using the silhouette. Finally the pose estimator predicts the intermediate representation for both the masked image ℐ^r_𝐡_1 and rendered synthetic image r_𝐡_1, that is Φ(ℐ^r_𝐡_1;θ)→𝐡_2 and Φ(r_𝐡_1;θ)→𝐡_3. As shown above, object poses are estimated multiple times across different domains, but the weights θ are shared. In the following, we present each component in detail:



Silhouette masking M. Its function is to transform the real input image by removing the background with the silhouette rendered using the estimation 𝐡_1: M(ℐ^r,m_𝐡_1) = ℐ^r_𝐡_1, where M is the masking function to remove the background given a silhouette. The masked image is used to estimate the pose 𝐡_2, which is encouraged to be consistent with 𝐡_3. The intuition is that in order to achieve this goal, the estimated pose 𝐡_1 needs to be correct, otherwise the foreground image would be removed resulting in the pose estimator not being able to estimate 𝐡_2 consistently.

Differentiable PnP and Differentiable renderer ℛ. In order to make the framework end-to-end trainable, we need a differentiable component to convert the output of the pose estimator to the 6D object pose.  We use the implementation from <cit.> for differentiable PnP. We use a differentiable renderer <cit.>, denoted as ℛ(·), to generate synthetic images and silhouettes given the object model, camera parameters and object pose.

Cross domain occlusion augmentation. To make the pose estimator robust to occlusions, randomly sized patches with Gaussian noise are placed on both the rendered image r_𝐡_1 and target domain image ℐ^r in each iteration during training. The part of the silhouette m_𝐡_1 occluded by the noise patch is excluded to avoid the masked real image ℐ^r_𝐡_1 containing the noise patch.

Pose consistency loss. The core idea is to encourage the pose estimator to estimate consistent poses for both synthetic and real domains. We define the pose consistency loss by penalising the discrepancy between the estimations from different domains. L_pose enforces consistency of the transformed vertices of the object model based on the estimated poses, and measure the alignment in 3D space:

    L_pose = PnP(𝐡_2,𝐊)ℳ_v-PnP(𝐡_3,𝐊))ℳ_v_2,

where ℳ_v are the positions of the vertices. L_pose is agnostic to the baseline pose estimator Φ used, since the output of the pose estimator is converted to homogeneous transformation matrix through differentiable PnP.

Synthetic domain supervised loss. The synthetic domain supervised loss exploits the ground truth pose label that is available in the synthetic domain:

    L_sup = 𝔼[Φ(r_𝐡_1;θ)-π(y^g_𝐡_1,ℳ,𝐊)_huber]
    
        +𝔼[Φ(ℐ^g;θ)-𝐡^g_huber],

where r_𝐡_1 and ℐ^g are the rendered image via the proposed pipeline and the synthetic images in the training set and huber denotes the Huber loss. 
The synthetic domain supervised loss helps to impose constraints on the real data self-supervision, as learning by the pose consistency alone can result in a singularity issue. For instance, the pose estimator can learn always to predict a trivial pose, e.g. frontal pose, regardless of the input data, in which the pose consistency loss is minimised.
It also helps to learn the prior geometric constraint, e.g. the output of the pose estimator always needs to form a valid bounding box in case the Φ is BB8. In general, this loss narrows the gap between synthetic supervision and real self-supervision, preventing the model parameter learning from diverging. 

Overall training objective. In addition to the expected loss over the consistency and the synthetic domain supervised loss, we also use an additional perceptual distance <cit.> to encourage the rendered image r_𝐡_1 and the masked real image ℐ^r_𝐡_1 to be perceptually similar. We compute the L_2 distance in the feature space of pre-trained VGG <cit.> network between the two images where feature stack from multiple layers are normalised in
the channel dimension and denote this loss L_percep. The overall training objective is given as:

    L_total = λ_poseL_pose+λ_supL_sup
            +λ_percepL_percep,

where λ_i are constant weights to control the interaction of the losses. For estimator which separately estimate object mask (e.g. Pix2Pose), L_1 loss is used for both domains with the ground-truth mask. With the above objective, the proposed framework allows us to learn the pose estimator Φ without the pose labels in the target domain (i.e. real RGB images). At the test time, only Φ and PnP are used which is light-weight and fast.





 §.§ Warp-Align Fine-Tuning Stage

Different to the self-supervised learning that learns to estimate poses by leveraging consistency in high-level features such as pose, the goal of the warp-alignment fine-tuning stage is to leverage consistency in lower-level photometric features.

Similar to the self-supervised framework, this step can be used with various off-the-shelf pose estimators without using any pose labels due to its unsupervised nature.

In Fig. <ref> we show a schematic of the warp-alignment framework. A pair of unlabelled real input images, source image (ℐ^r_s) and target image (ℐ^r_t), are compared and the model learns to minimise the difference between the two images. However, since the two images are taken from different viewpoints, they cannot be compared directly. The core idea is to transform (warp) ℐ^r_s to have the same viewpoint as ℐ^r_t, at which the images can be compared directly. 
In order to achieve the correct transformation and projection to 2D image, two ingredients are required: relative camera view transformation between the two images and depth value for the pixels of the image to be transformed, since the source image needs to be represented in 3D in order to apply 3D camera view transformation.


In 6D object pose estimation problems, the object pose is defined in the camera coordinate space. Hence the camera pose can be defined in the object coordinate space through a simple matrix inversion if the pose is represented as a homogeneous transformation matrix. Since the camera pose for both ℐ^r_s and ℐ^r_t can now be defined in the common object coordinate space, the relative view transformation is obtained as follows:


    T_s→t = PnP(Φ(ℐ^r_t;θ))(PnP(Φ(ℐ^r_s;θ)))^-1.



The depth values of the foreground pixels are required to warp the image to a different viewpoint and that can be obtained by rendering the 3D object model with the pose predicted by Φ. The masking function M is used to remove the background pixels.

3D points of the foreground pixels are then calculated using the rendered depth map and these points are later transformed by T_s→t to be projected onto an image plane.

We use the Multi-Scale Structural Similarity Index (𝖬𝖲_𝖲𝖲𝖨𝖬) <cit.> to calculate the similarity between the warped source image ℐ̃^r_s and the target image ℐ^r_t, defined as follows:


    L_𝖬𝖲_𝖲𝖲𝖨𝖬 = 1 - 𝖬𝖲_𝖲𝖲𝖨𝖬(ℐ̃^r_s, M(ℐ^r_t, m_𝐡_t)).
















§ EXPERIMENTS

We evaluate the efficacy of our framework through experiments on three datasets, and an ablation study is presented to investigate how different components contribute to the performance. We chose two baselines for Φ(·), namely BB8 <cit.> and Pix2Pose <cit.>. These methods are essentially different, which makes them good candidates to demonstrate the generalisability of the proposed framework. BB8 is a holistic pose estimator which regresses pixel locations of a projected 3D bounding box. Pix2Pose uses object coordinates as an intermediate representation and learns to estimate foreground object pixels (mask), object coordinates and confidence scores. In other words, BB8 estimates the pixel location given 3D points of bounding box whereas Pix2Pose estimates a 3D point given a pixel location. Note that we use our own implementations for the baselines and both baselines achieve a better performance than the ones reported in the original works. Training the baseline purely on synthetic images serves as a Lower Bound (LB), since the self-supervised training is initialised with the model pretrained with synthetic images. The baseline methods trained with both real and synthetic pose labels serves as an Upper Bound (UB). 


We evaluate our approach on the commonly used benchmarks LINEMOD <cit.> and LINEMOD OCCLUSION <cit.>, containing 13 and 8 objects respectively, as well as 3 objects from the HomebrewedDB dataset following Wang et al. <cit.>. All reported results are the percentage of correctly predicted poses using the AD{D|I} <cit.> metric. We set a threshold of 10% of the object's diameter following the BOP (Benchmark for 6D Object Pose Estimation) standard.




 §.§ Implementation Details

For self-supervised learning, a batch size of 16 is used with the ADAM optimiser <cit.> for both baselines. The initial learning rate is 1e–5 for the first 15 epochs and reduced to 1e–6 for the next 10 epochs. 

We kindly refer to the supplementary material for details on parameters for each baseline used, as well as the parameters for the warp-alignment framework. For the LINEMOD dataset, following the standard practice in the community, 15% of the images are used for training but without pose labels. The selection of training images follows the strategy in <cit.>. To be robust against inaccurate detection, we randomly perturb the scale and center position of the bounding box during training. We initialise the network by training with synthetic data only where the early layers are frozen to avoid overfitting to the synthetic dataset. 
All parameters are updated after initialisation. For all testing, Faster-RCNN <cit.> is used to detect objects and provides accurate bounding boxes with an accuracy of 99.24% and 97.29% with 50% and 75% IOU respectively, for all objects in the LINEMOD dataset. 

Warp-align fine-tuning starts training from the pose estimator trained with self-supervised framework. Source images and target images are sampled such that the estimated 3D pose difference between the two is less than 60^∘, as warping into a very different viewpoint degrades the quality of the warped image.



 §.§ Results

Ablation Study. Table <ref> shows the influence of different components for the self-supervised object pose estimation on camera object from the LINEMOD dataset with BB8 as the baseline method. It can be seen that silhouette masking is a critical component of the framework. Silhouette masking imposes a strong constraint as the pose consistency loss reduces if the first estimation 𝐡_1 is closer to the correct pose, thereby not masking out a portion of the foreground. This intuition is further reinforced in our experiments. Table <ref> shows that the objects with more distinctive silhouettes for different poses (e.g. Benchvise) obtain higher accuracies than those that are not (e.g. Duck). The silhouette of a spherical object is invariant to the rotation, which means a wrong 3D pose estimation may still produce the correct silhouette. Perception loss makes training more stable, and occlusion augmentation further boosts the performance since it enforces the estimator to learn more robust features.











Comparison with LB and UB.

Table <ref> (Top) indicates that the LB performance for the BB8 baseline is very low with the ADD metric on LINEMOD. The middle rows show how unsupervised domain adaptation methods improve from the LB. Although PixelDA <cit.> generates perceptually realistic images, it does not contribute to learning domain invariant features, as reported in <cit.>. This can be explained by the fact that the number of real images allowed in the standard LINEMOD protocol (15%) may not be enough to learn the distribution which results in overfitting on the training data. Using GRL <cit.> without real labels slightly improves the generalisability from synthetic training, as the only supervision is from the domain label which disregards the task. Our method significantly outperforms both methods and almost reaches the performance of the UB with real labels.

Table <ref> (Bottom) shows the performance on LINEMOD with Pix2Pose baseline. The performance with synthetic only training is much higher than BB8, and it can be explained by the fact that Pix2Pose regresses the object coordinates for every pixel and utilises a RANSAC framework which makes it more robust against false estimation, whereas BB8 regresses the location of eight corner points. Our proposed self-supervised learning framework improves significantly from the LB similar to the BB8 baseline experiment. Our warp-align stage boosts the performance further from the self-supervised framework except for some objects such as Ape, Duck or Lamp. Our alignment framework compares the texture between two images, but since those objects are almost monochromatic, alignment may not be able to provide meaningful supervision. Fig. <ref> shows the qualitative results with LB, self-supervised and ground truth. It shows that the alignment with ground truth is improved after self-supervision.



Comparison with state-of-the-art.
In Table. <ref>, we compare our proposed work with other state-of-the-art methods. We distinguished the methods based on whether the methods are supervised with real pose labels or not. Our method significantly outperforms SSD <cit.>, AAE <cit.>, MHP <cit.>, and DPOD <cit.> which all uses domain randomisation to learn from synthetic data. Notably, our method is on par with Self6D <cit.> which uses both visual and geometric alignment when utilising a depth map during training.

Table <ref> reports the performance on the LINEMOD OCCLUSION <cit.> dataset and all methods in the table do not use real pose labels during training. The results in the table follow the BOP <cit.> standard where 200 images from the original occlusion dataset are selected for the evaluation. Our method outperforms DPOD <cit.> and CDPN <cit.> which use RGB images during training, but performs lower than Self6D <cit.> when tested on images with strong occlusions, since using extra modalities such as depth in Self6D can help to learn more robust features in the presence of occlusions.


Table <ref> shows the results for the HomebrewedDB <cit.> dataset. HomebrewedDB is a suitable dataset to demonstrate whether the trained pose estimator is not overfitted on the data used for training and generalisable, as 3 objects from LINEMOD dataset are captured in different backgrounds and lighting conditions. The object estimator trained with 15% of unlabelled data from LINEMOD was tested on this dataset without retraining. The results show our self-supervised framework learns robust features that can generalise beyond the images in the training dataset. Our method outperforms DPOD <cit.> and SSD <cit.> by 59% and  20% respectively.








§ CONCLUSION

We proposed a framework that consists of two main components: a self-supervised training approach and a warp-align fine-tuning stage. Our experimental results show that our framework can learn a robust 6D pose estimator without the use of real pose labels. Furthermore, we have also experimentally shown that the framework is flexible to use off-the-shelf object pose estimators as a baseline method by choosing two popular but distinctive approaches. Our evaluations on multiple datasets show that the proposed framework significantly improves on models trained with synthetic data only, and that it can generalise to different datasets without retraining. Finally, our method shows state-of-the-art performance in the absence of real data and depth information surpassing current self-supervised approaches.


ieee







§ SUPPLEMENTARY MATERIAL



Pytorch <cit.> is used to train and test the proposed method on a machine with Intel i5 and GTX1080TI. The objec models(ℳ) used with the differentiable renderer(ℛ) <cit.> were downsampled to reduce the training time if the number of faces are too large.


  
Self-supervision Adam optimiser with The initial learning rate of 1e-5 was used for the first 15 epochs and  1e-6 for the next 10 epochs. Batch size of 16 was used. We followed the standard practice to generate training dataset <cit.>. More specifically, we copy-and-pasted the object image onto random background images. During training, images were cropped 1.3 times the size of shorter side of bounding box. The mean position of bounding box was perturbed by Gaussian distribution of mean value of 0 pixel and standard deviation of 3 pixel and randomly rescaled. Input images were augmented by: adding Gaussian noises, Contrast normalisation and Gaussian blur.



  
Warp-alignment Adam optimiser with learning rate of 0.0001 was used where learning rate were droped by 0.1 every 25 epoches. For every minibatch, 25 images of real images were sampled from the training dataset which produced 30300 possible pair combinations between the sampled images. Among them, 25 pairs with the minimum estimated 3D pose difference were selected to form a minibatch of batchsize 25. 


