MonoPCC: Photometric-invariant Cycle Constraint for Monocular Depth Estimation of Endoscopic Images
Zhiwei Wang Ying Zhou Shiquan He Ting Li Yitong Zhang Xinxia Feng Mei Liu Qiang Li

	
	[
    [
    31/12/2021
==============


		Photometric constraint is indispensable for self-supervised monocular depth estimation. It involves warping a source image onto a target view using estimated depth&pose, and then minimizing the difference between the warped and target images. However, the endoscopic built-in light causes significant brightness fluctuations, and thus makes the photometric constraint unreliable. Previous efforts only mitigate this relying on extra models to calibrate image brightness. In this paper, we propose MonoPCC to address the brightness inconsistency radically by reshaping the photometric constraint into a cycle form. Instead of only warping the source image, MonoPCC constructs a closed loop consisting of two opposite forward-backward warping paths: from target to source and then back to target. Thus, the target image finally receives an image cycle-warped from itself, which naturally makes the constraint invariant to brightness changes. Moreover, MonoPCC transplants the source image's phase-frequency into the intermediate warped image to avoid structure lost, and also stabilizes the training via an exponential moving average (EMA) strategy to avoid frequent changes in the forward warping. The comprehensive and extensive experimental results on three datasets demonstrate that our proposed MonoPCC shows a great robustness to the brightness inconsistency, and exceeds other state-of-the-arts by reducing the absolute relative error by at least 7.27%. 
		
	
	
	

	

§ INTRODUCTION

	
	Monocular endoscope is the key medical imaging tool for gastrointestinal diagnosis and surgery, but often provides a narrow field of view (FOV). 3D scene reconstruction helps enlarge the FOV, and also enables more advanced applications like surgical navigation by registration with preoperative computed tomography (CT). Depth estimation of monocular endoscopic images is prerequisite for reconstructing 3D structures, but extremely challenging due to absence of ground-truth (GT) depth labels. 
	
	
	
	The typical solution of monocular depth estimation relies on self-supervised learning, where the core idea is photometric constraint between real and warped images. Specifically, two convolutional neural networks (CNNs) have to be built; one is called DepthNet and the other is PoseNet. The two CNNs estimate a depth map of each image and camera pose changes of every two adjacent images, based on which a source frame in an endoscopic video can be projected into 3D space and warped onto a target view of the other frame. DepthNet and PoseNet are jointly optimized to minimize a photometric loss, which is essentially the pixel-to-pixel difference between the warped and target images. 
	
	However, the light source is fixed to the endoscope and moves along with the camera, resulting in significant brightness fluctuations between the source and target frames. Such problem can also be worsened by non-Lambertian reflection due to the close-up observation, as evidenced in Fig. <ref>(a)-(b).
	Consequently, between the target and warped source images, the brightness difference dominates as shown in Fig. <ref>(b)-(c), and thus misguides the photometric constraint in the self-supervised learning.
	
	Many efforts have been paid to enhance the reliability of photometric constraint under the fluctuated brightness. An intuitive solution is to calibrate the brightness in endoscopic video frames beforehand, using either a linear intensity transformation <cit.> or a trained appearance flow model <cit.>. However, the former only addresses the global brightness inconsistency, and the latter increases the training difficulty due to introduced burdensome computation. Moreover, the reliability of appearance flow model is also not always guaranteed due to the weak self-supervision, which leads to a risk of wrongly modifying areas unrelated to brightness changes.
	
	In this paper, we aim to address the bottleneck of brightness inconsistency without relying on any auxiliary model. Our motivation stems from a recent method named TC-Depth <cit.>, which introduced a cycle warping originally for solving the occlusion issue. TC-Depth warps a target image to source and then warps back to itself to identify every occluded pixel, since they assume the occluded pixel can not come back to the original position precisely. We find that such cycle warping can naturally overcome the brightness inconsistency, and yield a more reliable warped image compared to only warping from source to target, as shown in Fig. <ref>(b)-(d). However, directly applying the cycle warping often fails in photometric constraint because (1) the twice bilinear interpolation of cycle warping blurs the image too much, and (2) the networks of depth&pose estimation are learned actively, which makes the intermediate warping unstable and the convergence difficult.
	
	In view of the above analysis, we propose Monocular depth estimation based on Photometric-invariant Cycle Constraint (MonoPCC), which adopts the idea of cycle warping but significantly reshapes it to enable the photometric constraint invariant to inconsistent brightness. Specifically, MonoPCC starts from the target image, and follows a closed loop path (target-source-target) to obtain a cycle-warped target image, which inherits consistent brightness from the original target image. To make such cycle warping effective in the photometric constraint, MonoPCC employs a learning-free structure transplant module (STM) based on Fast Fourier Transform (FFT) to minimize the negative impact of the blurring effect. STM restores the lost structure details in the intermediate warped image by `borrowing' the phase-frequency part from the source image. Moreover, instead of sharing the network weights in both target-source and source-target warping paths, MonoPCC bridges the two paths using an exponential moving average (EMA) strategy to stabilize the intermediate results in the first path.
	
	In summary, our main contributions are as follows: 
	
		
  * We propose MonoPCC, which eliminates the inconsistent brightness inducing misguidance in the self-supervised learning by simply adopting a cycle-form warping to render the photometric constraint invariant to the brightness changes. 
		

		
  * We introduce two enabling techniques, i.e., structure transplant module (STM) and an EMA-based stable training. STM restores the lost image details caused by interpolation, and EMA stabilizes the forward warping. These together guarantee an effective training of MonoPCC under the cycle-form warping. 
		

		
  * Comprehensive and extensive experimental results on three public datasets demonstrate that MonoPCC outperforms other state-of-the-arts by decreasing the absolute relative error by at least 7.27%, and shows a strong ability to resist inconsistent brightness in the training.
	
	
	
	
	

§ RELATED WORK

	
	

 §.§ Self-Supervised Depth Estimation

	
	
	Self-supervised methods <cit.> of monocular depth estimation mostly, if not all, rely on the photometric constraint to train two networks, named DepthNet and PoseNet. 
	For example, as a pioneering work, SfMLearner <cit.> estimated depth maps and camera poses to synthesize a fake target image by warping another image from the source view, and calculated a reconstruction loss between the warped and target images as the photometric constraint. 
	
	For endoscopic cases, M3Depth <cit.> enforced not only the photometric constraint but also a 3D geometric structural consistency by Mask-ICP. For improving pose estimation in laparoscopic procedures, Li et al. <cit.> designed a Siamese pose estimation scheme and also constructed dual-task consistency by additionally predicting scene coordinates. These methods share a common flaw: brightness fluctuations caused by the moving light source in the endoscope, significantly reduce the effectiveness of photometric constraint in self-supervised learning, degrading the performance consequently.
	
	
	
	
	
	

 §.§ Brightness Inconsistency

	
	Recently, several efforts have been made to mitigate brightness inconsistency in endoscopic cases, the common solution is to calibrate the brightness inconsistency using either preprocessing or extra learned models. For example, Endo-SfMLearner <cit.> calculated the mean values and standard deviation of images, and then linearly aligned the brightness of the warped image with that of the target one. However, the real brightness fluctuations are actually local and non-linear, and are hardly calibrated by a simple linear transformation. Recently, AF-SfMLearner <cit.> tried to relax the requirement of brightness consistency by learning an appearance flow model, which estimates pixel-wise brightness offset between adjacent frames. Nevertheless, effectiveness relies on the quality of the learned appearance flow model, and the risk of wrongly modifying the image content occurs accompanying the model error. Also, introducing an extra model dramatically increases the training time and difficulty in self-supervised learning. 
	
	
	

§ METHOD

	
	Fig. <ref> illustrates the pipeline of MonoPCC, consisting of both forward and backward warping paths in the training phase. 
	We first explain how to warp images for self-supervised learning in Sec. <ref>, and then detail the photometric-invariant principle of MonoPCC in Sec. <ref>, as well as its two key enabling techniques in Sec. <ref> and Sec. <ref>, i.e., structure transplant module (STM) for avoiding detail lost and EMA between two paths for stabilizing the training.
	
	
	

 §.§ Warping across Views for Self-Supervision

	
	
	To warp a source image I_s to a target view I_t, DepthNet and PoseNet first estimate the target depth map D_t, and the camera pose changing from target to source T_t → s, respectively.
	
	The DepthNet is typically an encoder-decoder, which inputs a single endoscopic image and outputs an aligned depth map. In this work, we use MonoViT <cit.> as our backbone DepthNet. The PoseNet contains a lightweight ResNet18-based <cit.> encoder with four convolutional layers, which inputs a concatenated two images and predicts the pose change from the image in the tail channels to that in the front channels. 
	
	Using D_t and T_t → s, the warping can be performed from the source to target view, and each pixel in the warped image I_s → t can find its matching pixel in the source image using the following equation:
	
		
    p_s = K T_t → s D_t(p_s → t) K^-1 p_s → t,

		
	where p_s and p_s → t are the pixel's homogeneous coordinates in I_s and I_s → t, respectively, D(p) means the depth value of D at the position p, and K denotes the given camera intrinsic matrix. With the pixel matching relationship, I_s → t can be obtained by filling the color at each pixel position using differentiable bilinear sampling <cit.>:
	
    I_s → t(p_s → t)=𝙱𝚒𝚕𝚒𝚗𝚎𝚊𝚛𝚂𝚊𝚖𝚙𝚕𝚎𝚛(I_s(p_s)),

	where I(p) means the pixel intensity of I at the position p. Since p_s → t is discrete and p_s is continuous, 𝙱𝚒𝚕𝚒𝚗𝚎𝚊𝚛𝚂𝚊𝚖𝚙𝚕𝚎𝚛 is utilized to calculate the intensity of each pixel in I_s → t using the neighboring pixels around p_s and allow error backpropagation. 
	
	After warping, DepthNet and PoseNet can be constrained via the photometric loss, which is calculated as follows: 
	
		
    ℒ_pht(I_t, I_s → t)    = 
    				α·1-SSIM(I_t, I_s → t)/2
       + (1-α) ·|I_t-I_s → t|,

		
	where Structure Similarity Index Measure (SSIM) <cit.> constrains the image quality and L1 loss constrains the content. α is set to 0.85 according to <cit.>.
	
	One of the assumptions for the photometric constraint in Eq. (<ref>) is that the scene contains no specular reflections, and is temporally consistent in terms of brightness <cit.>. However, such assumption hardly holds true in the endoscopic cases as visualized in Fig. <ref>.
	
	
	

 §.§ Photometric-invariant Cycle Warping

	
	
	To overcome the brightness inconsistency, we construct a cycle loop path involving forward and backword warping, as shown in Fig. <ref>. To better explain the principle, we use the red and blue box contours to indicate the different brightness patterns carried by the image.
	
	To get a warped image inheriting the target's brightness, we start from the target itself, and first warp it to get I_t → s, and then warp back to get I_t ⇄ s. The pixel matching relationships across the three images are formulated as follows:
	
    ↓   p_t = K T_s → t D_s(p_t → s) K^-1 p_t → s, 
       p_t → s = K T_t → s D_t(p_t ⇄ s) K^-1 p_t ⇄ s,
    		.

	where p_t, p_t → s, and p_t ⇄ s are the pixel's homogeneous coordinates in I_t, I_t → s and I_t ⇄ s, respectively. D_s and D_t are DepthNet-predicted depth maps of the source and target images, respectively.
	
	Therefore, according to Eq. (<ref>), I_t → s and I_t ⇄ s can be obtained sequentially:
	
    ↓   I_t → s (p_t → s) = 𝙱𝚒𝚕𝚒𝚗𝚎𝚊𝚛𝚂𝚊𝚖𝚙𝚕𝚎𝚛(I_t(p_t)),
       I_t ⇄ s (p_t ⇄ s) = 𝙱𝚒𝚕𝚒𝚗𝚎𝚊𝚛𝚂𝚊𝚖𝚙𝚕𝚎𝚛(I_t → s(p_t → s)).
    		.

	
	Thus, we rewrite the previous photometric constraint Eq. (<ref>) to a cycle form, i.e., ℒ_pht(I_t, I_t ⇄ s), where I_t and I_t ⇄ s have the same brightness pattern as shown in Fig. <ref>. However, direct usage of such cycle warping brings two issues in the optimization of photometric constraint: (1) noticeable image detail lost due to twice image interpolation using 𝙱𝚒𝚕𝚒𝚗𝚎𝚊𝚛𝚂𝚊𝚖𝚙𝚕𝚎𝚛, which negatively affects the appearance-based photometric constraint, and (2) the networks learn too actively to give stable intermediate warping, making the training of two networks difficult to converge. Thus, we introduce two enabling techniques, i.e., structure transplant module to retain structure details and EMA strategy to stabilize forward warping.
	
	
	

 §.§ Structure Transplant to Retain Details

	
	
	
	
	
	
	The learning-free structure transplant module (STM) works based on two observations: (1) the warped frame I_t → s has homologous image style as I_t, and (2) the original frame I_s has real structure details, and also is roughly aligned with I_t → s. Inspired by the image-aligned style transformation proposed in <cit.>, we transplant fine structure of I_s onto the appearance of I_t → s via a learning-free style transformation approach illustrated in Fig. <ref>. 
	
	Specifically, STM utilizes Fast Fourier Transform (FFT) <cit.> to decompose an RGB image x into two components, i.e., amplitude 𝒜(x) and phase 𝒫(x). 𝒜(x) mainly controls the image style like brightness and 𝒫(x) contains the information of structure details <cit.>. Thus, after the forward warping, STM recombines 𝒜(I_t → s) and 𝒫(I_s), and produces the structure-restored warped image I_t → s^STM via Inverse Fast Fourier Transform (IFFT):  
	
		
    I_t → s^STM = IFFT(𝒜(I_t → s) * e^-j * 𝒫(I_s) ).

		
	
	Therefore, in the backward warping, we utilize I_t → s^STM to substitute I_t → s in Eq. (<ref>), which can be rewritten as follows:
	
		
    I_t ⇄ s (p_t ⇄ s) = 𝙱𝚒𝚕𝚒𝚗𝚎𝚊𝚛𝚂𝚊𝚖𝚙𝚕𝚎𝚛(I_t → s^STM (p_t → s)).

		
	
	As shown in Fig. <ref>, I_t → s^STM displays better structure details than I_t → s, and also maintains similar illumination as I_t → s.
	
	
	

 §.§ EMA to Stabilize Forward Warping

	
	
	As shown in Fig. <ref>, the two paths are connected in a cascaded fashion, where the result of the second warping relies on the output of the previous warping. Therefore, a well-initialized and steady intermediate warped image I_t → s is very important for a stable convergence of training. In view of this, we introduce an exponential moving average (EMA) to bridge the two paths and thus divide the whole training into warm-up and follow-up phases.
	
	During the warm-up training, EMA is disabled and we only train DepthNet and PoseNet in the forward warping by optimizing the photometric constraint between I_s and I_t → s, that is, θ = min_θℒ_pht(I_s, I_t → s), where θ represents the total learnable parameters of both DepthNet and PoseNet. Although the brightness inconsistency occurs in the warm-up training, a reasonably acceptable initialization of DepthNet and PoseNet can be reached. 
	
	
	
	The follow-up training with EMA enabled is the key step to resist brightness inconsistency based on the cycle warping. We duplicate the network parameters learned in the warm-up training as an EMA copy θ_EMA←θ, and use the models with θ_EMA to estimate depth and pose in the forward warping and those with θ in the backward warping. θ is updated actively by minimizing the cycle-form photometric loss L_pht(I_t,I_t ⇄ s), and θ_EMA is updated slowly by moving averaging θ. That is, the copy of DepthNet and PoseNet for the forward warping is not directly learned by the optimization of photometric constraint in the follow-up training, thus to avoid predicting frequently changed I_t → s. 
	
	Besides, we further enhance the learning robustness via a ready-made perception loss. As shown in Fig. <ref>, we backward warp the encoding map of I_s generated from DepthNet's encoder of its EMA version, yielding a warped feature map F_s → t. If we also extract the encoding map of I_t using the same encoder, F_s → t and F_t should be aligned and may contain high-level semantic information covering the low-level brightness changes. Thus, a perception loss is defined:
	
		
    ℒ_pcp(F_t, F_s → t) = |F_t - F_s → t|.

		
	
	In summary, the follow-up training of MonoPCC is formulated as follows:
	
    {   θ = min_θ (ℒ_pht(I_t, I_t ⇄ s) + ℒ_pcp(F_t, F_s → t)),
       θ_EMA = αθ_EMA + (1 - α) θ,
    		.

	where α is a hyperparameter to balance the stability and learnability of the forward warping, and set to 0.75 in MonoPCC by default. 
	
	
	

 §.§ Implementation Details

	
	
	We train MonoPCC using a single NVIDIA RTX A6000 GPU. We utilize MonoViT and ResNet18 pre-trained on ImageNet<cit.> for DepthNet and PoseNet initialization, respectively. We set the batch size to 12, and use AdamW <cit.> as the optimizer. The learning rate decay is exponential and set to 0.9. The number of epochs and learning rate differ in the warm-up and follow-up training phases. In warm-up training, the number of epochs is set to 20 epochs, and the initial learning rate is set to 1 × 10^-4 for DepthNet's encoder and 5 × 10^-5 for the rest weights. In follow-up training, the number of epochs is set to 10 epochs, and the initial learning rate is set to 5 × 10^-5 for all network weights.
	
	
	

§ EXPERIMENTAL SETTINGS

	
	

 §.§ Comparison Methods

	
	
	We compare MonoPCC with 8 state-of-the-art (SOTA) methods of monocular depth estimation, i.e., Monodepth2 <cit.>, FeatDepth <cit.>, HR-Depth <cit.>, DIFFNet <cit.>, Endo-SfMLearner <cit.>, AF-SfMLearner <cit.>, MonoViT <cit.>, and Lite-Mono <cit.>. 
	We evaluate their performance and make comparisons using their released codes.
	
	
	

 §.§ Datasets

	
	
	The experiment in this work involves 3 public endoscopic datasets, i.e., SCARED <cit.>, SimCol3D <cit.>, and SERV-CT <cit.>. 
	
	SCARED is from a MICCAI challenge and consists of 35 videos with the size of 1280 × 1024 collected from porcine cadavers using a da Vinci Xi surgical robot. 
	Structured light is used to obtain the ground-truth (GT) depth map for the first frame, which is then mapped onto the subsequent frames using camera trajectory recorded by the robotic arm.
	
	Since the original data is binocular, we only use the left view to simulate our focusing monocular situation. We split the dataset into 21,066 frames for training and 551 for test, which are consistent with the previous SOTA <cit.>. No videos are cross-used in both training and test. 
	
	
	SimCol3D is from MICCAI 2022 EndoVis challenge. It is a synthetic dataset which contains over 36,000 colonoscopic images and depth annotations with size of 475 × 475. Meanwhile, virtual light sources are attached to the camera. For the usage of SimCol3D dataset, we follow their official website
	
	and spilt the dataset into 28,776 and 9,009 frames for training and test, respectively. 
	
	SERV-CT is collected from two ex vivo porcine cadavers, and each has 8 binocular keyframes. We treat the two views independently and thus have 32 images in total with the size of 720 × 576. The GT depth map for each image is calculated by manually aligning the endoscopic image to the 3D anatomical model derived from the corresponding CT scan.
	
	
	
	
	
	
	
	
	
	
	
	
	
	

 §.§ Evaluation Metrics

	
	
	To be consistent with previous works, we employ 5 metrics in the evaluation, which are listed in Table <ref>. Note that, as the common evaluation routine of monocular depth estimation, the predicted depth map should be scaled beforehand since it is scale unknown. The scaling factor is the GT median depth divided by the predicted median depth <cit.>.
	
	
	

§ RESULTS AND DISCUSSIONS

	
	
	

 §.§ Comparison with State-of-the-arts

	
	
	
	

  §.§.§ Evaluation on SCARED and SimCol3D

	
	Table <ref> shows the comparison results on SCARED (left part) and SimCol3D (right part). As can be seen, our method achieves the best performance in terms of all five metrics on SCARED, and consistently outperforms the other SOTAs by different margins on SimCol3D.
	
	Likewise, Endo-SfMLearner and AF-SfMLearner also belong to the approach of addressing the brightness inconsistency in self-supervised monocular depth estimation. Endo-SfMLearner adopts the simple idea of brightness linear transformation, while AF-SfMLearner resorts to a more complex appearance flow model. 
	From their results, we can see that the appearance flow model is more effective, since the brightness changes are often non-linear in the endoscopic scene. Nevertheless, our MonoPCC still exceeds the AF-SfMLearner by reducing Sq Rel by 9.11% on SCARED, which implies that compared to the learning-based appearance flow model, our cycle-form warping can guarantee the brightness consistency in a more effective and reliable way. 
	
	
	The top four rows of Fig. <ref> present a qualitative comparison on the two datasets.
	Error maps are acquired by mapping pixel-wise Abs Rel value to different colors, and the red means large relative error and the blue indicates small error. 
	On SCARED, the color-coded error map of MonoPCC is clearly bluer than those of other SOTAs, especially for the regions near the boundaries or specular reflections, as indicated by the red dotted boxes in Fig. <ref>.
	
	On SimCol3D, MonoPCC provides lower relative depth errors at both the proximal and distal parts of the colon, which demonstrates the validity of MonoPCC for low-textured regions, e.g., colon in the digestive tract. 
	
	
	
	
	
	
	
	

  §.§.§ Generalization on SERV-CT

	
	We also evaluate the generalization of all methods, i.e., using the model trained on SCARED to directly test on SERV-CT. The comparison results are listed in Table <ref>. As can be seen, all methods experience a varying degree of degradation on SERV-CT. However, MonoPCC is still the best, and the only one maintaining the Abs Rel less than 0.1 and δ greater than 0.9. Also, MonoPCC significantly surpasses the second-best AF-SfMLearner by a relative decrease of 19.02% and 11.45% in the Sq Rel and RMSE log, respectively.
	The bottom two rows of Fig. <ref> exhibit two visual results from SERV-CT. 
	As can be seen, most methods yield higher errors compared to their visual results on SCARED. In comparison, MonoPCC still produces less red error maps as indicated by the dotted boxes in Fig. <ref>.
	
	
	

 §.§ Ablation Study

	
	
	We conduct ablation studies via 5-fold cross-validation on SCARED to verify the effectiveness of each component in MonoPCC and to compare MonoPCC with other related techniques against brightness fluctuations. 
	
	
	
	
		
		
		

  §.§.§ Effectiveness of Three Components
 

		We develop five variants of MonoPCC by disabling the three components, EMA and/or ℒ_pcp, and STM. Note that, the warping should be cycle-form to utilize the components, and if we disable EMA, only θ in the backward warping path will be updated, and θ_EMA in the forward warping remains unchanged. Table <ref> gives the comparison results between the variants and complete MonoPCC. We also include the backbone MonoViT in the first row of Table <ref> as the baseline, which adopts the non-cycle warping. 
		
		From the comparison results in Table <ref>, three key observations can be made:
		
		(1) By comparing the first two rows of Table <ref>, the inferior performance of the variant using none of the components indicates that the direct usage of cycle warping is not enough due to the image blurring and unstable gradient propagation mentioned in Sec. <ref>. 
		
		(2) As can be seen from the second to fourth rows, using EMA and ℒ_pcp solely brings no improvements. However, adding STM can significantly reduce Sq Rel and RMSE by 16.11% and 7.50%, respectively (p < 0.003). Thus, STM is the irreplaceable component for successfully training using the cycle-form warping.
		
		(3) Comparison between the last four rows reveals the effectiveness of EMA and ℒ_pcp. Specifically, with STM enabled, both EMA and ℒ_pcp bring remarkable reduction of Abs Rel by 8.08% (p = 0.012, p = 0.022). Meanwhile, EMA and ℒ_pcp are not mutually excluded, and the combination reduces Abs Rel by 14.14% (p = 0.001). 
		
		
		
		
				
				
				
				
				
				
				

  §.§.§ MonoPCC vs. Other Techniques against Brightness Fluctuations
 Several techniques have been developed to address inconsistent brightness in endoscopic images, e.g., affine brightness transformation (ABT) in Endo-SfMLearner, and appearance flow module (AFM) in AF-SfMLearner. For comparison, we train three variants of the backbone MonoViT, using ABT, AFM, and STM+EMA, respectively, to verify the robustness of different enabling techniques to the brightness inconsistency.
				
				Table <ref> lists the comparison results as well as the backbone MonoViT using none of these techniques. As can be seen, the variant using STM+EMA strategy exceeds that using AFM only with a reduction of Abs Rel and Sq Rel by 7.14% and 13.59%, respectively (p = 0.019, p = 0.014). It is worth noting that compared to AFM, our method requires no extra model to learn, exhibiting greater usability and scalability.
				
				
				

 §.§ Robustness to Severe Brightness Inconsistency

				
				
				Besides the current inconsistent brightness carried by the used dataset, we want to test the limit of MonoPCC under more severe brightness fluctuations. To this end, we create two copies of SCARED, and add global brightness perturbation to every adjacent frames of the first copy, and both global and local perturbations to the second. Specifically, the global perturbation is defined as the linear transformation on the image's brightness channel (HSV color model), that is, v^scale = k*v, k∈ [0.8, 0.9] ∪ [1.1, 1.2]. The local perturbation is the randomly placed Gaussian bright (or dark) spots. Fig. <ref> presents visual examples from the original SCARED and our created two copies with more severe brightness inconsistency. 
				
				
				
				
				Fig. <ref> illustrates Abs Rel errors of different methods on the two brightness-perturbed copies of and the original SCARED. As mentioned before, Endo-SfMLearner and AF-SfMLearner all tried to address the brightness inconsistency. As can be seen from the comparison results, Endo-SfMLearner shows a robustness to the global brightness perturbation thanks to its using brightness linear transformation, but degrades significantly when facing the local perturbation. AF-SfMLearner can address the local brightness perturbation more or less using a learned appearance flow model, but still presents a non-trivial increase of Abs Rel error. In comparison, MonoPCC achieves the lowest Abs Rel error on the three datasets, and even produces lower errors on the copy containing the most severe brightness inconsistency (global+local) compared to the other methods' results on the original SCARED.
				
				
				

§ CONCLUSION

				Self-supervised monocular depth estimation is challenging for endoscopic scenes due to the severe negative impact of brightness fluctuations on the photometric constraint. 
				In this paper, we propose a cycle-form warping to naturally overcome the brightness inconsistency of endoscopic images, and develop a MonoPCC for robust monocular depth estimation by using a re-designed photometric-invariant cycle constraint.
				To make the cycle-form warping effective in the photometric constraint, MonoPCC is equipped with two enabling techniques, i.e., structure transplant module (STM) and exponential moving average (EMA) strategy.
				STM alleviates image detail degradation to validate the backward warping, which uses the result of forward warping as input.
				EMA bridges the learning of network weights in the forward and backward warping, and stabilizes the intermediate warped image to ensure an effective convergence.
				The comprehensive and extensive comparisons with 8 state-of-the-arts on three public datasets, i.e., SCARED, SimCol3D, and SERV-CT, demonstrate that MonoPCC achieves a superior performance by decreasing the absolute relative error by at least 7.27%.
				Additionally, two ablation studies are conducted to confirm the effectiveness of three developed modules and the advancement of MonoPCC over other similar techniques against brightness fluctuations.
				In the future work, we plan to extend MonoPCC into the multi-frame based monocular depth estimation. 
				
				
				
				named
				
				
			