Commonsense Prototype for Outdoor Unsupervised 3D Object Detection
Hai Wu Shijia Zhao Xun Huang Chenglu Wen Xin Li Cheng Wang


Violation of Bell inequalities in an analogue black hole
    Mathieu Isoard
    
========================================================


The prevalent approaches of unsupervised 3D object detection follow cluster-based pseudo-label generation and iterative self-training processes. However, the challenge arises due to the sparsity of LiDAR scans, which leads to pseudo-labels with erroneous size and position, resulting in subpar detection performance. To tackle this problem, this paper introduces a Commonsense Prototype-based Detector, termed CPD, for unsupervised 3D object detection.  CPD first constructs Commonsense Prototype (CProto) characterized by high-quality bounding box and dense points, based on commonsense intuition. Subsequently, CPD refines the low-quality pseudo-labels by leveraging the size prior from CProto. Furthermore, CPD enhances the detection accuracy of sparsely scanned objects by the geometric knowledge from CProto. CPD outperforms state-of-the-art unsupervised 3D detectors on Waymo Open Dataset (WOD), PandaSet, and KITTI datasets by a large margin. Besides, by training CPD on WOD and testing on KITTI, CPD attains 90.85% and 81.01% 3D Average Precision on easy and moderate car classes, respectively. These achievements position CPD in close proximity to fully supervised detectors, highlighting the significance of our method. The code will be available at <https://github.com/hailanyi/CPD>.

    


§ INTRODUCTION


Autonomous driving requires reliable detection of 3D objects (e.g. vehicle and cyclist) in urban scenes for safe path planning and navigation. Thanks to the power of neural networks, numerous studies have developed high-performance 3D detectors through fully supervised approaches<cit.>. However, these models heavily depend on human annotations from diverse scenes to guarantee their effectiveness across various scenarios.
This data labeling process is typically laborious and time-consuming, limiting the wide deployment of detectors in practice <cit.>. 

Several studies have explored approaches to reduce labeling requirements by weakly supervised learning <cit.>, decreasing the label cost by over 80%. Notably, the objects within a 3D scene exhibit distinguishable attributes and can be easily identified through certain commonsense reasoning (see Fig. <ref>). 
For example, the objects are usually located on the ground surface with a certain shape; the object sizes are fixed across frames. This insight has prompted us to develop an unsupervised 3D detector that operates without using human annotations.



In recent years, traditional methods leveraged ground removal <cit.> and clustering technique <cit.> for unsupervised 3D object detection. However, these methods often struggle to achieve satisfactory performance due to the sparsity and occlusion of objects in 3D scenes. Advanced methods create initial pseudo-labels from point cloud sequences by clustering and bootstrap a good detector by iteratively training a deep network <cit.>. Nevertheless, the sparse and view-limited nature of LiDAR scanning leads to pseudo-labels with inaccurate sizes and positions, misleading the network convergence and resulting in suboptimal detection performance. A subset of objects, denoted as complete objects 𝒯, benefit from having at least one complete scan across the entire point cloud sequence, allowing their pseudo-labels to be refined through temporal consistency <cit.> (see  Fig. <ref> (a)). However, the majority of objects (e.g. 65% on WOD <cit.>, as shown in Fig. <ref> (c)), termed incomplete objects 𝒥, lack full scan coverage (see Fig. <ref> (b)), and cannot be recovered by temporal consistency. 



To tackle this issue, this paper proposes a Commonsense Prototype-based Detector, termed CPD, for unsupervised 3D object detection. CPD is built upon two key insights: (1) The ground truth of intra-class objects keeps a similar size (length, width, and height) distribution between incomplete objects and complete objects (see  Fig. <ref> (d)). (2) The nearby stationary objects are very complete in consecutive frames and can be recognized accurately by commonsense intuition (see  Fig. <ref> (f)(g)). 
Our idea is to construct a Commonsense Prototype (CProto) set representing accurate geometry and size from complete objects to refine the pseudo-labels of incomplete objects and improve the detection accuracy. 
To this end, we first design an unsupervised Multi-Frame Clustering (MFC) method that yields high-recall initial pseudo-labels. Subsequently, we introduce an unsupervised Completeness and Size Similarity (CSS) score that selects high-quality labels to construct the CProto set. Furthermore, we design a CProto-constrained Box Regularization (CBR) method to refine the pseudo-labels by incorporating the size prior from CProto. In addition, we develop CProto-constrained Self-Training (CST) that improves the detection accuracy of sparsely scanned objects by the geometry knowledge from CProto. 

The effectiveness of our design is verified by experiments on widely used WOD <cit.>, PandaSet <cit.>, and KITTI dataset <cit.>.
Besides, the individual components of our design are also verified by extensive experiments on WOD <cit.>. 
The main contributions of this work include:




  * We propose a Commonsense Prototype-based Detector (CPD) for unsupervised 3D object detection. CPD outperforms state-of-the-art unsupervised 3D detectors by a large margin. 


  * We propose Multi-Frame Clustering (MFC) and CProto-constrained Box Regularization (CBR) for pseudo-label generation and refinement, greatly improving the recall and precision of pseudo-label.


  * We propose CProto-constrained Self-Training (CST) for unsupervised 3D detection. It improves the recognition and localization accuracy of sparse objects, boosting the detection performance significantly.







§ RELATED WORK





Fully/weakly supervised 3D object detection.
Recent fully-supervised 3D detectors build single-stage <cit.>, two-stage <cit.> or multiple stage <cit.> deep networks for 3D object detection. However, these methods heavily rely on a large amount of precise annotations. Some weakly supervised methods replace the box annotation with low-cost click annotation<cit.>.  Other methods decrease the supervision by only annotating a part of scenes  <cit.> or a part of instances <cit.>. Unlike all of the above works, we aim to design a 3D detector that does not require human-level annotations.






Unsupervised 3D object detection.
Previous unsupervised pre-training methods discern latent patterns within the unlabeled data by masked labels <cit.> or contrastive loss <cit.>. But these methods require human labels for fine-turning.
Traditional methods <cit.> employ ground removal and clustering for 3D object detection without human labels, but suffer from poor detection performance. Some deep learning-based methods generate pseudo-labels by clustering and use the pseudo-labels to train a 3D detector <cit.> iteratively. Recent OYSTER <cit.> improves pseudo-label quality with temporal consistency. However, most pseudo-labels of incomplete objects cannot be recovered by temporal consistency. Our CPD addresses this problem by leveraging the geometry prior from CProto to refine the pseudo-label and guide the network convergence. 

Prototype-based methods.
The prototype-based methods are widely used in 2D detection <cit.> when novel classes are incorporated. Inspired by these methods, 
Prototypical VoteNet <cit.> constructs geometric prototypes learned from basic classes for few-shot 3D object detection.
GPA-3D <cit.> and CL3D <cit.> build geometric prototypes from a source-domain model for domain adaptive 3D detection. However, both the learning from basic class and training on the source domain require high-quality annotations. 
Unlike that, we construct CProto using commonsense knowledge and detect 3D objects in a zero-shot manner without human-level annotations.


§ CPD METHOD

This paper introduces the Commonsense Prototype-based Detector (CPD), a novel approach for unsupervised 3D object detection. As shown in Fig. <ref>, CPD consists of three main parts: (1) initial label generation; (2) label refinement; (3) self-training. We detail the designs as follows.



 §.§ Initial Label Generation



Recent unsupervised methods <cit.> detect 3D objects in a class-agnostic way. How to classify objects  (e.g. vehicle and pedestrian) without annotation is still an unsolved challenge.
Our observations indicate that some stationary objects in consecutive frames, appear more complete (see Fig.<ref> (f)) and can be classified by predefined sizes. 
This motivates us to design a Multi-Frame Clustering (MFC) method to generate initial labels. 
MFC involves motion artifact removal, clustering, and post-processing.

Motion Artifact Removal (MAR). 
Directly transforming and concatenating 2n+1 consecutive frames {x_-n, ..., x_n} (i.e., past n, future n, and the current frame) into a single point cloud x_0^* introduces motion artifacts from moving objects, leading to increased label errors as the n grows (see Fig. <ref>(a)). 
To mitigate this issue, we first transform the consecutive frames to global system and calculate the Persistence Point Score (PPScore)<cit.> by consecutive frames to identify the points in motion. 
We keep all the points from x_0 and remove moving points from the other frames x_-n,...,x_-1,x_1,...,x_n. 
After this removal, we concatenate the frames to obtain dense points x_0^*. 

Clustering and post-processing. 
In line with recent study <cit.>, we apply the ground removal<cit.>, DBSCAN <cit.> and bounding box fitting <cit.> on x_0^* to obtain a set of class-agnostic bounding boxes b̂. 
We observe that the objects of the same class typically have similar sizes in 3D space. Therefore, we pre-define class-specific size thresholds (e.g. the length of vehicle is generally larger than 0.5m) based on human commonsense to classify  b̂ into different categories. 
We then apply class-agnostic tracking to associate the small background objects with foreground trajectories, and enhance the consistency of objects' sizes by using temporal coherency <cit.>. This process results in a set of initial pseudo-labels b={b_j}_j, where b_j=[x,y,z,l,w,h,α,β,τ] represents position, width, length, height, azimuth angle, class identity, and tracking identity, respectively. 




 §.§ CProto-constrained Box Regularization for Label Refinement




As noted in Section <ref>, initial labels for incomplete objects often suffer from inaccuracies in sizes and positions. 
To tackle this issue, we introduce the CProto-constrained Box Regularization (CBR) method. The key idea is to construct a high-quality CProto set based on unsupervised scoring from complete objects to refine the pseudo-labels of incomplete objects. 
Different from OYSTER <cit.>, which can only refine the pseudo-labels of objects having at least one complete scan, our CBR can refine pseudo-labels of all objects, significantly decreasing the overall size and position errors. 

Completeness and Size Similarity (CSS) scoring.
Existing label scoring methods such as IoU scoring <cit.> are designed for fully supervised detectors.
In contrast, we introduce an unsupervised Completeness and Size Similarity scoring (CSS) method. It aims to approximate the IoU score using commonsense knowledge alone (see Fig. <ref>). 

Distance score. CSS first assesses the object completeness based on distance, assuming labels closer to the ego vehicle are likely to be more accurate. 
For an initial label b_j, we normalize the distance to the ego vehicle within the range [0,1] to compute the distance score as 

    ψ^1(b_j)=1-𝒩(‖ c_j‖),

where 𝒩 is the normalization function and c_j is the location of b_j. 
However, this distance-based approach has its limitations. For example, occluded objects near the ego vehicle, which should receive lower scores, are inadvertently assigned high scores due to their proximity. 
To mitigate this issue, we introduce a Multi-Level Occupancy (MLO) score, further detailed in Fig. <ref> (b). 


MLO score.
Considering the diverse sizes of objects, we divide the bounding box of the initial label into multiple grids with different length and width resolutions. The MLO score is then calculated by determining the proportion of grids occupied by cluster points, via

    ψ^2(b_j)=1/N^o∑_k O^k/(r^k)^2,

where N^o denotes resolution number, O^k is the number of occupied grids under k-th resolution, and r^k is the grid number of k-th resolution. 



Size Similarity (SS) score. 
While the distance and MLO scores effectively evaluate the localization and size quality, they fall short in assessing classification quality. 
To bridge this gap, we introduce the SS score. This score utilizes a class-specific template box a (average size of typical objects in Wikipedia) and calculates a truncated KL divergence <cit.>. Note that, this score is decided by ratio difference, rather than their specific values. Simple commonsense of l,w,h ratios (2:1:1 for Vehicle, 1:1:2 for Pedestrian, 2:1:2 for Cyclist) can also be used here.

    ψ^3(b_j)=1-min(0.05,∑_σ q^b_σ log(q^b_σ/q^a_σ))/0.05,

where q_σ^a∈{l^a,w^a,h^a}, q_σ^b∈{l^b,w^b,h^b} refer to the normalized length, width, and height of the template and label.

We linearly combine the three metrics 𝒮(b_j)=∑_i ω^i ψ^i(b_j) to produce final scoring, where ω^i is the weighting factor (in this study we adopt a simple average, ω^i=1/3). 
For each b_j∈b, we compute its CSS score s^css_j = 𝒮(b_j) and obtain a set of scores s={s_j^css}_j.


CProto set construction.
Regular learnable prototype-based methods require annotations <cit.>, which are unavailable in the unsupervised problem. 
We construct a high-quality CProto set P={P_k}_k, representing geometry and size centers based on the unsupervised CSS score.
Here, P_k={x^p_k,b^p_k}, where x^p_k indicates the inside points, and b^p_k refers to the bounding box. 
Specifically, we first categorize the initial labels b into different groups based on their tracking identity τ. 
Within each group, we select the high-quality boxes and inside points that meet a high CSS score threshold η (determined on validation set, using 0.8 in this study). 
Then, we transform all points and boxes into a local coordinate system, and obtain x^p_k by averaging the high-quality boxes and b^p_k by concatenating all the points. 



Box regularization.
We next regularize the initial labels by the size prior from CProto. Based on the statistics on WOD validation set <cit.>, we observe that the height of the initial labels is relatively correct than length and width (see Fig. <ref> (c)). Intuitively, the intra-class 3D objects with the same height have similar length and width. Therefore, we associate the initial label b_j with  CProto P_k by the minimum difference in box height. The initial pseudo-labels with the same P_k and similar length and width are naturally classified into the same group.
We then perform re-size and re-localization for each group to refine the pseudo-labels.
(1) Re-size. We directly replace the size of b_j using the length, width, and height of b_k^p ∈ P_k. 
(2) Re-location.  
Since points are mostly on the object's surface and boundary, we divide the object into different bins and align the box boundary and orientation to the boundary point of the densest part (see Fig. <ref>). 
Finally, we obtain improved pseudo-labels b^*={b^*_j}_j.






 §.§ CProto-constrained Self-Training (CST)

Recent methods <cit.> utilize pseudo-labels for training 3D detectors. 
However, even after refinement, 
some pseudo-labels remain inaccurate, diminishing the effectiveness of correct supervision and potentially misleading the training process.
To tackle these issues, we propose two designs:
(1) CSS-Weighted Detection Loss, which assigns different training weights based on label quality to suppress false supervision signals.
(2) Geometry Contrast Loss, which aligns predictions of sparsely scanned points with the dense CProto, thereby improving feature consistency.


Network architecture.
We adopt a dense-sparse alignment architecture (Fig. <ref> (c)), consisting of a prototype network ℱ^pro and a detection network ℱ^det, constructed from two-stage CenterPoint <cit.>. 
During training, for each b^*_j, we add its corresponding points x^p_k from CProto P_k to the scene to obtain a dense point cloud x^pro.
We feed x^pro to ℱ^pro to produce relatively good features and detections. We then feed randomly downsampled points x^det as a sparse sample to the ℱ^det. We align the features and detections from two branches by the detection loss and contrast loss. During testing, we feed points without downsampling to the detection network ℱ^det to perform detection.


CSS weight.
Considering that the false pseudo-labels may mislead the network convergence, we first calculate a loss weight based on different label qualities.
Formally, we convert a CSS score s^css_i of a pseudo-label to 

    ω_i = {[                   0         s^css_i<S_L; s^css_i-S_L/S_H-S_L     S_L<s^css_i<S_H;                   1         s^css_i>S_H ],.

where S_H and S_L are high/low-quality thresholds (we empirically set 0.7 and 0.4, respectively).

CSS-weighted detection loss. 
To decrease the influence of false labels, we formulate the CSS-weighted detection loss to refine N proposals 

    ℒ_det^css= 1/N∑_iω_i(ℒ^pro_i+ℒ^det_i),

where ℒ^pro_i and ℒ^det_i are detection losses <cit.> of ℱ^pro and ℱ^det, respectively. The losses are calculated by pseudo-labels b^* and network predictions.

Geometry contrast loss.
We formulate two contrast losses that minimize the feature and predicted box difference between the prototype and detection network.
 (1) Feature contrast loss.
For a foreground RoI r_i from the detection network, we extract features f^p_i from the prototype network by voxel set abstract <cit.>, and extract features f^d_i from detection network. We then formulate the contrast loss by cosine distance: 

    ℒ^css_feat=1/N^f∑_i ω_i f^d_i·f^p_i/‖f^d_i‖‖f^p_i‖,

where N^f is the foreground proposal number.
 (2) Box contrast loss.
For a box prediction d^p_i from the prototype network and a box prediction d^d_i from the detection network. We then formulate the box contrast loss by IoU, location difference, and angle difference: 

    ℒ^css_box=    1/N^f∑_i ω_i [1-I(d^d_i,d^p_i) 
       +‖ c^d_i-c^p_i‖ + |sin(α^d_i-α^p_i)|],

where I denote IoU function; c^d_i,α^d_i refers to position and angle of d^d_i; c^p_i,α^p_i refers to position and angle of d^p_i. 
We finally summat all losses to training the detector.







§ EXPERIMENTS




 §.§ Datasets

Waymo Open Dataset (WOD).
We conducted extensive experiments on the WOD <cit.> due to its diverse scenes. The WOD contains 798, 202 and 150 sequences for training, validation and testing, respectively. We adopted similar metrics (3D AP L1 and L2) as fully/weakly supervised methods <cit.>. No annotations were used for training. 







PandaSet dataset.
To compare with recent unsupervised methods <cit.>, we also conducted experiments on the PandaSet <cit.>. 
Like <cit.>, we split the dataset into 73 training and 30 validation snippets and use class-agnostic BEV AP and recall metrics with 0.3, 0.5, and 0.7 IoU thresholds. 

KITTI dataset. 
Since the KITTI detection dataset <cit.> did not provide consecutive frames, we only tested our method on the 3769 val split <cit.>. 
We used similar metrics (Car 3D AP R40 with 0.5 and 0.7 IoU thresholds) as employed in fully/weakly supervised methods <cit.>.



 §.§ Implementation Details

Network details.
Both prototype and detection networks adopt the same 3D backbone as CenterPoint <cit.> and the same RoI refinement network as Voxel-RCNN <cit.>.
For the WOD and KITTI datasets, we use the same detection range and voxel size as CenterPoint <cit.>. For the Pandaset, we use the same detection range as OYSTER <cit.>.

Training details.
We adopt the widely used global scaling and rotation data augmentation. 
We trained our network on 8 Tesla V100 GPUs with the ADAM optimizer. We used a learning rate of 0.003 with a one-cycle learning rate strategy. We trained the CPD for 20 epochs. 




 §.§ Comparison with Unsupervised Detectors


Results on WOD.
The results on the WOD validation set and test set are presented in Table <ref> and Table <ref>. 
All methods use identical size thresholds to define the object classes and use single traversal. 
Our method significantly outperforms existing unsupervised methods. Notably, under the 3D AP L2 with IoU thresholds of 0.7, 0.5, and 0.5, our CPD outperforms OYSTER <cit.> by 18.03%, 13.08%, and 4.55% on Vehicle, Pedestrian, and Cyclist, respectively. 
These advancements come from our MFC, CBR, and CST designs, which yield superior pseudo-labels and enhanced detection accuracy. 
CPD also surpasses the Proto-vanilla method, which uses class-specific prototype <cit.>. 






Results on PandaSet.
The class-agnostic results on PandaSet are presented in Table <ref>.
Our method outperforms OYSTER by 6.5% AP and 9.3% Recall under 0.7 IoU threshold. 
This improvement is largely due to our CPD's enhanced label quality. 
Unlike OYSTER, which suffers from the misleading effects of false labels during training, our CPD leverages the size prior from CProto to significantly improve these labels.




 §.§ Comparison with Fully/Weakly Supervised Detectors


Results on KITTI dataset.
To further validate our method, we pre-trained our CPD, along with OYSTER <cit.> and MODEST <cit.>, on WOD and tested them on the KITTI dataset using Statistical Normalization (SN) <cit.>. 
The car detection results are in Table <ref>. 
We first compared our method with a sparsely supervised method (weakly supervised with 2% labels) <cit.> that annotates a single instance per frame for training. 
Our unsupervised CPD outperforms this sparsely supervised method by 23.52% 3D AP @ IoU_0.7 on moderate car class. 
Additionally, our method attains 90.85% and 81.01% 3D AP for the easy and moderate car classes at a 0.5 IoU threshold. 
Notably, this performance is comparable to that of the fully supervised method CenterPoint <cit.>, demonstrating the advancement of our method.




Results on WOD.
We also compared our method with fully/weakly supervised methods on the WOD validation set <cit.>. The vehicle detection results are in Table <ref>. Our unsupervised CPD outperforms the sparsely supervised method (2% annotation) by 5.25% and 4.16% in terms of 3D AP L1 and L2 respectively. 




 §.§ Pseudo-label Comparison
 
To validate our pseudo-labels, we analyzed their 3D recall and precision on the WOD validation set. 
As shown in Table <ref>, our method surpasses the previous best-performing OYSTER with a 9.42% recall and 5.29% precision improvement (under a 0.7 IoU threshold). 
To understand the sources of this improvement, we examined the IoU between the pseudo-labels and ground truth, and compared the IoU distributions in Fig. <ref> (a)(b)(c). 
We also present the mean absolute error of size, position, and angle between different pseudo-labels in Fig. <ref> (d)(e)(f). The IoU distribution of our method is much closer to 1 than other methods, 
and it also exhibits lower errors in size, position, and angle. These results verify that our MFC and CBR significantly reduce label errors. 










 §.§ Ablation Study

Components analysis of CPD.
To evaluate the individual contributions of our designs, we incrementally added each component and assessed their impact on vehicle detection using the WOD validation set.
The results are shown in Table <ref>. 
Our MFC method surpasses Single Frame Clustering (SFC) by 2.52% in AP, attributed to the more complete point representation of objects across consecutive frames compared to a single frame.
The CBR further enhances performance by 19.27% in AP, as it reduces size and location errors in pseudo-labels. 
The CST contributes an 8.09% increase in AP, demonstrating the effectiveness of geometric features from CProto in detecting sparse objects. 


Frame number of MFC.
To examine the effect of frame count on initial pseudo-label quality, we experimented with different numbers of past and future point cloud frames on the WOD validation set. The BEV results, shown in Fig. <ref> (a)(b), indicate optimal performance with [-5, 5] frames (five past, five future, and the current frame). Additional frames did not significantly improve recall or precision. Consequently, we used 11 frames for initial pseudo-label generation in this study. 



Component analysis of CSS Scoring.
To assess the effectiveness of our scoring system, we calculated the BEV AP of initial pseudo-labels with different scores. 
These evaluations, reported in Table <ref>, show that incorporating all components (distance, MLO, and SS) yields the highest AP. 
The recall-precision curve, plotted in Fig. <ref> (c), also supports this finding. These indicate the significance of each component in accurately measuring pseudo-label quality. 

Components analysis of CBR.
To evaluate the impact of re-sizing and re-localization in CBR, we conducted experiments and analyzed pseudo-label performance. 
As shown in Table <ref>, 
re-sizing results in a 3.91% and 3.4% increase in BEV recall at the 0.5 and 0.7 IoU thresholds, respectively; 
re-localization further enhances recall by 12.68% and 6.43% at these thresholds, while also increasing precision. 
These results indicate the importance of both components, which effectively refine pseudo-labels. 



Components analysis of CST. 
To assess the effectiveness of each component in CST, we established a baseline using only CBR-generated pseudo-labels for training a two-stage CenterPoint detector, then incrementally added our loss components and evaluated vehicle detection performance on the WOD validation set.
As shown in Table <ref>, all loss components contribute to performance improvement. 
Specifically, our ℒ^css_det mitigates the influence of false pseudo-label using CSS weight, and improves the 3D AP L2 at IoU_0.7 by 4.79%. 
Our ℒ^css_feat and ℒ^css_box improve the 3D AP L2 at IoU_0.7 by 0.75% and 2.55% respectively, through leveraging geometric knowledge from dense CProto for more effective sparse object detection. 






 §.§ Visualization Comparison




To provide a more intuitive understanding of how our method improves detection performance, we visually compare our results with those of MODEST <cit.> and OYSTER <cit.>, as shown in Fig. <ref>. 
MODEST often misses distant, sparse objects (Fig. <ref>(1.1)), while OYSTER detects them but inaccurately reports their sizes and positions (Fig. <ref>(2.1)). 
In contrast, CPD, using our CProto-based design, not only recognizes these objects but also accurately predicts their sizes and positions (Fig. <ref>(3.1)). 
Furthermore, since our CST reduces the influence of false pseudo-labels, the false positives (Fig. <ref>(3.2)) are also much fewer than the previous methods (Fig. <ref>(1.2)(2.2)). 




§ CONCLUSION

This paper presents the CPD framework, a novel approach for accurate unsupervised 3D object detection. First, we develop an MFC method to generate initial pseudo-labels. Then, a CProto set is constructed using CSS scoring. Next, we introduce a CBR method to refine these pseudo-labels. Lastly, a CST is designed to enhance detection accuracy for sparse objects. Extensive experiments have verified the effectiveness of our design. Notably, for the first time, our unsupervised CPD method surpasses some weakly supervised methods, demonstrating the advancement of our approach.

Limitations. One notable limitation of our work is the significantly lower Average Precision (AP) for minority classes, such as cyclists (Table <ref>), compared to more prevalent classes like vehicles. This disparity is largely due to the scarce instances of these minority classes within the dataset. Future efforts to collect such objects could be a promising avenue to tackle this issue.

Acknowledgements.
This work was supported in part by the National Natural Science Foundation of China (No.62171393), the Fundamental Research Funds for the Central Universities (No.20720220064).



    ieeenat_fullname
    







§ MORE DETAILS OF METHOD

More details of MFC.
In our main paper section 3.1, we introduced the Multi-Frame Clustering (MFC) for initial label generation. For a more intuitive understanding, we provide a framework illustration in Fig. <ref>. 
Here we present more details of post-processing. As mentioned in our main paper, we pre-defined a set of class-specific size thresholds based on human commonsense to classify pseudo labels into different categories. Taking the WOD  as an example, we pre-define five categories: `Discard Small', `Pedestrian', `Cyclist', `Vehicle', and `Discard Large'.  Formally, for a cluster box b_j, we determine the class identity β by sequentially matching from the thresholds:


    β = {[                      Dis Small                         h≤0.8,;                        Vehicle       1<h≤3, 0.5<w≤3, 0.5<l≤8,;                     Pedestrian 0.8<h≤2.3, 0.2<w≤1., 0.2<l≤1.,;                        Cyclist  1.4<h≤2., 0.5<w≤1., 1.<l≤2.5,;                      Dis Large                        others.;                                ].

Where l,w,h refers to the length, width, and height of b_j, respectively.
The `Discard Large' boxes mostly with trees and buildings are directly removed. The `Discard Small' boxes contain both potential foreground objects and background objects. 
We then apply class-agnostic tracking to associate the small background objects with foreground trajectories, and enhance the consistency of objects' sizes by using temporal coherency.


More details of CSS scoring.
In our main paper section 3.2, we presented the CSS scoring. To better understand how the CSS scoring approximates the IoU score, we present the IoU-score carve in Fig. <ref>, where we show three methods: density scoring (s^den), distance scoring (s^dis) and our CSS scoring (s^css). Intuitively, good scoring should keep consistent with IoU scoring. In other words, with the increase of score, the selected pseudo labels should have larger IoUs with the ground truth. We found that our CSS scoring keeps the most consistent increase along with the IoU increase. Here we also provide the length, width and height of the template box for calculating the Size Similarity in the main paper Eq. 3: 


'Vehicle': [5.06, 1.86, 1.49],
'Pedestrian': [1.0, 1.0, 2.0], 
'Cyclist': [1.9, 0.85, 1.8]













§ MORE EXPERIMENTAL RESULTS

More visualization results.
To better understand how our method improves detection results, here we present more visualization results. From Fig. <ref>, we observe that both the recognition and localization performance of our method (3.1-3.4) are much better than previous methods(1.1-1.4, 2.1-2.4), thanks to our CProto-based design.

BEV AP and 3D APH results on WOD validation set.
Some fully supervised methods also reported the BEV AP L2 and 3D APH performance. Here we presented the results in Table <ref> and Table <ref>, respectively. 
Our CPD outperforms the previous MODEST and OYSTER in both BEV AP L2 and APH L2 by a large margin, further demonstrating the effectiveness of our method.




