Shaocong (Daniel) Xu

selected publications

^*equal contribution; ^♯corresponding author.

2024

1. 

   LiON: Learning Point-wise Abstaining Penalty for LiDAR Outlier DetectioN Using Diverse Synthetic Data

   Shaocong Xu, Pengfei Li, Qianpu Sun, Xinyu Liu, Yang Li, Shihui Guo, Zhen Wang, Bo Jiang, Rui Wang, Kehua Sheng, Bo Zhang, Li Jiang, Hao Zhao, and Yilun Chen

   In Proc. of the Association for the Advancement of Artificial Intelligence (AAAI) 2025

   Abs PDF Code Video Demo

   LiDAR-based semantic scene understanding is an important module in the modern autonomous driving perception stack. However, identifying Out-Of-Distribution points in a LiDAR point cloud is challenging as point clouds lack semantically rich features when compared with RGB images. We revisit this problem from the perspective of selective classification, which introduces a selective function into the standard closed-set classification setup. Our solution is built upon the basic idea of abstaining from choosing any known categories but learns a point-wise abstaining penalty with a marginbased loss. Synthesizing outliers to approximate unlimited OOD samples is also critical to this idea, so we propose a strong synthesis pipeline that generates outliers originated from various factors: unrealistic object categories, sampling patterns and sizes. We demonstrate that learning different abstaining penalties, apart from point-wise penalty, for different types of (synthesized) outliers can further improve the performance. We benchmark our method on SemanticKITTI and nuScenes and achieve state-of-the-art results. Risk-coverage analysis further reveals intrinsic properties of different methods.

2. 

   CoopDETR: A Unified Cooperative Perception Framework for 3D Detection via Object Query

   Zhe Wang, Shaocong Xu, Zhuang Xucai, Tongda Xu, Yan Wang, Jingjing Liu, Yilun Chen, and Ya-Qin Zhang,

   In Submission

   Abs Video Demo

   Cooperative perception enhances the individual perception capabilities of autonomous vehicles (AVs) by providing a comprehensive view of the environment. However, balancing perception performance and transmission costs remains a significant challenge. Current approaches that transmit regionlevel features across agents are limited in interpretability and demand substantial bandwidth, making them unsuitable for practical applications. In this work, we propose CoopDETR, a novel cooperative perception framework that introduces objectlevel feature cooperation via object query. Our framework consists of two key modules: single-agent query generation, which efficiently encodes raw sensor data into object queries, reducing transmission cost while preserving essential information for detection; and cross-agent query fusion, which includes Spatial Query Matching (SQM) and Object Query Aggregation (OQA) to enable effective interaction between queries. Our experiments on the OPV2V and V2XSet datasets demonstrate that CoopDETR achieves state-of-the-art performance and significantly reduces transmission costs to 1/782 of previous methods

3. 

   ECT: Fine-grained Edge Detection with Learned Cause Tokens

   Shaocong Xu, Xiaoxue Chen , Yuhang Zheng, Guyue Zhou, Yurong Chen, Hongbin Zha, and Hao Zhao

   Image and Vision Computing (IVC, JCR1, IF4.7) 2024

   Abs PDF Code Video Demo

   In this study, we tackle the challenging fine-grained edge detection task, which refers to predicting specific edges caused by reflectance, illumination, normal, and depth changes, respectively. Prior methods exploit multi-scale convolutional networks, which are limited in three aspects: (1) Convolutions are local operators while identifying the cause of edge formation requires looking at far away pixels. (2) Priors specific to edge cause are fixed in prediction heads. (3) Using separate networks for generic and fine-grained edge detection, and the constraint between them may be violated. To address these three issues, we propose a two-stage transformer-based network sequentially predicting generic edges and fine-grained edges, which has a global receptive field thanks to the attention mechanism. The prior knowledge of edge causes is formulated as four learnable cause tokens in a cause-aware decoder design. Furthermore, to encourage the consistency between generic edges and fine-grained edges, an edge aggregation and alignment loss is exploited. We evaluate our method on the public benchmark BSDS-RIND and several newly derived benchmarks, and achieve new state-of-the-art results.

4. 

   Intelligent Beehive Monitoring System based on Internet of Things and Colony State Analysis

   Yiyao Zheng, Xiaoyan Cao, Shaocong Xu, Shihui Guo, Rencai Huang, Yingjiao Li, Yijie Chen, Liulin Yang, Xiaoyu Cao, and Hongting Sun

   Smart Agricultural Technology (JCR1, IF6.3) 2024

   Abs PDF Video Demo

   CONTEXT: Bees play a crucial role in terrestrial ecosystems. However, Beekeepers are unable to monitor the state of beehives (bees and environment) all the time, which often results in Colony Collapse Disorder (CCD). Currently, some researchers provided the scheme of intelligent beehive monitoring system equipped with the Internet of Things (IoT). There remain two challenges: Accurately monitor the environmental status around the hive and accurately track and monitor bees in real time.
   OBJECTIVE: With the development of the IoT and computer vision algorithms, we hope to provide an automated and efficient system to meet the above challenges. In this paper, we propose a hive monitoring system and build a visualization module in the cloud to monitor the activity of bee colonies and the environmental dynamic changes.
   METHODS: We proposed a multi-bee tracking algorithm to solve the problem of monitoring bees at the door of the hive. We construct a dataset containing various complex scenes, named BEE22, for training and testing the performance of our algorithm. We design a bee counting rule, based on results of multi-bee tracking algorithm, to reasonably count the bees entering or leaving the beehive. We have deployed multiple sensors around (center, margin, door, and environment) the hive to accurately reflect the changes in the environment around the hive.
   RESULTS AND CONCLUSIONS: Experimental results demonstrate the effectiveness and superiority of our system. In particular, the tracking performance of the multi-bee tracking algorithm reaches 83.5% ± 0.7% accuracy (MOTA) and 77.3% ± 0.2% MOTP, speeds up to 16 frames per second, compared with other algorithms, MOTA and IDF1 are improved by 24.4% and 50.4% respectively. Moreover, our counting algorithm also achieved excellent results, with root mean square error (RMSE) of 0.17 ± 0.01, 1.55 ± 0.05, and 1.25 ± 0.06 in counting the number of bees entering, leaving, and the current scene in an episode, respectively. After that, the system will be deployed and monitored for a long time in the actual scenario, it was found that the activity of bees decreased significantly under heavy rainfall conditions. Additionally, the activity of the bee colony will also increase accordingly, when the amplitude is 500dB to 2000dB, the temperature of the center of the beehive is 25°C to 37°C, and the humidity is 48% to 67%.
   SIGNIFICANCE: Our system can provide valuable information for bee farmers to make control decisions on hives.

2023

1. 

   INT2: Interactive Trajectory Prediction at Intersections

   Zhijie Yan, Pengfei Li, Zheng Fu, Shaocong Xu, Yongliang Shi, Xiaoxue Chen, Yuhang Zheng, Yang Li, Tianyu Liu, Chuxuan Li, Nairui Luo, Xu Gao, Yilun Chen, Zuoxu Wang, Yifeng Shi , Pengfei Huang, Zhengxiao Han, Jirui Yuan, Jiangtao Gong, Guyue Zhou, Hang Zhao and Hao Zhao

   In Proc. of the IEEE International Conf. on Computer Vision (ICCV) 2023

   Abs PDF Code Project Page Dataset Video Demo

   Motion forecasting is an important component in autonomous driving systems. One of the most challenging problems in motion forecasting is interactive trajectory prediction, whose goal is to jointly forecasts the future trajectories of interacting agents. To this end, we present a large-scale interactive trajectory prediction dataset named INT2 for INTeractive trajectory prediction at INTersections. INT2 includes 612,000 scenes, each lasting 1 minute, containing up to 10,200 hours of data. The agent trajectories are auto-labeled by a high-performance offline temporal detection and fusion algorithm, whose quality is further inspected by human judges. Vectorized semantic maps and traffic light information are also provided. Additionally, the dataset poses an interesting domain mismatch challenge. For each intersection, we treat rush-hour and non-rush-hour segments as different domains. We benchmark the best open-sourced interactive trajectory prediction method on INT2 and Waymo Open Motion, under in-domain and cross-domain settings. The dataset, code and models will be made public.

2. 

   Foundation Models for Robotics: Best Known Practices

   Shaocong Xu, and Hao Zhao

   Proceedings of the 22nd Chinese National Conference on Computational Linguistics (Volume 4: Tutorial Abstracts) 2023

   Abs PDF

   Artificial general intelligence (AGI) used to be a sci-fi word but recently the surprising general-ization capability of foundation models have triggered a lot of attention to AGI, in both academiaand industry. Large language models can now answer questions or chat with human beings,using fluent sentences and clear reasoning. Diffusion models can now draw pictures of unprece-dented photo-realism, according to human commands and controls. Researchers have also madesubstantial efforts to explore new possibilities for robotics applications with the help of founda-tion models. Since this interdisciplinary field is still under fast development, there is no clearmethodological conclusions for now. In this tutorial, I will briefly go through best known prac-tices that have shown transformative capabilities in several sub-fields. Specifically, there are fiverepresentative paradigms: (1) Using foundation models to allow human-friendly human-car in-teraction; (2) Using foundation models to equip robots the capabilities of understanding vaguehuman needs; (3) Using foundation models to break down complex tasks into achievable sub-tasks; (4) Using foundation models to composite skill primitives so that reinforcement learningcan work with sparse rewards; (5) Using foundation models to bridge languge commands andlow-level control dynamics. I hope these best known practices to inspire NLP researchers.

3. 

   SLiP: Situated-oriented Localization in Point Clouds

   Shaocong Xu, Xiaoxue Chen, Hao Zhao, and Guyue Zhou

   Technical Report 2023

   PDF