Point and voxel cross perception with lightweight cosformer for large-scale point cloud semantic segmentation

> [!NOTE] > Click the Cite button above to demo the feature to enable visitors to import publication metadata into their reference management software.

Highlights

• PVCFormer: A novel point-and-voxel cross-perception network designed to effectively broaden the receptive field and improve object segmentation at various resolutions using cross-attention mechanisms.
• Lightweight Architecture: Introduction of CosFormer into point cloud processing to simplify computational complexity of transformers, reducing training time from 1200s to 800s per iteration without accuracy loss.
• SYSU9 Dataset: A new large-scale aerial point cloud dataset comprising approximately 200 million points, encompassing nine distinct feature classes and spanning over 7 square kilometers (Zhuhai campus of Sun Yat-sen University).
• State-of-the-art Performance:
  • SensatUrban: 92.4% OA, 61.5% mIoU
  • DALES: 94.6% OA, 73.6% mIoU
  • SYSU9: 91.1% OA, 62.4% mIoU

Methodology

The PVCFormer architecture consists of three main components:

1. Point-level Feature Extraction: Utilizes Point Transformer with frozen pre-trained parameters to extract fine-grained point features serving as Query (Q) in the cross-attention mechanism.

2. Voxel-level Feature Extraction (SPTNet Encoder):

   • Multi-resolution voxel sampling (e.g., 0.02m and 0.2m)
   • Sparse convolution blocks for local feature extraction
   • Transformer blocks for global context modeling
   • Outputs serve as Key (K) and Value (V)

3. CosFormer Block:

   • Cross-Attention: Fuses point and voxel features across different scales
   • Cos-Attention Mechanism: Replaces traditional softmax attention with cosine-based linear attention, reducing complexity from O(N²d) to O(Nd²)
   • Self-Attention & FFN: Standard transformer components for feature refinement

SYSU9 Dataset

A new benchmark dataset for large-scale outdoor point cloud semantic segmentation:

• Location: Zhuhai campus of Sun Yat-sen University
• Size: ~200 million points, 7 km² area
• Acquisition: RIEGL VUX-1LR UAV LiDAR (150m flight height, 400kHz scan frequency)
• Categories (9 classes):
  1. Roads
  2. Sidewalks
  3. Natural ground
  4. Trees
  5. Grass
  6. Vehicles
  7. Buildings
  8. Transportation facilities
  9. Man-made terrain

Experimental Results

Quantitative comparison on three benchmark datasets:

SensatUrban Dataset:

DALES Dataset:

SYSU9 Dataset:

Key Findings

1. Cross-attention Efficacy: Cross-attention between point and voxel features improves mIoU by 18.7% compared to voxel-only methods.
2. Pre-training Importance: Random initialization of Query leads to convergence failure; pre-trained Point Transformer is crucial.
3. Efficiency Gains: CosFormer reduces per-iteration time by 33% (1200s → 800s) compared to standard self-attention while maintaining comparable accuracy.
4. Small Object Detection: The method shows particular strength in detecting “Parking” and “Rail” categories due to cross-scale perception.