G2P augments points with Gaussian attributes while preserving geometry.(a)Point Cloud. Traditional point cloud representations lack
sufficient appearance information to distinguish objects with similar geometry.
As shown in the prediction, the existing model fails to segment coplanar windows
cleanly and misclassifies the metallic refrigerator
(yellow boxes), revealing geometric bias from
insufficient appearance cues. (b)GS-Augmented Point Cloud (ours).
Through our novel Gaussian-to-Point feature augmentation, G2P enriches points
with Gaussian attributes (Σ, α), which provide Gaussian-derived
visibility and scale cues. This enables successful segmentation of both
challenging cases, mitigating the geometric bias problem.
Abstract
Point cloud segmentation is critical for 3D scene understanding.
However, sparse and irregular point distributions provide limited appearance evidence,
making geometry-only features insufficient to distinguish objects with similar shapes but distinct appearances (\eg, color, texture, and material).
We propose Gaussian-to-Point (G2P), which transfers Gaussian attributes from 3D Gaussian Splatting to point clouds for more discriminative and appearance-consistent segmentation.
Our G2P addresses the misalignment between optimized Gaussians and original point geometry by establishing point-wise correspondences.
By distilling opacity-derived visibility cues, we mitigate the geometric ambiguity that limits existing models.
Additionally, Gaussian scale attributes enable precise boundary localization in complex 3D scenes.
Extensive experiments demonstrate that our approach achieves competitive performance on standard benchmarks and shows notable improvements on geometrically challenging classes,
without pretrained 2D features or language supervision in our segmentation pipeline.
Code will be released soon.
Method
G2P runs in two stages: an offline preparation stage that augments points with
Gaussian attributes and derives boundary pseudo-labels, and a training stage that
jointly learns semantics, boundaries, and distilled appearance cues. At inference, G2P is a
Gaussian-free, point-only model.
Preparation Stage
Gaussian-to-Point Feature Augmentation aligns each point
(μp, c, n) with its nearest 3D Gaussians (μg, Σ, α) using a
covariance-aware Mahalanobis distance, then aggregates their scale and opacity
into augmented features (S′, α′). An appearance encoder is pre-trained in a
self-supervised manner on (μp, c, α′), learning view-consistent appearance cues.
Scale-based Boundary Extraction
Points are color-coded by aggregated Gaussian scale magnitude (magenta = small,
yellow = large). Small scales concentrate along object boundaries and thin structures,
while large scales dominate planar regions. By pruning large-scale points on an
object-focused set, G2P derives scale-based boundary pseudo-labels that complement
semantic boundaries.
Training Stage
The point cloud (μp, c, n) is processed by a PT v3 backbone that jointly
predicts semantic segmentation and boundary maps. Boundary pseudo-labels
supervise the boundary head, while the pre-trained appearance encoder distills
opacity-derived visibility cues into the backbone — combining semantic, boundary,
and distillation supervision without any 2D or language input.
Experiments
Quantitative Results
ScanNet v2. G2P reaches 78.4 mIoU (+0.9 over PT v3), best among
geometric / GS-guided methods without external pre-training.
ScanNet200.36.6 mIoU / 83.8 OA (+1.4 over PT v3) on the 200-class
benchmark.
† reproduced by us · * external pre-training · Bold: best
Generalization to Other Benchmarks
G2P generalizes across datasets, reaching 48.7 mIoU on ScanNet++ and
55.9 mIoU on Matterport3D, both above the PT v3 baseline.
† reproduced by us · Bold: best
Qualitative Results on Challenging Classes
Compared with PT v3 and BFANet — which show category confusion (★) and incomplete
masks — G2P produces label-consistent and more complete predictions on doors,
windows, pictures, curtains, refrigerators, and shower curtains. Yellow boxes mark
regions where models differ.
Additional Visualizations
Distinguishable Classes (ScanNet v2)
On objects that share similar color or texture with their surroundings — cabinet, bed,
bookshelf, chair, sofa, and table — G2P separates objects more cleanly and
captures their full extent, where PT v3 and BFANet leak into adjacent structures or
leave regions unlabeled.
ScanNet++ and Matterport3D
From left to right: ground-truth labels, PT v3 baseline, G2P, and scale-based boundary
pseudo-labels (top: ScanNet++, bottom: Matterport3D). G2P yields clearer boundaries
and more accurate segmentation on thin and cluttered structures (yellow boxes).
Boundary Pseudo-labels and Predictions
Rows show input, scale-based boundary pseudo-labels, G2P's boundary predictions, final
segmentation, and ground-truth labels. The scale-based pseudo-labels closely follow
object contours, and joint semantic-boundary supervision suppresses texture-induced
noise in the final masks.
Citation
@article{song2025g2p,
title = {G2P: Gaussian-to-Point Attribute Alignment for Boundary-Aware 3D Segmentation},
author = {Song, Hojun and Song, Chae-yeong and Hong, Jeong-hun and Moon, Chaewon and
Kim, Soo Ye and Liao, Yiyi and Lee, Jaehyup and Park, Sang-hyo},
journal = {arXiv preprint arXiv:2601.03510},
year = {2025}
}
