While 3DGS achieves high fidelity on training viewpoints, it suffers from floating artifacts in novel
viewpoints,
indicating poor generalization. Our method enhances rendering quality in novel viewpoints (i.e.,
improves generalization).
Overview
We investigate the relationship between loss sharpness and generalization in novel view
synthesis, since pursuing flatter minima is widely known to improve generalization in deep learning.
However, loss sharpness and generalization are not strictly correlated in novel view synthesis.
Comparing with Sharpness-Aware Minimization (SAM), a representative flat-minima optimization method,
our method converges to sharper minima but achieves lower test loss, indicating better generalization.
This is because accurate reconstruction requires sharp minima in high-detail regions (e.g., edges)
but flat minima in low-detail regions for better generalization; thus, uniformly pursuing flat minima is
problematic.
To address this, we introduce structure-aware sharpness and adaptively regularize it:
in high-detail regions, we both reduce the regularization weight and the neighborhood radius
used for sharpness estimation (i.e., perturbation magnitude), while in low-detail regions, we increase them.
Our improvement is complementary to prior
methods
Slide to compare videos and click to play or pause
Eliminate flickering caused by the Gaussian near the camera.
Prevent sudden appearances of the Gaussian.
Correct the inaccurate geometry of the cable on the table.
Remove the floaters appearing in the top-left.
Consistent spots on the white concrete pot.
BibTeX
@misc{yun2025sasr,
title={Do Flat Minima Improve Sparse Novel View Synthesis?},
author={Youngsik Yun and Dongjun Gu and Youngjung Uh},
year={2025},
eprint={2511.17918},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2511.17918},
}