Figure 1. While 3DGS achieves high fidelity in training viewpoints, it suffers from floating artifacts in novel viewpoints,
indicating poor generalization. Our method enhances rendering quality in novel views.
3D Gaussian Splatting (3DGS) [1] lacks generalization across novel viewpoints in a few-shot scenario
because it overfits to the sparse observations (Fig. 1).
We reformulate the 3DGS training objective, guiding 3DGS to converge toward a better generalization solution (Fig. 2).
Figure 2. Given eight training views, our method maintains low Average Error across interpolated novel views,
whereas the baseline exhibits overfitting. Plots are means and standard deviations over ten runs.
Motivation
Figure 3. Conceptual 1D Loss Landscape of Flat and Sharp Minima [2].
Flat minimum is known to promote better generalization than sharp minimum (Fig. 3).
Motivated by this tendency, Sharpness-Aware Minimization (SAM) [3] jointly minimizes both empirical loss and sharpness,
which is equivalent to minimizing the worst-case loss within a neighborhood of radius ρ.
The model parameters are updated with the local maximum loss. In practice, the local maximum is approximated
via first-order Taylor expansion, perturbing the parameters in the gradient direction with magnitude ρ.
Figure 4. High-frequency regions, such as edges, requires sharp minima for accurate reconstruction,
while low-frequency regions favor flat minima.
However, directly applying SAM to 3DGS is suboptimal
because the importance of sharpness for accurate reconstruction varies across pixels (Fig. 4)
Therefore, finding a flat minimum for all Gaussians is problematic
as it over-penalizes high-frequency regions and under-penalizes low-frequency regions.
Figure 5. In the low-frequency region, the perturbation is too weak to be meaningful,
and in the high-frequency region, the perturbation is too strong, resulting in an overshoot.
Moreover, with a uniform neighborhood radius,
SAM fails to capture the worst-case, leading to inaccurate sharpness estimation (Fig. 5).
To this end, we increase the perturbation magnitude for the Gaussian representing low-frequency
and reduce the perturbation magnitude for high-frequency ones.
Thereby, appropriately estimating sharpness (Fig. 6).
Figure 7. Frequency-adaptive sharpness weighting.
And we increase the regularization weight for the Gaussian representing low-frequency
and reduce the regularization weight for high-frequency ones.
Thereby, encourage a flatter loss landscape while retaining sharpness for fine details (Fig. 7).
Figure 8. Algorithm of Frequency-Adaptive Sharpness Regularization (FASR).
Fig. 8 summarizes our algorithm.
Convergence Behavior
Figure 9. To analyze the convergence behaviors of 3DGS, SAM, and Ours,
we visualize the reconstruction loss landscape and the corresponding optimization trajectories.
Interestingly, our method converges to sharper minima than SAM but achieves lower test loss (Fig. 9). This suggests that sharpness is not strictly correlated with generalization, supporting our hyphothesis.
To better generalize, retaining sharpness for fine detail is crucial while regularizing sharpness,
as supported by visual comparisons (Fig. 10).
Figure 10. Our method preserves fine details that SAM tends to oversmooth.
Quantitative Results
Figure 11. Quantitative evaluation on LLFF dataset (3 views).
Considering the randomness of 3DGS, we conduct experiments with ten independent runs.
As shown in Fig. 11, our method outperforms baselines across all metrics.
Visual Comparisons
Please move the slider in the center to compare the videos.
Reduce floater artifacts appearing in the center and left.
Eliminate flickering caused by the Gaussian near the camera.
Prevent sudden appearances of the Gaussian.
Correct the inaccurate geometry of the cable on the table.
Remove the floaters appearing in the top-left.
Consistent spots on the white concrete pot.
BibTeX
@misc{yun2025fasr,
title={Frequency-Adaptive Sharpness Regularization for Improving 3D Gaussian Splatting Generalization},
author={Youngsik Yun and Dongjun Gu and Youngjung Uh},
year={2025},
eprint={2511.17918},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2511.17918},
}
References
[1] Kerbl, et al. "3D Gaussian Splatting for Real-Time Radiance Field Rendering" ACM ToG (2023)
[2] Keskar, et al. "On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima" ICLR (2017).
[3] Foret, et al. "Sharpness-Aware Minimization for Efficiently Improving Generalization" ICLR (2021).
