Segmentation Expert-Mixture Regularization: An Adaptive Learning Method for Imbalanced Regression Problems

Imbalanced regression poses a significant challenge for models across a diverse set of domains, where rare extreme cases are often the most important. Standard regression methods, which optimize global error objectives, tend to prioritize high-density regions of the target space, resulting in systematically degraded performance in low-density, extreme regions. Although prior work has focused on data-level strategies that modify the target distribution, comparatively little attention has been devoted to modifying the learning process itself, making it imbalance-aware. In this paper, we introduce Segmentation Expert-Mixture Regularization (SER), a novel algorithm-level framework for imbalanced regression. SER partitions the target space into regions of varying density and leverages a mixture-of-experts architecture to promote specialization across these regions. A regularization mechanism ensures smooth transitions between the built data partitions and provides a global coherence across segment boundaries. This ensures an adaptive and stable learning method over the entire target space. By integrating segmentation, expert specialization, and regularization within a unified learning framework, SER improves robustness and predictive performance, especially in the rare, extreme, and most important target cases. Our experiments show consistent improvements over standard models, particularly in extreme target quantiles. We further analyze the impact of segmentation design, parameter sensitivity, and performance variation across the target distribution. To foster reproducibility and future research, all our code is publicly released.