Causal Reinforcement Learning for Labelling Optimization in Cyber Anomaly Detection

The application of machine learning (ML) for cyber anomaly detection has attracted significant research attention. However, existing detection systems often face major challenges, including rigid feature discretisation, black-box classification, biased learning from confounded data, and lack of robustness, which collectively compromise interpretability, fairness, and predictive accuracy. Causal inference offers a robust approach to estimating intervention effects by isolating spurious correlations from true cause-effect relationships, crucial for reliable decision making under uncertainty. In contrast, reinforcement learning (RL) enables agents to learn optimal adaptive policies through interaction with dynamic environments. To address the aforementioned challenges, this work proposes a paradigm that leverages a RL framework to drive causal inference into the anomaly detection pipeline. Specifically, an RL agent is trained to optimize binning thresholds for confounded numerical features, guided by a reward function that incorporates both causal effect estimation and predictive accuracy. This approach enables the agent to learn feature discretisation strategies that avoid spurious associations induced by confounders, resulting in thresholds that are both causally aware and statistically effective. The optimized binning policy is then applied to transform the dataset, and a decision tree classifier is trained on the resulting unbiased features. This produces a model that is interpretable, robust to confounding, and sensitive to causal structures. Experimental results show that the proposed approach improves robustness and interpretability in unseen environments. This work highlights the potential of combining causal reasoning with adaptive learning to produce high-performance, transparent, optimal feature discretisation, and bias-aware cyber defence models.