Association-Based Fairness Measures under Class and Group Skew

Fairness audits in binary classification assess the differences between protected and unprotected groups, using measures such as statistical parity, equal opportunity, predictive equality, and predictive parity. However, prior work on algorithmic fairness has shown that several of these measures exhibit substantial behaviour changes as class imbalance and protected-group ratios vary. This paper presents ongoing work on whether association-based measures can provide more stable fairness evaluation under three forms of skew: class imbalance, protected-group ratio, and stereotypical ratio. In this context, we adapt classical association coefficients to fairness tables and propose three new metrics: Fairness $\varphi$, Marginal Y Association, and Conditional Y Association. The first two measure independence between the prediction and the protected attribute, whereas the third aggregates within-label associations and is intended to reflect separation-style disparities. We study these metrics on synthetic confusion matrices and empirically on the Adult dataset. Preliminary results indicate that the association-based metrics, in particular Fairness $\varphi$ and Conditional Y Association, are less sensitive to extreme class imbalance and protected-group skew. The trade-off is that the definitions of these measures are harder to interpret on a conceptual level and, for the conditional variant, require smoothing. Nevertheless, these results indicate that association-based fairness metrics may be useful audit-oriented tools when comparing models across skewed datasets.