Most bias auditing I’ve seen runs at the wrong end of the pipeline. You fine-tune, you stand up an expensive output benchmark or a judge model, and you hope the regression shows up in generated text. This paper argues the shift is often already sitting in the hidden states — before it ever reaches a token you can score.
| The obstacle is that you can’t compare raw hidden states across two checkpoints, because fine-tuning reshapes the representation geometry. So they encode each sentence by its similarity to a fixed set of anchor sentences, which drops both models into a shared comparison space, and then measure how a target group’s association with positive versus negative attributes moves. They call that delta the Representational Bias Shift. It correlates with output-level bias change in 15 of 18 settings — up to | r | = 0.84 under full fine-tuning — and thresholding it flags the checkpoints whose bias actually got worse with ROC AUC between 0.65 and 0.99. |
The reason I care is operational cost. This needs no task-specific eval data and audits a model in about three minutes, 3–50x cheaper than the output benchmarks it’s measured against. That’s the difference between a governance check you run once a quarter under duress and one that fires on every fine-tune in CI. The authors frame it as complementary to output-level auditing rather than a replacement, which is the honest read — a representational shift is a signal to go look, not a verdict. The HF paper page carries the stability analysis across anchor and attribute sets.
What I want settled before wiring this into a pipeline: does ΔB hold up under LoRA and other parameter-efficient adapters? The paper already admits it gets more model-dependent there — and PEFT is how most teams actually ship their fine-tunes.
tags: [ llm-ops ] [ research ]