Mitigating Content Effects on Reasoning in LLMs Through Fine-Grained Activation Steering

Mitigating Content Effects on Reasoning in Language Models Through Fine-Grained Activation Steering.

Research question

The paper asks whether reasoning errors caused by content effects can be reduced at inference time by steering a model’s internal activations. More specifically, it studies whether models can be pushed to rely more on formal logical validity and less on the superficial plausibility of the content, especially in syllogistic reasoning tasks. 

Methodology

The authors build a controlled syllogistic reasoning dataset designed to separate logical validity from content plausibility, then use probing to identify where these kinds of information are represented in the model. They apply fine-grained activation steering to modulate those internal representations at inference time and evaluate whether this improves formal reasoning performance without retraining the model. 

Findings

The paper finds that activation steering can reduce the influence of misleading content and improve reasoning based on formal structure rather than plausibility alone. The results suggest that content bias is at least partly tied to identifiable internal representations, and that targeted intervention on those representations can improve reasoning behavior. 

Limitations

A main limitation is that the study focuses on a controlled syllogistic setting, so it is not yet clear how well the approach generalizes to broader reasoning tasks or more realistic natural-language settings. It also depends on successfully locating and steering the relevant internal representations, which may be harder for more complex tasks or across different model families. 

Why it’s important

This paper matters because it shows a promising route for improving reasoning without retraining, using test-time control over internal model states. More broadly, it helps connect mechanistic interpretability to practical alignment by showing that some reasoning biases can be traced to, and mitigated within, the model’s internal computation.