Code models write code.

They can’t tell you why.

A model stumbles on buggy code.

Accuracy drops. Do the bugs cause it?

Correlation isn’t causation.

A hidden confounder, like the number of tokens, can drive both.

Introducing
doCode
Causal explanations for neural code models
✓ Robustness checked
01Model
02Identify
03Estimate
04Refute

Draw your assumptions.

A structural causal model links an intervention to an outcome, and names the confounders.

T

Intervention

Buggy → fixed code, comments, clones, layers, units.

Y

Outcome

Cross-entropy and next-token predictions.

Z

Confounders

Software engineering quality metrics.

Cut the back door.

Graph surgery isolates the intervention: what Y does under do(T), with Z adjusted for.

ADJUSTMENT FORMULA · BACK-DOOR CRITERION
p(Y | do(T)) =
Σz p(Y | T, z) · p(z)

Estimate the effect.

From observational data, with no need to retrain hundreds of models.

AVERAGE TREATMENT EFFECT
ATE = E[Y|do(T=1)] − E[Y|do(T=0)]
Propensity scoresLinear regression

Then try to break it.

Refutation tests check that the effect survives broken assumptions.

Random common causeadd a random covariate
Unobserved common causesimulate a hidden confounder
Placebo treatmentswap T for random noise
Data subsetsdrop a random subset
Case study · RNNs and Transformers trained on code
0
neural code models
0
interpretability scenarios
0
architecture families
Buggy vs. fixed code

Strong correlation. No causal effect.

CorrelationJS distance · RNN, cross-entropy
0.00
Causal effectATE, controlling for confounders
≈ 0

ATEs of −3E−4, −2.33E−5 and −2E−4 across three models: the correlation was confounding bias.

What doCode found.

Type II clones

Small syntax changes really do move the model.

Clones causally explain cross-entropy shifts: ATEs of 0.63 (RNN) and 0.56 (GPT-2), too.

0ATE on cross-entropy
GRU
Number of layers

Strongly correlated. Barely causal.

Layers vs. predictions of [blocks] tokens.

0 correlation
0 ATE
BERT · masking AST nodes

Grammar isn’t fully captured.

Masking random tokens hurts predictions more than masking grammar-based node types.

The model doesn’t entirely capture AST structure.

Toward a Theory of Causation for Interpreting Neural Code Models

David N. Palacio · Alejandro Velasco · Nathan Cooper · Alvaro Rodriguez · Kevin Moran · Denys Poshyvanyk
TSE 2024 · IEEE Transactions on Software Engineering