Change one method.
What else has to change?
Impact analysis is hard to automate.
Structural tools flag too much. History and execution data are often missing, or costly.
And the benchmarks were tangled.
Fewer than ten projects, built from commits that mix bug fixes with unrelated changes.
Introducing
Athena
Code semantics + dependence graphs
callssame class● colour = learned meaning
Map every dependence.
A static graph of the system’s methods: who calls whom, and who shares a class.
public boolean isValid(String domain) {
...
domain = unicodeToASCII(domain);
DomainValidator.isValid calls unicodeToASCII: a call edge, plus a same-class edge.
Read every method.
GraphCodeBERT, fine-tuned on code search, turns each method into a vector of its intent.
UrlValidator.isValid(String value)
Let meaning flow.
Each embedding absorbs its neighbors’, up to two hops away, so local semantics gain global context.
M′ = (I + w · D-½(Acall + Aclass)D-½) M
unicodeToASCII inherits meaning from isValid, the method that calls it.
Rank the impact.
Cosine similarity to the query ranks every method in the system.
WHERE DOES unicodeToASCII LAND?
LSIbest conceptual baseline
#589
GraphCodeBERT aloneno propagation
#137
Athenasemantics + dependence graph
#137
Alexandria · a new, untangled impact-analysis benchmark
0
open-source Java projects
0%
of tasks surface a truly impacted method in the top 10
+10.34%mRR over LSI, the best conceptual baseline
+9.55%mAP over LSI, with statistical significance
24 of 25projects where Athena improves mAP
Why untangling matters
Tangled commits blur the truth.
On a tangled copy of the benchmark, every model scores lower: mRR drops by 3.80–5.62% across the whole-system setting.
Enhancing Code Understanding for Impact Analysis by Combining Transformers and Program Dependence Graphs
Yanfu Yan · Nathan Cooper · Kevin Moran · Gabriele Bavota · Denys Poshyvanyk · Steve Rich
FSE 2024 · ACM International Conference on the Foundations of Software Engineering