µSE found 13 flaws in FlowDroid.

So we kept digging.

The journal extension of our USENIX Security ’18 study.

More apps. Many more mutants.

0
compilable mutants seeded in 8 new apps
0.00×
more compilable mutants from the original 7 apps
+1
new scheme: ScopeSink, leaks across visibility scopes
Two more detectors, analyzed in depth
Argus
formerly AmanDroid · v3.1.2
0
new flaws
5in RecyclerView adapters and view holders
4around Fragment click events
HornDroid
the first with a formal proof of soundness
0
new flaws
Missed a leak placed entirely inside Activity.onCreate()
A new flaw class: lifecycle callbacks

Only 4 of 31 mutated APKs could be analyzed without a crash or timeout.

0
undocumented flaws across three detectors
13FlowDroid
9Argus
3

13 from USENIX ’18 · 12 new: 9 in Argus, 3 in HornDroid

Every flaw shows up in more than one tool.

Even between tools that share no code. A shared goal is enough to share a blind spot.

And fixed flaws come back.

FlowDroid accepted our Fragment patch — yet every later release we tested is still affected.

v2.0PATCH ACCEPTED
v2.5STILL AFFECTED
v2.5.1STILL AFFECTED
v2.7.1STILL AFFECTED

Soundness needs continuous evaluation.

Systematic Mutation-based Evaluation of the Soundness of Security-focused Android Static Analysis Techniques

Amit Seal Ami · Kaushal Kafle · Kevin Moran · Adwait Nadkarni · Denys Poshyvanyk
TOPS 2021 · ACM Transactions on Privacy and Security