Built into nearly every stage of development. But they can miss what they claim to detect.
SOURCE
String q = request.getParameter("q");
log.info("search: " + q);
Cipher c = Cipher.getInstance("DES");NOT REPORTED
db.query(SQL_SEARCH, q);
return render(results);
SAST · ALERTS
!Possible injection · line 1FALSE POSITIVE
!Log forging · line 2FALSE POSITIVE
!SQL injection · line 4FALSE POSITIVE
!Unescaped output · line 5FALSE POSITIVE
A qualitative study · in-depth interviews
20 practitioners.
How do they choose security scanners, trust them, and work around their flaws?
FINDING · FALSE NEGATIVES FIRST
They’ll wade through false positives.
Nearly all preferred fewer false negatives, even at the cost of many false positives.
0%50%100%
20%upper bound proposed in the literature
80%+practical, for some practitioners
“I wouldn’t mind wading through 100 false positives, if I thought there were actually going to be genuine issues there.”— P02, open-source Java app server
FINDING · SELECTION
They pick tools by reputation.
Not by evidence that the tool finds real vulnerabilities.
recommendationsreputationease of usecorporate pressurecostcompliance
1 of 20
chose a SAST by testing 10–15 tools on a custom benchmark.
Benchmarks?
Distrusted: too basic to model real problems, or biased toward vendors’ tools.
FINDING · THE PARADOX
Someone else will catch it.
Security scanner
Manual review
covers reviewers’ blind spots
expected to catch what the tool missed
missed by both
Tools cover for reviewers.
Reviewers cover for tools.
So practitioners aren’t worried about unknown SAST flaws, and may assume the tools just work.
FINDINGS · AFTER THE MISS
Flaws stay hidden.
UNREPORTED
Few report SAST flaws.
Confidentiality, red tape, no incentive, and past reports that went nowhere.
EVASIVE DEVELOPERS
Bypassing checks is real.
Some had seen colleagues evade SAST checks under pressure, or had done it themselves.
0
findings that challenge long-held SAST design priorities