A real bug report
“Cannot enter any text in the SSID Filter field.”
Which screen is that?
Mobile bugs show up in the UI.
Fixing one starts with finding the buggy screen and component, often from a vague description.
≈26 screens. ≈17 components each.
That’s the average search space per bug report in the paper’s benchmark.
The first study of Buggy UI Localization
Can models find the buggy UI?
Four retrieval models, zero-shot, on 228 bug descriptions from 87 real bug reports.
QUERYCannot enter any text in the SSID Filter field
Rank the app’s screens.
The bug description is the query. Text models read each screen’s UI metadata; vision-language models look at its screenshot.
Lucene textSBERT textCLIP visionBLIP vision
Then its components.
Inside the buggy screen, components are ranked the same way. Here, the best models put the SSID field first.
Then find the code.
Source files tied to the localized screens get boosted in text-retrieval bug localization, fully automatically.
Studied on 39 Android apps
SCREENS · VISION WINS
0%
buggy screen in the top 3
BLIP, the best screen localizer
COMPONENTS · TEXT WINS
0%
buggy components in the top 3
SBERT, the best component localizer
And for buggy code localization
+0–0%
Hits@10, using the automatically localized screens
No single model wins both tasks: the paper calls for one that blends visual and textual UI information.
Toward the Automated Localization of Buggy Mobile App UIs from Bug Descriptions
Antu Saha · Yang Song · Junayed Mahmud · Ying Zhou · Kevin Moran · Oscar Chaparro
ISSTA 2024 · ACM SIGSOFT International Symposium on Software Testing and Analysis