Press space or tap to play to the next stop · ← back · F fullscreen
ISSTA 2026 · Oakland, California
Automated Program Repair of
UI-centric Android Bugs
How Far are We?
JW
Fifteen years of automated program repair
APR has been evaluated almost entirely on library code.
Success has been largely dictated by unit tests.
Effectiveness on
UI-centric programs?
The UI is nearly half of user-facing software: half the code, half the time, half the maintenance.
Myers, CHI 1992
11:49▾ ◢ ▮
https://telefoncek.si/2024/05/2024-05-30-grapheneos…
Telefoncek.si
GrapheneOS and forensic extraction of data
Objavljeno: 29. 05. 2024
Harmonic has stoppedjava.lang.NullPointerException
at CommentsFragment.onItemClick
at CommentsFragment.onItemClick
CLOSE APP
‹ Settings
Theme
Font size
Open links in
Account
STATUSLogging stopped
FORMATSGPX · KML · CSV
LAST POINT28.6024, −81.2001
General options
Logging details
Performance
Auto send, email and upload
Custom URL
GPSLogger has stoppedjava.lang.NullPointerException
at GpsMainActivity.launchPreferenceScreen
at GpsMainActivity.launchPreferenceScreen
CLOSE APP
onLongClick → onItemClick()onClickATag → openLink()onLongClick → onItemClick()onClick → expand()
comments: List<Comment>
expanded: true
scrollY: 0
disableATagClick: false
expanded: true
scrollY: 0
disableATagClick: false
onCreate()onResume()onPause()onStop()onDestroy()
Inside a UI-centric app
LayoutViews and their hierarchy, declared in XML
Event listenersCode the framework calls on a tap or a long press
StateValues the screen keeps between events
LifecycleCallbacks when the screen is paused, destroyed, re-created
11:49▾ ◢ ▮
<TextView android:id="@+id/comment_body"
android:layout_width="match_parent" />The user sees what is declared by developers.
When the user acts, the framework decides which listener runs, and when.
Changes made by user actions need to be tracked in state.
Apps need to handle users rotating the screen, or leaving the app and coming back.
Four kinds of UI bugs
Johnson et al., AndroR2+ · 180 real Android bug reports
Crash
Output
Cosmetic
Navigation
Harmonic HN · issue #114 · one of the 50 real bugs
bugHarmonic HN · #114reproduced · Pixel 2
Long-tapping URL (to open comment menu) also opens the URL
1Long tap on a URL2The menu opens, and the link opens too
Expected · only the menuObserved · menu and link together
Expected behavior: the long press opens only the menu.
Observed behavior: the menu and the link open simultaneously.
Two listeners on one view that the UI framework controls.
The fix: implement state that persists.
CommentsFragment.java · onItemClick(comment, pos, view) · builds the context menu
AlertDialog.Builder builder = new AlertDialog.Builder(ctx);builder.setAdapter(adapter, (dialog, which) -> { … }); CommentsRecyclerViewAdapter commentAdapter = this.adapter;builder.setOnDismissListener(new DialogInterface.OnDismissListener() { @Override public void onDismiss(DialogInterface dialog) { commentAdapter.disableCommentATagClick = false; }});AlertDialog dialog = builder.create();commentAdapter.disableCommentATagClick = true;dialog.show();
CommentsRecyclerViewAdapter.java · ItemViewHolder · one comment row
public boolean disableCommentATagClick = false;itemView.setOnLongClickListener(v -> { commentLongClickListener.onItemClick(comment, pos, v); return true; });commentBody.setOnClickATagListener((widget, text, href) -> { if (disableCommentATagClick) return true; Utils.openLinkMaybeHN(widget.getContext(), href); return true; });
✓
Five repair tools. Not one produced a patch that passed the tests.
·RewardRepair·NTR·D4C·ChatRepair·OpenHands
Part two
DroidFixBench
A benchmark for program repair of UI-centric Android bugs
Building DroidFixBench
What we need to evaluate UI-centric program repair
bug
Compilable snapshot
Statement-level localization
Correct patch
GUI tests
Bug description
How a real bug is built
Gathering real bugs
AndroR2+80 reportsour bug-localization dataset
F-Droid apps30 issuesfive recently updated apps
FBO4A22 reportsJohnson et al.
Harmonic HN · #114
1
Build the buggy commit
Gradle versions, dead dependencies. Many never build.
2
Reproduce on the emulator
✓
Pixel 2 · Android 9 · follow the report's steps.
3
Localize the fault
Diff buggy vs fixed commit. Two authors mark the statements.
4
Write two GUI tests
test 1 · fail → passtest 2 · fail → pass
Written by one author, reviewed by another.
5
Validate
✗ buggy✓ fixed
Both tests fail on the buggy build and pass on the fixed one.
0real bugs from 23 apps
three sources, several person-months
three sources, several person-months
12 crash25 output9 cosmetic4 navigation
| Developer fix complexity | across the 50 bugs |
|---|---|
| Fixes in one method | 27 bugs |
| Fixes across several methods | 23 bugs |
| Lines added · removed | 414 · 290 lines |
| Files changed | 63 files |
| Largest single fix | 92 lines |
| started | built | reproduced | selected | |
|---|---|---|---|---|
| AndroR2+ | 80 | 42 | 35 | 35 |
| Five F-Droid apps | 30 | 23 | 21 | 13 |
| FBO4A | 22 | – | – | 2 |
Two GUI tests per bug, written by hand and checked by two authors.
Script114.java · UiAutomator · the bug-revealing test, as written
mDevice.swipe(500, 1600, 500, 800, 20); // scrollallStories.get(i).click(); // open the busiest storymDevice.swipe(540, y, 540, y, 100); // long press on the linkUiObject2 select = mDevice.wait( Until.findObject(By.text("Copy text")), 2000); // the context menu is upassertNotNull(select);✕
✓
buggy build · the browser covers the menu · assertion fails
fixed build · the menu is found · passes
Both tests are fail-to-pass: each fails on the buggy build and passes on the fixed one. Test 1 reproduces the report directly; test 2 exercises the same behavior through the surrounding UI, so a patch has to fix the bug, not just the one path.
| Emulator | Pixel 2 · Android 9 | Time per test | ≈ 2.5 min |
| Written by | one author, reviewed by another | Agreement | 86% real · 98% synthetic |
Synthetic bugs: Android mutants injected into real code.
GPSLogger · GpsMainActivity.launchPreferenceScreen() · MDroid+ operator: NullIntent
new Handler().postDelayed(() -> { Intent targetActivity = new Intent(getApplicationContext(), MainPreferenceActivity.class); Intent targetActivity = null; targetActivity.putExtra("preference_fragment", whichFragment); startActivity(targetActivity);}, 250);Description written from what we observed on the emulator
"Launch preference screen has stopped." Open the menu, tap Auto send, email and upload. Observed: the app crashes. Expected: the auto-send settings open.
46 bugs · 10 operators · 5 apps · every fix is one statement in one method
NullIntent 5DifferentActivityIntentDefinition 5LengthyGUICreation 5NullValueIntentPutExtra 5ViewComponentNotVisible 5InvalidIDFindView 5FindViewByIdReturnsNull 5InvalidKeyIntentPutExtra 5LengthyGUIListener 3BuggyGUIListener 3
19 crash8 output9 cosmetic8 delay2 navigation
Five repair techniques
Each bug, five tools
bugAnglers' Log · #1445
Editing a trip's date fails
Create tripEditChange dateSaveerror
for (UserDefineObject o : getTrips())
if (trip.overlapsTrip((Trip) o))
return true;
failing GUI testbug reportlocalized line
RewardRepair
Neural machine translation · a pre-trained model, trained with compile and test feedback · ICSE 2022
seq2seq modelpre-trained · not tuned on UI code
Buggy line and method in, patch out. No report, no tests.
NTR
Fix templates guide an LLM · CodeLlama · 2024
if (▢ condition ▢!trip.getId().equals(tr.getId())) return true;
template: insert a condition → the model fills the hole
Pick a repair template, let the model fill the hole.
D4C
One prompt, the whole function rewritten · GPT-4 · 2024
promptbuggy method
bug report
failing test
→ rewrite the function
→
response · the whole method · real patch 62for (UserDefineObject object : trips)
if (object instanceof Trip &&
trip.overlapsTrip((Trip) object))
return true;
One prompt, the whole function back.
ChatRepair
A conversation with test feedback · GPT-3.5 · ISSTA 2024
1. propose a patch
2. run the tests
3. feed the failure back into the prompt
… up to 100 tries on a real bug
2. run the tests
3. feed the failure back into the prompt
… up to 100 tries on a real bug
Patch, test, feed the failure back. Up to 100 rounds.
OpenHands
A software engineering agent in a sandbox · Qwen2.5-Coder · 2025
sandbox
cat app/src/main/java/com/cohenadair/anglerslog/model/Logbook.java… public static boolean tripExists(Trip trip) { …edit Logbook.java tripExists()./gradlew assembleDebugBUILD SUCCESSFUL in 1m 12sRead, edit, run, repeat. One patch per bug.
Patch evaluation methodology
Applysplice into the buggy snapshot
BuildGradle, the whole app
Emulator2 GUI tests · Pixel 2 · Android 9
Correctmatches the developer's fix
0
Plausible, not correctpasses, does not match
0
Overfitpasses only by targeting the tests
0
Test failurebuilds, fails a GUI test
0
Build failureor no usable patch
0
0patches · 1,457 synthetic · 1,592 real · two authors judge every plausible one
Results
Effectiveness on synthetic bugs
0 / 46
bugs fixed by at least one tool. 14 were fixed by four of the five.
CorrectPlausible, not correctOverfitTest failureBuild failure or no patch
Effectiveness on real bugs
Results overview
0 / 50
bugs fixed by at least one tool. D4C fixed the most, 8. The fixed bugs average 1.09 methods and 7.9 lines.
CorrectPlausible, not correctOverfitTest failureBuild failure or no patch
41/46
synthetic bugs fixed
11/50
real bugs fixed
0%of the patches for real bugs did not build
1,074 of 1,592. The tools reference things that do not exist in the project.
1,074 of 1,592. The tools reference things that do not exist in the project.
TrainerApp 1641 · the tools guess the app's vocabulary
bugTrainerApp · #1641
The slider's training type is never stored
Open the editorSet the typeSavevalue lost
What the app actually has · developer fix
Spinner s = findViewById(R.id.trainSpinner);schedule.setTrainingsType( TrainingsTypes.values()[(int) s.getSelectedItemId()]);
The report says "slider". The widget is a Spinner, and the id is nowhere in the method the tools were given.
D4C · a component ID that does not exist
activeSchedule().setTrainingType( ((SeekBar) findViewById(R.id.trainingTypeSeekBar)) .getProgress());
cannot find symbol · build fails
Ten patches, five different guessed ids: trainingTypeSlider, training_type_slider, trainingTypeSpinner…
OpenHands · methods that do not exist
String t = getTrainingTypeFromProgress(progress);activeSchedule.setTrainingType(t);appManager.getApplicationModel().saveSchedule(…);
cannot find symbol · build fails
A plausible-sounding API, invented on the spot.
D4C · a fix that does nothing
fab.setOnClickListener(v -> { saveTrainingScheduleInXml(); updateScheduleView(); });private void saveTrainingScheduleInXml() { // Logic to save training schedule state in XML file }
test fails
Two new methods, both empty.
NTR · FieldBook 1299 · an Activity that does not exist
intent.setClassName(ConfigActivity.this, CollectDataActivity.class.getName());startActivity(intent);
cannot find symbol · build fails
The real screen is CollectActivity.
The tools see one method and a report. They guess the rest: ids, methods, screens.
PSLab 2678 · an overfitting patch
Developer fix
+ degree = Math.round(seekArc.getProgress() * 3.6f);
D4C · passes both tests
degree = 0;
test 1 ✓test 2 ✓
0overfitting patches on the real bugs
9 always return the same boolean · 9 hard-code a value · 1 deletes characters from a string
9 always return the same boolean · 9 hard-code a value · 1 deletes characters from a string
Why 379 sampled patches failed
A statistically significant sample across all tools and both datasets · three annotators · nine categories · one bar = four patches
0%
159 of 379 failures are specific to UI code
Non-existent method call 37Non-existent GUI component ID 35Syntax errors 28Non-existent object 26Thread.sleep lowered, not removed 21Non-existent Activity 18Patch identical to the buggy code 18
Implications
1
Give the tool the GUI vocabulary
Component IDs, Activities, and Fragments should be inputs, not guesses. 55 of 379 sampled failures hallucinated one of them.
R.id.comment_bodyCommentsFragmentR.id.story_comments
prompt
2
Expose the project's APIs
52 failures call methods that do not exist or call them with the wrong arguments. The fix often lives one class away from the buggy method.
Trip.getId()Utils.openLinkMaybeHN(…)Trip.overlapsTrip(Trip)
prompt
3
Use the UI itself as context
Screenshots and recordings of the bug and the screens around it carry information that the report and the code do not.
prompt
4
Put the build in the loop
1,711 of 3,049 patches never compiled. Build before returning a patch, and feed the compiler's errors back.
patch
build
cannot find symbol R.id.foopatch, againBUILD SUCCESSFULSummary
96DroidFixBench: real and synthetic UI-centric bugs with snapshot, localization, patch, GUI tests, and description
5repair techniques, from a pre-trained translation model to an agent, on 3,049 patches
41/46synthetic bugs fixed, but only 11 of 50 real bugs
42%of failures are specific to UI code: hallucinated IDs, Activities, and APIs
Replication package
Benchmark, a Docker harness per tool, every generated patch, the codebooks and the taxonomy.
github.com/SageSELab/
UI-Program-Repair
doi: 10.5281/zenodo.21521939
Paper
Slides
Replication packageISSTA 2026 · Oakland, California
Automated Program Repair for UI-centric Android Bugs: How Far are We?