A bug report is only as good as

its steps to reproduce.

No More Than 1 Service Interval at a Time⁇?

What steps will reproduce the problem?

1. Change the phone setting

2. Open Mileage Tracker

3. Navigate to the “Service Intervals” screen

4. Tap on “Add Service Interval” button

Actual behavior

[…] I entered the information for my next oil change. No problem. Then I added a second service… my yearly State Inspection. But, when I entered the second one, it erased the first one. […]

Reporters write steps in their own words.

Steps get skipped, merged, or don’t match the app. Linking that language to the GUI is where tools struggle.

Introducing
AstroBR
An LLM that checks steps against the real app.

No More Than 1 Service Interval at a Time⁇?

Steps

1. Change the phone setting

2. Open Mileage Tracker

3. Navigate to the “Service Intervals” screen

4. Tap on “Add Service Interval” button

Actual behavior

[…] I entered the information for my next oil change. No problem. Then I added a second service… my yearly State Inspection. But, when I entered the second one, it erased the first one. […]

01Find the steps
02Walk the app
03Grade each step

Find every step.

GPT-4 picks out the sentences that describe steps, even buried in the behavior, and splits them into single actions.

“I entered the information for my next oil change”
actionenter
objectinformation
prepositionfor
object 2next oil change

Walk the app.

On a graph of the app’s screens, GPT-4 guides a depth-first search for the actions each step means, then fills the gaps between them.

GPT-4-guided DFS

Grade every step.

1Change the phone settingVM
2Open Mileage TrackerCS
+Open the app menuMS
3Navigate to the “Service Intervals” screenCS
4Tap on “Add Service Interval” buttonCS
5I entered the information for my next oil changeAS
→ Enter “Oil change title” · Enter distance on odometer field · Tap “Add service interval”
CScorrectASambiguousVMvocabulary mismatchMSmissing step
Against Euler, the prior state of the art · 21 bug reports, 73 step sentences, 5 apps
+0%
better step-quality labels
F1 0.879 vs. 0.702
+0%
better missing-step suggestions
F1 0.639 vs. 0.373
158 missing steps suggested, 101 correct. Euler: 271 suggested, 80 correct.

Combining Language and App UI Analysis for the
Automated Assessment of Bug Reproduction Steps

Junayed Mahmud* · Antu Saha* · Oscar Chaparro · Kevin Moran · Andrian Marcus
* co-first authors
ICPC 2025 · IEEE/ACM International Conference on Program Comprehension