Five steps written.

Thirteen actions needed.

Open GnuCash · bug report

Bug: Long pressing the amount input brings up QWERTY keyboard

Software specifications
GnuCash 2.2.0 · Android 6.0 · Moto G (2nd gen)
Steps to reproduce
1Navigate to Transactions screen3 actions
2Tap the Add button1 action
3Enter Description (optional)optional
4Focus the Amount input1 action
5Long press to bring up the context menu1 action
GUI actions to reproduce it
0
found by reproducing it by hand
6 described7 in no step at all

To reproduce it, someone had to infer the rest.

An empirical study

What does it take to reproduce

a real Android bug report?

0
bug reports, each reproduced by hand
=
0
from users
+
0
from developers
EnvironmentSteps to reproduceObserved behavior
Finding 1 · Missing steps
0%
of reports skip at least one step.

The missing steps leave 43.2% of the needed GUI actions undescribed.

Finding 2 · Vague steps
0%
of reports have a step that hides several actions.

On average, 36% of a report’s steps map to more than one GUI action.

Finding 3 · Failure types

Only 28% are crashes.

Yet many bug report analysis techniques focus solely on crashes.

33%OUTPUT
31%COSMETIC
28%CRASH
8%NAV.
Finding 4 · Beyond the steps
54reports gave no environment details at all
6ways of reporting, from plain text to screen recordings
35discussion threads added details needed to reproduce

Automated reproduction must reason

about what reports leave out.

An Empirical Investigation into the Reproduction
of Bug Reports for Android Apps

Jack Johnson · Junayed Mahmud · Tyler Wendland · Kevin Moran · Julia Rubin · Mattia Fazzini
SANER 2022 · IEEE International Conference on Software Analysis, Evolution and Reengineering