Research built many Android testers.

Developers still test by hand.

Because most tools ask too much.

✕Instrument the app, or the OS
✕Fight a difficult setup
✕Decode raw logs and stack traces
Introducing
CrashScope
Upload an APK. Get crash reports.
01Upload
02Test at scale
03Read the report

Just an .apk.

No instrumentation of the app or the OS. Pick the strategies and press start.

Every strategy.
In parallel.

A message queue feeds a pool of devices and emulators. A crash doesn’t stop the run.

A report.
Not a log.

Screen flow, plain-English steps, a pruned stack trace, and a replay script.

crashscope · new taskcrashscope · testing dashboardcrashscope · crash report

New Task

Upload an Android app and choose how CrashScope explores it.
Drop an .apk here (source code optional)
app-release.apk
GUI TRAVERSAL
✓DFS top-down
✓DFS bottom-up
TEXT INPUT
✓No text
✓Expected
✓Unexpected
CONTEXTS
✓Network on / off
✓Sensor values
Start testing

Testing Dashboard

Submitted tasks and the progress of those running.
app-release.apkRUNNING
web appCouchDB queueexecution engine
Pool of physical devices or Android virtual devices (android-x86 + VirtualBox)

Crash Report

app-release.apk · strategy: DFS top-down · unexpected text
1Tap on “Add” button, which is located on the bottom right of the screen.
2Type “%#@!” on the “Amount” text field, which is located on the top of the screen.
java.lang.NumberFormatException at …AddEntryActivity.onSave(…)
Download replay script
Preliminary studies
0
open-source apps
0
tools compared, including Monkey
0
participants reproducing real crashes
About as many crashes as the state of the art.With fewer false positives caused by instrumentation.
As reproducible as human-written reports.And more readable and useful, from a developer’s perspective.

CrashScope: A Practical Tool for Automated Testing of Android Applications

Kevin Moran · Mario Linares-Vásquez · Carlos Bernal-Cárdenas · Christopher Vendome · Denys Poshyvanyk
ICSE 2017 · IEEE/ACM International Conference on Software Engineering · Demonstrations