Different apps.

The same uses.

Developers want tests for real uses.

Signing in. Adding to cart. Yet most test generators chase crashes and code coverage.

Reusing tests? Write them first.

Test transfer needs existing, hand-written tests, ideally from a similar app.

Introducing
AVGUST
Learn a usage from videos. Test it on a new app.
01Watch the videos
02Abstract the usage
03Guide a new test
3,000+
frames in a 2-minute sign-in video

Keep only what matters.

V2S pinpoints every touch in the recording. Typing frames are dropped, so passwords never leave the video.

Abstract the usage.

Vision and text classifiers suggest a canonical label for each screen and widget; a developer confirms. Out comes an app-agnostic state machine.

37canonical screens
74canonical widgets

Walk it on a new app.

On an app it has never seen, AVGUST matches each screen to a state and recommends the widget to use next, until the usage is done.

Evaluated on
0
videos
0
popular apps
0
common usages
0%
of generated tests accomplish the desired usage
35 of 51tests on unseen apps, fully correct
77.4%of widget recommendations correct
Beats S2Vat screen & widget classification

AVGUST: Automating Usage-based Test Generation from Videos of App Executions

Yixue Zhao · Saghar Talebipour · Kesina Baral · Hyojae Park · Leon Yee · Safwat Ali Khan · Yuriy Brun · Nenad Medvidovic · Kevin Moran
ESEC/FSE 2022 · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering