“I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it.”

— Fred Brooks, No Silver Bullet (1986)

So automate the representing.

Let developers think.

Software speaks three languages.

Moving between them is where developers struggle.

Natural language
Aa
{ }
Code
▢
GUIs
A dissertation, in three tools
Design. Build. Test.
MOCK-UP
IMPLEMENTATION
ImageView
TextView
Button
Unfortunately, the app has stopped.
TEXT POSITION
COLOR · POSITION
<ImageView …/>
<TextView …/>
<Button …/>
1Tap on the image.
2Tap on the button at the bottom of the screen.
!Crash: pruned stack trace, replay script
Design · mock-up vs. app
GVT

Compares a mock-up with the implemented screen, the way a human eye would, and reports every design violation.

1,000+designers and developers at Huawei actively use it
Build · mock-up to code
ReDraw

Detects GUI components, classifies them with a CNN, and assembles them into a prototype app’s code.

91%average GUI-component classification accuracy
Test · app to bug report
CrashScope

Explores the app, stress-tests crash-prone features, and writes reports developers can reproduce.

61 appsevaluated against five state-of-the-art tools
What comes next
SHORT TERMGUIs that document themselves
LONG TERMGUI-centric program understanding & synthesis
LONG TERMPractical, comprehensive GUI testing

Automating Software Development for Mobile Computing Platforms

Kevin Moran
ICSME 2018 · IEEE International Conference on Software Maintenance and Evolution · Doctoral Symposium