Designing an app screen is hard.
Start with real ones.
The ideas already exist.
Thousands of apps on Google Play, screen after screen.
But you can’t search them.
Image search can’t filter by screen type, GUI component, or color.
Introducing
Guigle
A search engine for Android app screens.
EXPLORING · DEPTH-FIRST
TOP-6 COLORS · HSL · PROPORTION
Guigle
TOP-10 RESULTS
Explore every app.
Each app is run and explored depth-first. Every GUI event saves a screenshot and its GUI hierarchy.
Screenshot
What the user sees.
<node class="EditText"
bounds="[40,610][680,700]">
uiautomator XML
Every component, its type, text and bounds.
Drop the noise.
Home screens, tutorial overlays and container-only layouts are filtered out.
Image filtering
Launcher screens, and overlays found by color-histogram analysis.
Layout filtering
Screens with nothing but containers are discarded.
Index what’s on screen.
Lucene indexes each screen’s app name, component types and text. Its top-6 colors are stored in HSL.
Searchable fields
appnameuitextcolor
Search in plain words.
A lightweight query language sorts every word into a field. AND is the default.
QUERY
red edittext pizza
↓ PARSED
color:red AND ui:edittext AND
(text:pizza OR appname:pizza)
Guigle indexes
0%
of returned screens were relevant, on average
Precision over the top-10 results of each query.
What 13 developers found, across five design tasks
Easy to learn.
Useful for designing new screens.
Similar screens and search-by-color stood out as most useful.
Guigle: A GUI Search Engine for Android Apps
Carlos Bernal-Cárdenas · Kevin Moran · Michele Tufano · Zichang Liu · Linyong Nan · Zhehan Shi · Denys Poshyvanyk
ICSE 2019 · ACM/IEEE International Conference on Software Engineering · Demonstrations