Research Projects

This page outlines my ongoing research projects. My current research focus, and the topic of my dissertation, relates to supporting mobile software developers in building, testing, and maintaining applications.  Additionally, I conduct research with other members of the SEMERU group in the general areas of software testing and maintenance, and applying deep learning techniques to software engineering tasks.

ReDraw - Using Artificial Intelligence to Automatically Prototype Mobile Applications

It is common practice for developers of user-facing software to transform a mock-up of a graphical user interface (GUI) into code. This process takes place both at an application’s inception and in an evolutionary context as GUI changes keep pace with evolving features. Unfortunately, this practice is challenging and time-consuming. In this paper, we present an approach that automates this process by enabling accurate prototyping of GUIs. Our approach called ReDraw combines automated software repository mining, dynamic analysis, and deep learning techniques to automatically convert images of a mobile app UI into highly accurate code.

MDroid+ Enabling Empirically Driven Mutation Testing for Android Apps

Mutation Analysis has arisen as a promising means of evaluating the effectiveness of software test suites.  However, traditional mutation operators tend to induce relatively simple bugs into software programs, and the extent to which the induced bugs are representative of real faults is an active topic of study.  With the MDroid+ project, we aim to help enable mutation testing for Android apps by empirically deriving a comprehensive set of mutation operators specific to the domain of Android apps.

Fusion - Improving Bug Reporting for Mobile Applications

Bug reporting systems have not changed significantly in recent years. Despite striking advancements in program analysis techniques, reporters typically enter textual information to describe a bug.  However, this type of report has been shown to be woefully inadequate for developers looking to reproduce and fix reported bugs.  The goal of the Fusion project is to leverage static and dynamic program analyses to improve the bug reporting process and produce higher quality reports with more detailed information, while requiring less effort from reporters.

ODBR - On Device Bug Reporting for Android Apps

Bugs that surface in mobile applications can be difficult to reproduce and fix due to several confounding factors such as varyiable contextual states, and device fragmentation.  Developers need support in the form of automated tools that allow for more precise reporting of application defects in order to facilitate more efficient and effective bug fixes.  The ODBR tool leverages the uiautomator framework and low-level event stream capture to offer support for recording and replaying a series of input gesture and sensor events that accuratley describe a bug in an Android application.

GVT - Automatically Verifying GUI Designs for Mobile Applications

The inception of a mobile app typically takes form of a mock-up of the Graphical User Interface (GUI), represented as a static image delineating the proper layout and location of GUI widgets that satisfy requirements. Following this initial mock-up, the design artifacts are then handed off to developers whose goal is to accurately implement these GUIs and the desired functionality. Given the sizable abstraction gap between mock-ups and code, developers often introduce mistakes related to the GUI that can negatively impact an app’s success in highly competitive marketplaces. The goal of the GVT project is to provide automated support for detecting and reporting these design violations in mobile apps.

GCat - Automated Summarization of GUI-Changes in Mobile Apps

Due to the rapid evolution of mobile apps, developers need automated support for documenting the changes made to their apps in order to aid in program comprehension. One of the more challenging types of changes to document in mobile apps are those made to the graphical user interface (GUI) due to its abstract, pixel-based representation. We developed a fully automated approach, called Gcat, for detecting and summarizing GUI changes during the evolution of mobile apps. Gcat leverages computer vision techniques and natural language generation to accurately and concisely summarize changes made to the GUI of a mobile app between successive commits or releases.

CrashScope- Effective Automated Testing for Android Applications

Automated testing techniques for Android exhibit notable shortcomings including: (i) A lack of expressive fault reports, (ii) lack of testing for contextual features (e.g. GPS, network), (iii) multiple input generation strategies.  CrashScope aims to overcome these shortcomings by using static analysis to identify GUI-specific locations where contextual features exist and multiple input generation strategies to effectively test these locations and uncover crashes.  When the tool crashes a target application, it generates an expressive report with the steps for reproduction and a repayable test script. Thus, CrashScope is an effective and practical automated testing tool for Android.

Research Funding

Press Coverage

This list of news stories outlines the press that my research has received from various media outlets.

• Smart Home Security - via The Washington Post

  • "This is just one scenario demonstrating one of many inherent flaws that computer scientists at the College of William and Mary discovered in internet-connected smart home devices during tests they conducted over the summer."

• Smart Home Security - via Quartz

  • "According to a new study by computer scientists at The College of William & Mary, even seemingly benign smart home devices, like smart plugs or lightbulbs, can provide entry points for hackers.”

• Smart Home Security - via NewsRadioWINA

  • "Researchers and study author Kaushal Kafle, of the College of William & Mary, talks with Les Sinclair about their discovery of potential vulnerabilities to hack your smart devices in your home, making your home it easier to break into your house."

• CrashScope - via iPRogrammer

  • "In testing it was discovered that CrashScope is as good as existing automatic testing tools in provoking a crash but it was better at reporting and reproducing the crash. The whole process is completely automated. You submit your apk to CrashScope and it generates suitable inputs which are recorded."

• Fusion - via The Register

  • "Auto-completion systems that attempt to finish your sentences when typing text messages or search queries can be a mixed blessing. Often, they save time. But they can also get in the way when they make incorrect guesses about intended input.  In the context of software bug reporting, however, auto-completion – adding additional information to bug report filings – doesn't have much of a downside."

• Fusion - via IEEE Software Blog

  • "The SEMERU research team has taken the first step towards addressing these problems by developing a novel bug reporting mechanism called Fusion that operates under the following key insight: automated program analysis techniques can be used to bridge the lexical knowledge gap between reporters and developers."

• SEMERU Lab - via W&M News

  • "The way humans use language informs how we process information and the same goes for computers, Poshyvanyk says. Just like human speech, computer language — or source code — has its own syntax and semantics. Poshyvanyk, an associate professor in William & Mary’s Department of Computer Science, has been working to bridge that human-to-computer language gap for the better part of the past decade. He and a team of nine W&M students are researching the ways code can mirror human communication."

Other Software & Side Projects

• OpenTerm-Mac

  • This is a small Mac app that can be placed in Finder and allows for the opening of a terminal window with a pwd that corresponds to the current Finder window.

• NewFile-Mac

  • This is a small Mac Application that can be placed in the Finder menu. When clicked on, the application creates a new text file in the directory displayed in the Finder window.

• OmniFocus Calendar Sync

  • This is a mac applescript that will sync Omnifocus tasks to a specified calendar using the built-in Mac calendar application.