Crepe: A Mobile Screen Data Collector Using Graph Query
Crepe is a credible CHI systems contribution: it reframes mobile screen data collection as selective, query-driven extraction rather than blanket capture. The Graph Query idea is the paper’s real novelty, and the evaluation is broad enough to show the approach works in practice, while the limitations are candid and important.
Axes Lens
Rare contribution shape, typical evidence profile. The point here is not a score. It is to show what kind of claim the paper makes, and whether the evidence pattern is unusual or baseline in this 268 -review set.
Contribution shape
- Knowledge form
- technical knowledge typical · 50/268
- Novelty type
- system architecture typical · 35/268
- Abstraction level
- system typical · 61/268
- Generalization target
- task class typical · 63/268
- Validation mode
- mixed methods typical · 136/268
Evidence profile
- Evidence strength
- strong typical · 158/268
- Claim alignment
- strong typical · 231/268
- Overclaim risk
- medium typical · 210/268
Review Summary
Crepe’s strongest contribution is not simply that it collects mobile screen data, but that it changes the unit of interaction from raw capture to selective, consent-aware extraction. The paper’s Graph Query idea is a meaningful systems contribution because it augments the accessibility-derived UI hierarchy into an enriched snapshot and then uses that structure to locate target content more precisely. That is a plausible and useful answer to a real CHI problem: academic researchers often lack access to screen-content datasets, while existing tools either focus on sensing or rely on manual screenshot workflows. The paper’s framing is therefore both practical and timely. The validation evidence is also reasonably convincing for a CHI paper of this type. The reported in-lab results show high collection accuracy across multiple app scenarios, and the field-style example suggests the approach can operate with limited perceived interruption. At the same time, the paper does not oversell universality: it explicitly notes that app redesigns, localization changes, unstable anchors, background termination, and non-textual rendering contexts can all constrain deployment. Those caveats matter because they define the method as a robust but bounded technique rather than a universal screen-mining solution. From a review perspective, the paper’s novelty is best understood as a system architecture and method package: a no-code Android collector plus a query layer for screen content. That is a solid CHI contribution because it combines technical design with a clear user-facing purpose and a privacy-sensitive deployment model. The main risk is overgeneralization beyond accessible, consented, mostly textual contexts. Within that boundary, however, the paper appears well grounded and appropriately positioned.
What Changed
Canon before
Prior CHI work on mobile data collection largely emphasized sensing streams, logging, or manual screenshot upload rather than a no-code, consent-centered collector for on-screen content. The paper positions itself against that baseline by targeting screen information extraction through demonstrations and graph-based UI querying.
Departure from common sense
The paper argues against the familiar choices of manual upload or continuous screen recording by collecting only when target data appears and only the target data itself. That is a notable shift in interaction and privacy framing, because it replaces broad capture with selective, event-triggered extraction.
Actual novelty
Crepe’s novelty is the Graph Query approach: it augments the Android accessibility-derived UI structure into an augmented UI Snapshot and uses that structure to identify, locate, and collect specific on-screen data. The contribution is not just a collector app but a queryable representation and extraction method for screen content.
Evidence
The evidence supports a system-level contribution with a concrete method for selective screen-data collection. The paper reports a controlled in-lab evaluation across three app scenarios with high collection accuracy, plus a field-style deployment example showing collection of Instagram Story ads with low perceived interruption. Limitations are explicitly acknowledged around app redesign, localization, background termination, and non-textual content.
“ Section A shows how to generate a Graph Query: Crepe first uses Android Accessibility Service to access the view hierarchy of a UI screen 1 and augment it into a UI Snapshot graph, using UI elements’ characteristics and their relations”
actual novelty · Abstract + Graph Query method description · confidence 0.52
“ Graph Query has the following characteristics: (1) Triggering a data collection only when the target data shows up; (2) Locating and accessing only the target data on screen; (3) Generalizing easily to diverse data types, especially dynamically changing data”
departure from common sense · Abstract/Introduction + Proposed Method overview · confidence 0.55
“ Information rendered in game or 3D engines cannot be collected, as they do not appear in Accessibility Service responses”
limitation · Limitations and Future Work (6.2, 6.3) · confidence 0.62
“ Breaking down by app, collection accuracy F-1 scores were consistently high: Uber (98”
validation scope · Evaluation (Study 2 and Study 3) · confidence 0.60
Limits
Method limits
The method depends on the stability and expressiveness of Android Accessibility-derived UI structures and on the correctness of Graph Query rules. The paper notes that app redesigns, localization changes, and unstable anchors can break queries, and that invisible UI elements can introduce noise.
Deployment limits
Crepe can be terminated by the system after inactivity, which can cause data loss. Its current scope is mainly textual data, and information rendered in game or 3D engines cannot be collected through Accessibility Service responses. The paper also frames use as research-only with explicit consent.
Boundary conditions
Best suited to consented research settings where target data appears in accessible UI hierarchies and where researchers can define demonstrations for the desired content. Less suitable when UI anchors shift frequently, when localization changes the structure, or when content is rendered outside Accessibility visibility.
Position in field
This sits at the intersection of mobile data collection, end-user programming/no-code tooling, and privacy-aware instrumentation. It extends beyond sensing frameworks by focusing on screen content and by proposing a structured query layer over mobile UI snapshots.