From MobileBDD scenarios to executable Maestro flows
Mobile app quality can affect more than implementation cost.
Reported for Android and iOS apps in Q3 2024.
The challenge is turning readable behavioural intent into executable mobile UI tests.
Mobile GUI testing exercises the application through visible behaviour, not through source-code access.
Test generation needs a model of intended interaction, but creating and maintaining that model is difficult.
Constrained scenarios keep intent readable while giving the compiler structure to validate and transform.
This thesis is a BDD-inspired, model-based test generation approach: scenarios become an interaction model and then Maestro UI test flows.
S1 analysis + S2 synthesis
Can the compiler process and translate BDD scenarios into schema-valid Maestro YAML?
S3 runtime execution
Can generated flows reach a terminal verdict on Android and iOS?
S4 oracle comparison
Do generated Pass/Fail verdicts match the manual reference oracle?
RQ1 outcomegenerates schema-valid Maestro flows
RQ2 outcomereaches terminal verdicts on each platform
RQ3 outcomematches the manual Pass/Fail oracle
SR = success rate · V = validation set · VG1 = validation gate before held-out evaluation
Feature: Grammar example
As an actor
I want to provide controlled input
So that I can verify parser behavior
Scenario: Controlled input and assertion
Given actor launches app
When actor enters field "value"
Then subject "value" should be displayedDocument shell: Feature, Scenario, Given/When/Then
Controlled step categories
Quoted values and placeholders
Scenario tags and screenshot directives
Live diagnostics, AIM-aware completions, hover/go-to-definition into AIM, and generate/run commands from the editor.
AIM = App Interaction Model · maps reusable scenario terms to app-specific targets, selectors, and Maestro recipes
config
InputAIM file, platform, corpus path
Appliedconfiguration and AIM validation
Outputvalidated run plan
semantic gate
InputMobileBDD feature files
Appliedparse, tag filter, semantic resolution
Outputaccepted scenario model
generation
Inputresolved scenarios and AIM recipes
AppliedMaestro backend synthesis
Outputschema-valid Maestro YAML
execution
Inputgenerated and manual flows
AppliedMaestro CLI runtime execution
Outputverdicts, logs, screenshots, reports
agreement
Inputgenerated and oracle verdicts
AppliedPass/Fail verdict comparison
Outputagreement report and metrics
The pipeline separates compilation evidence from runtime evidence before comparing generated and manual verdicts.
MobileBDD input
@capability-saved-attractions
Feature: Visitors maintain a personal list of Salzburg activities
As a visitor
I want to maintain a personal list of attractions
So that I can revisit activities I plan to visit
@SCN-F5-01-P @flow-interaction-response @expected-pass @refine
Scenario: Saving Getreidegasse changes its state to Saved
Given a visitor has opened details for "Getreidegasse"
When the visitor saves the activity
Then the activity state "Saved" should be displayedMaestro YAML
appId: ${APP_ID}
name: "SCN-F5-01-P Saving Getreidegasse changes its state to Saved"
tags:
- "generated-s2"
- "capability-saved-attractions"
- "SCN-F5-01-P"
- "flow-interaction-response"
- "expected-pass"
- "refine"
---
- launchApp:
clearState: true
- assertVisible:
text: "Salzburg activities"
- tapOn:
id: "search_input"
- inputText: "Getreidegasse"
- tapOn:
id: "poi_row_poi_7"
- tapOn:
text: "Save to My List"
- assertVisible:
text: "Saved"The generated flow keeps scenario metadata and emits executable Maestro commands.
| Set | Role | Scenarios | Oracle |
|---|---|---|---|
| V | development / validation | 32 × target platform16 pass + 16 fail per platform | none required |
| E | held-out evaluation | 32 × target platform16 pass + 16 fail per platform | manual Maestro |
| Supplement | Forbes + KotlinConf app corpora | 2 × 32 × target platformportability probe corpus | portability probe |
Android + iOSGenerated flows are considered per target platform.
Per-platform runsExecutability and agreement stay separated by platform.
Manual oracleManual Maestro flows provide the reference where reported.
The current reported result is bounded to the Android SalzburgCard evaluation run.
0 disagreed / 0 missing / 0 non-executable
feature files accepted; 32 scenarios extracted
schema-valid Maestro artifacts generated
generated and manual Android flows reached terminal verdicts
generated terminal verdicts matched the manual oracle
Source: build/pipeline-runs/android/pipeline-report.json
These issues do not answer the RQs directly, but they explain the final pipeline shape.
iOS TextField cursor jumps, Android first-start screens, and password-manager prompts affected runtime stability.
Clean app state, explicit launch behaviour, and isolated manual oracle flows.
Free-form descriptions could hide misspelled step keywords; ANTLR predicates coupled grammar to Java.
Keep grammar structural; move typo diagnostics into Kotlin validation.
Per-file CLI validation increased runtime; expected-fail flows look like failures in raw Maestro output.
Tag filtering, expected-status reporting, streamed logs, and per-flow evidence.
Some targets required scrolling; scrollUntilVisible behaviour depended on speed and current view state.
Represent acquisition strategies in AIM recipes, not in the grammar.
Parallel local Maestro runs could collide on emulator, simulator, and driver channels.
Serialize local S3 execution; treat scaling as device sharding or cloud infrastructure.
SUT = system under test · ANTLR = ANother Tool for Language Recognition
MobileBDD captures behaviour; AIM captures app-specific targets; Maestro YAML captures execution.
S1 and S2 check generation; S3 checks technical executability; S4 checks verdict-level agreement.
Reuse belongs at the behavioural-specification level, but correctness must still be checked at the automation level.
A constrained Mobile Behaviour-Driven Domain language for readable interaction intent.
A command-line compiler that turns MobileBDD scenarios into Maestro test flows.
Editor diagnostics and feedback for grammar-level and validation-level constraints.
Tooling that supports interaction with the transformation pipeline from the editor.
The prototype establishes the path; the remaining work is mostly about scale, platforms, and robustness.
1Evaluation evidence is bounded to the available scenario corpus and reported platform runs.
2Execution stability still depends on app state, platform prompts, selectors, and device runtime conditions.
3The approach depends on explicit AIM recipes instead of automatic selector discovery.
1Extend the evaluation to more apps, larger corpora, and both mobile platforms.
2Investigate stronger acquisition strategies for selectors, scrolling, and runtime state.
3Study scalable execution infrastructure for repeated generated-vs-manual comparisons.
4Validate the approach with a target group of mobile developers and QA practitioners.
Questions?
MobileBDD to MaestroA compiler path from behavioural scenarios to executable mobile UI test flows.