A numbered screen reference is a coordinate in an observation, not a permanent identity. If the UI moves, that reference can stop describing the element you meant.
Mobilebox rejects stale references rather than silently treating an old number as the new target. That is useful feedback, but the agent still needs a recovery rule. Repeating the same batch is rarely the best one.
Choose targets by what you know
| Target | Good use | Failure to watch for |
|---|---|---|
| text | A visible label such as “Save” | Duplicate or translated labels |
| id | A resource id exposed by the app | Different build or missing id |
| desc | An accessibility description | Missing or changed description |
| ref | A precise element in the latest screen | Screen changed since observation |
| x, y | Visual content absent from the tree | Layout or keyboard moved |
Text matching tries exact text, then contained text. Use nth when more than one element matches. Resource ids can be useful in an app you control. Coordinates should follow a screenshot of the current screen, not a remembered layout.
Recover from stale_ref
Observe again. Check the foreground app and the label or description of the intended element. Resolve a current target, then send the smallest next action. If you cannot identify the element from the tree, request a screenshot.
Do not create a new phone to repair a stale reference. The phone is still there; the observation became old. See the error guide for other failure classes.
Partial success changes the next call
Suppose a batch opens an app, taps a button, then fails to type because the field is not focused. The first two actions may already have happened. Repeating the full batch can navigate away or submit twice.
Read the per-action results and the returned screen. Tap or target the current field, type, then verify the value. A type_not_applied error exists to avoid declaring success when text did not reach the app.
A better agent instruction
“Use fresh observations. Prefer text or resource ids. After a failed action, inspect the returned state and continue from the last confirmed step. Before any irreversible action, verify the exact target and intended outcome.”
The agent guide gives the full control vocabulary. Keep the recovery policy alongside your workflow so a transient UI change does not become a repeated side effect.