AndroidNode

Interface

Accessibility tree nodes for UI element interaction

AndroidNode represents a node in the Android accessibility tree. Each node corresponds to a UI element on screen and contains properties like text, bounds, and state. Use methods like screenContent() to get the root node and traverse the tree.

Getting Screen Content

TypeScript

// Get the accessibility tree
const screen = await agent.actions.screenContent();

// Find elements
const button = screen.findTextOne("Submit");
const allInputs = screen.filterAdvanced(f => f.isEditText());

// Interact with elements
if (button) {
  const { left, top, right, bottom } = button.boundsInScreen;
  await agent.actions.tap((left + right) / 2, (top + bottom) / 2);
}

AndroidNodeFilter

Learn about the builder pattern for complex node queries

Properties

Identification

Property	Type	Description
`falseId`	`string`	UUID unique per request (changes each time screen content is fetched)
`viewId?`	`string`	Android view ID (e.g., 'com.example:id/button')
`className`	`string \| null`	Android class name (e.g., 'android.widget.Button')
`packageName`	`string`	Package name of the app owning this node

Layout

Property	Type	Description
`boundsInScreen`	`{left, top, right, bottom}`	Screen coordinates of the node's bounding box
`index`	`number`	Index within parent's children

Text Content

Property	Type	Description
`text?`	`string`	Text content of the node
`description?`	`string`	Content description (accessibility label)
`hintText`	`string \| null`	Hint text for input fields
`maxTextLength`	`number`	Maximum text length for input fields
`inputType`	`string`	Input type for text fields
`isShowingHintText`	`boolean`	Whether hint text is currently shown
`tooltipText?`	`string \| null`	Tooltip textAndroid 9+
`paneTitle?`	`string \| null`	Pane title for accessibilityAndroid 9+
`containerTitle?`	`string \| null`	Container titleAndroid 14+

State

Property	Type	Description
`isEnabled`	`boolean`	Whether the node is enabled
`clickable`	`boolean`	Whether the node is clickable
`isLongClickable`	`boolean`	Whether the node responds to long click
`isContextClickable`	`boolean`	Whether the node responds to context click
`isFocusable`	`boolean`	Whether the node can receive focus
`isFocused`	`boolean \| null`	Whether the node currently has focus
`isScrollable`	`boolean`	Whether the node is scrollable
`isSelected`	`boolean`	Whether the node is selected
`isChecked`	`boolean \| null`	Checked state (for checkboxes, switches)
`isEditable`	`boolean`	Whether text can be edited
`isDismissable`	`boolean`	Whether the node can be dismissed
`isPassword`	`boolean`	Whether this is a password field
`isMultiLine`	`boolean`	Whether this is a multi-line text field
`isVisibleToUser`	`boolean`	Whether the node is visible on screen
`isImportantForAccessibility`	`boolean`	Whether important for accessibility
`isHeading?`	`boolean`	Whether this node is a headingAndroid 9+
`isTextSelectable?`	`boolean`	Whether text can be selectedAndroid 13+
`isTextEntryKey?`	`boolean`	Whether this is a keyboard keyAndroid 10+

Collection Info

Property	Type	Description
`collectionInfo?`	`object`	Info when node is a collection (ListView, RecyclerView). Contains rowCount, columnCount, itemCount, hierarchical, selectionMode
`collectionItemInfo?`	`object`	Info when node is a collection item. Contains rowIndex, columnIndex, rowSpan, columnSpan, isSelected

Hierarchy

Property	Type	Description
`children`	`AndroidNode[]`	Child nodes
`parent`	`AndroidNode \| null`	Parent node (null for root)

Actions

Property	Type	Description
`actions`	`number[]`	Array of supported action IDs (use with agent.constants.ACTION_*)
`actionLabels`	`{id: number, label?: string}[]`	Action labels with optional custom text

Methods

`allNodes()`

TypeScript

allNodes(): AndroidNode[]

Returns all nodes in the subtree as a flat array, including this node and all descendants.

Returns

AndroidNode[]Flat array of all nodes

Examples

TypeScript

const screen = await agent.actions.screenContent();
const allNodes = screen.allNodes();
console.log("Total nodes:", allNodes.length);

// Find all visible text
const textNodes = allNodes.filter(n => n.text && n.isVisibleToUser);

`findById()`

TypeScript

findById(id: string): AndroidNode[]

Finds all nodes with the matching viewId.

Parameters

Name	Type	Description
`id`	`string`	View ID to search for (e.g., 'com.example:id/button')

Returns

AndroidNode[]Array of matching nodes

Examples

TypeScript

const buttons = screen.findById("com.example:id/submit_button");

`findByIdOne()`

TypeScript

findByIdOne(id: string): AndroidNode | null

Finds the first node with the matching viewId.

Parameters

Name	Type	Description
`id`	`string`	View ID to search for

Returns

AndroidNode | nullFirst matching node or null

Examples

TypeScript

const button = screen.findByIdOne("com.example:id/submit");
if (button) {
  await button.performAction(agent.constants.ACTION_CLICK);
}

`findText()`

TypeScript

findText(textOrRegex: string | RegExp): AndroidNode[]

Finds all nodes containing the text in description, text, or hintText (case-insensitive). Supports strings and regular expressions.

Parameters

Name	Type	Description
`textOrRegex`	`string \| RegExp`	Text to search for or regex pattern

Returns

AndroidNode[]Array of matching nodes

Examples

Find by string

TypeScript

const nodes = screen.findText("Submit");

Find by regex

TypeScript

const priceNodes = screen.findText(/\$\d+\.\d{2}/);

`findTextOne()`

TypeScript

findTextOne(textOrRegex: string | RegExp): AndroidNode | null

Finds the first node containing the text (case-insensitive).

Parameters

Name	Type	Description
`textOrRegex`	`string \| RegExp`	Text to search for or regex pattern

Returns

AndroidNode | nullFirst matching node or null

Examples

TypeScript

const submit = screen.findTextOne("Submit");
if (submit) {
  const { left, top, right, bottom } = submit.boundsInScreen;
  await agent.actions.tap((left + right) / 2, (top + bottom) / 2);
}

`find()`

TypeScript

find(predicate: (node: AndroidNode) => boolean): AndroidNode | null

Finds the first node matching a custom predicate function.

Parameters

Name	Type	Description
`predicate`	`(node: AndroidNode) => boolean`	Function to test each node

Returns

AndroidNode | nullFirst matching node or null

Examples

TypeScript

// Find first clickable button with specific text
const btn = screen.find(node =>
  node.className === "android.widget.Button" &&
  node.clickable &&
  node.text === "Submit"
);

// Find first visible input field
const input = screen.find(node =>
  node.isEditable && node.isVisibleToUser
);

`filter()`

TypeScript

filter(predicate: (node: AndroidNode) => boolean): AndroidNode[]

Filters all nodes matching a custom predicate function.

Parameters

Name	Type	Description
`predicate`	`(node: AndroidNode) => boolean`	Function to test each node

Returns

AndroidNode[]Array of matching nodes

Examples

TypeScript

// Find all buttons
const buttons = screen.filter(node =>
  node.className === "android.widget.Button"
);

// Find all clickable, visible nodes
const clickables = screen.filter(node =>
  node.clickable && node.isVisibleToUser
);

// Find nodes with text containing dollar amounts
const prices = screen.filter(node =>
  node.text?.includes("$")
);

`findAdvanced()`

TypeScript

findAdvanced(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): AndroidNode | null

Finds the first node using the AndroidNodeFilter builder pattern.

Parameters

Name	Type	Description
`filterBuilder`	`(f: AndroidNodeFilter) => AndroidNodeFilter`	Function to build the filter

Returns

AndroidNode | nullFirst matching node or null

Examples

TypeScript

// Find the submit button
const submit = screen.findAdvanced(f => f.isButton().hasText("Submit"));

// Find first editable text field
const input = screen.findAdvanced(f => f.isEditText().isEditable());

// Find by view ID
const header = screen.findAdvanced(f => f.hasId("com.example:id/header"));

`filterAdvanced()`

TypeScript

filterAdvanced(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): AndroidNode[]

Filters all nodes using the AndroidNodeFilter builder pattern.

Parameters

Name	Type	Description
`filterBuilder`	`(f: AndroidNodeFilter) => AndroidNodeFilter`	Function to build the filter

Returns

AndroidNode[]Array of matching nodes

Examples

TypeScript

// Find all clickable buttons
const buttons = screen.filterAdvanced(f => f.isButton().isClickable());

// Find text nodes with specific text
const labels = screen.filterAdvanced(f => f.isText().hasText("Welcome"));

// Find nodes in a specific list
const listItems = screen.filterAdvanced(f =>
  f.anyParent(p => p.hasId("com.example:id/list"))
);

`matches()`

TypeScript

matches(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): boolean

Checks if this node matches the given filter.

Parameters

Name	Type	Description
`filterBuilder`	`(f: AndroidNodeFilter) => AndroidNodeFilter`	Function to build the filter

Returns

booleantrue if node matches the filter

Examples

TypeScript

// Check if a node is a clickable button
if (node.matches(f => f.isButton().isClickable())) {
  await node.performAction(agent.constants.ACTION_CLICK);
}

// Filter children manually
const clickableChildren = node.children.filter(child =>
  child.matches(f => f.isClickable())
);

`toJSON()`

TypeScript

toJSON(): object

Converts the node to a plain JSON object. Useful for debugging or serialization.

Returns

objectPlain object representation

Examples

TypeScript

console.log(JSON.stringify(node.toJSON(), null, 2));

Action Methods

These methods allow you to perform actions directly on nodes. Available since app version 2.119.

`performAction()`

TypeScript

performAction(actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{ actionPerformed: boolean }>

Performs an accessibility action on this node. This is the preferred way to interact with UI elements as it uses the accessibility system.

Parameters

Name	Type	Description
`actionInt`	`number`	Action constant (use agent.constants.ACTION_*)
`data?`	`object`	Additional action data (optional, used for ACTION_SET_TEXT etc.)
`fieldsToIgnore?`	`string[]`	Fields to ignore when matching node (optional)

Returns

Promise<{ actionPerformed: boolean }>Whether the action was performed successfully

Examples

Click a button

TypeScript

const button = screen.findTextOne("Submit");
if (button) {
  await button.performAction(agent.constants.ACTION_CLICK);
}

Set text on an input field

TypeScript

const input = screen.findAdvanced(f => f.isEditText());
if (input) {
  await input.performAction(agent.constants.ACTION_SET_TEXT, {
    [agent.constants.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE]: "Hello World"
  });
}

Scroll a list

TypeScript

const scrollView = screen.findAdvanced(f => f.isScrollable());
if (scrollView) {
  await scrollView.performAction(agent.constants.ACTION_SCROLL_FORWARD);
}

`adbPerformAction()`

TypeScript

adbPerformAction(actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{ actionPerformed: boolean }>

Performs an accessibility action on this node via ADB. This is the ADB equivalent of performAction(). Uses UiAutomation through the ADB touch server instead of the accessibility service.

Parameters

Name	Type	Description
`actionInt`	`number`	Action constant (use agent.constants.ACTION_*)
`data?`	`object`	Additional action data (optional, used for ACTION_SET_TEXT etc.)
`fieldsToIgnore?`	`string[]`	Fields to ignore when matching node (optional)

Returns

Promise<{ actionPerformed: boolean }>Whether the action was successfully performed

Examples

Click a button via ADB

TypeScript

const button = screen.findTextOne("Submit");
if (button) {
  await button.adbPerformAction(agent.constants.ACTION_CLICK);
}

Set text via ADB

TypeScript

const input = screen.findAdvanced(f => f.isEditText());
if (input) {
  await input.adbPerformAction(agent.constants.ACTION_SET_TEXT, {
    [agent.constants.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE]: "Hello World"
  });
}

`randomClick()`

TypeScript

randomClick(): void

Performs a tap at a random position within this node's bounds. Uses boundsInScreen to determine the click area. Useful for more natural-looking automation.

Examples

TypeScript

const button = screen.findTextOne("Submit");
if (button) {
  button.randomClick();
}

Click multiple items

TypeScript

const items = screen.filterAdvanced(f => f.isClickable().hasText(/Item \d+/));
for (const item of items) {
  item.randomClick();
  await sleep(500);
}

`randomSwipe()`

TypeScript

randomSwipe(direction: "up" | "down" | "left" | "right"): void

Performs a swipe gesture within this node's bounds. The swipe starts and ends at random positions within the node for more natural-looking automation.

Parameters

Name	Type	Description
`direction`	`"up" \| "down" \| "left" \| "right"`	Direction to swipe

Examples

Scroll a list down

TypeScript

const listView = screen.findAdvanced(f => f.isScrollable());
if (listView) {
  listView.randomSwipe("up"); // Swipe up to scroll down
}

Swipe through a carousel

TypeScript

const carousel = screen.findAdvanced(f => f.hasId("carousel"));
if (carousel) {
  carousel.randomSwipe("left"); // Swipe left to see next item
}

`randomClickAdb()`

TypeScript

randomClickAdb(): void

Performs a tap at a random position within this node's bounds using ADB shell input instead of the accessibility service. Fire-and-forget like randomClick().

Examples

TypeScript

const button = screen.findTextOne("Submit");
if (button) {
  button.randomClickAdb();
}

`randomSwipeAdb()`

TypeScript

randomSwipeAdb(direction: "up" | "down" | "left" | "right"): void

Performs a swipe gesture within this node's bounds using ADB shell input instead of the accessibility service. Fire-and-forget like randomSwipe().

Parameters

Name	Type	Description
`direction`	`"up" \| "down" \| "left" \| "right"`	Direction to swipe

Examples

Scroll a list down via ADB

TypeScript

const listView = screen.findAdvanced(f => f.isScrollable());
if (listView) {
  listView.randomSwipeAdb("up"); // Swipe up to scroll down
}

Note: randomClickAdb() and randomSwipeAdb() require app version 2.141 (153)+. They use ADB shell commands instead of the accessibility service, which can be more reliable in some scenarios.

Highlight Methods

`showHighlight()`

TypeScript

showHighlight(color?: string, style?: "bordered" | "filled", blink?: boolean): void

Draws a highlight over this node's boundsInScreen using the HTML overlay. Same as agent.utils.showHighlight(node.boundsInScreen, color, style, blink). Touches pass through the highlight. Only one overlay exists, so this replaces any highlight or HTML already shown with agent.display.displayHTMLCode().

Parameters

Name	Type	Description
`color?`	`string`	Any CSS color, e.g. "red", "#00ff00" or "rgba(255,0,0,0.4)"(default: `"red"`)
`style?`	`"bordered" \| "filled"`	"bordered" draws a 3px outline, "filled" fills the node's bounds(default: `"bordered"`)
`blink?`	`boolean`	When true, the highlight is visible for 1 second out of every 2(default: `false`)

Examples

Highlight the button about to be tapped

TypeScript

const button = screen.findTextOne("Continue");
if (button) {
  button.showHighlight("lime", "bordered", true);
  await sleep(3000);
  button.hideHighlight();
  button.randomClick();
}

`hideHighlight()`

TypeScript

hideHighlight(): void

Removes the highlight. Same as agent.utils.hideHighlight(), which calls agent.display.hideHTMLCode().

Common Patterns

Click a button (random position)

TypeScript

const button = screen.findTextOne("Submit");
if (button) {
  button.randomClick();
}

Click using accessibility action

TypeScript

const button = screen.findAdvanced(f => f.isButton().hasText("OK"));
if (button) {
  await button.performAction(agent.constants.ACTION_CLICK);
}

Type in an input field

TypeScript

const input = screen.findAdvanced(f => f.isEditText().isEditable());
if (input) {
  await input.performAction(agent.constants.ACTION_CLICK);
  await agent.actions.writeText("Hello World");
}

Scroll a list

TypeScript

const scrollView = screen.findAdvanced(f => f.isScrollable());
if (scrollView) {
  // Using accessibility action
  await scrollView.performAction(agent.constants.ACTION_SCROLL_FORWARD);

  // Or using random swipe for more natural scrolling
  scrollView.randomSwipe("up");
}