Python API

JavaAccessBridge

exception RPA.JavaAccessBridge.ElementNotFound

Bases: ValueError

No matching elements were found.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception RPA.JavaAccessBridge.InvalidLocatorError

Bases: AttributeError

Invalid locator string.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class RPA.JavaAccessBridge.JavaAccessBridge(ignore_callbacks: bool = False, access_bridge_path: str = None)

Bases: object

Java application UI automation library using Java Access Bridge technology.

The library utilizes java-access-bridge-wrapper package to interact with Java UI. Currently only the 64-bit Windows OS is supported.

Inspecting elements

The recommended tool for inspecting Java application is:

The Accessibility Insights for Windows can show element properties if application framework supports Windows UI Automation (UIA), see more at using Accessibility Insights. Then recommended library would be RPA.Windows library.

Steps to enable

  1. Enable the Java Access Bridge in Windows

  2. Set environment variable RC_JAVA_ACCESS_BRIDGE_DLL as an absolute path to WindowsAccessBridge-64.dll. It is also possible to give DLL location as library initialization parameter access_bridge_path.

C:\path\to\java\bin\jabswitch -enable
set RC_JAVA_ACCESS_BRIDGE_DLL=C:\path\to\Java\bin\WindowsAccessBridge-64.dll
*** Settings ***
Library   RPA.JavaAccessBridge   access_bridge_path=C:\path\to\Java\bin\WindowsAccessBridge-64.dll

About Java wrapper callbacks and actions

There might be a compability issue with callbacks and actions on target Java application. Possible reasons:

  • target application is executed with 32-bit Java

  • target application does not support callbacks and/or actions

Workaround for this situation is to initialize JavaAccessBridge library with parameter ignore_callbacks=True. Then application’s element information is still accessible and any actions on those elements can be performed with RPA.Desktop library.

Note. There are still keywords, for example. Call Element Action, which will cause error if used in this situation. To be fixed in future release.

*** Settings ***
Library   RPA.JavaAccessBridge   ignore_callbacks=True

Locating elements

To automate actions on the Java application, the robot needs locations to various elements using a feature called locators. Locator describes properties of an element.

At the moment library contains basic level support for locators.

The common locator types are name and role.

To identify element with more than one property and can be used, for example:

role:push button and name:Clear

To address element within parent element > can be used, for example:

name:Find Purchase Orders > name:NumberField

Some keywords accept element as an parameter in place of locator.

New locator type strict has been added in rpaframework==12.5.0. Currently property values of string type have been evaluated with startsWith which can match several property values. With strict set in the locator string, all locator on the right side of this definition will be matched using strict (equal matching), example:

# without strict, name can be 'Type', 'Type1', 'Type of'...
Get Elements   role:push button and name:Type
# name must be equal to 'Type'
Get Elements  role:push button and strict:True and name:Type

Keyword Get Elements has extra parameter strict, which when set to True forces all locator value matches to be strict, example:

# without strict, name can be 'Type', 'Type1', 'Type of'...
Get Elements  role:push button and name:Type
# name must be equal to 'Type' and role must be equal to 'text'
Get Elements  role:text and name:Type  strict=True

About JavaElement object

The JavaElement was added in rpaframework==12.3.0 for easy access into ContextNode objects which have been returned by Get Elements keyword.

Keyword Get Elements still returns ContextNode objects, but with parameter java_elements=True the keyword returns JavaElement objects instead (they still contain reference to ContextNode object via node property, e.g. JavaObject.node).

Properties and methods included in the JavaElement:

  • name: str

  • role: str

  • states: list # list presentation of states (string)

  • checked: bool

  • selected: bool

  • visible: bool

  • enabled: bool

  • states_string: str

  • x: int # left coordinate of the element

  • y: int # top coordinate of the element

  • width: int

  • height: int

  • node: ContextNode # original ContextNode

  • row: int # table row, -1 if element is not member of table

  • col: int # table column, -1 if element is not member of table

  • text: str # text content of the element

  • column_count: int # table column count

  • visible_children: list #visible_children elements of this element

  • click() # method for clicking element center

  • type_text() # method for typing text into element (if possible)

Interacting with elements

By default application elements are interacted with Actions supported by the element. Most common example is click action supported by an button element.

But because application and technology support for the actions might be limited, it is also possible to opt for interaction elements by their coordinates by giving keyword parameter action=False if parameter is available.

Examples

robotframework

*** Settings ***
Library   RPA.JavaAccessBridge
Library   Process

*** Tasks ***
Write text into Swing application
    Start Process    java -jar BasicSwing.jar
    ...              shell=${TRUE}
    ...              cwd=${CURDIR}
    Select Window    Chat Frame
    Type Text    role:text
    ...          text for the textarea
    Type Text    role:text
    ...          text for the input field
    ...          index=1
    ...          clear=${TRUE}
    Click Element    role:push button and name:Send

Python

from RPA.JavaAccessBridge import JavaAccessBridge
import subprocess

jab = JavaAccessBridge()

subprocess.Popen(
    ["java", "-jar", "BasicSwing.jar"],
    shell=True,
    cwd=".",
    close_fds=True
)
jab.select_window("Chat Frame")
jab.type_text(
    "role:text",
    "text for the textarea",
    enter=True
)
jab.type_text(
    "role:text",
    "text for the input field",
    index=1,
    clear=True
)
jab.click_element("role:push button and name:Send")
ROBOT_AUTO_KEYWORDS = False
ROBOT_LIBRARY_DOC_FORMAT = 'REST'
ROBOT_LIBRARY_SCOPE = 'GLOBAL'
application_refresh()

Refresh application element tree

Might be required action after application element structure changes after window refresh.

call_element_action(locator: str, action: str)

Call element action

Parameters
  • locator – target element

  • action – name of the element action to call

click_coordinates(x: int, y: int, click_type: str = 'click', delay: float = 0.5)

Keyword to mouse click at specific coordinates.

Parameters
  • x – horizontal coordinate

  • y – vertical coordinates

  • click_type – default click, see RPA.Desktop for different click options

  • delay – how much in seconds to delay after click, defaults to 0.5

click_element(locator: Union[JABWrapper.context_tree.ContextNode, RPA.JavaAccessBridge.JavaElement, str], index: int = 0, action: bool = True, timeout: int = 10)

Click element

Parameters
  • target – element to click

  • index – target element index if multiple are returned

  • action – call click action on element (default), or use coordinates

  • timeout – timeout in seconds to find element

click_push_button(button_name: str)

Click element of role push button

Parameters

button_name – name of the button to click

close_java_window()

Close active Java window which has been accessed via `Select Window keyword.

get_element_actions(locator: Union[JABWrapper.context_tree.ContextNode, RPA.JavaAccessBridge.JavaElement, str])

Get list of possible element actions

Parameters

locator – target element

get_element_text(locator: Union[JABWrapper.context_tree.ContextNode, RPA.JavaAccessBridge.JavaElement, str], index: int = 0)

Get element text

Parameters
  • locator – target element

  • index – target element index if multiple are returned

get_elements(locator: str, java_elements: bool = False, strict: bool = False)

Get matching elements

Parameters
  • locator – elements to get

  • java_elements – if True will return elements as JavaElement on False will return Java ContextNodes

  • strict – on True all locator matches need to match exactly, on False will be using startsWith matching on non-integer properties

Returns

list of ContextNodes or JavaElements

Python example.

elements = java.get_elements("name:common", java_elements=True)
for e in elements:
    print(e.name if e.name else "EMPTY", e.visible, e.x, e.y)
    if e.role == "check box":
        e.click()
    else:
        java.type_text(e, "new content", clear=True, typing=False)

# following does NOT return anything because search is strict
# and there are no 'push butto' role
elements = java.get_elements("role:push butto", strict=True)

Robotframework example.

${elements}=    Get Elements
...    role:push button and name:Send
...    java_elements=True
Evaluate   $elements[0].click()
Click Element    ${elements}[0]    action=False
Type Text
...    ${elements}[0]
...    moretext
...    clear=True
...    typing=False
get_version_info()

Get Java Access Bridge version information

highlight_element(locator: Union[JABWrapper.context_tree.ContextNode, RPA.JavaAccessBridge.JavaElement, str], index: int = 0)

Highlight an element

Parameters
  • locator – element to highlight

  • index – target element index if multiple are returned

press_keys(*keys)

Press multiple keys down simultaneously

See Desktop library documentation for supported keys

Parameters

keys – keys to press

print_element_tree(filename: str = None)

Print current element into log and possibly into a file

Parameters

filename – filepath to save element tree

read_table(locator: Union[JABWrapper.context_tree.ContextNode, RPA.JavaAccessBridge.JavaElement, str])

Return Java table as list of lists (rows containing columns).

Each cell element is represented by JavaElement class.

Parameters

locator – locator to match element with type of table

Returns

list of lists

Example.

table = java.read_table(locator_table)
for row in table:
    for cell in row:
        if cell.role == "check box":
            print(cell.row, cell.col, str(cell.checked))
        else:
            print(cell.row, cell.col, cell.name)
select_menu(menu: str, menuitem: str)

Select menu by clicking menu elements

Parameters
  • menu – name of the menu

  • menuitem – name of the menu item

select_window(title: str, bring_foreground: bool = True, timeout: int = 30)

Selects Java application window as target for the automation

Parameters
  • title – application window title

  • bring_foreground – if application is brought to foreground or not

  • timeout – selection timeout

set_display_scale_factor(factor: float)

Override library display scale factor.

Keyword returns previous value.

Parameters

factor – value for the new display scale factor

Returns

previous display scale factor value

set_mouse_position(element: JABWrapper.context_tree.ContextNode)

Set mouse position to element center

Parameters

element – target element

shutdown_jab()

Call Java Access Bridge process shutdown

toggle_drop_down(locator: Union[JABWrapper.context_tree.ContextNode, RPA.JavaAccessBridge.JavaElement, str], index: int = 0)

Toggle dropdown action on element

Parameters
  • locator – element locator

  • index – target element index if multiple are returned

type_text(locator: Union[JABWrapper.context_tree.ContextNode, RPA.JavaAccessBridge.JavaElement, str], text: str, index: int = 0, clear: bool = False, enter: bool = False, typing: bool = True)

Type text into coordinates defined by locator

Parameters
  • locator – target element

  • text – text to write

  • index – target element if multiple are returned

  • clear – should element be cleared before typing

  • enter – should enter key be pressed after typing

  • typing – if True (default) will use Desktop().type_text() if False will use Desktop().press_keys()

wait_until_element_is_focused(locator: Union[JABWrapper.context_tree.ContextNode, RPA.JavaAccessBridge.JavaElement, str], index: int = 0, timeout: float = 0.5)

Wait until element is focused

Parameters
  • locator – target element

  • index – target element index if multiple are returned

  • timeout – timeout in seconds to wait, default 0.5 seconds

wait_until_element_text_contains(locator: Union[JABWrapper.context_tree.ContextNode, RPA.JavaAccessBridge.JavaElement, str], text: str, index: int = 0, timeout: float = 0.5)

Wait until element text contains expected text

Parameters
  • locator – target element

  • text – element text should contain this

  • index – target element index if multiple are returned

  • timeout – timeout in seconds to wait, default 0.5 seconds

wait_until_element_text_equals(locator: Union[JABWrapper.context_tree.ContextNode, RPA.JavaAccessBridge.JavaElement, str], text: str, index: int = 0, timeout: float = 0.5)

Wait until element text equals expected text

Parameters
  • locator – target element

  • text – element text should match this

  • index – target element index if multiple are returned

  • timeout – timeout in seconds to wait, default 0.5 seconds

class RPA.JavaAccessBridge.JavaElement(node, scaling_factor=None, internal_node=None, index=0, column_count=None)

Bases: object

Abstraction for Java object properties

checked: bool = None
click(click_type: str = 'click')
col: int = None
column_count: int = None
enabled: bool = None
height: int = None
name: str = None
node: ContextNode = None
role: str = None
row: int = None
selected: bool = None
states: list = None
states_string: str = None
text: str = None
type_text(text: str, clear: bool = False) → None
visible: bool = None
visible_children: list = None
width: int = None
x: int = None
y: int = None
exception RPA.JavaAccessBridge.JavaWindowNotFound

Bases: ValueError

No active Java window were found.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

RPA.JavaAccessBridge.get_scaled_coordinate(coordinate, scaling_factor)