Python API

Base64AI

class RPA.DocumentAI.Base64AI.Base64AI

Library to support Base64.ai service for intelligent document processing (IDP).

Library requires at the minimum rpaframework version 19.0.0.

Service supports identifying fields in the documents, which can be given to the service in multiple different file formats and via URL.

Robot Framework example usage

*** Settings ***
Library   RPA.DocumentAI.Base64AI
Library   RPA.Robocorp.Vault

*** Tasks ***
Identify document
    ${secrets}=   Get Secret  base64ai-auth
    Set Authorization  ${secrets}[email-address]   ${secrets}[apikey]
    ${results}=  Scan Document File
    ...   ${CURDIR}${/}invoice.pdf
    ...   model_types=finance/check/usa,finance/invoice/usa
    # Scan response contains list of detected models in the document
    FOR  ${result}  IN  @{results}
        Log To Console  Model: ${result}[model]
        Log To Console  Field keys: ${{','.join($result['fields'].keys())}}
        Log To Console  Fields: ${result}[fields]
        Log To Console  Text (OCR): ${result}[ocr]
    END

Python example usage

from RPA.DocumentAI.Base64AI import Base64AI
from RPA.Robocorp.Vault import Vault

secrets = Vault().get_secret("base64ai-auth")
baselib = Base64AI()
baselib.set_authorization(secrets["email-address"], secrets["apikey"])
result = baselib.scan_document_file(
    "invoice.pdf",
    model_types="finance/invoice,finance/check/usa",
)
for r in result:
    print(f"Model: {r['model']}")
    for key, props in r["fields"].items():
        print(f"FIELD {key}: {props['value']}")
    print(f"Text (OCR): {r['ocr']}")

Portal example: https://github.com/robocorp/example-idp-base64

BASE_URL = 'https://base64.ai'

ROBOT_LIBRARY_DOC_FORMAT = 'REST'

ROBOT_LIBRARY_SCOPE = 'GLOBAL'

filter_matching_signatures(match_response: Optional[Union[Dict[Hashable, Optional[Union[str, int, float, bool, list, dict]]], List[Optional[Union[str, int, float, bool, list, dict]]], str, int, float, bool, list, dict]], confidence_threshold: float = 0.8, similarity_threshold: float = 0.8) → Dict[Tuple[int, Tuple[int, ...]], List[Dict[str, Any]]]

Gets through all the recognized signatures in the queried image and returns only the ones passing the confidence & similarity thresholds.

Additionally, this keyword simplifies the original input match_response structure and returns a dictionary with all the detected and accepted reference signatures as keys, and lists of similar enough query signatures as values.

Each reference signature (key) is a tuple of (index, coordinates).
Each query signature (sub-value) is a dictionary of {index, coords, similarity}.
The coordinates describe the bounding-box enclosing the detected signature portion from the original image, as follows: (left, top, right, bottom) corners.

Use the original match_response object and the indexes from here if you need to retrieve extra details not found here (e.g. confidence score). Use the Get Signature Image to save and preview the image crop belonging to the signature of choice.

Parameters

match_response – The raw JSON-like response retrieved with the Get Matching Signatures keyword.
confidence_threshold – The minimum accepted confidence score (0.0-1.0) for a candidate to be considered a signature. (to avoid false-positives)
similarity_threshold – The minimum accepted similarity score (0.0-1.0) for a query signature to be considered an alike signature. (to discard different or fraudulent signatures)

Returns

A dictionary of accepted reference signatures and their similar ones found in the queried image.

Example: Robot Framework

*** Tasks ***
Match Signatures
    &{matches} =   Filter Matching Signatures      ${sigs}
    Log Dictionary    ${matches}

Example: Python

matches = lib.filter_matching_signatures(sigs)
print(matches)

get_fields_from_prediction_result(prediction: Optional[Union[Dict[Hashable, Optional[Union[str, int, float, bool, list, dict]]], List[Optional[Union[str, int, float, bool, list, dict]]], str, int, float, bool, list, dict]]) → List

Helper keyword to get found fields from a prediction result. For example see Scan Document File or Scan Document URL keyword.

Parameters: prediction – prediction result dictionary
Returns: list of found fields

get_matching_signatures(reference_image: Union[Path, str], query_image: Union[Path, str]) → Optional[Union[Dict[Hashable, Optional[Union[str, int, float, bool, list, dict]]], List[Optional[Union[str, int, float, bool, list, dict]]], str, int, float, bool, list, dict]]

Returns a list of matching signatures found from the reference into the queried image.

The input images can be paths to the files or URLs.

The output JSON-like dictionary contains all the details from the API, like the detected signatures in both the reference and query image and for every such signature, its bounding-box geometry, confidence and similarity score. Use the Filter Matching Signatures over this value to get a simpler structure.

Parameters

reference_image – The reference image (jpg/png) to check query signatures against. (e.g. driving license, ID card)
query_image – The query image containing signatures similar to the ones from the reference image. (e.g. signed contract, bank check)

Returns

A JSON-like dictionary revealing recognized signatures and how much they resemble with each other.

Example: Robot Framework

*** Tasks ***
Match Signatures
    ${ref_image} =  Set Variable    driving-license.jpg
    ${query_image} =  Set Variable    signed-check.png
    ${sigs} =   Get Matching Signatures     ${ref_image}    ${query_image}

Example: Python

from RPA.DocumentAI.Base64AI import Base64AI

lib = Base64AI()
sigs = lib.get_matching_signatures(
    "driving-license.jpg", "signed-check.png"
)

Portal example: https://github.com/robocorp/example-signature-match-assistant

get_signature_image(match_response: Optional[Union[Dict[Hashable, Optional[Union[str, int, float, bool, list, dict]]], List[Optional[Union[str, int, float, bool, list, dict]]], str, int, float, bool, list, dict]], *, index: int, reference: bool = False, path: Optional[Union[Path, str]] = None) → str

Retrieves and saves locally the image cut belonging to the provided index.

The image data itself is provided with the original match_response object as base64 encoded content. This utility keyword retrieves, decodes and saves it on the local disk customized with the path parameter. By default, the searched index is considered a query image, switch to the reference type by enabling it with the reference parameter.

Parameters

match_response – The raw JSON-like response retrieved with the Get Matching Signatures keyword.
index – The image ID (numeric) found along the coordinates in the output of the Filter Matching Signatures keyword. (the list order is stable)
reference – Set this to True if you’re looking for a reference (not query) image instead. (off by default)
path – Set an explicit output path (including file name) for the locally saved image. (uses the output directory as default)

Returns

The image path of the locally saved file.

Example: Robot Framework

*** Tasks ***
Match Signatures
    @{ref_sigs} =   Get Dictionary Keys    ${matches}
    @{qry_sigs} =    Get From Dictionary    ${matches}    ${ref_sigs}[${0}]
    &{qry_sig} =    Set Variable    ${qry_sigs}[${0}]
    ${path} =   Get Signature Image     ${sigs}     index=${qry_sig}[index]
    Log To Console    Preview query signature image crop: ${path}

Example: Python

qry_sig = list(matches.values())[0][0]
path = lib.get_signature_image(sigs, index=qry_sig["index"])
print("Preview query signature image crop: ", path)

get_user_data() → Dict

Get user data including details on credits used and credits remaining for the Base64 service.

Returned user data contains following keys:

givenName

familyName

email

hasWorkEmail

companyName

numberOfCredits

numberOfPages

numberOfUploads

numberOfCreditsSpentOnDocuments (visible if used)

numberOfCreditsSpentOnFaceDetection (visible if used)

numberOfCreditsSpentOnFaceRecognition (visible if used)

hasActiveAwsContract

subscriptionType

subscriptionPeriod

tags

ccEmails

status

remainingCredits (calculated by the keyword)

Returns: object containing details on the API user

Robot Framework example:

${userdata}=   Get User Data
Log To Console  I have still ${userdata}[remainingCredits] credits left

Python example:

userdata = baselib.get_user_data()
print(f"I have still {userdata['remainingCredits']} credits left")

scan_document_file(file_path: str, model_types: Optional[Union[str, List[str]]] = None, mock: bool = False) → Optional[Union[Dict[Hashable, Optional[Union[str, int, float, bool, list, dict]]], List[Optional[Union[str, int, float, bool, list, dict]]], str, int, float, bool, list, dict]]

Scan a document file. Can be given a model_types to specifically target certain models.

Parameters

file_path – filepath to the file
model_types – single model type or list of model types
mock – set to True to use /mock/scan endpoint instead of /scan

Returns

result of the document scan

Robot Framework example:

${results}=    Scan Document File
...    ${CURDIR}${/}files${/}IMG_8277.jpeg
...    model_types=finance/check/usa,finance/invoice
FOR    ${result}    IN    @{results}
    Log To Console    Model: ${result}[model]
    Log To Console    Fields: ${result}[fields]
    Log To Console    Text (OCR): ${result}[ocr]
END

Python example:

result = baselib.scan_document_file(
    "./files/Invoice-1120.pdf",
    model_types="finance/invoice,finance/check/usa",
)
for r in result:
    print(f"Model: {r['model']}")
    for key, val in r["fields"].items():
        print(f"{key}: {val['value']}")
    print(f"Text (OCR): {r['ocr']}")

scan_document_url(url: str, model_types: Optional[Union[str, List[str]]] = None, mock: bool = False) → Optional[Union[Dict[Hashable, Optional[Union[str, int, float, bool, list, dict]]], List[Optional[Union[str, int, float, bool, list, dict]]], str, int, float, bool, list, dict]]

Scan a document URL. Can be given a model_types to specifically target certain models.

Parameters

url – valid url to a file
model_types – single model type or list of model types
mock – set to True to use /mock/scan endpoint instead of /scan

Returns

result of the document scan

Robot Framework example:

${results}=    Scan Document URL
...    https://base64.ai/static/content/features/data-extraction/models//2.png
FOR    ${result}    IN    @{results}
    Log To Console    Model: ${result}[model]
    Log To Console    Fields: ${result}[fields]
    Log To Console    Text (OCR): ${result}[ocr]
END

Python example:

result = baselib.scan_document_url(
    "https://base64.ai/static/content/features/data-extraction/models//2.png"
)
for r in result:
    print(f"Model: {r['model']}")
    for key, props in r["fields"].items():
        print(f"FIELD {key}: {props['value']}")
    print(f"Text (OCR): {r['ocr']}")

set_authorization(api_email: str, api_key: str) → None

Set Base64 AI request headers with email and key related to API.

Parameters

api_email – email address related to the API
api_key – key related to the API

Robot Framework example:

${secrets}=   Get Secret  base64ai-auth
Set Authorization    ${secrets}[email-address]    ${secrets}[apikey]

Python example:

secrets = Vault().get_secret("base64ai-auth")
baselib = Base64AI()
baselib.set_authorization(secrets["email-address"], secrets["apikey"])