Frequently Asked Questions

Here you can find answers to some frequently asked questions regarding the use of PaceWords in QVision and tips how to overcome some issues you may encounter when developing test automation scripts.

How to use icons?

Icons needs to be saved in advance to a .png format to the reference folder. The reference folder can be set with the SetReferenceFolder -keyword.

Note that the icon image size affects it’s recognition. Importantly the icon image size should match the size of the icon on the screen. Best way to ensure that is to use screenshots or the snipping tool to save the icon image directly from the application under automation.

How can I make it run faster?

Machine Vision algorithms will require a bit more from your hardware compared to API based automation solutions. If you find yourself hoping for a faster execution here are some things you can do (besides upgrading your hardware):

Enable faster OCR engine

You can enable a faster OCR engine by calling in your automation script

SetOCREngine    RevisionEngine

This is a faster OCR engine without a fallback to a slower one when it does not find the requested text (as in the default engine). In many cases it will perform just as well as the default engine but be much faster.

Lower your resolution

Smaller resolution makes the screenshots smaller. This makes the computer vision algorithms having to process less image data resulting into faster execution.

What to do when text is not found

There are cases when QVision doesn’t find a word or reads it incorrectly. In these cases one has to use a workaround.

Adjust scaling

Scaling helps when sentences are recognized as a single word or a single word is detected as multiple words (e.g. “Paragraph” is detected as “Par” “agr” “aph”).

If a sentence is recognized as a single word, you should make the scaling larger than 1.0 to help the detector to separate the words:

VerifyText    Sentence detected as one word    scale=2.0

If a word is detected as multiple words, you should make the scaling smaller than 1.0:

VerifyText    Paragraph    scale=0.5

The scale parameter is supported by all the PaceWords dealing with text detection.

If you notice that you have to do this for most of the words, you should tune the ocr_scale configuration appropriately with SetConfig. Scaling configuration is dependent on the resolution and scaling of your operating system. Remember to match your screen settings with the screen settings of your production environment when developing automation scripts.

Adjust detection tolerance

Lowering detection tolerance helps when some letters are incorrectly recognized in the text. The tolerance can be set to a value in (0.0, 1.0], where 1.0 means that all letters must match.

To change the tolerance temporarily for a detection you can write:

VerifyText    Incorrectly Detected    tol=0.5

The tol parameter is supported by all the PaceWords dealing with text recognition. Please note that letter matching is case sensitive.

It is also possible to set the default tolerance value with the tolerance_text configuration. Note however, that using a too low default tolerance can cause incorrect verifications or interactions with unintended texts.

Use nearby text with offsets

You can use a nearby text or icon to interact with a text/element that cannot be reached otherwise. Many of the PaceWords support offset parameters that can be used to interact with elements near another element:

ClickText    Text above    below=1.5
TypeText     Name          John           right=5.0

This method is also useful for clicking icons or text fields next to a text. See Offsets for more information.

Use Icon

When nothing above works, you can also crop the element/UI component image that you wish to click and save it as an icon:

ClickIcon    text_or_element_icon

Use the incorrectly detected text

As a last resort you can use the incorrectly detected text:

ClickText    Par agr aph

As long as your screen setting remain the same, you can trust that the detection works the same way. Alternatively you can use a variable to mark what the text should be at that location:

ClickText    ${paragraph}

How can I adjust scaling?

QVision is dependent on the resolution and scaling settings of your operating system. For optimal performance, it is recommended to use the following settings:

  • Resolution 1920 x 1080

  • Windows scale 100%

Alternatively, you can also adjust the ocr_scale parameter which will scale the screenshots before detection:

SetConfig     ocr_scale         1.5

Setting a good scale parameter can help dramatically in the recognition performance for your application.

The method below can be used in finding a good scale parameter. Open the application under test in the production environment. Navigate to a state which is representative of the complexity (e.g. has dialogs/buttons visible). Then run the following test case:

*** Test Cases ***
Find Optimal Ocr Scale
    SetOcrScale       0.25
    LogScreenshot     annotated
    SetOcrScale       0.50
    LogScreenshot     annotated
    SetOcrScale       0.75
    LogScreenshot     annotated
    SetOcrScale       1.0
    LogScreenshot     annotated
    SetOcrScale       1.25
    LogScreenshot     annotated
    SetOcrScale       1.50
    LogScreenshot     annotated
    SetOcrScale       1.75
    LogScreenshot     annotated

Open the produced log.html file and determine the best scale parameter by investigating the results. You should choose the value that has marked single words the best (note, not sentences). To further tune the value, you can rerun the test above with more granular scale values.