Version 2.9.0 Date: 23 Apr, 2020

Automated Test Design: No corner case is left untested

The quality of your test automation is measured by its ability to find flaws in the application under test. Therefore, you certainly need to strengthen the fault detection capability of your test automation. Automated test design holds the solution to your problem.

We are excited to introduce two very exciting and powerful automated test design methods in Qentinel Pace. These methods standalone or in conjunction with each other will enable you to automatically design test scenarios to test your application’s behavior, which has so far been left untouched and untested.

You can learn to use automated test design here. You may want to read our blog.

Combinatorial testing

Combinatorial testing is based on the premise that many errors in software can only arise from the interaction of two or more parameters. Let’s take an easy example to understand this powerful concept, you have to test an end-to-end process in Salesforce, wherein you need to populate a form with first name, last name and company name. Your testing objective is to test your application’s response to various combination of data. In Qentinel Pace, a sample test script would look like as shown below

My test case
    [Tags]      testgen       nwise=2
    TypeText    First name    [John, Jane]
    TypeText    Last name     [Johnson, Janeson]
    TypeText    Company       [Qentinel, ACME, Inc Incorporated]

The test case above will generate 6 test cases with data values (John, Johnson, Qentinel), (Jane, Janeson, Qentinel), (Jane, Johnson, ACME), (John, Janeson, ACME), (John, Janeson, Inc Incorporated) and (Jane, Johnson, Inc Incorporated).

Drawing from our vast datasets, we recommend pairwise testing (nwise=2) for conducting combinatorial testing. However should there be a need to have more than two interacting variables, nwise can be configured accordingly, for example nwise=3 or nwise=4.

Test data generation

We are often faced with problems where we need to test the application with valid and invalid email addresses, IBAN bank account numbers, social security numbers, string encoded IP addresses, and so on. Traditionally, this has been extremely difficult to create sufficient and proper test data that comprehensively covers the software logic for any non-trivial software system. Well the good news is that with Qentinel Pace you can systematically and automatically generate test cases for covering each “corner case” for such case or combination of such cases.

Drawing from our Salesforce test example, let’s say you have to test that your form excepts all valid email addresses and show a message ‘Valid email address provided’. Your test case will look like as shown below

My test case
    [Tags]        testgen
    TypeText      Email    VALID_EMAIL_ADDRESS
    VerifyText    Valid email address provided

Here, the predefined test data modifier VALID_EMAIL_ADDRESS will create a collection of test cases for you where each generated test case contains a valid email address in such a fashion that each distinct test case verifies a unique feature or a corner case of email address pattern.

Furthermore, the test data generation expands to negative and invalid test data as well. This ensures that your application behaves properly not only for valid data but also handles invalid data values properly. Invalid test data is generated simply by prefixing the test data modifier with INVALID instead of VALID as shown below

My test case
    [Tags]       testgen
    TypeText     Email    INVALID_EMAIL_ADDRESS
    VerifyText   Invalid email address provided

Combinatorial testing and test generation

As you have guessed correctly by now, you dont need to stop here, the test data modifiers can be used in conjunction with combinatorial test generation. And this will empower your test script to explore a whole new landscape of potential data combination to test your application against. Let’s take a test case wherein, you first want to enter a first name and then a last name from combination of two data values each, a valid email address and an invalid IBAN number.

My test case
    [Tags]      testgen       nwise=2
    TypeText    First name    [John, Jane]
    TypeText    Last name     [Johnson, Janeson]
    TypeText    Email         VALID_EMAIL_ADDRESS

You could define a maximum number of tests you want Qentinel Pace to generate for you, by default it is set to 100. Moreover, should you realize that a set of generated test case is of special significance, you could include it in your regression test set, for good.

More speed

We have significantly speeded up the process of project creation and robot addition. You will be amazed to notice that project and robot creation is 10x faster than before.

Quick view on your robots at work

Earlier, when you have had multiple robots running concurrently, if not difficult it was rather tedious to know the exact robots in action. Now, a dynamic list will show up on the top-left of your screen with all the suites that are currently being executed. Hence, you can have a grasp at the state of your robots within a project.

2901

Other improvements

  1. Recorded videos can be watched inline or in a different tab. You can click on the video icon to enable/disable them.

2902
  1. Scheduler is moved inline with other suite editing controls.

2903
  1. Detailed log for a specific test result can be opened directly by clicking on the test name.

testgeneration4

Happy testing!