We have moved

This is a legacy version of Copado Robotic Testing help. This page will not be updated anymore.

See https://docs.copado.com/ for up-to-date documentation.

How to create an Insights dashboard?

You can create dashboards and add widgets for interesting metrics and metric trees. Let’s first create a simple Insights dashboard with one example widget as follows: 1. Create a [data] source for the project you want to obtain data from. 2. Define or modify a metric using that source. 3. Create a dashboard. 4. Add a widget on that dashboard using the metric you defined.

Creating a source for a test result metric

  1. Click the Quality Intelligence icon in the tool bar to navigate to Quality Intelligence.

quality intelligence icon

  1. Select sources in the top-left toolbar.

sources icon

  1. Click ‘+’ to add source.

add source icon

  1. Give a name to the source and a brief description. Select the project from the drop-down list and select Test results as the type of the source. Press Save.

devops source save dialog

as each source is tied to a project in Pace, using the project name in the source name makes it easier to recognize the sources if there are later sources from several projects.

Creating metrics with the DevOps metrics tree template

The fastest way to setup the Quality Intelligence for DevOps metrics is to apply the template that creates the metrics tree skeleton that you can modify.

  1. Navigate to Metrics

devops metrics icon

  1. Start with a template and create a skeleton for a DevOps metrics tree and the required sources.

devops add metric

Note: In addition to metrics, the template generates the required sources:

devops required sources

Now that you have the sources and metrics defined you can elaborate metric definitions in terms of target values, sources or formulas used.

Let’s edit the Test Pass Rate (QA) metric:

devops test passrate qa

The metric form that opens looks like this:

devops test passrate qa edit

We can see here that:

  • The bottom value has been set to 30%. So if the test pass rate is lower than that, the index value will be zero.

  • The target value is set to 95% which would give the index value 100.

  • Normalized value is used as the calculation method, so we want to calculate some value (test pass rate) and a normalized index score for it according to the normalization rules explained earlier. The alternative calculation method ‘index’ can be applied for metric trees (e.g. Flow of Value) only, so in cases where we calculate an index by averaging the index values of child metrics.

  • Calculation mode is set to formula. The alternative would be reading the values directly from a source.

  • The source is set to ‘Test Results (162)’ in this example. That is a test results source linked to the Quick start project earlier.

  • You can use rich mathematical formulas to define a metric. The underscore ( _ ) and number can refer to a source, or in case of a parent metric in a metric tree, also to a child metric, which allows you to define metrics using other metrics. In this formula: (_162.test_passed / _162.test_sum) * 100

    • 162 is the _source ID for a stream of test results connected to the Quick start project

    • _162.test_passed gives the number of passed test cases

    • _162.test_sum gives the number of all test cases run

So the formula gives the test pass rate % over time. ID’s for sources and metrics can be found by selecting a source or a metric. You can find the possible data fields you can use in the formula under the formula field. You can use categories to filter data, like by jobName and/or suiteName in this case.

devops formula

By defining a suite name you can e.g. define separate run results for QA and Production environments. Note that the metrics template does not define this, so you need to do that. Now that you have created the metric, you can view the history of existing data (if data exists, of course) using View history.

devops show history

Run a test to get your first data point. This kind of window will open:

devops graph first run

From now on you will see a graph of test results as new results arrive from the test runs.

Creating a dashboard

  1. Navigate to Insights page.

  2. Add a dashboard with ‘+’

devops add dashboard icon
  1. Select Quality Intelligence for DevOps dashboard.

  2. Give a descriptive name and a description if needed. Now you have created and empty dashboard. Next we will add a metric widget to it.

Adding a widget to a dashboard

  1. Select Add widget.

  2. Fill in widget details: name and optional description.

  3. For the widget type you can select:

    1. Metric value: a box showing the latest value for a single metric.

    2. Metric tree: a hierarchic presentation of related metrics

    3. Value Creation Model: Quality Intelligence for DevOps visualization.

    4. Chart: a graph chart showing a trend of a metric.

    5. Custom HTML content: basically any HTML embed. Select Chart as we want to illustrate the trend of the test results.

  4. Next you need to select:

    1. the chart type: either line or stacked area graph.

    2. The metric(s) for which you want the graph to be drawn.

    3. Aggregation in case you want data to be aggregated. Select Line, Test Pass Rate (QA) and no aggregation as we don’t want to summarize pass rate scores.

devops add widget

  1. Press Save and you should see a graph with your historical counts of passed test cases. After running the Starter suite of Quick start project twice with 100% pass rate the graph would look like this:

devops new widget with data

Rearranging widgets

Add another widget. Press Rearrange widgets to resize or move widgets to different positions by dragging the widget edges.

Adding a manual metric and data

You can add manual data sources that you update by entering the data values manually. Let’s create an example manual source for Employee Experience (EX) survey results with which we track Happiness of the employees.

  1. Navigate to Sources.

  2. Add a new source with ‘+’.

  3. Fill in the name for the data source and an optional description.

  4. Select ‘Manual’ as data type.

All data sources need to be linked to a single project so select a project to which this data relates to.

devops add manual source

  1. Now you can add the data fields for the manual data source with ‘+’ on the rights of Fields. In this example we add two fields: eNPS for Employee Net Promoter Score and Team spirit.

devops source fields

Adding data to a manual source

Open the data source and click add data icon on the right.

devops add data

Provide the time stamp for when the data you enter was valid and then the data values.

devops timestamp

Press Send to submit the data to the data warehouse of Copado Robotic Testing. You can now use these two data fields in the metrics you want to create using them.