Xray Test plan execution status

Questions & Answers
,
21

Hi Zane,

Thank you so much, now we had this measure, and it is super faster with large xRay test plans, and it gives the same statics as we are expecting.

But what I found with this measure is that it doesn’t filter the values when a custom field dimension added in rows. (Same we did with above MDX)

Best regards,
Seif

22

This issue of performance is resolved, it is not due to the MDX calculation mentioned above but instead it is because of another selected dimension for status execution selected in columns, once that dimension is removed that would no longer be any performance issue even with test plan having large tests.

23

Unfortunately, I noticed an issue with the statics in the new reports when I used the “TestRunStatus” dimension instead of the “Xray Test Run Status” dimension. The “TestRunStatus” doesn’t give the last run only instead it gives the cumulative execution of a test. (Also mentioned in 12150/3)

24

Hi @zane.baranovska,

I hope you’re doing well.

We noticed an error in the calculation when we selected multiple test plans at the same time.

The report context is as following:

Pages:

  • Test Plan.

Rows:

  • A Jira custom field.

Columns:

  • Xray Test Plan Last Run Status (Your calculation taken from above)
  • Xray Test Run Status dimension

The issue happens when we select multiple more than one test plan then the calculation will be wrong.

For e.g. I selected two test plans and I had in the column Pass: 11 (instead of having 15)

The error happens with the Fail column as well. Can you please check this from your side? Any help would be really appreciated. It is as urgent issue for us.

Thank you,
Seif

25

Since eazyBI version 7.2 and on Cloud, there are new measures “Xray Test Runs created first date” and “Xray Test Runs created last date” representing the date when a Test was added to a Test Execution. It might be handy to consider tests that are only scheduled and the test run has not started yet.
More details on how to obtain new measures described in the documentation: Xray Test Management

For example, you can retrieve the test run creation date if there is no test execution date yet (for scheduled test runs):

--get the execution date or test run creation date if the former is not present
CoalesceEmpty(
  [Measures].[Xray Tests executed last date],
  [Measures].[Xray Test Runs created last date]
)

Then, one of the variations to see the last status for completed and not completed execution results might be as below:

CASE WHEN
  [Xray Test Run Status].CurrentMember.Level.Name = "Status"
THEN
  --count tests by last execution status
  Cache(NonZero(Count(
    Filter(
      --itesrate throguh individual Tests
      Descendants([Xray Test].CurrentMember, [Xray Test].[Test]),
      --test has execution in report context
      [Measures].[Xray Tests with executions] > 0 AND
      Order(
        --iterate through execution statuses for a selected Test
        Filter([Xray Test Run Status].[Status].Members,
          [Measures].[Xray Tests with executions] > 0),
        --order statuses descending by execution date or test run creation date
        CoalesceEmpty(
          [Measures].[Xray Tests executed last date],
          [Measures].[Xray Test Runs created last date]),
        BDESC
        --name of the last execution matches the status in columns
      ).item(0) IS [Xray Test Run Status].CurrentMember
  ))))   
ELSE
  ([Measures].[Xray Tests with executions],
  [Xray Test Run Status].DefaultMember)
END

Please note that results might not match with Test Plan Overall Execution Status in Jira as that field gets calcauted on the issue screen, and its logic is determined by Xray settings for the project and filter criteria on Test Plan issue (final stauses, etc.).

Best,
Zane / support@eazyBI.com