Hello,
The issue type “Test Plan” is presenting the latest tests execution status and also make some calculations, it looks like that:
Thanks, Adi
I’ve tried that command, but it didn’t work:
[Xray Test Plan].CurrentHierarchyMember.Get(‘Overall Execution Status’)
Hi @ValorHeart,
If you want to see the summary of all test executions for the Test plan, you can use the default measure “Xray Tests execution summary” in combination with Xray Test Plan dimension.
In case you want to see only the status of the current test execution, please copy the formula of the “Xray Tests execution summary” and alter it to create a new calculated measure, that has filter conditions only for the measure [Measures].[Xray Tests executing].
Kind regards,
Robert // eazyBI staff
Thanks @roberts.cacus for the answer,
Somewhy, the “Xray Test Plan” dimension shows no data in my eazyBI, even when i remove all filters and select “All test plans” in the dimension, Any idea?
Thanks, Adi
Hi @ValorHeart,
I recently encountered something quite similar. Although in my case no Xray dimension displayed any data with the Xray measures. I resolved this with selecting and deselecting the Xray import in the eazyBI import settings and performed a full import. Please try it and report back.
Kind regards,
Robert // eazyBI staff
Hi,
The “Overall Execution Status” in Jira has added value over the “Xray Tests execution summary”.
Suppose I have a test plan with 14 tests. In a first test execution, I execute the 14 tests and get 3 failed. I do some fixes, then re execute the 3 previous failed tests in another test execution. They become pass.
Only the Overall Execution Status gives the balance sheet of the test plan execution.
How to get the same result with EaszyBI ?
Regards
Hi,
eazyBI does not replicate Xray functionality as those are two different tools. But you may build custom reports and calculations similar to reports and calculations in Xray.
Calculated measure Xray Test execution summary would match overview in the Xray for each Test Execution. When measure Xray Test execution summary is used with a Test Plan, then it returns all execution results within the test plan. If the test is executed twice (has two Test Executions) within a test plan, then it will appear twice in summary.
To get tests only by the last execution status in the selected Test Plan, you may create a new calculated measure in Measures. The calculation would go through all tests within a test plan and for each test get the last execution status in that test plan. The formula might look like this:
UPDATE: The formula is updated to use new dimension name “Xray Test Run Status”. Dimension “Xray Test Execution Status” was renamed to “Xray Test Run Status” in version 6.5.
CASE WHEN
[Xray Test Run Status].CurrentMember.Level.Name = "Status"
THEN --count tests by last execution status
Cache(NonZero(Count(
Filter(
--iterate throug all Tests
Descendants([Xray Test].CurrentMember, [Xray Test].[Test]),
[Measures].[Xray Tests with executions] > 0 AND
Order(
--iterate through execution statuses for a selected Test
Filter([Xray Test Run Status].[Status].Members,
[Measures].[Xray Tests with executions] > 0),
--order statuses by execution date descending
DateParse(
Generate(
Tail(
--iterate through all dates when a Test was executed
Filter([Time].[Day].Members,
[Measures].[Xray Tests with executions] > 0
)).Item(0),
Format([Time].CurrentHierarchyMember.StartDate, 'yyyy-mm-dd'),
",")),
BDESC
--name of the last execution matches status in columns
).item(0) IS [Xray Test Run Status].CurrentMember)
)))
ELSE --total count of tests with executions
([Measures].[Xray Tests with executions],
[Xray Test Run Status].DefaultMember)
END
This calculation is quite resourceful. I recommend selecting option Nonempty and running a report for a single selected Test Plan (see picture below).
Best,
Zane / support@eazyBI.com
Hello, I used this formula and it looks ok in most cases, but there are cases where a test is failed and also exists as To Do in a new test execution. In that case it is shown as failed in the “overall execution status” in the issue navigator, but it is shown as To Do in the above EazyBI formula.
To fix that I’ve added the following condition (AND [Xray Test Run Status].CurrentMember.name <> ‘TODO’)), and I’ve added a new calculated member for To Do:
[Measures].[Xray Test Count by Last Execution Status in Test Plan] is defined as:
CASE WHEN
[Xray Test Run Status].CurrentMember.Level.Name = "Status"
THEN --count tests by last execution status
Cache(NonZero(Count(
Filter(
--iterate throug all Tests
Descendants([Xray Test].CurrentMember, [Xray Test].[Test]),
[Measures].[Xray Tests with executions] > 0 AND
Order(
--iterate through execution statuses for a selected Test
Filter([Xray Test Run Status].[Status].Members,
[Measures].[Xray Tests with executions] > 0
-- ADDITIONAL CONDITION to exclude planned executions
AND [Xray Test Run Status].CurrentMember.name <> 'TODO'),
--order statuses by execution date descending
DateParse(
Generate(
Tail(
--iterate through all dates when a Test was executed
Filter([Time].[Day].Members,
[Measures].[Xray Tests with executions] > 0
)).Item(0),
Format([Time].CurrentHierarchyMember.StartDate, 'yyyy-mm-dd'),
",")),
BDESC
--name of the last execution matches status in columns
).item(0) IS [Xray Test Run Status].CurrentMember)
)))
ELSE --total count of tests with executions
([Measures].[Xray Tests with executions],
[Xray Test Run Status].DefaultMember)
END
[Measures].[Execution Status Pass] =
(
[Measures].[Xray Test Count by Last Execution Status in Test Plan],
[Xray Test Run Status].[PASS]
)
[Measures].[Execution Status Fail] =
(
[Measures].[Xray Test Count by Last Execution Status in Test Plan],
[Xray Test Run Status].[FAIL]
)
To Do is defined as follows:
[Measures].[Xray Tests created] -
[Measures].[Execution Status Pass] -
[Measures].[Execution Status Fail]
UPDATE: The formulas are updated to use the new dimension name “Xray Test Run Status”. Dimension “Xray Test Execution Status” was renamed to “Xray Test Run Status” in version 6.5.
Hi @zane.baranovska!
This is a great example, however, I don’t understand the term
and how it affects the ordering.
Can you explain, please!
Thx,
Johannes
Hi @zane.baranovska!
How to model this specific Xray configuration:
I.e., to model the precedence of final statuses over non-final statuses?
In our configuration PASS, FAIL, ABORTED and CONDITIONAL_PASS are final, while TODO, EXECUTING, BLOCKED, N/A are non-final.
I think it would be the general answer to the extension from @ran.lavi above.
Best regards,
Johannes
Hi Johannes,
The first question about the order
This statement helps to find tests with the latest status, which is the same status as selected in the rows.
The question about the status configuration
Currently, XRay status configuration (order, final/not final) is not taken into account in calculations.
Tests are counted to “XRay Tests executed”, “XRay Tests executing” and “XRay Tests scheduled” by the existence of dates “Started on” and “Finished on” (https://docs.eazybi.com/display/EAZYBIJIRA/Xray+Test+Management#XrayTestManagement-XrayMeasures).
Ilze, support@eazybi.com
Please see also the related topic on how to count Xray tests by the last execution status:
Hi Zane,
I just tried your code but unfortunately, I had the following error:
Formula is not valid:
MDX object ‘[Xray Test Execution Status]’ not found in cube ‘Issues’
could you please help?
Thank you,
Seif
Thank you @Seif_Eddine for mentioning the error.
Dimension “Xray Test Execution Status” is renamed to “Xray Test Run Status” since eazyBI version 6.5. I update expression with the new dimension name.
You’re welcome, Zane.
I tried it. It works but it still doesn’t give the same Xray Overall Execution Status.
Any suggestion, please?
@Seif_Eddine, use the property “Xray Test Plan Overall Execution Status” to see the overall execution status as Xray has calculated it in Jira.
I would recommend adding a parameter to check if the field value is updated on every data import. As it is calculated by Xray based on execution results, the field value might change regularly even Test Plan issue is not updated. See picture below.
UPDATE: since eazyBI version 7.0 the Xray calcauted fields are checked by eazyBI automatically. No need to add the parameter.
On the other hand, the dimension “Xray Test Run Status” would group tests and test executions by the run results and the rest of the report context - used dimensions on report rows, columns, pages, and the selected measure.
Hello @zane.baranovska,
We tried you’re calculation and it works perfectly. Thank you so much.
But for some reports where we used (Xray Test Run Status) in columns.
And we’ve encountered a high execution timeout issue as following:
Do you please have any suggestion for this case?
Best regards,
Seif
@Seif_Eddine the calculation posted in Xray Test plan execution status - #7 by zane.baranovska is very resourceful and might end with the timeout for accounts with larger data volume (more tests and tests executions). And there is not much place left for improvements.
Therefore we suggest using the new property “Xray Test Plan Overall Execution Status” to the summary at the Test Plan level.
Could you please share how we can get this measure into our env?
Unfortunately, we are not able to find it.
Thank you,
Seif
Hi @Seif_Eddine,
If you have eazyBI app on Jira Data center or Server, then go to import options, tab Add ons, and select the new field “Test Plan Status” for data import.
Unfortunately, the new property “Xray Test Plan Overall Execution Status” is not supported for Jira Cloud clients. For more details on available Xray fields, see the documentation: Xray Test Management.
Best,
Zane / support@eazyBI.com
