Hi Zane,
Thank you so much, now we had this measure, and it is super faster with large xRay test plans, and it gives the same statics as we are expecting.
But what I found with this measure is that it doesn’t filter the values when a custom field dimension added in rows. (Same we did with above MDX)
Best regards,
Seif
This issue of performance is resolved, it is not due to the MDX calculation mentioned above but instead it is because of another selected dimension for status execution selected in columns, once that dimension is removed that would no longer be any performance issue even with test plan having large tests.
Unfortunately, I noticed an issue with the statics in the new reports when I used the “TestRunStatus” dimension instead of the “Xray Test Run Status” dimension. The “TestRunStatus” doesn’t give the last run only instead it gives the cumulative execution of a test. (Also mentioned in 12150/3)
Hi @zane.baranovska,
I hope you’re doing well.
We noticed an error in the calculation when we selected multiple test plans at the same time.
The report context is as following:
Pages:
Rows:
Columns:
The issue happens when we select multiple more than one test plan then the calculation will be wrong.
For e.g. I selected two test plans and I had in the column Pass: 11 (instead of having 15)
The error happens with the Fail column as well. Can you please check this from your side? Any help would be really appreciated. It is as urgent issue for us.
Thank you,
Seif
Since eazyBI version 7.2 and on Cloud, there are new measures “Xray Test Runs created first date” and “Xray Test Runs created last date” representing the date when a Test was added to a Test Execution. It might be handy to consider tests that are only scheduled and the test run has not started yet.
More details on how to obtain new measures described in the documentation: Xray Test Management
For example, you can retrieve the test run creation date if there is no test execution date yet (for scheduled test runs):
--get the execution date or test run creation date if the former is not present
CoalesceEmpty(
[Measures].[Xray Tests executed last date],
[Measures].[Xray Test Runs created last date]
)
Then, one of the variations to see the last status for completed and not completed execution results might be as below:
CASE WHEN
[Xray Test Run Status].CurrentMember.Level.Name = "Status"
THEN
--count tests by last execution status
Cache(NonZero(Count(
Filter(
--itesrate throguh individual Tests
Descendants([Xray Test].CurrentMember, [Xray Test].[Test]),
--test has execution in report context
[Measures].[Xray Tests with executions] > 0 AND
Order(
--iterate through execution statuses for a selected Test
Filter([Xray Test Run Status].[Status].Members,
[Measures].[Xray Tests with executions] > 0),
--order statuses descending by execution date or test run creation date
CoalesceEmpty(
[Measures].[Xray Tests executed last date],
[Measures].[Xray Test Runs created last date]),
BDESC
--name of the last execution matches the status in columns
).item(0) IS [Xray Test Run Status].CurrentMember
))))
ELSE
([Measures].[Xray Tests with executions],
[Xray Test Run Status].DefaultMember)
END
Please note that results might not match with Test Plan Overall Execution Status in Jira as that field gets calcauted on the issue screen, and its logic is determined by Xray settings for the project and filter criteria on Test Plan issue (final stauses, etc.).
Best,
Zane / support@eazyBI.com
Hello all
I wanted to share final result of a test plan report created in the cloud, with filter by test plan and requirement
please notice that it is also possible to have requirement hierarchy according to the documentation.
{
"cube_name": "Issues",
"cube_reports": [ {
"name": "Test plan report",
"result_view": "table",
"definition": {"columns":{"dimensions":[{"name":"Measures","selected_set":["[Measures].[Last run result according to scope]","[Measures].[Sucess rate]"],"members":[{"depth":0,"name":"Latests","full_name":"[Measures].[Latests]","format_string":"#,##0","report_specific":true,"calculated":true},{"depth":0,"name":"Test Status Summary","full_name":"[Measures].[Test Status Summary]","format_string":"#,##0","report_specific":true,"calculated":true},{"depth":0,"name":"Last run result according to scope","full_name":"[Measures].[Last run result according to scope]","format_string":"#,##0","calculated":true,"drilled_into_dimension":"Xray Test Run Status"}]},{"name":"Xray Test Run Status","selected_set":["[Xray Test Run Status].[Status].Members"],"members":[],"bookmarked_members":[]}]},"rows":{"dimensions":[{"name":"Xray Requirement","selected_set":["[Xray Requirement].[Requirement].Members"],"members":[],"bookmarked_members":[]}],"nonempty_crossjoin":true},"pages":{"dimensions":[{"name":"Xray Test Plan","selected_set":["[Xray Test Plan].[All Xray Test Plans]"],"members":[{"depth":0,"name":"All Xray Test Plans","full_name":"[Xray Test Plan].[All Xray Test Plans]","drillable":true,"type":"all","expanded":true,"drilled_into":false},{"depth":1,"name":"scrum team","full_name":"[Xray Test Plan].[scrum team]","drillable":true,"key":"ST","expanded":true,"drilled_into":false,"parent_full_name":"[Xray Test Plan].[All Xray Test Plans]"},{"depth":2,"name":"ST-25","full_name":"[Xray Test Plan].[scrum team].[ST-25]","key":"ST-25","parent_full_name":"[Xray Test Plan].[scrum team]"}],"bookmarked_members":[],"current_page_members":["[Xray Test Plan].[scrum team].[ST-25]"]}]},"options":{},"view":{"current":"table","maximized":false,"table":{"freeze_header":false}},"calculated_members":[]}
} ],
"calculated_members": [{"dimension":"Measures","name":"Xray Test Runs created last date","format_string":"yyyy-mm-dd hh:nn:ss","formula":"TimestampToDate([Measures].[Xray Test Runs created last timestamp])"},{"dimension":"Measures","name":"Xray Tests executed last date","format_string":"yyyy-mm-dd hh:nn:ss","formula":"TimestampToDate([Measures].[Xray Tests executed last timestamp])"},{"name":"Sucess rate","dimension":"Measures","formula":"case when\n [Measures].[Last run result according to scope] \u003e 0\nTHEN\n ([Measures].[Last run result according to scope], [Xray Test Run Status].[PASS]) /\n [Measures].[Last run result according to scope]\nEND\n","format_string":"#0.00%"},{"name":"Last run result according to scope","dimension":"Measures","formula":"CASE WHEN\n [Xray Test Run Status].CurrentMember.Level.Name = \"Status\"\nTHEN\n --count tests by last execution status\n Cache(NonZero(Count(\n Filter(\n --itesrate throguh individual Tests\n Descendants([Xray Test].CurrentMember, [Xray Test].[Test]),\n --test has execution in report context\n [Measures].[Xray Tests with executions] \u003e 0 AND\n Order(\n --iterate through execution statuses for a selected Test\n Filter([Xray Test Run Status].[Status].Members,\n [Measures].[Xray Tests with executions] \u003e 0),\n --order statuses descending by execution date or test run creation date\n CoalesceEmpty(\n [Measures].[Xray Tests executed last date],\n [Measures].[Xray Test Runs created last date]),\n BDESC\n --name of the last execution matches the status in columns\n ).item(0) IS [Xray Test Run Status].CurrentMember\n )))) \nELSE\n ([Measures].[Xray Tests with executions],\n [Xray Test Run Status].DefaultMember)\nEND\n","format_string":""}]
}
Hello All , Again , this is Aya to update that once we started to have more and more requirments and more and more plans and runs the report got a lot of time to run, we use Data center and we had some preformace issues.
luck for us now we can use AI report builder and optimizer built in eazybi and I was able to improve the report preformance , here is the information needed in case youcome across the case .
measure. Last run status by Plan
CASE WHEN
[Xray Test Run Status].CurrentMember.Level.Name = “Status”
THEN
Cache(
NonZero(
Sum(
Filter(
Descendants([Xray Test].CurrentMember, [Xray Test].[Test]),
[Measures].[Xray Tests with executions] > 0
),
CASE WHEN
[Measures].[new Xray Test last execution status] =
[Xray Test Run Status].CurrentMember.Name
THEN
1
END
)
)
)
ELSE
(
[Measures].[Xray Tests with executions],
[Xray Test Run Status].DefaultMember
)
END
inner measure
[Measures].[new Xray Test last execution status]
CASE WHEN [Measures].[Xray Tests with executions] > 0
THEN
TopCount(
Filter(
[Xray Test Run Status].[Status].Members,
[Measures].[Xray Tests with executions] > 0
),
1,
CoalesceEmpty([Measures].[Xray Tests executed last date],[Measures].[Xray Test Runs created last date])
).Item(0).Name
END
and now I just copy pase the explain from the AI improver -
Order() and Filter() that was ordering statuses by execution date for each test[Measures].[new Xray Test last execution status] instead of recalculating the last status.item(0) IS comparison with a simple string equality checkThis optimization should reduce the execution time of this measure by 60-80%, as it eliminates the most expensive operations while maintaining identical functionality. The overall report execution time should improve from 74ms to approximately 45-55ms.
thank you so much Eazybi team for your effort in combining the AI with Eazybi it really help to learn more and more .