Provide ability to share same report portal launch ID

[sumit-gupta91]

Playwright provides support for grouping tests using annotations. Using these annotations I can group tests and run them in the order we want.

For example :
yarn run e2e -- --grep '@flow=auth' --project 'chromium'
yarn run e2e -- --grep-invert '@flow=auth'

Above, all the auth test cases are run first followed by all the other test cases which can leverage already established auth sessions.

The problem is, report portal integration generates different launch ID for each of these runs even though they are for the same commitID, which causes problems in reporting on report portal.

[AmsterGet]

As the Playwright has excellent support of running tests in parallel or sequentially out of the box, we didn't consider this feature for agent-js-playwright.
Recently (in version 1.31.1) the Playwright introduced the project dependencies, seems like this can help you make the dependency on establishing auth sessions for other tests.
See also Playwright release notes.

[sumit-gupta91]

@AmsterGet Sure, let me experiment with new version of playwright and see if it fulfills all the needs. If not will revert back on this issue.

[sterschl]

I have another use case that goes in a similar direction:

I use GitlabCI to run Playwright tests for five different browsers. To increase performance, each browser runs parallel in a separate job. However, this also gives me five different launches in the ReportPortal. I would like to combine the results into one launch.

[AmsterGet]

Hello @sterschl !
Since there are independent runs on different machines, Playwright does not provide an orchestration mechanism for their reports.
From our side, I can offer two options:

1. Trigger a launch before all tests manually via the API, then specify its id in the reporter config for each job.
   As a run post-step (when all tests finished), launch also needs to be finished manually.
   Our client-javascript already supports the existing launch id for reporting, but it's not yet implemented in this agent, so it needs to be implemented by our team. ETA is unknown at the moment, because our priorities are concentrated on other things.
2. Specify a unique CI run ID as a launch attribute, which will be the same for different jobs in the same run (this could be a commit hash or something else).
   As a follow up step, you can use the API to filter the required launches by the provided attribute, collect their ids, and then simply call the "merge" launch API endpoint to combine the launches into one.

Please let me know if there are any questions.

[sterschl]

Hi @AmsterGet,
thanks for your help and suggestions. I have implemented your second option and it works very well.

The only disadvantage is that the merge can only be done after the tests are finished. But this is not a problem. It would be great if in the future the Id could also be set via the playwright client. Then I would probably try the first option.

[AmsterGet]

Hi @sterschl !
That's great!
I've added an improvement with providing an existing launch id to the agent config in our roadmap.
I will let you know as soon as it is implemented.
Thanks for your feedback!

[AmsterGet]

@sumit-gupta91 Hello!
Have you tried my suggestion for your request?
Is it works?

[sumit-gupta91]

@AmsterGet We are in process to upgrade to latest version of playwright and use the projects approach. If we have any more such requirements then I would prefer 1st approach as we are doing something similar with our unit tests as well. We use report portal for creating reports of unit tests runs as well and it has a similar issue.

reportportal/agent-js-jest#121

We use agent-js-jest ability to accept report portal launch ID.

[AmsterGet]

Hello guys!
@sumit-gupta91 @sterschl
I've added support of existing launch ID to this agent.
I plan to release it this week with some small other improvements.
I'll notify you when it will be ready.

[AmsterGet]

@sumit-gupta91 @sterschl
Published in agent-js-playwright 5.1.0.