feat: add junit formatter

[ricalbuquerque]

🤔 What's changed?

• Added JUnit formatter

⚡️ What's your motivation?

Needed to get proper reporting in Azure Pipelines so had to develop a JUnit formatter that would not treat each step as a different test. After testing and started to use it I decided to make it available for the community,

🏷️ What kind of change is this?

• ⚡ New feature (non-breaking change which adds new behaviour)

♻️ Anything particular you want feedback on?

• Feel free to suggest any change or possible improvement!

📋 Checklist:

• I agree to respect and uphold the Cucumber Community Code of Conduct
• I've changed the behaviour of the code
  • I have added/updated tests to cover my changes.
• My change requires a change to the documentation.
  • I have updated the documentation accordingly.
• Users should know about my change
  • I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

---

This text was originally generated from a template, then edited by hand. You can modify the template here.

[coveralls]

Coverage increased (+0.06%) to 98.317% when pulling effb22a on ricalbuquerque:feat/junit-formatter into efd77e3 on cucumber:main.

[davidjgoss]

Thanks for raising this @ricalbuquerque, we do indeed need a JUnit formatter and this looks like a good implementation at first glance.

There's a broader effort in cucumber/common#871 to have a JUnit formatter for all Cucumber implementations that behave the same (share a test suite etc), so I'm going to bring this up there and see if it could be a vehicle to get that piece moving a bit in the right direction, and I'll probably come back with some detailed review comments soon.

[ricalbuquerque]

Thank you @davidjgoss. I don't mean to interfere with any plans for a new formatter. I did try a few formatters (some of them seem to be a bit old) and none would do what I was expecting to (treating scenarios as testcases) so this was my motivation.
I did see that issue on the common repo but since it seemed to be a bit blocked I decided to implement a version of it in javascript since I don't have that much experience in Go.
Feel free to suggest any improvements you would like to see!

[davidjgoss]

@ricalbuquerque not an interference at all, I think this could help move it forward. I don't think there will be a single implementation in Go, that model didn't work great in practise for other components. More like, a shared set of tests and then native implementations in JavaScript, Java, Ruby etc that all pass those same tests. So this is definitely in line with that.

[mpkorstanje]

Looking at the test cases, I feel there may be a lot of fidelity missing. I can't provide concrete examples right now but contrast with the Cucumber JVM test cases:

https://github.com/cucumber/cucumber-jvm/blob/main/cucumber-core/src/test/java/io/cucumber/core/plugin/JUnitFormatterTest.java

And so it might be good to compare against the XSD.

https://github.com/cucumber/cucumber-jvm/blob/main/cucumber-core/src/test/resources/io/cucumber/core/plugin/surefire-test-report-3.0.xsd

[davidjgoss]

Following on from previous comments, just calling out things that are different from the style seen in cucumber-jvm which I think we should align with:

• All steps from the scenario with their statuses are printed with a consistent line width in the <system-out> tag, see https://github.com/cucumber/cucumber-jvm/blob/ee6b693184b463c023a265fe98fa9ab5ab2ce819/cucumber-core/src/test/java/io/cucumber/core/plugin/JUnitFormatterTest.java#L69
• There is a single <testsuite> (not one for each feature), and each <testcase> uses the feature name as classname and scenario name as name respectively
• Failures have the type and message as attrs on the <failure> tag and the stack in the body of that tag - I don't think we can do this with cucumber-js because the message and stack are not separated, so we'll have to omit message attr

I'm happy to jump in and help out with any of this if you'd like, just let me know. Thanks again for putting this together.

[tim-yao]

We need this as CircleCI will stop support for JSON format in 20 days. Here is the announcement: https://discuss.circleci.com/t/cucumber-test-results-format-deprecation/45776

Starting November 20th, all projects using the Cucumber test runner must store test results using the junit format.

This PR is the only way to continue using cucumber with CircleCI. The existing JUnit formatted plugin is not working with new cucumber-js versions:
mvtm-dn/cucumber-junit-formatter#10

[davidjgoss]

@ricalbuquerque are you happy for me to make some of the changes discussed above so we can get this in?

[mpkorstanje]

Looks good! Spotted a few complicated problems.

[davidjgoss]

@mpkorstanje I've made some updates to bring this pretty much in line with what the cucumber-jvm one does, and added more tests. Would you mind taking another look before I merge?

[davidjgoss]

Huge thanks @ricalbuquerque for contributing this!

Changes done

[aslakhellesoy]

Hi @ricalbuquerque,

Thanks for your making your first contribution to Cucumber, and welcome to the Cucumber committers team! You can now push directly to this repo and all other repos under the cucumber organization! 🍾

In return for this generous offer we hope you will:

• ✅ Continue to use branches and pull requests. When someone on the core team approves a pull request (yours or someone else's), you're welcome to merge it yourself.
• 💚 Commit to setting a good example by following and upholding our code of conduct in your interactions with other collaborators and users.
• 💬 Join the community Slack channel to meet the rest of the team and make yourself at home.
• ℹ️ Don't feel obliged to help, just do what you can if you have the time and the energy.
• 🙋 Ask if you need anything. We're looking for feedback about how to make the project more welcoming, so please tell us!

On behalf of the Cucumber core team,
Aslak Hellesøy
Creator of Cucumber

[mpkorstanje]

@mpkorstanje I've made some updates to bring this pretty much in line with what the cucumber-jvm one does, and added more tests. Would you mind taking another look before I merge?

Looks good!

[davidjgoss]

Released in https://github.com/cucumber/cucumber-js/releases/tag/v8.9.0