comments should not change parser state

[jacob-wieland-gematik]

🤔 What's changed?

⚡️ What's your motivation?

🏷️ What kind of change is this?

• 📖 Documentation (improvements without changing code)
• 🏦 Refactoring/debt/DX (improvement to code design, tooling, etc. without changing behaviour)
• 🐛 Bug fix (non-breaking change which fixes a defect)
• ⚡ New feature (non-breaking change which adds new behaviour)
• 💥 Breaking change (incompatible changes to the API)

♻️ Anything particular you want feedback on?

📋 Checklist:

• I agree to respect and uphold the Cucumber Community Code of Conduct
• I've changed the behaviour of the code
  • I have added/updated tests to cover my changes.
• My change requires a change to the documentation.
  • I have updated the documentation accordingly.
• Users should know about my change
  • I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

---

This text was originally generated from a template, then edited by hand. You can modify the template here.

[mpkorstanje]

The parser is generated. To change the parser you'll have to change the grammar.

gherkin/gherkin.berp

Lines 35 to 38 in 00fcc43

	// we need to explicitly mention comment, to avoid merging it into the description line's #Other token
	// we also eat the leading empty lines, the tailing lines are not removed by the parser to avoid lookahead, this has to be done by the AST builder
	DescriptionHelper := #Empty* Description? #Comment*
	Description! := #Other+

To regenerate the parsers you can follow these instructions:

https://github.com/cucumber/gherkin/blob/main/CONTRIBUTING.md#generating-parsers

[jacob-wieland-gematik]

I have changed the grammar to allow any mixture of empty lines, comments and other-lines as the description.

Extracting only the other-lines from the Description-AST-Node (as before) should yield the description-text without the comments and empty lines.

Only difference is now that a collection of ony empty-lines and comments would yield an empty description text instead of a non-existing Description node.

[jacob-wieland-gematik]

Changed the grammar in a way that is more consistent with the former test cases (treating empty lines inside description blocks as '#Other' lines).

[kieran-ryan]

@jacob-wieland-gematik, installing berp on the test-codegen workflow and running the tests on the test-dotnet workflow are failing owing to incompatibilities with ubuntu-24.04 which has now been promoted as the ubuntu-latest GitHub runner and is being picked up in the pipeline. I have pinned those workflows to ubuntu-22.04 for now (#348); and you may want to pull in those changes to allow those workflow steps to proceed.

[jacob-wieland-gematik]

@kieran-ryan I have rebased on latest main and updated a tokens file. Can you approve the checks again?

[jacob-wieland-gematik]

Added dialect.c which was missing for some reason. Please approve checks @kieran-ryan

[jacob-wieland-gematik]

I am not able to generate Parser.php locally, so not sure what to do here @kieran-ryan

[mpkorstanje]

@jacob-wieland-gematik are you able to generate the php parser using the docker container?

https://github.com/cucumber/gherkin/blob/main/CONTRIBUTING.md#generating-parsers

[jacob-wieland-gematik]

@jacob-wieland-gematik are you able to generate the php parser using the docker container?

https://github.com/cucumber/gherkin/blob/main/CONTRIBUTING.md#generating-parsers

No, there seems to be a problem doing that under windows (probably CRLF vs LF related) that hinders this generation.

When change the CRLF of the generated classes.php to LF and then do the generation steps manually, I was able to generate Parser.php

@mpkorstanje