Implement the official gherkin parser

[jsa34]

Implementation of the official Gherkin parser.
Trying to maintain the public API and current codebase as much as possible. I was thinking little steps, and introduce new features/refactoring after we know it works like-for-like with current feature set.

Credit to @elchupanebrej for his implementation I referenced in pytest-bdd-ng

Breaking changes
Some of the previous features that are not Gherkin-compliant I had to remove. I have tried to find workarounds where possible.

Changes include:

• multiline steps not using triple quotes no longer work
• tags cannot have spaces in them

NB: This simply replaces the parsing logic with the Gherkin official parser, but the current parser does some additional logic and makes custom entities that are needed for test execution. I have coerced the Gherkin Document data to the existing model (minimising change), but I have raised a new MR (#699) that completely removes the existing parser to replace with Gherkin Document data generated. Not sure which is best route to take.

[sourcery-ai]

🧙 Sourcery has finished reviewing your pull request!

---

Tips

• Trigger a new Sourcery review by commenting @sourcery-ai review on the pull request.
• Continue your discussion with Sourcery by replying directly to review comments.
• You can change your review settings at any time by accessing your dashboard:
  • Enable or disable the Sourcery-generated pull request summary or reviewer's guide;
  • Change the review language;
• You can always contact us if you have any questions or feedback.

[sourcery-ai]

We've reviewed this pull request using the Sourcery rules engine. If you would also like our AI-powered code review then let us know.

[jsa34]

@youtux please let me know if this is the right way to go.
I also see that in the parser, a few of the entities seem to reference each other, and I wonder if we might be able to avoid it by refactoring (e.g. Step has an optional Background, and Background has a list of steps) - this would be a future endeavour. I'm just thinking it's best to clean up the parser and entities before adding features if this is needed.

[jsa34]

Finished tinkering and fixing things for the CI jobs :)

[jsa34]

@sourcery-ai review

[sourcery-ai]

We've reviewed this pull request using the Sourcery rules engine. If you would also like our AI-powered code review then let us know.

[youtux]

Good stuff! A few comments, but we are in the right track! Thank you for your effort

[jsa34]

I'll take a look into your comments.

Most I can already see I agree with.

I was in 2 minds about Pydantic - it's made model validation much easier when ingesting the gherkin document dict, but I can absolutely see your point. I will have a look :)

[jsa34]

Thank you for your feedback and time to look at this, @youtux!

I've resolved most comments - just a couple of things:

• given_when_then comments' response -> just need a steer
• the escaping I have fixed the way I could best see, but would appreciate a second pair of eyes.

[youtux]

can you also add a changelog entry in CHANGES.rst with all the backwards incompatible changes?
So far I could see that we don't accept spaces in tags and that now Feature: is required in the feature file definitions.

[codecov]

Codecov Report

Attention: Patch coverage is 96.75926% with 14 lines in your changes missing coverage. Please review.

Project coverage is 95.23%. Comparing base (cec07df) to head (572934c).
Report is 41 commits behind head on master.

Files with missing lines	Patch %	Lines
src/pytest_bdd/gherkin_parser.py	97.53%	2 Missing and 2 partials ⚠️
src/pytest_bdd/parser.py	96.42%	1 Missing and 3 partials ⚠️
src/pytest_bdd/scenario.py	77.77%	1 Missing and 1 partial ⚠️
src/pytest_bdd/utils.py	0.00%	2 Missing ⚠️
src/pytest_bdd/generation.py	66.66%	0 Missing and 1 partial ⚠️
src/pytest_bdd/steps.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #698      +/-   ##
==========================================
- Coverage   95.37%   95.23%   -0.14%     
==========================================
  Files          50       53       +3     
  Lines        1838     1994     +156     
  Branches      201      228      +27     
==========================================
+ Hits         1753     1899     +146     
- Misses         53       58       +5     
- Partials       32       37       +5     

Flag	Coverage Δ
	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[youtux]

Congrats! We are now using the official parser!

[jsa34]

Thank you, @youtux 🙌