Skip to content

feat: testing framework in python and microsoft fabric support with expression grammar and evaluation support#30

Merged
arjendev merged 131 commits into
mainfrom
feature/python-codebase
Dec 8, 2023
Merged

feat: testing framework in python and microsoft fabric support with expression grammar and evaluation support#30
arjendev merged 131 commits into
mainfrom
feature/python-codebase

Conversation

@arjendev
Copy link
Copy Markdown
Collaborator

@arjendev arjendev commented Nov 15, 2023 •
edited
Loading

This PR transforms our framework from dotnet to python with an insane amount of new features like:

  1. Support for Fabric and Azure Data Factory with a unified model to support any activity and be resilient for future changes in activities
  2. Expression parsing with lark-parser with our own grammar definition and evaluation
  3. 100% expression function call coverage with a few missing cases around xml / date formats)
  4. All iteration activities are implemented (e.g. switch, foreach, filter, set-variable etc..)
  5. Child activity evaluation and pipelineReturnValue support

And a lot more...

arjendev and others added 30 commits November 8, 2023 10:37
@arjendev
feat: introduce initial python code base and patching script to fix a…
354b349 
…zure-sdk datafactory models
@arjendev
fix: fix imports of scripts
82058f9
@arjendev
chore: migrate to pytest
3a1a016
@arjendev
feat: model and data factory repositories, dynamically patching gener…
149d995 
…ated models
@arjendev
feat: testframework pipeline evaluation logic without iteration activ…
ea7a209 
…ities
@arjendev
chore: move examples to example folder
29c9ff6
@arjendev
feat: foreach support and refactor model patching
314a1f4
@arjendev
refactor: automatically patching of models
bbc159a
@arjendev
feat: if condition activity
d731665
@arjendev
feat: until activity
d85fa3a
@arjendev
feat: setVariable activity
036c370
@arjendev
feat: pipeline activity
4a5f63f
@arjendev
feat: function_call, function_argument and expression_parsing
a01458d
@arjendev
feat: function argument expression evaluation
9b2b2f3
@arjendev
feat: function argument activity output evaluation
e83afbb
@arjendev
feat: function_call evaluation so far
386c7b6
@arjendev
feat: function_call evaluation, optional parameters, apply leonards f…
f3e8c47 
…eedback and bunch of improvements
@arjendev
test: add function_call tests
5fe72b0
@arjendev
fix: function_call error argument naming
853db41
@LeonardHd
chore(python): add ruff linting
b8ac613
@LeonardHd
style(python): apply ruff auto fixes
30c7178
@LeonardHd
style(python): apply ruff unsafe auto fixes
b8ddb34
@LeonardHd
style(python): ruff fix E501 and ruff formatting
d4eb7ac
@LeonardHd
style(python): ruff fixes ANN001,ANN201,ANN401
9e9354e
@LeonardHd
style(python): fix new line formatting
30c5f44
@LeonardHd
style(python): format test files
efa809a
@LeonardHd
style(python): ruff E741
85dc7c4
@LeonardHd
fix(python): import docstring parse
aa0921b
@LeonardHd
style(python): fix N806
3c3ab64
@LeonardHd
style(python): various ruff fixes on style
62d74e3
@arjendev arjendev changed the title feat: testing framework in python and microsoft fabric support feat: testing framework in python and microsoft fabric support with expression grammar and evaluation support Dec 1, 2023
@arjendev arjendev marked this pull request as ready for review December 1, 2023 18:12
@arjendev arjendev self-assigned this Dec 1, 2023
@arjendev arjendev added the enhancement New feature or request label Dec 1, 2023
@arjendev arjendev added this to the 2023-12 milestone Dec 1, 2023
This was linked to issues Dec 1, 2023
Implement all functions in functions_repository in python version #31
Closed
Support system variables #6
Closed
Evaluate child pipelines #7
Closed
jayachithra
jayachithra previously approved these changes Dec 8, 2023
View reviewed changes
Comment thread .github/workflows/ci-cd.yml Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jayachithra jayachithra jayachithra approved these changes

@sreedhar-guda sreedhar-guda Awaiting requested review from sreedhar-guda

@ydaponte ydaponte Awaiting requested review from ydaponte

Assignees

@arjendev arjendev

Labels

enhancement New feature or request

Projects

Azure Data Factory Testing Framework
Status: Done

Milestone

2023-12

3 participants

@arjendev @jayachithra @LeonardHd