Add LocalToAzureDataLakeStorageOperator

[ephraimbuddy]

This PR adds LocalToAzureDataLakeStorageOperator. This operator will help to add system test for ADLSToGCSOperator.
Also, the upload and download methods of AzureDataLakeHook was improved to accommodate extra arguments since the documentation allows for more arguments.

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

[ephraimbuddy]

@mik-laj, I plan to work on the missing example dags. This Operator is needed to complete the example dag and system test for adls_to_gcs. It will upload a file to Data Lake which will then be transferred to GCS. Kindly review. Thanks

[mik-laj]

@michalslowikowski00 Can you look at it?

[ephraimbuddy]

@mik-laj, @michalslowikowski00, @turbaszek please check if this is necessary :)

[michalslowikowski00]

Do you need to pass empty dict in execute?

[ephraimbuddy]

Got it now, thanks

[ephraimbuddy]

This empty dictionary is necessary so I could use keyword args for the upload_file method

[michalslowikowski00]

Great. I see that this param is needed.

[michalslowikowski00]

We are trying to avoid full stops in documentation.

Suggested change

	are not supported.
	are not supported

[michalslowikowski00]

Looks good to me.
Personally I would love to see:

• example DAG (https://github.com/apache/airflow/tree/master/airflow/providers/microsoft/azure/example_dags)
• system test (https://github.com/apache/airflow/tree/master/tests/providers/microsoft/azure/operators)
• documentation (https://github.com/apache/airflow/tree/master/docs/howto/operator)

There are no docs about Azure and Microsoft operators at all. You could be a pioneer and add docs and make a good example for future contributors. :)

I am sorry for such late review. :(

[ephraimbuddy]

Looks good to me.
Personally I would love to see:

• example DAG (https://github.com/apache/airflow/tree/master/airflow/providers/microsoft/azure/example_dags)
• system test (https://github.com/apache/airflow/tree/master/tests/providers/microsoft/azure/operators)
• documentation (https://github.com/apache/airflow/tree/master/docs/howto/operator)

There are no docs about Azure and Microsoft operators at all. You could be a pioneer and add docs and make a good example for future contributors. :)

I am sorry for such late review. :(

Yeah, I plan to actively work on example dags on this provider and some others but I'm thinking of starting it up on a separate PR. Currently, there's no SystemTest class for this provider. Do you suggest I start SystemTests with this PR or Create another one and do example dags with tests in separate PRs?

[michalslowikowski00]

Looks good to me.
Personally I would love to see:

• example DAG (https://github.com/apache/airflow/tree/master/airflow/providers/microsoft/azure/example_dags)
• system test (https://github.com/apache/airflow/tree/master/tests/providers/microsoft/azure/operators)
• documentation (https://github.com/apache/airflow/tree/master/docs/howto/operator)

There are no docs about Azure and Microsoft operators at all. You could be a pioneer and add docs and make a good example for future contributors. :)
I am sorry for such late review. :(

Yeah, I plan to actively work on example dags on this provider and some others but I'm thinking of starting it up on a separate PR. Currently, there's no SystemTest class for this provider. Do you suggest I start SystemTests with this PR or Create another one and do example dags with tests in separate PRs?

IMHO SystemTest could be in separate PR but example dag should be in this PR. For SystemTest you need example dag anyway.

[ephraimbuddy]

Alright. Thanks

[michalslowikowski00]

if you need help, let me know.

[ephraimbuddy]

I have added the example dag. Will add a doc and system test to verify the example in a separate PR as discussed @michalslowikowski00

[michalslowikowski00]

I have added the example dag. Will add a doc and system test to verify the example in a separate PR as discussed @michalslowikowski00

Great. Looks good for me.
Now someone from the community has to accept it.

[turbaszek]

Can we add type annotations, please? 🙏

[turbaszek]

Same here, otherwise those changes are not related

[turbaszek]

Can you add a reference to how to guide?

[turbaszek]

I would suggest adding those fields to template fields.

[ephraimbuddy]

Hi @turbaszek, please can you take a look once more, thanks.