[AIRFLOW-17200] Add Alibaba Cloud OSS support #17201
Conversation
Gabriel39
force-pushed
the
support_alibabacloud
branch
from
d2b5143 to
141518a
Compare
|
FYI: This PR is WIP. For now, OSS support is almost done except documents, and I will follow up soon. So feel free to comment for current work. Thx. |
Gabriel39
force-pushed
the
support_alibabacloud
branch
from
141518a to
11f5615
Compare
Gabriel39
force-pushed
the
support_alibabacloud
branch
2 times, most recently
from
7ffd2c1 to
ad08cdd
Compare
Gabriel39
changed the title
Gabriel39
force-pushed
the
support_alibabacloud
branch
from
ad08cdd to
eca772f
Compare
|
What do you think about adding system tests to these operators? |
Gabriel39
force-pushed
the
support_alibabacloud
branch
from
eca772f to
9fe6022
Compare
Hi @mik-laj , I� updated my code and resubmitted this PR. Thank you for your advises! |
Gabriel39
force-pushed
the
support_alibabacloud
branch
from
9fe6022 to
c74716b
Compare
|
cc: @vikramkoka thx :) |
Gabriel39
force-pushed
the
support_alibabacloud
branch
from
c74716b to
24121a0
Compare
|
@vikramkoka @mik-laj need review. kind remind. Thx :) |
Gabriel39
force-pushed
the
support_alibabacloud
branch
3 times, most recently
from
f8930b0 to
6bc3460
Compare
Gabriel39
force-pushed
the
support_alibabacloud
branch
14 times, most recently
from
5b1abc5 to
a842146
Compare
Gabriel39
force-pushed
the
support_alibabacloud
branch
from
a842146 to
8f0ea49
Compare
potiuk
left a comment
potiuk
left a comment
There was a problem hiding this comment.
Looks fantastic. Just one small NIT - the provider version should be 1.0.0
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
@potiuk Updated. Thx :) |
|
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
github-actions
bot
added
the
full tests needed
|
This makes every (or almost every) test run try to reach out to alibaba cloud in the oss-cn-hangzhou region, and is currently failing builds on main https://github.com/apache/airflow/runs/3290445260 From what I can tell this bucket region is unreachable outside China, which means in the best case these tests will be skipped, and in the worst case as now they will be skipped. I think we have to revert this PR until we have a fix for this. Or at the very least we will skip these tests and we can't release this provider until the tests are actually running. |
6 participants
Alibaba Cloud Object Storage Service (OSS) is an encrypted, secure, cost-effective, and easy-to-use object storage service that enables you to store, back up, and archive large amounts of data in the cloud, with a guaranteed durability of 99.9999999999%(12 9’s). (refer to https://www.alibabacloud.com/product/oss)
This PR is to enable Airflow users using OSS.