Tracking issues of storage backend integration tests

[Xuanwo]

• aws s3: tested by minio now. We can add real s3 bucket in with sponsor.
• aliyun oss: need an oss bucket (better to locate near us-east-1)
• azure adlsv2: can be tested by Azurite. And I'm willing to provide test infra as Microsoft MVP.
• gcp gcs: need a gcs bucket (better to locate near us-east-1)
• hadoop hdfs: can setup in CI directly (thanks open source!)

I agree that we can label these features as experimental. Setting up the CI infrastructure requires time, more so than implementing those features.

Originally posted by @Xuanwo in #408 (comment)

[jdockerty]

@Xuanwo I can take a shot at GCP GCS if nobody else is working on that at the moment 👍

[liurenjie1024]

@Xuanwo I can take a shot at GCP GCS if nobody else is working on that at the moment 👍

Hi, @jdockerty thanks for you interest. How do you plan to do it? Is there any open source containers that could mock gcs?

[jdockerty]

How do you plan to do it? Is there any open source containers that could mock gcs?

@liurenjie1024

There is this mock which I found earlier: https://github.com/fsouza/fake-gcs-server

Failing this, my plan was to use an actual GCS bucket for testing the functionality - given this was mentioned at being potentially experimental for now. Granted, this isn't ideal for CI, but the mock from above looks relatively well-used at first glance 👍

[liurenjie1024]

How do you plan to do it? Is there any open source containers that could mock gcs?

@liurenjie1024

There is this mock which I found earlier: https://github.com/fsouza/fake-gcs-server

Failing this, my plan was to use an actual GCS bucket for testing the functionality - given this was mentioned at being potentially experimental for now. Granted, this isn't ideal for CI, but the mock from above looks relatively well-used at first glance 👍

Thanks @jdockerty , using this mock is fine to me.

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'