Skip to content

feat: Implement stat dir correctly based on RFC-3243 List Prefix#3651

Merged
Xuanwo merged 53 commits intomainfrom
fix-stat
Nov 23, 2023
Merged

feat: Implement stat dir correctly based on RFC-3243 List Prefix#3651
Xuanwo merged 53 commits intomainfrom
fix-stat

Conversation

@Xuanwo
Copy link
Copy Markdown
Member

@Xuanwo Xuanwo commented Nov 22, 2023 •
edited
Loading

Part of #3247

This PR changed the behavior of stat as described in RFC-3243 List Prefix. In short:

  • We will return file not found if we are trying to stat file abc with abc/.
  • We will handle dir correctly by calling list with prefix.

This change will make sure we return correct dir metadata on s3 alike storage services.

I will update upgrade.md in following PRs since this PR is so large that any single change will trigger all CI to run.

Xuanwo added 23 commits November 21, 2023 15:57
@Xuanwo
polish stat docs
fc125ff 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Add behavior docs
bb1a6ac 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Add more test cases for stat
46b9040 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix kv adapter
fadf4e6 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix kv on new test cases
e8181fd 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Implement for s3
744c5cf 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix fs
435e548 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix azblob
501b718 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix azdls
592161b 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Merge remote-tracking branch 'origin/main' into fix-stat
7441aaa
@Xuanwo
Polish azfile
bc903fc 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix cos
b66c7cd 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix dropbox
238b081 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix ftp
8b3405f 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix gcs
eb953e6 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix gdrive
fb9c92f 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix ghac
6e778e5 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix hdfs
e9e9c7f 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix obs
cfdd0ea 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
FIx oss
e39939b 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
FIx test
44f9447 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix swift
e746e74 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
FIx webhdfs
c0af347 
Signed-off-by: Xuanwo <github@xuanwo.io>
@github-actions github-actions Bot added the releases-note/feat The PR implements a new feature or has a title that begins with "feat" label Nov 22, 2023
@Xuanwo Xuanwo mentioned this pull request Nov 22, 2023
Closed
1 task
@Xuanwo Xuanwo marked this pull request as draft November 22, 2023 12:10
@Xuanwo
Fix ghac
697449c 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Address at complete layer side
3fc595e 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Address root
1af8ead 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Address alluxio
c7a26a6 
Signed-off-by: Xuanwo <github@xuanwo.io>
Xuanwo added 10 commits November 23, 2023 12:10
@Xuanwo
Merge remote-tracking branch 'origin/main' into fix-stat
af638db
@Xuanwo
Fix test
242b3bf 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix bindings test
0a65800 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Format code
6601a90 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
FIx services
950409b 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Remove create_dir for services that not supportted
177c9d7 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix test
d4c95ce 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix azfile
b80c519 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix java code format
4e0ea92 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix webdav
c848d91 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo Xuanwo marked this pull request as ready for review November 23, 2023 05:41
@Xuanwo
Fix etcd
43dbe1e 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Fix swift
34020e5 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Merge remote-tracking branch 'origin/main' into fix-stat
da97115
@Xuanwo
Fix swift
dcbfb78 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Copy link
Copy Markdown
Member Author

Xuanwo commented Nov 23, 2023

dropbox failed for too_many_write_operations.

suyanhanx
suyanhanx reviewed Nov 23, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@suyanhanx suyanhanx left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This must be a breaking change. Please update the upgrade guide.

Comment thread core/src/raw/oio/list/api.rs
Comment thread core/tests/behavior/write.rs
Comment thread core/src/types/operator/operator.rs
@Xuanwo Xuanwo merged commit 0711e6c into main Nov 23, 2023
@Xuanwo Xuanwo deleted the fix-stat branch November 23, 2023 08:27
@Xuanwo Xuanwo mentioned this pull request Nov 23, 2023
Closed
@CookiePieWw CookiePieWw mentioned this pull request Nov 23, 2023
Closed
@G-XD G-XD mentioned this pull request Nov 27, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@suyanhanx suyanhanx suyanhanx left review comments

@PsiACE PsiACE PsiACE approved these changes

@G-XD G-XD Awaiting requested review from G-XD

@ClSlaid ClSlaid Awaiting requested review from ClSlaid

@messense messense Awaiting requested review from messense messense is a code owner

@Zheaoli Zheaoli Awaiting requested review from Zheaoli Zheaoli is a code owner

@tisonkun tisonkun Awaiting requested review from tisonkun tisonkun is a code owner

Assignees

No one assigned

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat"

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Invalid metadata for dir objects in s3 is_exist always return true for key end with '/', in S3 service

3 participants

@Xuanwo @suyanhanx @PsiACE