Skip to content

RFC-2133: Introduce Append API#2133

Merged
Xuanwo merged 5 commits intoapache:mainfrom
ShadowySpirits:append_operation
Apr 29, 2023
Merged

RFC-2133: Introduce Append API#2133
Xuanwo merged 5 commits intoapache:mainfrom
ShadowySpirits:append_operation

Conversation

@ShadowySpirits
Copy link
Copy Markdown
Member

Discussed in #2115

Originally posted by ShadowySpirits April 25, 2023
OpenDAL provides append-alike operation, which is based on multipart API. But this approach has some limitations:

  • Data could be lost and not readable before w.close() returned Ok(())
  • File can't be append again after w.close() returned Ok(())

Currently, there are not many docs about this. To ensure proper usage, it is important to inform users about these risks. Furthermore, for services that support append API natively, using append API instead of multipart API is a better choice.

I propose to separate the write and append operations. The files created by the write operation can not be modified. And the files created by the append operation can be appended via multipart API or append API for services that natively support append, such as Azure blob and Alibaba cloud OSS.

@ShadowySpirits ShadowySpirits changed the title RFC: Introduce Append API RFC-2133: Introduce Append API Apr 26, 2023
Comment thread core/src/docs/rfcs/2133_introduce_append_api.md Outdated
Comment thread core/src/docs/rfcs/2133_introduce_append_api.md Outdated
Comment thread core/src/docs/rfcs/2133_introduce_append_api.md Outdated
Comment thread core/src/docs/rfcs/2133_introduce_append_api.md Outdated
Comment thread core/src/docs/rfcs/2133_append_api.md Outdated
Comment thread core/src/docs/rfcs/2133_append_api.md Outdated
Comment thread core/src/docs/rfcs/2133_append_api.md Outdated
Comment thread core/src/docs/rfcs/2133_append_api.md Outdated
Comment thread core/src/docs/rfcs/2133_append_api.md Outdated
Comment thread core/src/docs/rfcs/2133_append_api.md Outdated
Comment thread core/src/docs/rfcs/2133_append_api.md Outdated
Comment thread core/src/docs/rfcs/2133_append_api.md Outdated
@ShadowySpirits ShadowySpirits requested a review from Xuanwo April 28, 2023 03:07
Comment thread core/src/docs/rfcs/2133_append_api.md Outdated
Copy link
Copy Markdown
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot!

@Xuanwo Xuanwo merged commit ddad905 into apache:main Apr 29, 2023
@Xuanwo Xuanwo mentioned this pull request May 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants