feat(core): add capability check

[unixzii]

Fixed #1752.

In this PR, we will only implement the capability check at operation-level.

Implementation Notes

Static dispatching

with_strict method is added on OperatorBuilder type to eliminate dynamic dispatching & extra heap allocations. That will make the built Operator be strict in its whole lifetime, therefore with_strict method is also added on Operator for flexibility.

Dedicated layer for checking

The capability checking is performed in CapabilityCheckLayer to guarantee the single responsibility principle. It's also hard to mix them with any existing layer while allowing the flexible controls.

Manually checking against all ops

The capability metadata is defined separately from ops (in different modules). We have to enumerate all the cases when checking, and map them manually. Fortunately, this is simplified by an internal macro, which can avoid a large amount of redundant code.

Checklist

• Confirm the design
• Expose APIs for users to enable the strict mode
• Implement the supporting layer
• Implement checks for all operations
• Add tests

[unixzii]

Hi @Xuanwo, maybe we can have a quick review of the current design, so we can make sure the direction is good to follow when implementing the rest things.

[Xuanwo]

The capability checking is performed in CapabilityCheckLayer to guarantee the single responsibility principle. It's also hard to mix them with any existing layer while allowing the flexible controls.

All those related tasks are handled at CompleteLayer which only used internally.

[unixzii]

The capability checking is performed in CapabilityCheckLayer to guarantee the single responsibility principle. It's also hard to mix them with any existing layer while allowing the flexible controls.

All those related tasks are handled at CompleteLayer which only used internally.

One question for this, how should we pass the strict mode flag to the CompleteLayer? Once Operator is built, its layers are type-erased and no longer accessible from the operator side. Maybe we need to introduce extra heap allocation to share a boolean between operator and layer? Or use mechanism like task local, scoped variables, etc.

[Xuanwo]

One question for this, how should we pass the strict mode flag to the CompleteLayer?

We can store a boolean strict in Capability which can set by services config like what we does for batch_max_operations in s3.

[unixzii]

We can store a boolean strict in Capability which can set by services config like what we does for batch_max_operations in s3.

Should we expose the configuration to users? Like overriding it when building the operator.

[Xuanwo]

Should we expose the configuration to users? Like overriding it when building the operator.

I'm a bit hesitant about this. Maybe we can provide a with_strict in opendal::Builder.

[unixzii]

Should we expose the configuration to users? Like overriding it when building the operator.

I'm a bit hesitant about this. Maybe we can provide a with_strict in opendal::Builder.

No worrying about this, we can implement the mandatory operation-level checks first.

[unixzii]

Just updated the PR, it's now much simpler :)

[Xuanwo]

Mostly looks good to me! Only a few minor changes.

[Xuanwo]

Kv and TypedKv's blokcing cap is not set which results in our tests failed: https://github.com/apache/incubator-opendal/blob/main/core/src/raw/adapters/typed_kv/backend.rs#L65

Can you help fix it in this PR?

[unixzii]

Kv and TypedKv's blokcing cap is not set which results in our tests failed: https://github.com/apache/incubator-opendal/blob/main/core/src/raw/adapters/typed_kv/backend.rs#L65

Can you help fix it in this PR?

Yep, bringing up the commits.

[unixzii]

There is still something wrong, I'm fixing it.

[Xuanwo]

Thanks!