Advanced HTTP ACL

[veeshi]

In (bytecodealliance/wasmtime#6808 / https://github.com/nethunterslabs/wasmtime/tree/wasi-http-acl) I integrated a HTTP ACL to be used for outgoing HTTP requests. We are still using this custom fork (which I'm hopefully going to update soon) until we can possibly create a component generator or another solution.

The HTTP ACL has the following flow:

• Is the HTTP method allowed
• Is the URL Scheme allowed (i.e. HTTP or HTTPS)
• Is the port allowed (if set)
• Check the authority:
  • If it is a domain/hostname:
    • Check if there is a static DNS record in the ACL
    • If no static record then check if it is allowed
    • Resolve the host/domain and check the IP address/es is/are allowed
  • If it is an IP address check the IP address is allowed
• Check the URL path is allowed

In the PR it was mentioned that a HTTP ACL could be solved by a component generator. I can see how this can be done for all the steps aside from the IP address resolution without resolving it twice, once here and once in wasi-http. Is there a plan or scope to create an advanced HTTP ACL like this in wasi-virt or is there another approach that would be better?

[guybedford]

This is a good fit for WASI-Virt as a configurable HTTP virtualization.

For the DNS check, that would effectively extend the world support to also expect support for WASI sockets DNS lookup, perhaps this should itself be configurable. And yes you're right it would effectively be a double lookup, I don't see a way to get around that either.

I don't know of anyone else needing this feature right now, but if you are interested, I can make sure to properly review any work done here.

[veeshi]

I'm unsure why it isn't something people are more interested in. Most FAAS platforms using Wasmtime have built rudimentary ACLs for HTTP requests but they all miss out on the part of checking the IP address from a DNS lookup is actually an external IP. To prevent this class of attacks https://github.com/stripe/smokescreen is usually used but we're looking at doing as much checking as we can at the source and simplifying our stack.

Happy to contribute! What are the overall steps to add this as a virtualised component?

[guybedford]

WASI-Virt is still relatively bleeding edge at this point, so while it was designed for these use cases, it hasn't been driven to support them yet. It's more just that no one has go around to it yet, as we all have other things to do.

The IO virtualization already wraps the outgoing request process in https://github.com/bytecodealliance/WASI-Virt/blob/main/virtual-adapter/src/io.rs#L633, so a virtualization would be a gate at that function call that would deny the request before it is made by returning an HTTP failure.

In terms of managing the configuration, the IO configuration object is defined in https://github.com/bytecodealliance/WASI-Virt/blob/main/virtual-adapter/src/io.rs#L145, which could be extended to support HTTP ACL fields. Then that can be populated and managed similar to the FS Virt, which is set in https://github.com/bytecodealliance/WASI-Virt/blob/main/src/virt_io.rs#L551, and configured in the API at https://github.com/bytecodealliance/WASI-Virt/blob/main/src/lib.rs#L47 and the CLI at https://github.com/bytecodealliance/WASI-Virt/blob/main/src/bin/wasi-virt.rs#L69.

For testing, we have test components built in for example https://github.com/bytecodealliance/WASI-Virt/tree/main/tests/components/file-read, so we'd probably need an HTTP outgoing test. Then the test configuration files are in https://github.com/bytecodealliance/WASI-Virt/tree/main/tests/cases, where the configurations are the serde config of the main API and the test component to run the virtualization against just referenced by name. The test expectation is based on configuration from https://github.com/bytecodealliance/WASI-Virt/blob/main/tests/virt.rs#L50, which would need some kind of http request configuration if the component should do an HTTP request, and that call is then done in eg https://github.com/bytecodealliance/WASI-Virt/blob/main/tests/virt.rs#L260 which could do a real call to an exported function like do_http_call(url: string). The WASI test world for this exported function is defined in https://github.com/bytecodealliance/WASI-Virt/blob/main/wit/virt.wit#L191, where test components export empty versions of these functions when they don't test them.

Probably too much info for now, but if you're really keen, those are the steps!

[veeshi]

Thanks, that's in depth enough for me to have a real stab at it hopefully in the next week or so!