feat: bump bindle to v0.9.0-rc.1

[FrankYang0529]

resolve #689

Setup

# clone this PR and build spin
cargo build

# clone bindle dockerize PR https://github.com/deislabs/bindle/pull/343 and build a docker image
docker build -t bindle .

# create a bindle directory and a key
BINDLE_TEMP=$(mktemp -d)
echo $BINDLE_TEMP
export BINDLE_KEYRING=$BINDLE_TEMP/keyring.toml
./target/debug/bindle keys create "VishnuJin<me@example.com>" -f $BINDLE_TEMP/secret_keys.toml

# start bindle server
docker run --name bindle -d --restart=unless-stopped -e RUST_LOG=debug -v $BINDLE_TEMP:/bindle-data -p 8080:8080 bindle
# show bindle server logs
docker logs bindle -f

Test spin bindle prepare

# export BINDLE_TEMP to the folder which we created in the last terminal
export BINDLE_TEMP=

export BINDLE_KEYRING_FILE=$BINDLE_TEMP/keyring.toml
export BINDLE_SECRET_FILE=$BINDLE_TEMP/secret_keys.toml
export BINDLE_LABEL="VishnuJin<me@example.com>"

# create a spin app
./target/debug/spin new http-rust myapp

# build spin app
cd myapp
# remember to update spin_sdk version to v0.5.0 before building it
../target/debug/spin build

# run spin bindle prepare
export STAGING_DIR=$(mktemp -d)
../target/debug/spin bindle prepare --staging-dir $STAGING_DIR

Test spin bindle push

export BINDLE_URL=http://localhost:8080/v1
../target/debug/spin bindle push

Test integration with hippo

# start consul
consul agent -dev

# start nomad
nomad agent -dev

# clone the latest hippo and start hippo
export Database__Driver=sqlite
export ConnectionStrings__Database="Data Source=hippo.db;Cache=Shared"
export ConnectionStrings__Bindle="Address=http://127.0.0.1:8080/v1"
export Nomad__Driver="raw_exec"
export Jwt__Key="ceci n'est pas une jeton"
export Jwt__Issuer="localhost"
export Jwt__Audience="localhost"
cd src/Web
dotnet build
dotnet run

# Run spin deploy in myapp
export HIPPO_USERNAME="admin"
export HIPPO_PASSWORD="p@ssword"
export HIPPO_URL="http://localhost:5309"
../target/debug/spin deploy --deploy-existing-bindle

[vdice]

Thanks for this much-needed PR @FrankYang0529! I'm excited to test.

The updates look good/correct at a high level. The one question I had was around the scenario when spin (specifically its bindle client) doesn't share the keyring file with the bindle server, eg when the bindle server is fully remote. Do we have logic here to fetch the public key from the bindle server in this case? (... speaking of, I wonder if it would be worth adding a new test for this scenario as well -- I love seeing how we already have existing tests that have been updated!)

[FrankYang0529]

Hi @vdice, I agree we need to fetch the public key. However, even if I use different folders for client and server, I still can run spin up --bindle myapp/0.1.0. Could you help me check where is wrong? I can't figure out why it works.

Setup

# clone this PR and build spin
cargo build

# clone bindle dockerize PR https://github.com/deislabs/bindle/pull/343 and build a docker image
docker build -t bindle .

# create a bindle client directory and a key
BINDLE_TEMP=$(mktemp -d)
echo $BINDLE_TEMP
export BINDLE_KEYRING=$BINDLE_TEMP/client/keyring.toml
./target/debug/bindle keys create "VishnuJin<me@example.com>" -f $BINDLE_TEMP/client/secret_keys.toml

# create a bindle server directory
mkdir $BINDLE_TEMP/server
cp $BINDLE_TEMP/client/keyring.toml $BINDLE_TEMP/server/keyring.toml

# start bindle server
docker run --name bindle -d --restart=unless-stopped -e RUST_LOG=debug -v $BINDLE_TEMP/server:/bindle-data -p 8080:8080 bindle

# show bindle server logs
docker logs bindle -f

Push spin app

# export BINDLE_TEMP to the folder which we created in the last terminal
export BINDLE_TEMP=

export BINDLE_KEYRING_FILE=$BINDLE_TEMP/client/keyring.toml
export BINDLE_SECRET_FILE=$BINDLE_TEMP/client/secret_keys.toml
export BINDLE_LABEL="VishnuJin<me@example.com>"

# create a spin app
./target/debug/spin new http-rust myapp

# build spin app
cd myapp
# remember to update spin_sdk version to v0.5.0 before building it
../target/debug/spin build

# run spin bindle push
export BINDLE_URL=http://localhost:8080/v1
../target/debug/spin bindle push

Run spin app

export BINDLE_URL=http://localhost:8080/v1
./target/debug/spin up --bindle myapp/0.1.0

[FrankYang0529]

Hi @vdice, I agree we need to fetch the public key. However, even if I use different folders for client and server, I still can run spin up --bindle myapp/0.1.0. Could you help me check where is wrong? I can't figure out why it works.

I know the reason! We didn't update bindle package version to v0.9.0-rc.1 in creates/loader. I will fix it.

[FrankYang0529]

I know the reason! We didn't update bindle package version to v0.9.0-rc.1 in creates/loader. I will fix it.

Hi @vdice, I updated creates/loader and used different keyring files for testing. If you want to run make test-e2e on your computer, please remember to update bindle-server to v0.9.0-rc.1. Thank you.

[FrankYang0529]

I will resolve conflicts tomorrow.

[vdice]

I will resolve conflicts tomorrow.

👍 I'll be away next week (10/3-10/7) but hopefully another maintainer can take a look during that time as well. I do know that we'll probably want to wait to merge until after the next v0.6.0 release. Thanks!

[FrankYang0529]

Resolve conflicts with #794. Add spin deploy in test steps of this PR.

Setup

# clone this PR and build spin
cargo build

# clone bindle dockerize PR https://github.com/deislabs/bindle/pull/343 and build a docker image
docker build -t bindle .

# create a bindle client directory and a key
BINDLE_TEMP=$(mktemp -d)
echo $BINDLE_TEMP
export BINDLE_KEYRING=$BINDLE_TEMP/client/keyring.toml
./target/debug/bindle keys create "VishnuJin<me@example.com>" -f $BINDLE_TEMP/client/secret_keys.toml

# create a bindle server directory
mkdir $BINDLE_TEMP/server
cp $BINDLE_TEMP/client/keyring.toml $BINDLE_TEMP/server/keyring.toml

# start bindle server
docker run --name bindle -d --restart=unless-stopped -e RUST_LOG=debug -v $BINDLE_TEMP/server:/bindle-data -p 8080:8080 bindle

# show bindle server logs
docker logs bindle -f

Push spin app

# export BINDLE_TEMP to the folder which we created in the first terminal
export BINDLE_TEMP=

export BINDLE_KEYRING_FILE=$BINDLE_TEMP/client/keyring.toml
export BINDLE_SECRET_FILE=$BINDLE_TEMP/client/secret_keys.toml
export BINDLE_LABEL="VishnuJin<me@example.com>"

# create a spin app
./target/debug/spin new http-rust myapp

# build spin app
cd myapp
# remember to update spin_sdk version to v0.5.0 before building it
../target/debug/spin build

# run spin bindle push
export BINDLE_URL=http://localhost:8080/v1
../target/debug/spin bindle push

Run spin app

export BINDLE_URL=http://localhost:8080/v1
./target/debug/spin up --bindle myapp/0.1.0

Start Hippo

# start consul
consul agent -dev

# start nomad
nomad agent -dev

# clone the latest hippo and start hippo
export Database__Driver=inmemory
export ConnectionStrings__Bindle="Address=http://127.0.0.1:8080/v1"
export Nomad__Driver="raw_exec"
export Jwt__Key="ceci n'est pas une jeton"
export Jwt__Issuer="localhost"
export Jwt__Audience="localhost"
cd src/Web
dotnet build
dotnet run

Login and deploy

# export BINDLE_TEMP to the folder which we created in the first terminal
export BINDLE_TEMP=

export BINDLE_KEYRING_FILE=$BINDLE_TEMP/client/keyring.toml
export BINDLE_SECRET_FILE=$BINDLE_TEMP/client/secret_keys.toml
export BINDLE_LABEL="VishnuJin<me@example.com>"
export BINDLE_URL=http://localhost:8080/v1

export HIPPO_USERNAME="admin"
export HIPPO_PASSWORD="p@ssword"
export HIPPO_URL="http://localhost:5309"

# Login
./target/debug/spin login

# Run spin deploy in myapp
cd myapp
../target/debug/spin deploy --deploy-existing-bindle

[vdice]

@FrankYang0529 Thanks so much for updating to work with the latest spin deploy changes! I hope to revisit this PR asap.

One other item I was thinking about... thoughts on how we'd surface spin-bindle incompatibilities? For instance, if a user is attempting to deploy using spin v0.5.0 and the bindle server is 0.9.0+, how best to relay that this combination isn't supported. Conversely, when we release a version of spin updated to talk to 0.9.0+, how best to make it obvious that the newer spin client can't talk to previous bindle server versions?

[FrankYang0529]

One other item I was thinking about... thoughts on how we'd surface spin-bindle incompatibilities? For instance, if a user is attempting to deploy using spin v0.5.0 and the bindle server is 0.9.0+, how best to relay that this combination isn't supported. Conversely, when we release a version of spin updated to talk to 0.9.0+, how best to make it obvious that the newer spin client can't talk to previous bindle server versions?

Yeah, this is a good question. If we can determine bindle-server version, we can give a proper message to users. Do you think it's good to return bindle-server version in /v1/healthz API? If yes, I can help to create a PR in bindle. Thanks.

[vdice]

Do you think it's good to return bindle-server version in /v1/healthz API? If yes, I can help to create a PR in bindle. Thanks.

Indeed, if bindle-server doesn't yet return its version in another endpoint, that seems like a good candidate. I'll create an issue there just in case other bindle maintainers have input.

[FrankYang0529]

The CI will fail since we haven't updated the bindle version in CI pipeline, but I add the bindle health check first. I will change the bindle version in Cargo.toml back to https://github.com/fermyon/bindle once we fetch the latest commit and release a new version v0.9.1-rc.2. Currently, we are still blocked by deislabs/bindle#354.

[vdice]

@FrankYang0529 thanks so much for your effort here. It's looking like Spin moving towards using OCI registries as its default form of artifact support. Full details can be seen in SIP-008: Using OCI Registries and a working (albeit in-progress) prototype can be seen here.

It remains to be seen whether Spin deprecates support for using Bindle or not. However, I'm anticipating that we'll be sticking with Bindle 0.8.x for the forseeable future, at least while we gain experience and collect feedback around OCI support. Lastly, if we do indeed update Bindle at some point, we'll most likely wait until a non-rc release of the next version is available and hardened.

With the notes above, I suggest we close this PR. However, I wanted to check in with you first before doing so.

[itowlson]

As this was suggested to be closed in January, and there's been no feedback to keep it open, I'm going to close it. If anyone wants it furthering then do feel free to reopen though!