macOS and Linux API versions differ from Windows

[ryanblenis]

The b2 CLI app behaves differently on macOS/Linux than it does on Windows when creating a key.
Versions tested: Up to 4.5.1

Reproducible example:

1. Authorize with master account.
2. Try to create a key with the following permissions:

b2 key create --bucket my-bucket-name-here Test-Write-Only-Key-Files-1 listBuckets,readBuckets,listFiles,readFiles,writeFiles,readBucketEncryption,readBucketReplications,readBucketRetentions,readFileRetentions,writeFileRetentions,readFileLegalHolds

When attempting a b2 account authorize with the key information yields the following error:
ERROR: unable to authorize account: This request is not currently supported on API version number 2 (bad_request)

Additionally, the key is hidden from the "Application Keys" portion of the web portal.

Running the same command on the Windows version creates the correct key as expected, allows authorization, and shows itself in the web portal.

[ppolewicz]

@olzhasar-reef I think we might want to add an integration test for this particular case, if we don't have one already.

I suspect this is somehow due to multi-bucket keys not being supported by old login endpoint and the web portal, but why would a new version of the cli on linux/mac create a multi-bucket key with a single bucket while Windows does something else?

@ryanblenis are you sure you used the same CLI & sdk version on all 3 environments? Perhaps it's a matter of sdk version and not the CLI? That would make a lot of sense actually.

Still, I don't think it should just be like this - if we can make some kind of a workaround for this, that would be great.

[ryanblenis]

I'm not using/changing anything with an SDK (not sure if what else installed on the system would affect the inner workings of the b2 executables, you'll have to enlighten me on that).

@ppolewicz I used the CLI's available from the github release pages. I've just downloaded the newest files from github for linux with the following results:

Linux:
b2v3-linux: creates the key, is able to log in with that key (b2 account authorize) via CLI, but does not appear in the Application Key list on the web.
b2-linux: creates the key, is able to log in with that key (b2 account authorize) via CLI, but does not appear in the Application Key list on the web.

On Windows:
b2-windows: works as expected (creates the key and appears in Application Key list on the web)

On macOS:
Installed the latest version (4.5.1) via homebrew (brew install b2-tools). creates the key, is able to log in with that key (b2 account authorize) via CLI, but does not appear in the Application Key list on the web.
I also noticed that this installs b2, b2v3, and b2v4 on macOS. I tested key creation in each and none of them appear in the Application Key list on the web.

This is, of course, creating a key for a particular bucket. I can try this with multi-bucket keys as well if needed. I'm not sure which version originally got me the issue of This request is not currently supported on API version number 2 (bad_request) and if it was multi bucket or scoped, but I cannot replicate that error at this time on the newest version with a single bucket.

My main use-case for this is most of our infrastructure is on linux servers, and ideally we'd like to be able to create a key with those permissions using the master key on our infra, and the client utilizing the limited keys are mostly Windows machines. This is how we discovered the discrepancy in the web portal not showing the new keys and having clients fail... unless we used the master key on a Windows system to create the limited keys.

I'll continue testing and let you know if I can replicate it the bad_request, but I think the telltale sign at this point is that the key does not appear in the web portal when created on linux/macOS.

Please let me know if there is anything specific on my end I can test to help solve this. Thank you.

[olzhasar-reef]

@ryanblenis Thank you for reporting your issue!

B2 cli, starting with version 4.4.0, uses B2 native API v4 for all it's API requests.
Unfortunately, any api keys with bucket restrictions (whether it's a single- or multi-bucket restricted key) do not appear to be backwards compatible with earlier API versions. You can read more about it in the Version History of the Backblaze Native API.

Therefore, if you try to create a key using a recent CLI version (>4.4.0), that is restricted to one or multiple buckets, you will not be able to use that key with an older CLI version (that uses B2 Native api v3 or v2 under the hood).

In your particular case, please make sure that the CLI installed on a Windows machine(s) is of recent version (preferably the latest one). Compatibility shouldn't be dependent on the OS type you use at all, only the CLI version. Please, let us know if you'll experience the same problem with the latest version of the CLI on Windows. If, for some reason, you are unable to update the CLI on Windows clients, you can try creating a new key with B2 CLI 4.3.3, which uses an older Native API version for creating them, and there shouldn't be any compatibility issues as a result.

As for the application keys created using the CLI not being displayed in the Web UI, this appears to be a bug on the UI side. It only applies to keys created with Native API v4 (b2 cli >= 4.4.0). The issue doesn't seem to affect anything else, you can still use the keys as usual. Hopefully, the bug in the UI will be fixed soon, but for now, you can use the cli to query your keys (using a master key):

b2 key list

or for a more detailed output:

b2 key list --long

[olzhasar-reef]

@ppolewicz the fundamental problem here is that the server treats any keys with bucket restrictions as multi-bucket (or of a new kind, if you will). An integration test checking that compatibility will just fail all the time.
Therefore, if the server behavior stays the same, the workaround options on the CLI and the sdk side seem to be pretty limited.

One potential tiny improvement of the user experience, is perhaps, warning users that their newly created application key will not be compatible with older CLI (and sdk) versions?