diff --git a/README.md b/README.md index f9f4ce15..87763438 100644 --- a/README.md +++ b/README.md @@ -36,10 +36,17 @@ difference of factor **100** which is not going to be easy to beat. ## Trying -Trying Keystone (assuming you have the Rust build environment or you are in the +Trying Keystone (assuming you have the Rust build environment or you are in the possession of the binary is as easy as `keystone -c etc/keystone.conf -vv` +Alternatively you can try it with `docker compose -f docker-compose.yaml up`. + +## Documentation + +Comprehensive (as much as it can be at the current stage) is available +[here](https://gtema.github.io/keystone). + ## Talks -Detailed introduction of the project was given as +Detailed introduction of the project was given as [ALASCA tech talk](https://www.youtube.com/watch?v=0Hx4Q22ZNFU). diff --git a/doc/src/SUMMARY.md b/doc/src/SUMMARY.md index 44bb06c1..2739b232 100644 --- a/doc/src/SUMMARY.md +++ b/doc/src/SUMMARY.md @@ -2,14 +2,19 @@ [Introduction](./intro.md) [Installation](./install.md) -[Policy](./policy.md) --- -- [API](./swagger-ui.html) +[Architecture](./architecture.md) +[Policy enforcement](./policy.md) + +--- + - [Federation](./federation.md) + - [Oidc RP mode](./oidc.md) + - [JWT](./jwt.md) - [Passkeys](./passkey.md) - +- [API](./swagger-ui.html) --- diff --git a/doc/src/architecture.md b/doc/src/architecture.md new file mode 100644 index 00000000..4b6c8b2f --- /dev/null +++ b/doc/src/architecture.md @@ -0,0 +1,57 @@ +# Architecture + +Keystone requires 2 additional components to run: + +- database (the same as the py-keystone uses) + +- OpenPolicyAgent, that implements API policy enforcement + +```mermaid +architecture-beta + + service db(database)[Database] + service keystone(server)[Keystone] + service opa(server)[OpenPolicyAgent] + + db:L -- R:keystone + opa:L -- T:keystone + +``` + +## Database + +Python keystone uses the sqlalchemy as ORM and the migration tool. It cannot be +used from Rust efficiently, therefore keystone-ng uses the `sea-orm` which +provides async support natively and also allows database type abstraction. +Current development focuses on the PostgreSQL database. The MySQL should be +supported, but is not currently tested against. + +New API and resources are being added. This requires database changes. sea-orm +also comed with the migration tools. However there is a slight difference +between sqlalchemy and sea-orm. The later suggests doing database schema first. +In the next step object types are created out of the database. That means that +the database migration must be written first and cannot be automatically +generated from the code (easily, but there is a way). Current migrations do not +create database schema that is managed by the py-keystone. Therefore in order +to get a fully populated database schema it is necessary to apply +`keystone-manage db_sync` and `keystone-db up` independently. + +Target of the keystone-ng is to be deployed in pair with the python keystone of +"any" version. Due to that it is not possible to assume the state of the +database, nor to apply any changes to the schema manaaged by the py-keystone. A +federation rework assumes model change. To keep it working with the +python-keystone artificial table entries may be created (in the example when a +new identity provider is being created automatically sanitized entries are +being added for the legacy identity provider and necessary protocols) A +federation rework assumes model change. To keep it working with the +python-keystone artificial table entries may be created (in the example when a +new identity provider is being created automatically sanitized entries are +being added for the legacy identity provider together with necessary idp +protocols). + +## Fernet + +keystone-ng uses the same mechanism for tokens to provide compatibility. The +fernet-keys repository must be provided in the runtime (i.e. by mounting them +as a volume into the container). There is no tooling to create or rotate keys +as the py-keystone does. diff --git a/doc/src/database.md b/doc/src/database.md new file mode 100644 index 00000000..f1f9cc0c --- /dev/null +++ b/doc/src/database.md @@ -0,0 +1 @@ +# Database diff --git a/doc/src/federation.md b/doc/src/federation.md index e4969698..55cc0cf3 100644 --- a/doc/src/federation.md +++ b/doc/src/federation.md @@ -19,169 +19,36 @@ pretty big number of limitations (not limited to): - Client authentication right now is complex and error prone (every public provider has implementation specifics that are often even not cross-compatible) - -In order to address those challenges and complete reimplementation is being -done here. This leads to a completely different design opening doors for new -features. +In order to address those challenges a complete reimplementation is being done +with a different design. This allows implementing features not technically +possible in the py-keystone: - Federation is controlled on the domain level by the domain managers. This means that the domain manager is responsible for the configuration of how users should be federated from external IdPs. -- Keystone serves as a relying party in the OIDC authentication flow. This - moves the complex logic from client to the the Keystone side. This allows - making client applications much simpler and more reliable. - -## Authentication using the Authorization Code flow and Keystone serving as RP - -```mermaid -sequenceDiagram - - Actor Human - Human ->> Cli: Initiate auth - Cli ->> Keystone: Fetch the OP auth url - Keystone --> Keystone: Initialize authorization request - Keystone ->> Cli: Returns authURL of the IdP with cli as redirect_uri - Cli ->> User-Agent: Go to authURL - User-Agent -->> IdP: opens authURL - IdP -->> User-Agent: Ask for consent - Human -->> User-Agent: give consent - User-Agent -->> IdP: Proceed - IdP ->> Cli: callback with Authorization code - Cli ->> Keystone: Exchange Authorization code for Keystone token - Keystone ->> IdP: Exchange Authorization code for Access token - IdP ->> Keystone: Return Access token - Keystone ->> Cli: return Keystone token - Cli ->> Human: Authorized - -``` - -## Authenticating with the JWT - -It is possible to authenticate with the JWT token issued by the federated IdP. -More precisely it is possible to exchange a valid JWT for the Keystone token. -There are few different use scenarios that are covered. - -Since the JWT was issued without any knowledge of the Keystone scopes it -becomes hard to control scope. In the case of real human login the Keystone may -issue unscoped token allowing user to further rescope it. In the case of the -workflow federation that introduces a potential security vulnerability. As such -in this scenario the attribute mapping is responsible to fix the scope. - -Login request looks following: - -```console - - curl https://keystone/v4/federation/identity_providers/${IDP}/jwt -X POST -H "Authorization: bearer ${JWT}" -H "openstack-mapping: ${MAPPING_NAME}" -``` - -### Regular user obtains JWT (ID token) at the IdP and presents it to Keystone - -In this scenario a real user (human) is obtaining the valid JWT from the IDP -using any available method without any communication with Keystone. This may -use authorization code grant, password grant, device grant or any other enabled -method. This JWT is then presented to the Keystone and an explicitly requested -attribute mapping converts the JWT claims to the Keystone internal -representation after verifying the JWT signature, expiration and further -restricted bound claims. - -### Workflow federation - -Automated workflows (Zuul job, GitHub workflows, GitLab CI, etc) are typical -workloads not being bound to any specific user and are more regularly -considered being triggered by certain services. Such workflows are usually in -possession of a JWT token issued by the service owned IdP. Keystone allows -exchange of such tokens to the regular Keystone token after validating token -issuer signature, expiration and applying the configured attribute mapping. -Since in such case there is no real human the mapping also need to be -configured slightly different. - -- It is strongly advised the attribute mapping must fill `token_user_id`, - `token_project_id` (and soon `token_role_ids`). This allows strong control of - which technical account (soon a concept of service accounts will be introduced - in Keystone) is being used and which project such request can access. - -- Attribute mapping should use `bound_audiences`, `bound_claims`, - `bound_subject`, etc to control the tokens issued by which workflows are - allowed to access OpenStack resources. - -### GitHub workflow federation - -In order for the GitHub workflow to be able to access OpenStack resources it is -necessary to register GitHub as a federated IdP and establish a corresponding -attribute mapping of the `jwt` type. - -IdP: - -```json -"identity_provider": { - "name": "github", - "bound_issuer": "https://token.actions.githubusercontent.com", - "jwks_url": "https://token.actions.githubusercontent.com/.well-known/jwks" -} -``` - - -Mapping: - -```json -"mapping": { - "type": "jwt", - "name": "gtema_keystone_main", - "idp_id": , - "domain_id": , - "bound_audiences": ["https://github.com"], - "bound_subject": "repo:gtema/keystone:pull_request", - "bound_claims": { - "base_ref": "main" - }, - "user_id_claim": "actor_id", - "user_name_claim": "actor", - "token_user_id": -} -``` - -TODO: add more claims according to [docs](https://docs.github.com/en/actions/reference/security/oidc#oidc-token-claims) - -A way for the workflow to obtain the JWT [is described here](https://docs.github.com/en/actions/reference/security/oidc#methods-for-requesting-the-oidc-token). - -```yaml -... -permissions: - token: write - contents: read +- Identity providers and/or attribute mappings can be reused by different + domains allowing implementing social logins. -job: - ... - - name: Get GitHub JWT token - id: get_token - run: | - TOKEN_JSON=$(curl -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \ - "$ACTIONS_ID_TOKEN_REQUEST_URL&audience=https://github.com") +- Keystone serves as a relying party in the OIDC authentication flow. It + decreases amount of different flows to the minimum making client applications + much simpler and more reliable. - TOKEN=$(echo $TOKEN_JSON | jq -r .value) - echo "token=$TOKEN" >> $GITHUB_OUTPUT - ... - # TODO: build a proper command for capturing the actual token and/or write a dedicated action for that. - - name: Exchange GitHub JWT for Keystone token - run: | - KEYSTONE_TOKEN=$(curl -H "Authorization: bearer ${{ steps.get_token.outputs.token }}" -H "openstack-mapping: gtmema_keystone_main" https://keystone_url/v4/federation/identity_providers/IDP/jwt) - -``` ## API changes A series of brand new API endpoints have been added to the Keystone API. -- /v3/federation/identity_providers (manage the identity providers) +- **/v4/federation/identity_providers** (manage the identity providers) -- /v3/federation/mappings (manage the mappings tied to the identity provider) +- **/v4/federation/mappings** (manage the mappings tied to the identity provider) -- /v3/federation/auth (initiate the authentication and get the IdP url) +- **/v4/federation/auth** (initiate the authentication and get the IdP url) -- /v3/federation/oidc/callback (exchange the authorization code for the Keystone token) +- **/v4/federation/oidc/callback** (exchange the authorization code for the Keystone token) -- /v3/federation/identity_providers/{idp_id}/jwt (exchange the JWT token issued by the referred IdP for the Keystone token) +- **/v4/federation/identity_providers/{idp_id}/jwt** + (exchange the JWT token issued by the referred IdP for the Keystone token) ## DB changes @@ -229,3 +96,5 @@ need to rely on Selenium. At the moment following integrations are tested automatically: - Keycloak (login using browser) +- Keycloak (login with JWT) +- GitHub (workload federation with JWT) diff --git a/doc/src/install.md b/doc/src/install.md index 724f7ce6..fce28b79 100644 --- a/doc/src/install.md +++ b/doc/src/install.md @@ -1,14 +1,61 @@ # Installation -TODO: +The easiest way to get started with the keystone-ng is using the container +image. It is also possible to use the compiled version. It can be either +compiled locally or downloaded from the project artifacts. -- Prepare the binary (download from GH releases, build yourself, use the - container image, ...) +## Using pre-compiled binaries -- Perform the DB migration `keystone-db up` +As of the moment of writing there were no releases. Due to that there are no +pre-compiled binaries available yet. Every release of the project would include +the pre-compiled binaries for a variety of platforms. -- Start the binary as `keystone -c ` +## Compiling +In order to compile the keystone-ng it is necessary to have the rust compiler +available. It may be installed from the system packages or using the +`rustup.rs` + +```console +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh +``` + +Afterwards in the root of the project source tree following command may be +executed to invoke the `cargo` + +```console + +cargo build --release + +``` + +It produces 2 binaries: + +- target/release/keystone (the api server) + +- target/release/keystone-db (the database management tool) + +Currently keystone depends on the openssl (through one of the dependencies). +Depending on the environment it may be a statically linked or dynamically. +There are signals that that may be not necessary anymore once all dependencies +transition to the use of rustls. + +## Using containers + +It is possible to run Keystone-ng inside containers. A sample Dockerfile is +present in the project source tree to build container image with the Keystone +and the `keystone-db` utility. When no ready image is available it can be build +like that: + +```console + +docker build . -t keystone:rust + +``` + +Since keystone itself communicates with the database and OpenPolicyAgent those +must be provided separately. `docker-compose.yaml` demonstrates how this can be +done. ## Database migrations @@ -16,3 +63,30 @@ Rust Keystone is using different ORM and implements migration that co-exist together with alembic migrations of the python Keystone. It also ONLY manages the database schema additions and does NOT include the original database schema. Therefore it is necessary to apply both migrations. + +```console +keystone-db -u +``` + +It is important to also understand that the DB_URL may differ between python +and rust due to the optional presence of the preferred database driver in the +url. keystone-ng will ignore the the driver in the application itself, but the +migration may require user to manually remove it since it is being processed by +the ORM itself and not by the keystone-ng code. + +## OpenPolicyAgent + +keystone-ng relies on the OPA for policy enforcement. Default policies are +provided with the project and can be passed directly to the OPA process or +compilied into the bundle. + +```console + +opa run -s policies + +``` + +**NOTE:** by default OPA process listens on the localhost only what lead to +unavailability to expose it between containers. Please use `-a 0.0.0.0:8181` to + +start listening on all interfaces. diff --git a/doc/src/jwt.md b/doc/src/jwt.md new file mode 100644 index 00000000..2d3331d2 --- /dev/null +++ b/doc/src/jwt.md @@ -0,0 +1,111 @@ +# Authenticating with the JWT + +It is possible to authenticate with the JWT token issued by the federated IdP. +More precisely it is possible to exchange a valid JWT for the Keystone token. +There are few different use scenarios that are covered. + +Since the JWT was issued without any knowledge of the Keystone scopes it +becomes hard to control scope. In the case of real human login the Keystone may +issue unscoped token allowing user to further rescope it. In the case of the +workflow federation that introduces a potential security vulnerability. As such +in this scenario the attribute mapping is responsible to fix the scope. + +Login request looks following: + +```console + + curl https://keystone/v4/federation/identity_providers/${IDP}/jwt -X POST -H "Authorization: bearer ${JWT}" -H "openstack-mapping: ${MAPPING_NAME}" +``` + +## Regular user obtains JWT (ID token) at the IdP and presents it to Keystone + +In this scenario a real user (human) is obtaining the valid JWT from the IDP +using any available method without any communication with Keystone. This may +use authorization code grant, password grant, device grant or any other enabled +method. This JWT is then presented to the Keystone and an explicitly requested +attribute mapping converts the JWT claims to the Keystone internal +representation after verifying the JWT signature, expiration and further +restricted bound claims. + +## Workload federation + +Automated workflows (Zuul job, GitHub workflows, GitLab CI, etc) are typical +workloads not being bound to any specific user and are more regularly +considered being triggered by certain services. Such workflows are usually in +possession of a JWT token issued by the service owned IdP. Keystone allows +exchange of such tokens to the regular Keystone token after validating token +issuer signature, expiration and applying the configured attribute mapping. +Since in such case there is no real human the mapping also need to be +configured slightly different. + +- It is strongly advised the attribute mapping must fill `token_user_id`, + `token_project_id` (and soon `token_role_ids`). This allows strong control of + which technical account (soon a concept of service accounts will be introduced + in Keystone) is being used and which project such request can access. + +- Attribute mapping should use `bound_audiences`, `bound_claims`, + `bound_subject`, etc to control the tokens issued by which workflows are + allowed to access OpenStack resources. + +### GitHub workflow federation + +In order for the GitHub workflow to be able to access OpenStack resources it is +necessary to register GitHub as a federated IdP and establish a corresponding +attribute mapping of the `jwt` type. + +IdP: + +```json +"identity_provider": { + "name": "github", + "bound_issuer": "https://token.actions.githubusercontent.com", + "jwks_url": "https://token.actions.githubusercontent.com/.well-known/jwks" +} +``` + +Mapping: + +```json +"mapping": { + "type": "jwt", + "name": "gtema_keystone_main", + "idp_id": , + "domain_id": , + "bound_audiences": ["https://github.com"], + "bound_subject": "repo:gtema/keystone:pull_request", + "bound_claims": { + "base_ref": "main" + }, + "user_id_claim": "actor_id", + "user_name_claim": "actor", + "token_user_id": +} +``` + +TODO: add more claims according to [docs](https://docs.github.com/en/actions/reference/security/oidc#oidc-token-claims) + +A way for the workflow to obtain the JWT [is described here](https://docs.github.com/en/actions/reference/security/oidc#methods-for-requesting-the-oidc-token). + +```yaml +... +permissions: + token: write + contents: read + +job: + ... + - name: Get GitHub JWT token + id: get_token + run: | + TOKEN_JSON=$(curl -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \ + "$ACTIONS_ID_TOKEN_REQUEST_URL&audience=https://github.com") + + TOKEN=$(echo $TOKEN_JSON | jq -r .value) + echo "token=$TOKEN" >> $GITHUB_OUTPUT + ... + # TODO: build a proper command for capturing the actual token and/or write a dedicated action for that. + - name: Exchange GitHub JWT for Keystone token + run: | + KEYSTONE_TOKEN=$(curl -H "Authorization: bearer ${{ steps.get_token.outputs.token }}" -H "openstack-mapping: gtmema_keystone_main" https://keystone_url/v4/federation/identity_providers/IDP/jwt) + +``` diff --git a/doc/src/oidc.md b/doc/src/oidc.md new file mode 100644 index 00000000..a902878b --- /dev/null +++ b/doc/src/oidc.md @@ -0,0 +1,36 @@ +# Authentication using the Authorization Code flow and Keystone serving as RP + +```mermaid +sequenceDiagram + + Actor Human + Human ->> Cli: Initiate auth + Cli ->> Keystone: Fetch the OP auth url + Keystone --> Keystone: Initialize authorization request + Keystone ->> Cli: Returns authURL of the IdP with cli as redirect_uri + Cli ->> User-Agent: Go to authURL + User-Agent -->> IdP: opens authURL + IdP -->> User-Agent: Ask for consent + Human -->> User-Agent: give consent + User-Agent -->> IdP: Proceed + IdP ->> Cli: callback with Authorization code + Cli ->> Keystone: Exchange Authorization code for Keystone token + Keystone ->> IdP: Exchange Authorization code for Access token + IdP ->> Keystone: Return Access token + Keystone ->> Cli: return Keystone token + Cli ->> Human: Authorized + +``` + +## TLDR + +The user client (cli) sends authentication request to Keystone specifying the +identity provider, the preferred attribute mapping and optionally the scope (no +credentials in the request). In the response the user client receives the time +limited URL of the IDP that the user must open in the browser. When +authentication in the browser is completed the user is redirected to the +callback that the user also sent in the initial request (most likely on the +localhost). User client is catching this callback containing the OIDC +authorization code. Afterwards this code is being sent to the Keystone together +with the authentication state and the user receives regular scoped or unscoped +Keystone token. diff --git a/doc/src/policy.md b/doc/src/policy.md index 4e9cf75d..edd108ad 100644 --- a/doc/src/policy.md +++ b/doc/src/policy.md @@ -80,13 +80,36 @@ As can be guessed such policy would permit the API request when `admin` role is present in the current credentials roles or the mapping in scope is owned by the domain the user is currently scoped to with the `manager` role.` -Additional improvement from the legacy Keystone is the time and data when the -policies are evaluated. For `list` operation policy input is populated with the -credentials and all query parameters. For `show` operation the input -additionally contain the target object previously fetched so that the policy -can additionally consider current resource attributes. `create` operation also -gets the complete input. `update` operation first fetch the target resource and -pass it as the target, while the updated properties are passed as the "update" -object into the policy. The `delete` operation also fetches the to be deleted -object passing it into the policy. This approach allow advanced cases where -operations may need to be prohibited by certain resource attributes. +## List operation + +All query parameters are passed into the policy engine to be provide capability +of making decision based on the parameters passed. For example an admin user +may specify `domain_id` parameter when the current authentication scope is not +matching the given `domain_id` or a user with the `manager` role being able to +list shared federated identity providers. + +Policy is being evaluated before the real data is being fetched from the backend. + +## Show operation + +Policy evaluation for GET operations on the resource are executed with the +requested entity in the scope. This allows policy to deny the operation if the +user requested resource it is should not have access to. This means that 404 +error may be raised before the validation of whether the user is allowed to +perform such operations. + +## Create operation + +Resource creation operation would pass the whole object to be created in the +context to the policy enforcement engine. + +## Update operation + +For the update operation the context contain the current state of the resource +and the new one. This allows defining policies preventing resource update upon +certain conditions (i.e. when tag "locked" is added). + +## Delete operation + +Resource deletion also passes the current resource state in the context to +allow comprehensive logic.