Skip to content

2025.1 sapcc cherry picks and required changes#38

Merged
stanislav-zaprudskiy merged 46 commits intostable/2025.1-m3from
2025.1-cherry-picks
Nov 28, 2025
Merged

2025.1 sapcc cherry picks and required changes#38
stanislav-zaprudskiy merged 46 commits intostable/2025.1-m3from
2025.1-cherry-picks

Conversation

@stanislav-zaprudskiy
Copy link
Copy Markdown
Collaborator

@stanislav-zaprudskiy stanislav-zaprudskiy commented Nov 25, 2025

The previous changeset stable/2024.1...sapcc:keystone:stable/2024.1-m3

diff command (collapsible) 👇
$ diff \
  <(git log --no-decorate --oneline origin/stable/2024.1...origin/stable/2024.1-m3 | cut -d ' ' -f 2-) \
  <(git log --no-decorate --oneline origin/stable/2025.1...2025.1-cherry-picks | cut -d ' ' -f 2-)
1,3d0
< Merge pull request #35 from sapcc/cherry_pick_ossa-2025-002
< Add service user authentication to ec2 and s3 endpoints
< Add test with noauth for s3tokens and ec2tokens
5d1
< Merge pull request #33 from sapcc/bmemcached
7d2
< Merge pull request #31 from sapcc/chery-pick-pci-dss-invalid-password-reporting
9d3
< Merge pull request #29 from bbobrov/stable/2024.1-m3
15d8
< Correct format for token expiration time
21d13
< Catch CredentialNotFound and raise a proper exception

lost the following commits:

Additions:

Rudolf Vriend and others added 15 commits November 25, 2025 15:22
Change-Id: I628a3a6b524cd099345463e3a6bafe16df450581
Change-Id: Ic236212dfda70a28ea6fece177c05308c12936d1
…instead of a hard-coded admin check (which is definitely not enough)

Change-Id: Ia6071a0ba7c698ee2a425096888f06a12c1e236e
We are using ldap and our max_size is bigger.
Also update tests

Change-Id: I13dc7cf77dbad236492f7504033d0cb41a5656cd
…t to cloud_admin for now

Co-authored-by: Maurice Escher <maurice.escher@sap.com>
Change-Id: I765e4d2b5999f282a3b324312fb2485bc38ad914
configuration option to set tags that get added to newly created projects

Change-Id: Icac8d54506082816bdaeacb73853a20b49735c16
- add policy_id to target
- it was possible to create ec2 creds to different projects
without a policy check
@stanislav-zaprudskiy stanislav-zaprudskiy changed the title 2025.1 cherry picks 2025.1 sapcc cherry picks Nov 26, 2025
Carthaca and others added 12 commits November 26, 2025 10:18
Change-Id: I7b81f7d16987f0e633cd999923bdfe19b4e0d3da
tox running on python2 causes weird issues with string encoding. Instead
of fixing those issues, switch it to python3
Change-Id: I120ef1c0c8259c85b6030f2db0a649c71b990879
closes-bug: 1877393
If project list is requested via /v3/projects with domain-scoped
token, only the project in the domain are being returned. It makes no
sense when is_domain filter is used, because domains are top-level the
request always returned an empty list.

Return domain list the same way as if /v3/domains is being used and do
not filter out anything. Note: the policies still need to be adjusted by
the operators if they wish to allow this kind of request.

Closes-Bug: 1950325
Change-Id: I77ed200d1a222659abd1e2f00b9984647b310c43
Change-Id: I271bf96ea0a0bde3ca91c6de6ee90ed55e3fd72a
Instead, just use whatever python3 there is.

Also, use py38 for unit tests

Change-Id: I48d5995ac249516bac547e01e7f67ccddac0a7d7
Change-Id: I4be1b3e888f5ef7d3a557695d3b103374cebe106
tox had a lot of changes that break test runs. Instead of adapting to
these chances, cap tox. Uncap later, when OpenStack upstream fixes it

Change-Id: Ib7338e937fb49d08ce1890626a50996747bd34c9
Change-Id: I26b1db96a20895c851089458b6ab9bcd4223829c
Change-Id: Ia9d1e8f803824e96fcfb3541d142b41a54eea1a6
Change-Id: I436fdc526316648098bb31dbe62a8d863af2f2eb
bbobrov and others added 9 commits November 26, 2025 10:18
Change-Id: I7ed981a74e9f89327659fb46972e87b79951ae85
Change-Id: Iad1ef565c04d2a2d20a929c4ccd0bf4e7d492afc
Due to architectural reasons,
bug https://bugs.launchpad.net/keystone/+bug/1878438 appeared.
There is no good way to fix it, upstream also cannot get to it.
If someone hits the bug, they get error 500 and keystone crashes.

Fix this hard crash and return an Unauthorized response instead. This
will not break any existing usecases, because things are not working
already. This change should be reverted after upstream fixes the bug.

Change-Id: I0d7802ddcdef7646f43fd57a0cf9ae94686d58e9
The ratelimiting middleware seems to bring more maintainance than use,
which is why it should be disabled until we figure out how to properly
set it up.

Change-Id: If01714058982e64bb58bccf7cc853a22fc0c0ac7
Change-Id: I915e3128ff02bde2f00fdefeafde3f6f46b04c5d
Keystone uses many non-standard names for credentials and we need to
explicitly list them.

Change-Id: Icaaa785f0dd5fb25f3831aafe420b6db731574b1
These messages are too spammy and do not bring any value

Change-Id: I13e4dbcc3333cb8e3154e5a463f2614a47dd70da
stanislav-zaprudskiy and others added 5 commits November 26, 2025 10:53
Add support of configuration allowing inclusion of partial hash of
invalid password in event notifications to facilitate anyalysis of
failed login attemps.

SecurityImpact
Related-Bug: 2060972
Depends-On: https://review.opendev.org/c/openstack/keystone-specs/+/915482
Closes-Bug: 2060972

Change-Id: I0f34d90660a4a915c9c3f9512dc6d794b8415cd5
Install the python-binary-memcached package into the Keystone image

SALS works only over a binary protocol, and it can only work with
`dogpile.cache.bmemcached` backend, which requires pip package
`python-binary-memcached` to be present.
We also set `nosec` for one use of `random.sample()` to make `bandit` -
which runs as part of `tox -e pep8` - happy. Upstream has that same line
in _without_ `nosec` and I don't get why pep8 tests work for them.

Change-Id: Id26894f2da2877c2455e02a6d3290bccb2b6decd
@stanislav-zaprudskiy
Copy link
Copy Markdown
Collaborator Author

I was able to execute py312 tests locally

Ran: 5723 tests in 154.6647 sec.
 - Passed: 5006
 - Skipped: 715
 - Expected Fail: 2
 - Unexpected Success: 0
 - Failed: 0

* tox: do not exclude-regex tests

It doesn't cause failures without the argument.

* Do not clone from remote but use Concourse provided repo

Concourse already has the repo version to be tested cloned under
`source` path.

* Do not run memcached

Executing without memcached doesn't change the number of tests and the
outcome.

* Do not set WATCHER_DISABLED=true

It looks like runtime parameter, not changing tests behavior.

* Make it a normal bash scipt

It doesn't have to be one-line, given the way Concourse runs it.

But working with it as with a regular script is much easier in
development and testing.

* Install tools in `tools` venv

Installing them on Ubunbut 24.04 fails -
https://stackoverflow.com/a/75696359. Whereas an alternative option
could be to use `--break-system-packages` - which is less nicer than
venv IMHO.

* Use latest tox

Upstream tests run with v4+, and our could too.

* Install OS requirements using bindep

The OS requirements packages change over time (some of which are no
longer relevant for e.g. Ubuntu 24.04), and are maintained in
`bindep.txt`. So just install them with `bindep`, similar to how Zuul
does that upstream
- https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/bindep/tasks/packages.yaml.

The disadvantage could be that it could install more than required -
e.g. mysql, postgresql or graphviz, which we don't need for our test
run.

* Add `build-essential` to fix psycopg2, python-ldap

* Install and configure git

Without `safe.directory` it fails to use `git` under `/source`, and
fails to recognize pbr version correspondingly
(https://docs.openstack.org/pbr/latest/user/features.html#version).

* Maintain LDAP tests environment variables

With or without them the number of executed tests and the outcome are
the same. Not sure why so - needs more investigation. But just maintain
them as in the documentation -
https://docs.openstack.org/keystone/latest/contributor/testing-keystone.html#ldap-tests

* Make tox to NOT skip-missing-interpreters

This is to make sure that it won't succeed in case the interpreter could
not be found.
Copy link
Copy Markdown

@JoJoPuppe JoJoPuppe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked how you arrived at the proposed cherry-picked commits and your reasoning behind why certain commits are no longer needed. It makes sense to me, and running the test locally with py312 resulted in nearly the same outcome as yours. Somehow, one more test was run, but it was also successful.

======
Totals
======
Ran: 5724 tests in 94.0914 sec.
 - Passed: 5007
 - Skipped: 715
 - Expected Fail: 2
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 1029.0674 sec.

@stanislav-zaprudskiy stanislav-zaprudskiy changed the title 2025.1 sapcc cherry picks 2025.1 sapcc cherry picks and required changes Nov 27, 2025
Copy link
Copy Markdown
Member

@tz3 tz3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR.

I ran through the commits LGTM could run the test locally (with docker)

Totals
======
Ran: 3269 tests in 109.8747 sec.
 - Passed: 2868
 - Skipped: 400
 - Expected Fail: 1
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 861.7773 sec.

==============
Worker Balance
==============
 - Worker 0 (407 tests) => 0:01:49.812731
 - Worker 1 (407 tests) => 0:01:49.209030
 - Worker 2 (409 tests) => 0:01:49.016440
 - Worker 3 (409 tests) => 0:01:48.744222
 - WARNING: missing Worker 4!
 - WARNING: missing Worker 5!
 - WARNING: missing Worker 6!
 - WARNING: missing Worker 7!
 - Worker 8 (409 tests) => 0:01:47.475433
 - Worker 9 (409 tests) => 0:01:45.047961
 - WARNING: missing Worker 10!
 - WARNING: missing Worker 11!
 - Worker 12 (409 tests) => 0:01:45.182973
 - Worker 13 (410 tests) => 0:01:49.447896
  py312: OK (208.53=setup[80.58]+cmd[127.95] seconds)
  congratulations :) (208.82 seconds)```

@stanislav-zaprudskiy stanislav-zaprudskiy merged commit e306b1e into stable/2025.1-m3 Nov 28, 2025
3 checks passed
@stanislav-zaprudskiy stanislav-zaprudskiy deleted the 2025.1-cherry-picks branch November 28, 2025 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants