YAML Defined Tests Scorecard Plugin #1241
Conversation
AlexNPavel
added
kind/feature
AlexNPavel
requested review from
estroz,
hasbro17,
joelanford,
lilic,
shawn-hurley and
theishshah
openshift-ci-robot
added
do-not-merge/work-in-progress
AlexNPavel
force-pushed
the
scorecard-usertest-proposal-2
branch
2 times, most recently
from
27f76e9 to
697c985
Compare
|
This is ready for review. |
openshift-ci-robot
removed
the
do-not-merge/work-in-progress
AlexNPavel
force-pushed
the
scorecard-usertest-proposal-2
branch
from
697c985 to
8db07b2
Compare
| For `Status` and `Resources` fields, we can implement a bit of extra computation instead of simple string checking. For instance, | ||
| in the memcached-operator test, we should expect that the length of the `nodes` field (which is an array) has a certain length. To implement functions like | ||
| these, we can create some functions for these checks that are prepended by `scorecard_function_` and take an array of objects. For instance, in the above | ||
| example, `scorecard_function_length` would check that each field listed under it matches the specified length (like `nodes: 4`). If the yaml key does not |
There was a problem hiding this comment.
Can you give an example of how to implement a scorecard_function_?
There was a problem hiding this comment.
The scorecard functions are simply a part of the main yaml parser and just change the way we handle a submap. I have the scorecard_function_length in my old implementation.
Parser to get the interfaces we are checking: https://github.com/AlexNPavel/operator-sdk/blob/scorecard-usertest//commands/operator-scorecard/lib/lib.go#L284:L296
Length function: https://github.com/AlexNPavel/operator-sdk/blob/scorecard-usertest//commands/operator-scorecard/lib/lib.go#L198:L213
The only functions I can think of that we would need are array length and regex
There was a problem hiding this comment.
Ok it would be good to at least list those two in this section.
|
Issues go stale after 90d of inactivity. Mark the issue as fresh by commenting If this issue is safe to close now please do so with /lifecycle stale |
openshift-ci-robot
added
the
lifecycle/stale
|
@jmccormick2001 is this doc still relevant after the changes you've made? If so we should probs get this merged @AlexNPavel. |
|
/retest |
I'd probably wait to merge it since we have just entered new stories to define a new 'custom test' design/interface that would work with the other scorecard changes in the queue all as part of v1alpha2 API changes. |
camilamacedo86
left a comment
camilamacedo86
left a comment
There was a problem hiding this comment.
Since it is old proposal thas been open since 29 May, I am ok with too.
But the new ones we should be doing following the template which as added in the project.
|
/hold |
openshift-ci-robot
added
the
do-not-merge/hold
|
Why @estroz /hold this one? What is missing for we merge this? |
camilamacedo86
added
the
approved
|
Stale issues rot after 30d of inactivity. Mark the issue as fresh by commenting If this issue is safe to close now please do so with /lifecycle rotten |
openshift-ci-robot
added
lifecycle/rotten
|
Rotten issues close after 30d of inactivity. Reopen the issue by commenting /close |
|
@openshift-bot: Closed this PR. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
6 participants
Description of the change: Add proposal for a YAML Defined Tests plugin for the scorecard
Motivation for the change: This was initially proposed in PR #1049, but has since been split off into this PR as the goals of that proposal changed. There is a lot of discussion over this design on that PR.
This PR will initially be created as a draft as I have not yet implemented various improvements that I've planned. It will be changed to a standard PR once I make those changes.
This PR also depends on PR #1049 getting merged first, and it may be wise to wait for the implementation of that proposal before fully moving forward with this PR.