Skip to content

change wait calls for check calls in tests#11680

Merged
knative-prow-robot merged 5 commits intoknative:mainfrom
leetcope:replace-waits-for-checks-on-e2e-tests
Aug 31, 2021
Merged

change wait calls for check calls in tests#11680
knative-prow-robot merged 5 commits intoknative:mainfrom
leetcope:replace-waits-for-checks-on-e2e-tests

Conversation

@leetcope
Copy link
Copy Markdown
Contributor

Fixes #1178

Proposed Changes

  • Bring in changes from knative/pkg to use CheckEndpointState

@google-cla google-cla bot added the cla: yes Indicates the PR's author has signed the CLA. label Jul 14, 2021
@knative-prow-robot knative-prow-robot added area/test-and-release It flags unit/e2e/conformance/perf test issues for product features size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 14, 2021
@codecov
Copy link
Copy Markdown

codecov bot commented Jul 14, 2021

Codecov Report

Merging #11680 (ff7cd6e) into main (0be381e) will decrease coverage by 0.07%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             main   #11680      +/-   ##
==========================================
- Coverage   87.85%   87.78%   -0.08%     
==========================================
  Files         196      196              
  Lines        9373     9430      +57     
==========================================
+ Hits         8235     8278      +43     
- Misses        884      893       +9     
- Partials      254      259       +5     
Impacted Files Coverage Δ
pkg/reconciler/route/reconcile_resources.go 76.66% <0.00%> (-4.94%) ⬇️
pkg/reconciler/route/controller.go 100.00% <0.00%> (ø)
pkg/apis/serving/v1/route_lifecycle.go 100.00% <0.00%> (ø)
pkg/reconciler/route/route.go 79.89% <0.00%> (+0.19%) ⬆️
pkg/reconciler/configuration/configuration.go 88.46% <0.00%> (+1.53%) ⬆️
pkg/reconciler/route/resources/service.go 93.02% <0.00%> (+1.77%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0be381e...ff7cd6e. Read the comment docs.

@markusthoemmes
Copy link
Copy Markdown
Contributor

Have you considered #11404 (and by extension #5573) in this work?

@knative-prow-robot knative-prow-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 15, 2021
@leetcope leetcope force-pushed the replace-waits-for-checks-on-e2e-tests branch from 9cf4a5b to 16b1b7a Compare July 15, 2021 19:52
@knative-prow-robot knative-prow-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 15, 2021
@leetcope
Copy link
Copy Markdown
Contributor Author

/retest

@leetcope
Copy link
Copy Markdown
Contributor Author

@markusthoemmes thanks for the remainder, I had forgotten about the route state check

@leetcope leetcope force-pushed the replace-waits-for-checks-on-e2e-tests branch from 67b9d78 to 8ab62d5 Compare July 16, 2021 02:38
@leetcope
Copy link
Copy Markdown
Contributor Author

/assign @dprotaso

@markusthoemmes
Copy link
Copy Markdown
Contributor

@markusthoemmes thanks for the remainder, I had forgotten about the route state check

I'm not quite parsing that response 🤔. My question was whether this impacts the discussion in the issue I shared (I think it does?).

@leetcope
Copy link
Copy Markdown
Contributor Author

@markusthoemmes well I didn't change every place where a WaitFor function is used, at first I did but some actually need the WaitFor function, so I don't think it made the tests any narrower

@dprotaso
Copy link
Copy Markdown
Member

I still see a few more waits after we check/wait for our resources to be Ready

ie. here's a raw dump - not all apply

$ rg WaitForEndpointState

test/conformance/runtime/util.go
58:	resp, err := pkgTest.WaitForEndpointState(

test/conformance/runtime/readiness_probe_test.go
114:				if _, err = pkgtest.WaitForEndpointState(

test/e2e/scale.go
186:				_, err = pkgTest.WaitForEndpointState(
195:					t.Error("WaitForEndpointState(expected text) =", err)
196:					return fmt.Errorf("WaitForEndpointState(expected text) failed: %w", err)

test/conformance/api/v1/util.go
131:	_, err := pkgTest.WaitForEndpointState(

test/conformance/api/v1/route_test.go
43:	_, err := pkgtest.WaitForEndpointState(

test/e2e/autoscale.go
237:	if _, err := pkgTest.WaitForEndpointState(

test/e2e/logging_test.go
108:		// A request was sent to / in WaitForEndpointState.

test/e2e/autoscale_hpa_test.go
94:	if _, err := pkgTest.WaitForEndpointState(

test/e2e/tagheader/tag_header_based_routing_test.go
118:			if _, err := pkgTest.WaitForEndpointState(

test/performance/benchmarks/scale-from-zero/continuous/main.go
198:		_, err := pkgTest.WaitForEndpointStateWithTimeout(

test/ha/ha.go
66:	if _, err := pkgTest.WaitForEndpointState(

test/upgrade/upgrade.go
53:	if _, err := pkgTest.WaitForEndpointState(

$ rg WaitForRouteState

test/e2e/subroutes_test.go
160:	if err = v1test.WaitForRouteState(clients.ServingClient, resources.Route.Name, func(r *v1.Route) (bool, error) {
185:	if err := v1test.WaitForRouteState(clients.ServingClient, resources.Route.Name, func(r *v1.Route) (bool, error) {
271:	if err = v1test.WaitForRouteState(clients.ServingClient, resources.Route.Name, func(r *v1.Route) (bool, error) {
303:	if err = v1test.WaitForRouteState(clients.ServingClient, resources.Route.Name, func(r *v1.Route) (b bool, e error) {
347:	if err = v1test.WaitForRouteState(clients.ServingClient, resources.Route.Name, v1test.IsRouteReady, "Route is ready"); err != nil {

test/e2e/destroypod_test.go
72:	if err := v1test.WaitForRouteState(clients.ServingClient, names.Route, v1test.IsRouteReady, "RouteIsReady"); err != nil {

test/conformance/api/v1/generatename_test.go
191:	if err := v1test.WaitForRouteState(clients.ServingClient, names.Route, v1test.IsRouteReady, "RouteIsReady"); err != nil {

test/conformance/api/v1/route_test.go
36:	if err := v1test.WaitForRouteState(clients.ServingClient, names.Route, v1test.IsRouteReady, "RouteIsReady"); err != nil {

test/e2e/route_service_test.go
161:			if err = v1test.WaitForRouteState(clients.ServingClient, serviceresourcenames.Route(svc), hasPrivateRoute, "RouteIsClusterLocal"); err != nil {

test/e2e/autotls/auto_tls_test.go
74:	if err = v1test.WaitForRouteState(clients.ServingClient, names.Route, routeTLSDisabled, "RouteTLSDisabled"); err != nil {
78:	if err = v1test.WaitForRouteState(clients.ServingClient, names.Route, routeURLHTTP, "RouteURLIsHTTP"); err != nil {
136:		if err = v1test.WaitForRouteState(clients.ServingClient, names.Route, routeTrafficHTTPS, "RouteTrafficIsHTTPS"); err != nil {

test/v1/route.go
69:// WaitForRouteState polls the status of the Route called name from client every
73:func WaitForRouteState(client *test.ServingClients, name string, inState func(r *v1.Route) (bool, error), desc string) error {
74:	span := logging.GetEmitableSpan(context.Background(), fmt.Sprintf("WaitForRouteState/%s/%s", name, desc))
97:// This is the non-polling variety of WaitForRouteState

@leetcope leetcope force-pushed the replace-waits-for-checks-on-e2e-tests branch from 8ab62d5 to 14f917b Compare August 19, 2021 19:04
@knative-prow-robot knative-prow-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 19, 2021
@leetcope leetcope force-pushed the replace-waits-for-checks-on-e2e-tests branch from 14f917b to 0c71c53 Compare August 19, 2021 19:37
@leetcope
Copy link
Copy Markdown
Contributor Author

/retest

@leetcope leetcope force-pushed the replace-waits-for-checks-on-e2e-tests branch from 0c71c53 to bb1c75b Compare August 19, 2021 20:40
@knative-prow-robot knative-prow-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 25, 2021
Signed-off-by: Fabian Lopez <lfabian@vmware.com>
Signed-off-by: Fabian Lopez <lfabian@vmware.com>
Signed-off-by: Fabian Lopez <lfabian@vmware.com>
Signed-off-by: Fabian Lopez <lfabian@vmware.com>
@leetcope leetcope force-pushed the replace-waits-for-checks-on-e2e-tests branch from bb1c75b to 231f058 Compare August 25, 2021 21:52
@knative-prow-robot knative-prow-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 25, 2021
Copy link
Copy Markdown
Member

@dprotaso dprotaso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there's a new test file added with the rebase test/e2e/emptydir_test.go that uses WaitFor when it can be a Check

I still see Waits in the following files - any reason why those can't be switch over?

test/conformance/api/v1beta1/domain_mapping_test.go
test/conformance/api/v1/util.go
test/conformance/runtime/readiness_probe_test.go
test/e2e/autoscale.go

Comment thread test/e2e/logging_test.go
}

_, err = pkgtest.WaitForEndpointState(
_, err = pkgtest.CheckEndpointState(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the comments in line 108 in this file?

Comment thread test/e2e/scale.go
latencies.Add("time-to-ready", start)

_, err = pkgTest.WaitForEndpointState(
_, err = pkgTest.CheckEndpointState(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the error strings in this file - lines 196 and 197 to Match this new method call

Signed-off-by: Fabian Lopez <lfabian@vmware.com>
@dprotaso
Copy link
Copy Markdown
Member

/lgtm
/approve

@knative-prow-robot knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Aug 31, 2021
@knative-prow-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dprotaso, shinigambit

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow-robot knative-prow-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 31, 2021
@knative-prow-robot knative-prow-robot merged commit 68dba4a into knative:main Aug 31, 2021
@nak3
Copy link
Copy Markdown
Contributor

nak3 commented Sep 1, 2021

The Check() func used by CheckEndpointState drops DefaultErrorRetryChecker so this change made e2e narrower 😢
We probably need to make DefaultErrorRetryChecker available by some option?

@dprotaso
Copy link
Copy Markdown
Member

dprotaso commented Sep 1, 2021

Interesting we should probably add that to the Check call in knative.dev/pkg - cc @shinigambit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/test-and-release It flags unit/e2e/conformance/perf test issues for product features cla: yes Indicates the PR's author has signed the CLA. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"Wait" is overly used in e2e/conformance

5 participants