Skip to content

Downstream only: Rework of Makefile and incusion of lint#339

Merged
weshayutin merged 1 commit into
openshift:konveyor-devfrom
mpryc:makefile_rework
Sep 4, 2024
Merged

Downstream only: Rework of Makefile and incusion of lint#339
weshayutin merged 1 commit into
openshift:konveyor-devfrom
mpryc:makefile_rework

Conversation

@mpryc
Copy link
Copy Markdown

@mpryc mpryc commented Aug 22, 2024

The rework of Makefile to make it more readable and inclusion of lint as a target.

  • Removed unnecessary installation of code-generator which was not used in the Makefile.prow
  • Removed copy of the code to separate folder, which was required by code-generator
  • Removed installation of kubectl, which isn't used at the moment by unit and lint jobs
  • Rework of code to be eye-candy
  • KUBEBUILDER_ASSETS moved inside test target, so it's not invoked for lint job
  • Added fake verify-modules target so now make ci is calling exactly same targets as upstream Makefile
  • Replaced docker with podman for running tests locally as docker is not available by default in RHEL/Fedora.

Tests performed:

$ podman run -it --rm -v `pwd`:`pwd`:Z -w `pwd` quay.io/konveyor/builder:ubi9-v1.22.2 bash -c "make -f Makefile.prow ci"
$ podman run -it --rm -v `pwd`:`pwd`:Z -w `pwd` quay.io/konveyor/builder:ubi9-v1.22.2 bash -c "make -f Makefile.prow lint"

Comment thread Makefile.prow Outdated
Comment thread Makefile.prow Outdated

.PHONY: verify-modules
verify-modules:
@echo "verify-modules target is a placeholder to use same targets as upstream CI"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how this calls the target in the other Makefile?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not, it's just a placeholder that does nothing, but the make ci target is exactly the same as upstream:

velero/Makefile

Line 330 in 32e4c95

ci: verify-modules verify all test

equals to our Makefile.prow target.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it does nothing, while other makefile does anotherthing, it is not the same...

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok will update to call original upstream.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be resolved. Please check.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will try to produce a test locally

@mpryc
Copy link
Copy Markdown
Author

mpryc commented Aug 22, 2024

@mateusoliveira43 while adding lint target I did notice that this PR introduced lint error, because we never ran lint jobs, should I fix it with this PR or address with another. We will unfortunately need to backport this to other branches of downstream Velero:

32e4c95#diff-23f2d480d6500154fe7ea296a25696240962d708378c7d8f4648958e3579a864R34

The linting error:

pkg/repository/udmrepo/kopialib/backend/common_kopia_algorithms_test.go:33:48: unnecessary leading newline (whitespace)

@mateusoliveira43
Copy link
Copy Markdown

@mpryc first, the errors needs to show in our CI, so adding it to makefile target is needed

We need to confirm the errors are only in our patches, as upstream files should already be linted

and we need to find a way to always run the same version as upstream. currently it is the same, but after rebase, it will be outdated https://github.com/openshift/release/blob/master/ci-operator/config/openshift/velero/openshift-velero-konveyor-dev.yaml#L5

@mpryc
Copy link
Copy Markdown
Author

mpryc commented Aug 22, 2024

@mateusoliveira43

@mpryc
and we need to find a way to always run the same version as upstream. currently it is the same, but after rebase, it will be outdated https://github.com/openshift/release/blob/master/ci-operator/config/openshift/velero/openshift-velero-konveyor-dev.yaml#L5

I don't understand that part? We do have go version only in the above file, so after which rebase it will be outdated? We don't run manually container execution of quay.io/konveyor/builder:ubi9-v1.22.2, this is hashed and just a comment on the Makefile.prow

@mateusoliveira43
Copy link
Copy Markdown

wrong link, sorry

right one https://github.com/vmware-tanzu/velero/blob/main/hack/build-image/Dockerfile#L97

golangci-lint version is my concern, not container

@weshayutin
Copy link
Copy Markdown

ok  	github.com/vmware-tanzu/velero/internal/volume	0.021s	coverage: 48.9% of statements
ok  	github.com/vmware-tanzu/velero/internal/volumehelper	0.026s	coverage: 37.4% of statements
Success!
make[1]: Leaving directory '/home/whayutin/OPENSHIFT/git/OADP/velero'

WFM :)

@mpryc
Copy link
Copy Markdown
Author

mpryc commented Aug 22, 2024

wrong link, sorry

right one https://github.com/vmware-tanzu/velero/blob/main/hack/build-image/Dockerfile#L97

golangci-lint version is my concern, not container

How about we do something smart inside a Makefile and if it's not found then fail:

$ grep -oP 'golangci-lint/master/install.sh.*\Kv[0-9]+\.[0-9]+\.[0-9]+' hack/build-image/Dockerfile
v1.57.2

Comment thread Makefile.prow
KUBEBUILDER_ASSETS=$(shell echo $(shell $(GOBIN)/setup-envtest use -p path) | sed 's/ /\\ /g')
.PHONY: envtest
envtest: $(GOBIN)/setup-envtest
.PHONY: lint
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this

we are adding without using it

Copy link
Copy Markdown
Author

@mpryc mpryc Aug 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to introduce prow job for that target, but first we need to have it in the Makefile so prow CI job can be successful. It's chicken-egg problem.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on OADP, we have everything under unit-test job

would do the same approach here

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very much different from the upstream and usually lint is not part of unit. Having it separate gives clean information that the code is OK, just needs reformatting without checking job logs.

@kaovilai
Copy link
Copy Markdown
Member

needs rebase

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 22, 2024
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 27, 2024
@mpryc
Copy link
Copy Markdown
Author

mpryc commented Aug 27, 2024

needs rebase

done

@sseago
Copy link
Copy Markdown

sseago commented Aug 27, 2024

@mpryc I'm concerned that this is going to lead to rebase merge conflict headaches every time upstream Makefile changes. Not sure what the right answer is, though.

@kaovilai
Copy link
Copy Markdown
Member

@sseago this is Makefile.prow our downstream only file which do not merge conflict with upstream Makefile.

@mpryc
Copy link
Copy Markdown
Author

mpryc commented Aug 27, 2024

Corresponding release change: openshift/release#55969

@sseago
Copy link
Copy Markdown

sseago commented Aug 27, 2024

@kaovilai @mpryc oops. Sorry, I misread the filename. You're right. No concerns here.

@weshayutin
Copy link
Copy Markdown

/retest

@kaovilai
Copy link
Copy Markdown
Member

kaovilai commented Sep 3, 2024

I don't like how this pr is showing 41 commits.. did you fetch openshift before rebasing against openshift/konveyor-dev?

The rework of Makefile to make it more readable and
inclusion of lint as a target as well extract
golangci-lint version from the upstream Dockerfile,
so we test in PROW or locally on the same version as upstream.

Signed-off-by: Michal Pryc <mpryc@redhat.com>
@mpryc
Copy link
Copy Markdown
Author

mpryc commented Sep 4, 2024

I don't like how this pr is showing 41 commits.. did you fetch openshift before rebasing against openshift/konveyor-dev?

Fixed, that was based before rebase of velero, that's why.

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Sep 4, 2024

@mpryc: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@weshayutin weshayutin merged commit b78d068 into openshift:konveyor-dev Sep 4, 2024
@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Sep 4, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: kaovilai, mpryc, sseago

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mpryc
Copy link
Copy Markdown
Author

mpryc commented Sep 5, 2024

/cherry-pick oadp-1.4

@openshift-cherrypick-robot
Copy link
Copy Markdown

@mpryc: new pull request created: #345

Details

In response to this:

/cherry-pick oadp-1.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants