Skip to content

Conversation

@euanh
Copy link
Contributor

@euanh euanh commented Jun 20, 2018

Previously the build container did not provide protoc or protoc-gen-gogo; now these are available in the container. There is also a make pattern rule to regenerate .pb.go files from .proto files.

I regenerated the .pb.go files. agent.go.pb was regenerated recently and does not change, however older .pb.go files change slightly when regenerated with newer tools.

@euanh euanh force-pushed the update-protobuf branch from 7c3e905 to abe7420 Compare June 20, 2018 16:32
@euanh
Copy link
Contributor Author

euanh commented Jun 20, 2018

@fcrisciani, @ctelfer: This should make regenerating protocol buffers more consistent. I'll try to come up with a nice regenerate-protobufs target in the Makefile as well. In the meantime, just touch the .proto file or delete the .pb.go and run make <path/foo>.pb.go from within the build container.

Copy link
Contributor

@ctelfer ctelfer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely a worthwhile thing to add and the output looks sane from a brief glance. Have you tried building moby with the resulting libnetwork. Best double-check would be to replace one of the moby's in a cluster with one w/ the updated libnetwork and then run some test load of services to make sure all nodes can still communicate cleanly w/ each other. (I can't imagine the PB mods will break ... but need to verify of course.)

@euanh
Copy link
Contributor Author

euanh commented Jun 21, 2018

I haven't tried building moby with these changes yet. Is there a convenient set of tests which particularly exercises libnetwork?

@euanh euanh force-pushed the update-protobuf branch 2 times, most recently from 1d15e95 to ebc2fda Compare June 22, 2018 13:15
euanh added 2 commits June 22, 2018 15:03
Signed-off-by: Euan Harris <euan.harris@docker.com>
agent.pb.go is unchanged, but the files in networkdb and drivers
are slightly different when regenerated using the current versions
of protoc and gogoproto.    This is probably because agent.pb.go
was last regenerated quite recently, in February 2018, whereas
networkdb.pb.go and overlay/overlay.pb.go were last changed in 2017,
and windows/overlay/overlay.pb.go was last changed in 2016.

Signed-off-by: Euan Harris <euan.harris@docker.com>
@euanh euanh force-pushed the update-protobuf branch from ebc2fda to e12dd44 Compare June 22, 2018 14:03
@euanh euanh changed the title WIP: Add support for regenerating protocol buffers, and refresh .pb.go files Add support for regenerating protocol buffers, and refresh .pb.go files Jun 22, 2018
@euanh
Copy link
Contributor Author

euanh commented Jun 22, 2018

I tried using the vendored version of gogoproto, but protoc generated bad import statements which go build couldn't resolve:

import _ "gogoproto"

instead of

import _ "github.com/gogo/protobuf/gogoproto"

Copy link

@fcrisciani fcrisciani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PB_FILES=$(PROTO_FILES:.proto=.pb.go)

%.pb.go: %.proto
${docker} protoc -I=. -I=/go/src -I=/go/src/github.com/gogo/protobuf -I=/go/src/github.com/gogo/protobuf/protobuf --gogo_out=./ $<

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't not have to be in this PR, but would be great to fail if the PR while compiled by the CI has a diff with the file generated compared to the files coming from the PR itself. This way we can enforce that the PR contains all the proper information.
Also does it makes sense to have build target depend on the protobuf generation?

@selansen
Copy link
Contributor

LGTM

@euanh
Copy link
Contributor Author

euanh commented Jun 26, 2018

@fcrisciani @selansen @ctelfer I'm currently running the tests that @ctelfer suggested and tracking down a couple of failures.

@euanh
Copy link
Contributor Author

euanh commented Jun 27, 2018

I reproduced the same failures running a mixed-version cluster with moby master (without my changes) on the leader and 17.06 on two other nodes. Updating all nodes to moby master made them pass. Similarly, with my changes only on the leader I had a couple of failures; with my changes on all nodes the tests passed.

Copy link
Contributor

@ctelfer ctelfer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants