-
Notifications
You must be signed in to change notification settings - Fork 2.1k
plugin: update/improve process lifecycle documentation #4960
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm... and I just realised we may have picked a very wrong name for this env-var, or at least the GoDoc should probably be very clear that this is an internal env-var, used between the CLI (acting as "host" for plugins) and binaries running as CLI-plugin.
DOCKER_XXXXXXformat for user-facing env-vars; in this case, we don't want users to (e.g.)DOCKER_CLI_PLUGIN_SOCKET=<whatever> /usr/libexec/docker/cli-plugins/docker-compose ....rmthe given path if it's a macOS binary?)Should we change it to something more clearly meant for "internal"? Any good conventions for that?
docker_cli_plugin_socket)_/__(__docker_cli_plugin_socket); could even have a_addrsuffix to indicate it's the address (__docker_cli_plugin_socket_addr).__docker_cli_plugin_socket_addr_v1), but I guess we can do so if we would need av2__docker_cli_plugin_internal_socket_addr)On topic of
_addre; should we have included the scheme / protocol? (unix:///); would there ever be a reason for us to change this? (grpc://)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with these points. I was especially curious about including the scheme/protocol in the env var value as well, as i guess right now we just assume the socket is
unixin the various plugins etcUh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Re: naming, we haven't been strict in the past, e.g.
DOCKER_TEST_exists in several places in the daemon. I'd suggest we just go with something like@DOCKERas the convention for internal variables, as the@makes it very hard to interact with these variables from shells.Making that change now would be moderately breaking, but I think it would be more acceptable now rather than later (if we want to bite the bullet) as mismatched versions would only result in a reversion to the less-than-graceful lifecycle, instead of anything more drastic (e.g. ability to talk to a RPC server being broken).
Regarding the protocol, I don't think that makes sense, and I think it's just unnecessary complexity -- we control both the producer and the consumer here (in one codebase), and while cross-version compatibility is a concern, I think we just version the environment variable if we ever make a breaking change.
Adding e.g. gRPC or TTRPC here will not be breaking as older clients won't actually try to push anything over the socket. Also,
unix://would indicate a transport layer, whilegrpc://would indicate a protocol -- so something likeunix+grpc://would be a lot more appropriate.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So
DOCKER_TESTwas used for "CI-specific" env-vars, so "some consistency" there, but agreed, we haven't always been good at it. Still,DOCKER_is common for user-facing ones, so what I want to avoid.I'm slightly concerned about using characters outside of the recommended character set https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html;
"Environment variable names used by the utilities in the Shell and Utilities volume of POSIX.1-2017 consist solely of uppercase letters, digits, and the ( '_' ) from the characters defined in Portable Character Set and do not begin with a digit."
Hence thinking of lowercase (indicating "local" / "non exported"), and the underscore prefix.
Yup! This was a bit of an oversight I think, at least it didn't occur to me at the time.
w.r.t. compatibility; we can add a transitional situation; set both "old" and "new" env-var name for 1 release; this allows us to ship the new name fast (existing plugins continue to function, and don't even have to fall back), and then remove the "duplicate" env-var in the release after.
Yes, I haven't given that a lot of thought yet, other then IF we decide to make this change, "now" may be the right moment to do so as well.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Going outside the recommended character set would more or less be the point; the kernel/OS actually have very little in the way of opinions here outside the significance of '=' (see an example implementation in e.g. bionic libc), and the POSIX recommendations primarily relate to shell support.
With regard to compatibility, I think that's somewhat more optimistic than is realistic. I don't think that older plugins with a newer CLI are going to be that unusual, though it's not really an edge case I am too concerned about.
In any case, I'd rather not layer more complexity here... If we want to change the variable (and any changes should be in a separate PR), let's do that; but let's keep the simple 'sun_name of a Unix socket' pattern, and let's recognize that this may cause regressions for some users unless we maintain both variables in perpetuity (partially defeating the point).
Oh, and I forgot to address the 'what if the user sets this in standalone mode' question -- nothing, really. The
unlink(2)logic is in the parent CLI, so the worst they can do is cause some spurious attempts to connect and log messages.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Re: the overall point though, I don't see a need for changing the env-var right now. It looks kind of like a user-facing env var, which we could/should have avoided, but now that it's there, the downsides of changing it don't seem worth any positives we get from that? If it leaks through anywhere (where would that happen? it's set on the CLI process, and only used for the plugins it execs, and dies after) then there isn't really any consequences to that, since the socket is only used for signalling right now, and the socket gets cleaned up before the CLI exits regardless.
If in the future we make a change to use the socket for more things/communication, I think then would be a good time to reevaluate and maybe rename. Until then, I don't see a point.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@neersighted But why take the risk? Do we really need to? Go stdlib also supports abstract sockets on macOS, and we've seen how well that went
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Go stdlib does not support abstract sockets on macOS -- the convention of '@' to indicate a NUL byte at the start of
sun_pathis explicitly not part of the macOS implementation. It was our oversight/not checking what was supported that led to the creation of files with an '@' (and indeed, the 'skip unlink' is universal -- this was a shortcut taken by the Go team).See https://cs.opensource.google/go/go/+/master:src/syscall/syscall_linux.go;l=569-574;drc=87056756141798b4dfd51dcaaa3e4ce63633a884;bpv=0;bpt=1 and https://cs.opensource.google/go/go/+/master:src/syscall/syscall_windows.go;l=870-875 for the Linux and Windows implementations.
I don't see the risk here -- by consulting the sources of libc and Go on the platforms of interest, the decades-old convention can be trivially verified.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OH DUH. Things got mixed up in my head, when talking about socket name vs socket ENV VAR name 😓 Yeah, "@" should be fine here.
I still do think that we don't need to change this right now (we'll be doing breaking changes to older plugins that are out there for virtually no benefit).
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, you say shortcut, I say oversight 😅 – it's a bug, since now it doesn't unlink real sockets on macOS because they start with "@".