image: port registries.conf to pkg/configfile#773
Conversation
|
@jankaluza @mtrmac PTAL |
|
libimage tests set
This now fails since we cannot open this as dir, the only reason it worked before is due the use of a IMO incorrect WalkDir() function which does not error if the passed in path is not a dir. Should we keep supporting this? I guess I could special case /dev/null to set DoNotLoadDropInFiles? Or make pkg/configfile ignore the case when we do not get a directory? Or just fix the tests and hope no actual users depend on this option? I guess this ties back to the behavior that setting a custom main file will always still try to parse drop ins. So I assume the /dev/null as used to disable that? |
Doing this in tests might have started as a habit of mine — POSIX specifies properties of remarkably few paths, and I used to prefer writing POSIX-portable software. (See
That’s how I read this, and what various other tests have had to do — the goal is to use only the test config file without any of the system-wide drop-ins which might have been added by Just fix the tests please, I don’t think we ever promised that this would work. |
| } | ||
|
|
||
| if conf.EnvironmentName != "" && !conf.DoNotLoadDropInFiles { | ||
| if conf.EnvironmentName != "" && !conf.DoNotLoadDropInFiles && conf.CustomConfigFileDropInDirectory == "" { |
There was a problem hiding this comment.
A sort of semantics question — should this be driven by CustomConfigFilePath, the DropInDirectory, or both?
I started thinking that CustomConfigFilePath should turn this off (I think base on some renamed version of shouldLoadMainFile) … but, on reflection, the override is sort of a drop-in.
There was a problem hiding this comment.
I am not to sure either, I guess the simplest would be to always use it even when CustomConfigFilePath or CustomConfigFileDropInDirectory set. Then we could always have a env override but then it means cli options may not win over the env which is unexpected.
And yes since I considered this a drop in I only turned it off on CustomConfigFileDropInDirectory
There was a problem hiding this comment.
I agree that the most important part is that a caller can opt out of the environment variables in a documented way.
I’m fine with leaving it as is, but I’ll leave the thread marked unresolved to allow an opportunity for other opinions/views.
|
Ok added docs and some other minor fixes I found. I will go thourhg your comments tomorrow. Then one thing that is still missing here is the use of configfile.Slice over []string when parsing the toml but that seem not so trivial with the way the custom overwrites are already handled. |
mtrmac
left a comment
There was a problem hiding this comment.
Just a fairly quick look, I didn’t carefully read the configfile parts nor all of the tests.
| }, | ||
| }, | ||
| }, | ||
| // TODO: add more test cases |
There was a problem hiding this comment.
(It would be sort of elegant for Read and GetSearchPaths to share the test cases — but, as is, that would be a lot of duplication.)
|
|
||
| // configWrapper is used to store the paths from ConfigPath and ConfigDirPath | ||
| // and acts as a key to the internal cache. | ||
| type configWrapper struct { |
There was a problem hiding this comment.
configWrapper changes from “resolved paths” to “inputs to path resolution”. Is that safe?
It might very well be fine in practice, but e.g. a change to CONTAINERS_REGISTRIES_CONF (or HOMEDIR or XDG…) at runtime won’t have effect. That would probably only affect in-process tests, but I’m not sure.
OTOH it’s not clear that we can safely do much better. We could run ToConfigFileOptions().GetSearchPaths() at construction time, but that would be racy WRT later additions/removals of files on the filesystem (much more likely than environment variable changes during the lifetime of a process), and we’d need to somehow turn the result into a valid map key, yuck.
There was a problem hiding this comment.
yeah I mean we could try to resolve the envs here here and store the value as part of the hash key, but then we duplicate some intentionally abstracted details and also still have a race. And yes GetSearchPaths() does not feel any better either.
Also the fact that you cannot use []string as map hash key, so that woudl need to be turned into a string first as well.
I can add a comment that this is the best we could do realistically.
There was a problem hiding this comment.
We already have a public InvalidateCache, so that should be sufficient. It’s sort of an API break, but, meh.
|
I do not see anything obviously wrong here. I had the very similar patch ready, but then replaced it with |
7e21306 to
99fd74f
Compare
|
Packit jobs failed. @containers/packit-build please check. |
3 similar comments
|
Packit jobs failed. @containers/packit-build please check. |
|
Packit jobs failed. @containers/packit-build please check. |
|
Packit jobs failed. @containers/packit-build please check. |
|
Podman PR containers/podman#28552 |
containers/container-libs#773 Signed-off-by: Paul Holzinger <pholzing@redhat.com>
containers/container-libs#773 Signed-off-by: Paul Holzinger <pholzing@redhat.com>
mtrmac
left a comment
There was a problem hiding this comment.
A full review now. ACK overall, just nits.
| } | ||
|
|
||
| if conf.EnvironmentName != "" && !conf.DoNotLoadDropInFiles { | ||
| if conf.EnvironmentName != "" && !conf.DoNotLoadDropInFiles && conf.CustomConfigFileDropInDirectory == "" { |
There was a problem hiding this comment.
I agree that the most important part is that a caller can opt out of the environment variables in a documented way.
I’m fine with leaving it as is, but I’ll leave the thread marked unresolved to allow an opportunity for other opinions/views.
| shouldLoadDropIns = false | ||
| } else { | ||
| // default search paths | ||
| mainFiles = append(mainFiles, userConfig, overrideConfig, defaultConfig) |
There was a problem hiding this comment.
Something somewhere should filter out the "" entries; if not in this subpackage, then at the latest in ConfigurationSourceDescription.
Absolutely non-blocking: Doing it here would also allow centralizing the filepath.Join(…, configFileName) code. (Well, getDropInPaths would need to be updated to add configFileName +"." + dropInSuffix, but I think that would simplify the handling of specialName.)
Deprecate V1RegistriesConf and no longer accept the V1 syntax as part of the config file. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
These should mirror the current registries.conf behavior for SystemRegistriesConfPath and SystemRegistriesConfDirPath so I can port this over to the new package. CustomConfigFilePath sets the single config path to the main file which we parse first, we must error if that path does not exists. Also if set we still must read normal drop ins from the default locations to keep the current registries.conf behavior. CustomConfigFileDropInDirectory sets the directory from which we read the drop in files instead of the default locations. If that path does not exists no drop in will be parsed. In addition these options have higher priority then the environment variables as they are often used for cli options which should be more important then the env. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
We need an API to get a list of all search paths to display it as part of error messages for registries.conf. This will also be needed to get the containers.conf module directories for the shell completion logic. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Rewrite registries.conf parsing to use our new configfile package. https://github.com/containers/podman/blob/34a4633d5fd4a502cef289b4f3f449535a7e1067/contrib/design-docs/config-file-parsing.md Signed-off-by: Paul Holzinger <pholzing@redhat.com>
/dev/null is not a valid directory, the new reworked logic correctly fails wehn given a non directory path while the old just ignored it. As such make sure we pass an valid empty directory here. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Update the docs for the new registries.conf parsing behavior. Remove the old containers-registries.conf.d.5 man page and just link to the main one to dedup some content. The install-docs Makefile target should install this "link" just fine as is. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
mtrmac
left a comment
There was a problem hiding this comment.
Still #773 (comment) I think, not blocking — users are likely to only hit this once.
Feel free to merge as is, or update+merge without another review.
Ah I though I dropped the commit in a rebase, I must have forgotten. Should be a quick fix up. |
The config file name is registries.conf not registry.conf. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
We already respect the CONTAINERS_REGISTRIES_CONF in the actual file reading code so this function should not be used anymore. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The CONTAINERS_REGISTRIES_CONF env is already read by the config parser, support for REGISTRIES_CONFIG_PATH is removed. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The file name is registries.conf. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
containers/container-libs#773 Signed-off-by: Paul Holzinger <pholzing@redhat.com>
containers/container-libs#773 Signed-off-by: Paul Holzinger <pholzing@redhat.com>
|
@Luap99 please merge whenever the consumers are ready. |
Podman 6 is going to drop support for reading the old v1 format in containers/container-libs#773 The v2 format is supported for a long time already so this can just be switched and still works with older versions. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
see commits