libct: close the mount source fd ASAP!#5177
Conversation
There was a problem hiding this comment.
Pull request overview
Close mount source file descriptors as soon as they are no longer needed during rootfs setup, reducing peak FD usage in runc (especially under high mount counts / concurrency).
Changes:
- Refactors mount source-fd acquisition logic into a new
initMountEntryhelper. - Moves lifetime management of the mount source FD into
mountToRootfsto close it immediately after each mount is processed.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
rata
left a comment
There was a problem hiding this comment.
Just curious, are you seeing a lot fds and is causing issues for you?
0d45db9 to
7244287
Compare
This can fail under a restrictive |
|
Guess you'd want 1.5 backport @lifubang ? |
Yes, it looks like a small bug. |
rata
left a comment
There was a problem hiding this comment.
Thanks! Let's tune the error handling and simplify a little bit the code
Signed-off-by: lifubang <lifubang@acmcoder.com>
1c3f36f to
1334297
Compare
kolyshkin
left a comment
There was a problem hiding this comment.
For review purposes, it would be nice to split the second commit into two
- Factor out setupAndMountToRootfs.
- Implement the fix.
Otherwise it is harder to review what exactly are you changing since you also move the code around.
Still LGTM though
|
Yeah I agree @kolyshkin, it took me a couple of read-throughs to see what was being changed. |
This commit factors out setupAndMountToRootfs without changing any logic. Use "Hide whitespace changes" during review to focus on the actual changes. The refactor ensures the mount source file descriptor is closed via defer in each loop iteration, reducing the total number of open FDs in runc. This helps avoid hitting the file descriptor limit under high concurrency or when handling many mounts. Signed-off-by: lifubang <lifubang@acmcoder.com>
Signed-off-by: lifubang <lifubang@acmcoder.com>
1334297 to
7fdab1c
Compare
In fact, no second fix is needed. As @rata suggested, factoring out You can use “Hide whitespace changes” to make the review easier. |
Thank you for clarification! Still LGTM ) |
|
@lifubang Do we want 1.4 backport for this one? Seems like a bug to me and 1.4 is to be supported for 6+ months. |
| update_config '.process.rlimits = [{ | ||
| "type": "RLIMIT_NOFILE", | ||
| "soft": 20, | ||
| "hard": 20 |
There was a problem hiding this comment.
The test is using exactly 20, right?
I think there are many things that can use fds and this might be flaky or failing in the future. But if this is important for your environment, I'm fine having it. If it causes CI issues in the future, we can decide then what to do
There was a problem hiding this comment.
The test is using exactly 20, right?
Yes, the test explicitly adds 20 mounts. Before this patch, this would have caused a failure due to the limit.
Since runc theoretically supports an RLIMIT_NOFILE value of 1, this test case is deterministic and should not be flaky in the future.
| destname="${2:-}" | ||
| setup_idmap_single_mount 0:100000:65536 0:100000:65536 "$mountname" "$destname" | ||
| } |
There was a problem hiding this comment.
Why are you changing this? It's not really need it in the test, right?
There was a problem hiding this comment.
This is necessary because I need to mount to more than three target directories within the container.
| } | ||
|
|
||
| // setupAndMountToRootfs sets up the mount for a single mount point and mounts it to the rootfs. | ||
| func setupAndMountToRootfs(pipe *syncSocket, config *configs.Config, mountConfig *mountConfig, m *configs.Mount) error { |
There was a problem hiding this comment.
nit: I think something like doMount or applyMount would be a little less specific (I think setupAndMountToRootfs is too descriptive without telling you what stage in the process it is) but I don't really like any of those names to be honest... 🤔
This PR contains the following updates: | Package | Type | Update | Change | OpenSSF | |---|---|---|---|---| | [github.com/opencontainers/runc](https://github.com/opencontainers/runc) | require | patch | `v1.4.1` → `v1.4.2` | [](https://securityscorecards.dev/viewer/?uri=github.com/opencontainers/runc) | --- >⚠️ **Warning** > > Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/23) for more information. --- ### Release Notes <details> <summary>opencontainers/runc (github.com/opencontainers/runc)</summary> ### [`v1.4.2`](https://github.com/opencontainers/runc/releases/tag/v1.4.2): runc v1.4.2 -- "Я — Земля! Я своих провожаю питомцев" [Compare Source](opencontainers/runc@v1.4.1...v1.4.2) This is the second patch release of the 1.4.z release series of runc. ##### Fixed - A regression in runc v1.3.0 which can result in a stuck `runc exec` or `runc run` when the container process runs for a short time. ([#​5208](opencontainers/runc#5208), [#​5210](opencontainers/runc#5210), [#​5216](opencontainers/runc#5216)) - Mount sources that need to be open on the host are now closed earlier during container start, reducing the total amount of used file descriptors and helping to avoid hitting the open files limit when handling many such mounts. ([#​5177](opencontainers/runc#5177), [#​5201](opencontainers/runc#5201)) ##### Static Linking Notices The `runc` binary distributed with this release are *statically linked* with the following [GNU LGPL-2.1][lgpl-2.1] licensed libraries, with `runc` acting as a "work that uses the Library": [lgpl-2.1]: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html - [libseccomp](https://github.com/seccomp/libseccomp) The versions of these libraries were not modified from their upstream versions, but in order to comply with the LGPL-2.1 (§6(a)), we have attached the complete source code for those libraries which (when combined with the attached runc source code) may be used to exercise your rights under the LGPL-2.1. However we strongly suggest that you make use of your distribution's packages or download them from the authoritative upstream sources, especially since these libraries are related to the security of your containers. *** Thanks to the following contributors for making this release possible: - Ayato Tokubi <atokubi@redhat.com> - Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp> - Aleksa Sarai <cyphar@cyphar.com> - Kir Kolyshkin <kolyshkin@gmail.com> - Li Fubang <lifubang@acmcoder.com> - Rodrigo Campos Catelin <rodrigo@amutable.com> Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At 12:00 AM through 04:59 AM and 10:00 PM through 11:59 PM, Monday through Friday ( * 0-4,22-23 * * 1-5 ), Only on Sunday and Saturday ( * * * * 0,6 ) (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My44Ni4xIiwidXBkYXRlZEluVmVyIjoiNDMuODYuMSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiS2luZC9EZXBlbmRlbmNpZXMiXX0=--> Reviewed-on: https://altlinux.space/stapler/stplr/pulls/387 Co-authored-by: Renovate Bot <stapler-helper-bot@noreply.altlinux.space> Co-committed-by: Renovate Bot <stapler-helper-bot@noreply.altlinux.space>
This commit factors out setupAndMountToRootfs without changing any
logic. Use "Hide whitespace changes" during review to focus on the
actual changes.
The refactor ensures the mount source file descriptor is closed via
defer in each loop iteration, reducing the total number of open FDs
in runc. This helps avoid hitting the file descriptor limit under
high concurrency or when handling many mounts.