Skip to content
This repository was archived by the owner on Oct 13, 2023. It is now read-only.

[18.03] ExportContainer: do not panic#468

Merged
andrewhsu merged 4 commits into
docker-archive:18.03from
thaJeztah:18.03-do-not-panic
Apr 18, 2018
Merged

[18.03] ExportContainer: do not panic#468
andrewhsu merged 4 commits into
docker-archive:18.03from
thaJeztah:18.03-do-not-panic

Conversation

@thaJeztah
Copy link
Copy Markdown
Member

@thaJeztah thaJeztah commented Mar 14, 2018

Cherry-picks for 18.03 of

fixes moby/moby#36561
fixes docker/for-mac#2679

no conflicts

@thaJeztah thaJeztah added this to the 18.03.0 milestone Mar 14, 2018
@thaJeztah
Copy link
Copy Markdown
Member Author

ping @kolyshkin @tonistiigi PTAL

@tonistiigi
Copy link
Copy Markdown
Contributor

LGTM

1 similar comment
@kolyshkin
Copy link
Copy Markdown
Contributor

LGTM

@thaJeztah
Copy link
Copy Markdown
Member Author

thaJeztah commented Mar 16, 2018

We need to also add:

@thaJeztah
Copy link
Copy Markdown
Member Author

Added moby/moby#36610

@thaJeztah
Copy link
Copy Markdown
Member Author

thaJeztah commented Mar 16, 2018

Added moby/moby#36459 (which moves tests to the API), and moby/moby#36606 (adding an integration test)

This should be ready now 👍

@thaJeztah
Copy link
Copy Markdown
Member Author

hmf, more changes needed in the integration-test suite;

11:31:34 integration/container/export_test.go:31:17: undefined: "github.com/docker/docker/integration/internal/container".IsStopped
11:31:34 integration/container/export_test.go:63:20: testEnv.IsRemoteDaemon undefined (type *environment.Execution has no field or method IsRemoteDaemon)
11:31:35 Makefile-docker-ce:57: recipe for target 'test-integration-cli' failed

@thaJeztah thaJeztah force-pushed the 18.03-do-not-panic branch from 421e976 to c692d7c Compare March 16, 2018 13:48
@thaJeztah
Copy link
Copy Markdown
Member Author

Added moby/moby#36393 "Move containerIsStopped/containerIsInState to integration/internal/container"

Copy link
Copy Markdown
Contributor

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🐯

@thaJeztah
Copy link
Copy Markdown
Member Author

thaJeztah commented Mar 16, 2018

We're not there yet 😞 (sorry for the noise);

4:06:12 integration/container/export_test.go:63:20: testEnv.IsRemoteDaemon undefined (type *environment.Execution has no field or method IsRemoteDaemon)
14:06:12 Makefile-docker-ce:57: recipe for target 'test-integration-cli' failed

Adding:

@thaJeztah thaJeztah force-pushed the 18.03-do-not-panic branch from c692d7c to c243b57 Compare March 16, 2018 14:36
@kolyshkin
Copy link
Copy Markdown
Contributor

integration/container/export_test.go:63:20: testEnv.IsRemoteDaemon undefined (type *environment.Execution has no field or method IsRemoteDaemon)

One possible fix for this (other than the backporting) is

-skip.If(t, testEnv.IsRemoteDaemon())
+skip.If(t, !testEnv.IsLocalDaemon())

@thaJeztah
Copy link
Copy Markdown
Member Author

Ah, dang, did I skip one?

@thaJeztah
Copy link
Copy Markdown
Member Author

It's expecting this one;

14:56:30 # github.com/docker/docker/integration/container
14:56:30 integration/container/remove_test.go:41:66: undefined: "github.com/docker/docker/integration/internal/container".WithBind
14:56:30 integration/container/remove_test.go:64:66: undefined: "github.com/docker/docker/integration/internal/container".WithVolume

@thaJeztah
Copy link
Copy Markdown
Member Author

Looks like this is the one we'd need for that moby/moby@d896f87 (moby/moby#36292)

@thaJeztah thaJeztah force-pushed the 18.03-do-not-panic branch from c243b57 to 788f460 Compare March 17, 2018 01:03
@thaJeztah
Copy link
Copy Markdown
Member Author

Okay, looks like I have them all now. I reorganised the commits, putting all the test-changes at the start; I can split those to a separate PR if we want to (changes to testing are also being tracked, related to running the tests in the e2e suite)

Windows failure is a flaky test (would be fixed by moby/moby#36571);

04:20:37 ----------------------------------------------------------------------
04:20:37 FAIL: docker_cli_run_test.go:4374: DockerSuite.TestSlowStdinClosing
04:20:37 
04:20:37 docker_cli_run_test.go:4390:
04:20:37     c.Fatal("running container timed out") // cleanup in teardown
04:20:37 ... Error: running container timed out
04:20:37 

Windows also timed out 😞

04:26:15 PASS: docker_cli_volume_test.go:398: DockerSuite.TestVolumeCLIRmForceUsage	0.974s
04:26:15 SKIP: docker_cli_volume_test.go:468: DockerSuite.TestVolumeCliInspectWithVolumeOpts (unmatched requirement DaemonIsLinux)
04:26:19 Build timed out (after 200 minutes). Marking the build as failed.
04:26:19 Set build name.
04:26:19 New build name is 'PR-468'
04:26:19 Build was aborted

PowerPC doesn't look related https://jenkins.dockerproject.org/job/docker-ce-pr-ppc64le/449/execution/node/452/log/

01:39:23 FAIL: check_test.go:366: DockerSwarmSuite.TearDownTest
01:39:23 
01:39:23 unmount of /tmp/docker-execroot/db206cf1eaae9/netns failed: invalid argument
01:39:23 unmount of /tmp/docker-execroot/db206cf1eaae9/netns failed: no such file or directory
01:39:23 check_test.go:371:
01:39:23     d.Stop(c)
01:39:23 daemon/daemon.go:389:
01:39:23     t.Fatalf("Error while stopping the daemon %s : %v", d.id, err)
01:39:23 ... Error: Error while stopping the daemon d1e07298d4d75 : exit status 130
01:39:23 
01:39:23 
01:39:23 ----------------------------------------------------------------------
01:39:23 PANIC: docker_api_swarm_test.go:462: DockerSwarmSuite.TestAPISwarmManagerRestore
01:39:23 
01:39:23 [db206cf1eaae9] waiting for daemon to start
01:39:23 [db206cf1eaae9] daemon started
01:39:23 
01:39:23 [db206cf1eaae9] exiting daemon
01:39:23 [db206cf1eaae9] waiting for daemon to start
01:39:23 [db206cf1eaae9] daemon started
01:39:23 
01:39:23 [d1e07298d4d75] waiting for daemon to start
01:39:23 [d1e07298d4d75] daemon started
01:39:23 
01:39:23 [d1e07298d4d75] exiting daemon
01:39:23 [d1e07298d4d75] waiting for daemon to start
01:39:23 [d1e07298d4d75] daemon started
01:39:23 
01:39:23 [deb5adde9e799] waiting for daemon to start
01:39:23 [deb5adde9e799] daemon started
01:39:23 
01:39:23 [deb5adde9e799] exiting daemon
01:39:23 [deb5adde9e799] waiting for daemon to start
01:39:23 [deb5adde9e799] daemon started
01:39:23 
01:39:23 [deb5adde9e799] waiting for daemon to start
01:39:23 [deb5adde9e799] daemon started
01:39:23 
01:39:23 [db206cf1eaae9] exiting daemon
01:39:23 [d1e07298d4d75] daemon started
01:39:23 Attempt #2: daemon is still running with pid 26249
01:39:23 Attempt #3: daemon is still running with pid 26249
01:39:23 Attempt #4: daemon is still running with pid 26249
01:39:23 [d1e07298d4d75] exiting daemon
01:39:23 ... Panic: Fixture has panicked (see related PANIC)
01:39:23 

@andrewhsu andrewhsu modified the milestones: 18.03.0, 18.03.1 Mar 23, 2018
@thaJeztah
Copy link
Copy Markdown
Member Author

DockerSwarmSuite.TestAPISwarmManagerRestore is being tracked in moby/moby#34993

@thaJeztah thaJeztah changed the title [18.03] ExportContainer: do not panic [WIP] [18.03] ExportContainer: do not panic Apr 17, 2018
kolyshkin and others added 4 commits April 17, 2018 15:09
In case ContainerExport() is called for an unmounted container, it leads
to a daemon panic as container.BaseFS, which is dereferenced here, is
nil.

To fix, do not rely on container.BaseFS; use the one returned from
rwlayer.Mount().

Fixes: 7a7357d ("LCOW: Implemented support for docker cp + build")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 81f6307)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 7a7357d ("LCOW: Implemented support for docker cp + build")
changed `container.BaseFS` from being a string (that could be empty but
can't lead to nil pointer dereference) to containerfs.ContainerFS,
which could be be `nil` and so nil dereference is at least theoretically
possible, which leads to panic (i.e. engine crashes).

Such a panic can be avoided by carefully analysing the source code in all
the places that dereference a variable, to make the variable can't be nil.
Practically, this analisys are impossible as code is constantly
evolving.

Still, we need to avoid panics and crashes. A good way to do so is to
explicitly check that a variable is non-nil, returning an error
otherwise. Even in case such a check looks absolutely redundant,
further changes to the code might make it useful, and having an
extra check is not a big price to pay to avoid a panic.

This commit adds such checks for all the places where it is not obvious
that container.BaseFS is not nil (which in this case means we do not
call daemon.Mount() a few lines earlier).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit d6ea46c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test case checks that a container created before start
of the currently running dockerd can be exported (as reported
in #36561). To satisfy this condition, either a pre-existing
container is required, or a daemon restart after container
creation.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 6e7141c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit 0f5fe3f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah thaJeztah force-pushed the 18.03-do-not-panic branch from 788f460 to a978050 Compare April 17, 2018 22:12
@thaJeztah thaJeztah changed the title [WIP] [18.03] ExportContainer: do not panic [18.03] ExportContainer: do not panic Apr 17, 2018
@thaJeztah
Copy link
Copy Markdown
Member Author

#512 was merged, so this one is rebased and now only contains the changes related to this fix (last four commits)

@thaJeztah
Copy link
Copy Markdown
Member Author

@andrewhsu Windows CI got stuck; can you restart?

Copy link
Copy Markdown
Contributor

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🐸

@andrewhsu andrewhsu merged commit 68acf3c into docker-archive:18.03 Apr 18, 2018
@thaJeztah thaJeztah deleted the 18.03-do-not-panic branch April 18, 2018 22:00
docker-jenkins pushed a commit that referenced this pull request May 4, 2020
[master] deb: use DEBIAN_FRONTEND=noninteractive for all dockerfiles
Upstream-commit: f062a0d
Component: packaging
akrasnov-drv pushed a commit to drivenets/docker-ce that referenced this pull request Apr 23, 2023
[master] deb: use DEBIAN_FRONTEND=noninteractive for all dockerfiles
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants