There was a problem entering the container

[1084077463]

docker version:20.10.6
Use the docker exec command to enter the container:
failed to resize tty, using default size

docker version:

-bash-4.2$ docker version
Client: Docker Engine - Community
 Version:           20.10.6
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        370c289
 Built:             Fri Apr  9 22:45:33 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.6
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       8728dd2
  Built:            Fri Apr  9 22:43:57 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.4
  GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc:
  Version:          1.0.0-rc93
  GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info:

-bash-4.2$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc.)

Server:
 Containers: 9
  Running: 9
  Paused: 0
  Stopped: 0
 Images: 38
 Server Version: 20.10.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-862.9.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.26GiB
 Name: ecs-633a.novalocal
 ID: OSWU:T67Q:7GME:BF3M:OHIH:ZSHT:3VWP:C7TQ:AAOD:JUTL:EG3U:ACVN
 Docker Root Dir: /xx/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors:
  https://xxxxxxx.mirror.aliyuncs.com/
 Live Restore Enabled: false

After restarting the docker, you can enter the container through exec temporarily, but after a period of time or a few days, you can't enter the container using exec and report 'failed to resize TTY, using default size'. Sometimes, even restarting the docker will get stuck and there will be no response。

Is this problem because the kernel version of CentOS is incompatible with the new version of docker or something? How do you think this problem can be solved?

[thaJeztah]

Is this happening with a specific container, or with any container? Is the container fully functional otherwise?

Is this problem because the kernel version of CentOS is incompatible with the new version of docker or something?

I don't directly think this would be a kernel issue (based on the description), but that said, kernel 3.10.0-862.9.1.el7.x86_64 looks to correspond with CentOS 7.5. CentOS uses a "rolling release" model, which means that previous releases become EOL when a new version is released, and older version are not supported. The current CentOS 7 version is 7.9, with kernel 3.10.0-1160, so if possible (and you have a test-environment to test upgrades before deploying to production), I'd definitely recommend upgrading to an actively supported CentOS 7.

[1084077463]

This problem occurs in all containers. Once the problem of unable to enter the container occurs, all containers cannot enter. Only after restarting the docker service can you enter temporarily. Sometimes, even the 'systemctl restart docker' doesn't respond and gets stuck. Finally, I choose to restart the server to try to solve this problem. After restarting, it can work normally for a period of time, but after a week, this problem occurs again.

When installing the docker version, I installed it according to the command 'yum -y install docker-ce docker-ce-cli containerd.io'. I also installed this docker version on other servers. Other servers use CentOS 8.3. This problem does not occur at present. So I'm not sure whether this problem is the docker problem or the server problem.

If this problem is caused by the higher version of docker, can I reduce the version of docker to solve this problem?

[thaJeztah]

Other servers use CentOS 8.3. This problem does not occur at present. So I'm not sure whether this problem is the docker problem or the server problem.

So the problem doesn't occur on the CentOS 8.3 servers? And those are identical otherwise?

With your description that the problem introduces itself after some time; is is possible the machine is running out of resources on something? (e.g. memory pressure, cpu load), and therefore slow to respond?

[1084077463]

At present, the server with the problem is used in production. Moreover, its CPU resources and memory are not overloaded. Several other CentOS 8.3 servers are only used as test servers at present. Therefore, it is uncertain whether this problem will occur in CentOS 8.3, because this server has not been put into production.

[thaJeztah]

Clear! Thanks for the extra details.

I don't have a direct idea at present what can cause this, but let me also include my comment from your original issue in case it helps others with coming to possible ideas; moby/moby#42707 (comment)

This error comes from the docker cli;

cli/cli/command/container/tty.go

Lines 49 to 68 in a32cd16

	func initTtySize(ctx context.Context, cli command.Cli, id string, isExec bool, resizeTtyFunc func(ctx context.Context, cli command.Cli, id string, isExec bool) error) {
	rttyFunc := resizeTtyFunc
	if rttyFunc == nil {
	rttyFunc = resizeTty
	}
	if err := rttyFunc(ctx, cli, id, isExec); err != nil {
	go func() {
	var err error
	for retry := 0; retry < 5; retry++ {
	time.Sleep(10 * time.Millisecond)
	if err = rttyFunc(ctx, cli, id, isExec); err == nil {
	break
	}
	}
	if err != nil {
	fmt.Fprintln(cli.Err(), "failed to resize tty, using default size")
	}
	}()
	}
	}

The cli will resize the terminal inside the VM to match the terminal Window (when starting the exec, and if your terminal window is resized). This is on a "best effort" base, because it's possible that the container is slow to respond, or other reasons. Possibly the CLI should print the actual error it encountered (but that would only be useful for debugging purposes)

[1084077463]

When you have a solution, please reply to me when you can. Thank you！

[thaJeztah]

Will do (but not sure at this moment if it will be easy to triage further). Also keep us posted if you discover more details that could help narrowing down (and if the problem does not occur on CentOS 8)

[thaJeztah]

Let me close this ticket as the resizing issue should be fixed on current versions of docker (23.0 and 24.0), see #3554