Explain the SIZE column in "docker ps -s" and what "virtual" keyword means

[Niceplace]

File: engine/reference/commandline/ps.md

Under the SIZE column of the outpout of "docker ps -s". Is it possible to explain what the "virtual" keyword means ?

Example : for a container I am runing I can see 2.045 MB (virtual 540.5 MB)
Does this mean that the container takes 2.045 MB on disk, or 540.5 MB ?

[mdlinville]

We may be able to explain it. This is not a Docker-specific concept. For now, there is a great explanation here: http://serverfault.com/questions/48582/how-is-memory-usage-reported-in-linux

[itsafire]

I'm confused. I understand the concept of virtual memory but @mstanleyjones are you sure the -s switch is about virtual memory ? Why is it reporting reporting a virtual size for stopped containers then ?

I understand the -s switch is about file sizes, right ?

[mdlinville]

@thaJeztah can you help here? I may be missing something.

[itsafire]

As I see it, the -s options outputs the size of additonal layers on top of the given image the container runs. The virtual size of all image layers plus the first value. Volumes are not counted. I see this behaviour with aufs on linux and on docker on mac. Right ?

[thaJeztah]

The "size" and "virtual size" describe the amount of disk space used by a container. Let me try and explain:

When starting a container, the image that the container is started from is mounted read-only. On top of that, a writable layer is mounted, in which any changes made to the container are written.

The read-only layers of an image can be shared between any container that is started from the same image, whereas the "writable" layer is unique per container (because: you don't want changes made in container "a" to appear in container "b" 😄)

Back to the docker ps -s output;

• The "size" information shows the amount of data (on disk) that is used for the writable layer of each container
• The "virtual size" is the total amount of disk-space used for the read-only image data used by the container and the writable layer.

The reason it's named "virtual size", is that (as described earlier), the disk space for the read-only layer(s) can be shared between containers, so only take up disk space once (perhaps a different name ("shared" size?) would have been better in hindsight, but naming is hard 😄). edit: virtual actually shows the combined size of the readonly layer (the image), and the writable layer of the container.

In the example below, I started 10 nginx containers;

CONTAINER ID        IMAGE                      COMMAND                  CREATED              STATUS              PORTS               NAMES                                         SIZE
6ca0cef8db8d        nginx                      "nginx -g 'daemon ..."   55 seconds ago       Up 54 seconds       80/tcp, 443/tcp     pedantic_mahavira                             2B (virtual 183MB)
0b75532c34d2        nginx                      "nginx -g 'daemon ..."   56 seconds ago       Up 55 seconds       80/tcp, 443/tcp     hardcore_haibt                                2B (virtual 183MB)
a86d8818795d        nginx                      "nginx -g 'daemon ..."   56 seconds ago       Up 55 seconds       80/tcp, 443/tcp     happy_pasteur                                 2B (virtual 183MB)
3e9151f0332d        nginx                      "nginx -g 'daemon ..."   57 seconds ago       Up 56 seconds       80/tcp, 443/tcp     pensive_khorana                               2B (virtual 183MB)
429c418114a1        nginx                      "nginx -g 'daemon ..."   57 seconds ago       Up 56 seconds       80/tcp, 443/tcp     nostalgic_bose                                2B (virtual 183MB)
01f6a0818b32        nginx                      "nginx -g 'daemon ..."   58 seconds ago       Up 57 seconds       80/tcp, 443/tcp     zen_davinci                                   2B (virtual 183MB)
9ed93ee43912        nginx                      "nginx -g 'daemon ..."   58 seconds ago       Up 58 seconds       80/tcp, 443/tcp     unruffled_ptolemy                             2B (virtual 183MB)
c7c17bf50744        nginx                      "nginx -g 'daemon ..."   59 seconds ago       Up 58 seconds       80/tcp, 443/tcp     zen_khorana                                   2B (virtual 183MB)
a7b0502d95ce        nginx                      "nginx -g 'daemon ..."   About a minute ago   Up About a minute   80/tcp, 443/tcp     gallant_leavitt                               2B (virtual 183MB)
8b9e9ee6c4e8        nginx                      "nginx -g 'daemon ..."   About a minute ago   Up About a minute   80/tcp, 443/tcp     clever_pasteur                                2B (virtual 183MB)

All these containers use the same image, so the "Virtual size" (183MB in the example) is used only once, irregardless of how many containers are started from the same image - I can start 1 container or a thousand; no extra disk space is used. The "Size" (2B in the example) is unique per container though, so the total space used on disk is:

183MB + 10 * 2B

Be aware that the size shown does not include all disk space used for a container. Things that are not included currently are;

• disk space used for log-files (if you use the json-file logging driver) - which can be quite a bit if your container generates a lot of logs, and log-rotation (max-file / max-size logging options) is not configured
• volumes used by the container
• disk space used for the container's configuration files (hostconfig.json, config.v2.json, hosts, hostname, resolv.conf) - although these files are small
• memory written to disk (if swapping is enabled)
• checkpoints (if you're using the experimental checkpoint/restore feature)

[aryamsft]

Is there anyway to determine the total amount of disk space used by each container including log files, volumes, container config files, memory written to disk and checkpoints?

Can this be done via using docker restful API's? See link here:
https://docs.docker.com/engine/api/v1.37/#operation/ContainerStats

[yonglam]

@thaJeztah I have two containers created from the same image, but why their virtual size is different?
One is 'size 185.9 GB (virtual 196.7 GB)', the other is 'size 74.83 GB (virtual 85.6 GB)'.

[thaJeztah]

@yonglam "same image", but also same version of the image? Were they started at different points in time? For example;

(using docker image tag just for illustration);

myimage:latest is created, and a container is started with it;

docker image tag busybox myimage:latest 
docker run -dit --name container1 myimag

docker ps -s

CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS               NAMES               SIZE
7767a6e5b649        myimage             "sh"                     About a minute ago   Up About a minute                       container1          0B (virtual 1.15MB)

myimage is updated with a new version, and a second container is started

docker image tag ubuntu myimage:latest
docker run -dit --name container2 myimage

docker ps -s 

CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS               NAMES               SIZE
74f0e474467c        myimage             "/bin/bash"              About a minute ago   Up About a minute                       container2          0B (virtual 79.6MB)

Both were stared from "the same image"

CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS               NAMES               SIZE
74f0e474467c        myimage             "/bin/bash"              About a minute ago   Up About a minute                       container2          0B (virtual 79.6MB)
7767a6e5b649        8c811b4aec35        "sh"                     About a minute ago   Up About a minute                       container1          0B (virtual 1.15MB)

[yonglam]

@thaJeztah Thank you for your response. I don't think they are different image versions. Because the image tagged latest was created at 2017-12-04. It is far before I pull the image. I created the first container about 3 months ago, and the second one about 13 days ago. What I did was:

docker pull docker.xx.com:8080/my-jenkins-node-full:latest
docker run -d --net=host --name the_first docker.xx.com:8080/my-jenkins-node-full:latest  
docker run -d --net=host --name the_second docker.xx.com:8080/my-jenkins-node-full:latest 

the docker version is 1.10.3 (linux/amd64)