Memory restricted containers on Linux

[Sarke]

Both LXC and Docker containers seem to be reporting host system memory instead of available memory. See:

timescale/timescaledb-tune#58

[M0rdecay]

Same issue

[pbnjay]

This package uses standard linux system calls, so if LXC and Docker are reporting unexpected values it isn't really something in scope here unfortunately. It sounds like there's a discrepency between the values in /proc/meminfo and Sysinfo_t is that correct?

[M0rdecay]

Yes, looks like sysinfo_t returns host parameters, it doesn't count cgroup mem limit.

Upd. My bad, cgroup limit should be available in sys/fs/cgroup/memory/memory.limit_in_bytes

[M0rdecay]

Well, this works for me

func sysTotalMemory() uint64 {
	in := &syscall.Sysinfo_t{}
	err := syscall.Sysinfo(in)
	if err != nil {
		return 0
	}
	// If this is a 32-bit system, then these fields are
	// uint32 instead of uint64.
	// So we always convert to uint64 to match signature.
	memTotal := uint64(in.Totalram) * uint64(in.Unit)

	// cgroup v1 detection
	if limitRaw, err := os.ReadFile("/sys/fs/cgroup/memory/memory.limit_in_bytes"); err == nil {
		if limit, err := strconv.ParseUint(strings.TrimSuffix(string(limitRaw), "\n"), 10, 64); err == nil {
			if limit < memTotal {
				return limit
			}
		}
	}

	// cgroup v2 detection
	if limitRaw, err := os.ReadFile("/sys/fs/cgroup/memory.max"); err == nil {
		if limit, err := strconv.ParseUint(strings.TrimSuffix(string(limitRaw), "\n"), 10, 64); err == nil {
			if limit < memTotal {
				return limit
			}
		}
	}

	return memTotal
}

I thnk, free memory can be calculated as memory.limit_in_bytes - memory.usage_in_bytes or memory.max - memory.current.

[pbnjay]

For cgroup v2 it looks like we'll need to get the process relative cgroup path from /proc/self/cgroup but this approach should work.

I will want to make this secondary check consistent across platforms, so I'll have to test equivalent code for BSD limits and Windows as well.

[Sarke]

If I log into a LXC container and run free or cat /proc/meminfo or top, it will report the 8GB that it is allocated, and not the 128GB of the host. I am surprised it is even aware if how much the host has.

[Sarke]

Actually, it seems it's only inside the Docker container that it is the issue.

Host:

❯ cat /proc/meminfo | grep MemTotal
MemTotal:       131920188 kB

LXC:

❯ cat /proc/meminfo | grep MemTotal                 
MemTotal:        8388608 kB

Docker on host:

❯ docker run --rm --memory=2g ubuntu sh -c 'cat /proc/meminfo | grep MemTotal'
MemTotal:       131920188 kB

Docker inside LXC (this one surprised me the most):

❯ docker run --rm --memory=2g ubuntu sh -c 'cat /proc/meminfo | grep MemTotal'
MemTotal:       131920188 kB