Skip to content

Node Exporter confusing 2 partitions #2894

@hermann-osi

Description

@hermann-osi

Host operating system: output of uname -a

Linux amberlyst 5.10.0-26-amd64 #1 SMP Debian 5.10.197-1 (2023-09-29) x86_64 GNU/Linux

node_exporter version: output of node_exporter --version

Docker, latest, here is my configuration :

  node-exporter:
    image: quay.io/prometheus/node-exporter:latest
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
      - /data:/data:ro
      - /data2:/data2:ro
      - /:/data3:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.sysfs=/host/sys'
      - '--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|host|etc|home/ftpuser/video_folder)($$|/)'
    networks:
      - monitor-net
    ports:
      - '127.0.0.1:9100:9100'
    restart: always

node_exporter command line flags

node_exporter log output

Nothing weird in the logs but here are they :

grafana-node-exporter-1  | ts=2024-01-06T22:28:50.572Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)"
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.572Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)"
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.573Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.573Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.573Z caller=filesystem_common.go:94 level=warn collector=filesystem msg="--collector.filesystem.ignored-mount-points is DEPRECATED and will be removed in 2.0.0, use --collector.filesystem.mount-points-exclude"
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.573Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(sys|proc|dev|host|etc|home/ftpuser/video_folder)($|/)
grafana-node-exporter-1  | ts=2024-01-06T22:28:50.574Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$

Are you running node_exporter in Docker?

Yes

What did you do that produced an error?

I added the following request in grafana to obtain the disk usage of / :
1 - (sum(node_filesystem_avail_bytes{mountpoint=~"/"}) / sum(node_filesystem_size_bytes{mountpoint=~"/"}))
1 - (sum(node_filesystem_avail_bytes{device=~"/dev/sdb3"}) / sum(node_filesystem_size_bytes{device=~"/dev/sdb3"}))
Same result with both : 77% of usage shown (exactly the same as /dev/sda1 and exactly the same maximum size)
df -h returns 30% of usage.

What did you expect to see?

30% of usage and 10 time less maximum size.

What did you see instead?

The exact same data as another partition in another disk of 1TB.

I know docker is not advised for node-exporter but it's strange to see that being off. Everything else is working fine, my other 2 partitions are correctly detected with their correct data. It's only the main partition that seem to be confused with /dev/sda1.
Does anyone know what could cause this behaviour ?

Thank you in advance for looking at my post.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions