About Metrics “node_nfsd_disk_bytes_read_total” and "node_nfsd_disk_bytes_written_total" type

[aojdong]

Host operating system: output of uname -a

Linux ubuntu-173 4.15.0-29-generic #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

node_exporter version:

prom/node-exporter:v1.2.2

What did you expect to see?

About Metrics “node_nfsd_disk_bytes_read_total” and "node_nfsd_disk_bytes_written_total", their data type are "Counter". Then, the value of this parameter should continue to increase and not decrease before I restart the NFS service.
This value comes from “/proc/net/rpc/nfsd”，run command "cat /proc/net/rpc/nfsd | grep io" to obtain the value. However, in my K8S cluster environment, this value drops. Due to the large number of users in the environment, unfortunately we cannot find out exactly what action is causing the value to drop.

while true;do cat /proc/net/rpc/nfsd | grep io; sleep 10; done
io 1818669107 4011772609
io 1818669107 4168772412
io 1818669107 4247404892
io 1818669107 4247404892
io 1818669107 16437700
io 1818669107 40773018
io 1818669107 69130195
io 1818669107 75535438
...
io 721461246 333671902
io 2721723773 333671902
io 485557602 333671902
io 2543275600 333671902
io 87462768 333671902
io 2024188388 333671902
io 3927507548 333671902
io 1611898452 333671902
io 3551011011 333671902
io 1116179405 333671902
io 2929335665 333671902
io 535310326 333671902

I can assure you that the NFS service has not been restarted.

ps -eo pid,lstart,cmd,etime | grep nfsd
 1817 Tue Dec 21 10:25:47 2021 [nfsd]                       2-06:29:55
 1818 Tue Dec 21 10:25:47 2021 [nfsd]                       2-06:29:55
 1819 Tue Dec 21 10:25:47 2021 [nfsd]                       2-06:29:55
 1820 Tue Dec 21 10:25:47 2021 [nfsd]                       2-06:29:55
 1821 Tue Dec 21 10:25:47 2021 [nfsd]                       2-06:29:55
 1822 Tue Dec 21 10:25:47 2021 [nfsd]                       2-06:29:55
 1823 Tue Dec 21 10:25:47 2021 [nfsd]                       2-06:29:55
 1824 Tue Dec 21 10:25:47 2021 [nfsd]                       2-06:29:55
68759 Wed Dec 29 16:55:42 2021 grep --color=auto nfsd            00:00

Practice has proved that there is a real decline in the value. Is it unreasonable to still consider it a counter type?

Are Other Metrics Required to Monitor the NFS Read and Write Statistics of the Cluster?

[ventifus]

Your numbers indicate that the kernel's metric is a 32-bit unsigned number and it's rolling over at 4294967295. This should be fine: Prometheus's rate() and irate() functions can handle counter rollovers.

[discordianfish]

Exactly, this is exactly why they are defined as counters.