node_exporter crash: panic: "node_rapl_package-15_joules_total" is not a valid metric name #1808
Description
Host operating system: output of uname -a
Linux xxx 3.10.0-1127.13.1.el7.x86_64 #1 SMP Tue Jun 23 15:46:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
node_exporter version: output of node_exporter --version
node_exporter, version 1.0.1 (branch: HEAD, revision: 3715be6)
build user: root@1f76dbbcfa55
build date: 20200616-12:44:12
go version: go1.14.4
node_exporter command line flags
./node_exporter --log.level=debug
Are you running node_exporter in Docker?
no
What did you do that produced an error?
$ ./node_exporter --log.level=debug
level=info ts=2020-08-06T11:04:41.735Z caller=node_exporter.go:177 msg="Starting node_exporter" version="(version=1.0.1, branch=HEAD, revision=3715be6ae899f2a9b9dbfd9c39f3e09a7bd4559f)"
level=info ts=2020-08-06T11:04:41.735Z caller=node_exporter.go:178 msg="Build context" build_context="(go=go1.14.4, user=root@1f76dbbcfa55, date=20200616-12:44:12)"
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:105 msg="Enabled collectors"
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=arp
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=bcache
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=bonding
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=btrfs
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=conntrack
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=cpu
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=cpufreq
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=diskstats
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=edac
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=entropy
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=filefd
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=filesystem
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=hwmon
level=info ts=2020-08-06T11:04:41.736Z caller=node_exporter.go:112 collector=infiniband
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=ipvs
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=loadavg
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=mdadm
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=meminfo
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=netclass
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=netdev
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=netstat
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=nfs
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=nfsd
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=powersupplyclass
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=pressure
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=rapl
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=schedstat
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=sockstat
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=softnet
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=stat
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=textfile
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=thermal_zone
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=time
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=timex
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=udp_queues
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=uname
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=vmstat
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=xfs
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:112 collector=zfs
level=info ts=2020-08-06T11:04:41.737Z caller=node_exporter.go:191 msg="Listening on" address=:9100
level=info ts=2020-08-06T11:04:41.737Z caller=tls_config.go:170 msg="TLS is disabled and it cannot be enabled on the fly." http2=false
level=debug ts=2020-08-06T11:04:43.211Z caller=node_exporter.go:73 msg="collect query:" filters="unsupported value type"
level=debug ts=2020-08-06T11:04:43.216Z caller=collector.go:165 msg="collector succeeded" name=conntrack duration_seconds=0.000101629
level=debug ts=2020-08-06T11:04:43.216Z caller=collector.go:165 msg="collector succeeded" name=softnet duration_seconds=0.000755879
level=debug ts=2020-08-06T11:04:43.216Z caller=infiniband_linux.go:112 collector=infiniband msg="infiniband statistics not found, skipping"
level=debug ts=2020-08-06T11:04:43.217Z caller=collector.go:159 msg="collector returned no data" name=infiniband duration_seconds=0.000617915 err="collector returned no data"
level=debug ts=2020-08-06T11:04:43.217Z caller=collector.go:165 msg="collector succeeded" name=textfile duration_seconds=0.000112786
level=debug ts=2020-08-06T11:04:43.217Z caller=ipvs_linux.go:117 collector=ipvs msg="ipvs collector metrics are not available for this system"
level=debug ts=2020-08-06T11:04:43.217Z caller=time.go:50 collector=time msg="Return time" now=1.5967118832177792e+09
level=debug ts=2020-08-06T11:04:43.217Z caller=collector.go:159 msg="collector returned no data" name=ipvs duration_seconds=0.000943754 err="collector returned no data"
level=debug ts=2020-08-06T11:04:43.217Z caller=collector.go:165 msg="collector succeeded" name=time duration_seconds=0.000117125
level=debug ts=2020-08-06T11:04:43.217Z caller=collector.go:165 msg="collector succeeded" name=powersupplyclass duration_seconds=0.000384807
level=debug ts=2020-08-06T11:04:43.218Z caller=collector.go:165 msg="collector succeeded" name=sockstat duration_seconds=0.00064038
level=debug ts=2020-08-06T11:04:43.218Z caller=diskstats_linux.go:195 collector=diskstats msg="Ignoring device" device=xvda1
level=debug ts=2020-08-06T11:04:43.219Z caller=diskstats_linux.go:195 collector=diskstats msg="Ignoring device" device=xvda2
level=debug ts=2020-08-06T11:04:43.219Z caller=collector.go:165 msg="collector succeeded" name=cpufreq duration_seconds=0.001087537
level=debug ts=2020-08-06T11:04:43.219Z caller=collector.go:165 msg="collector succeeded" name=diskstats duration_seconds=0.001541529
level=debug ts=2020-08-06T11:04:43.219Z caller=collector.go:165 msg="collector succeeded" name=hwmon duration_seconds=9.0266e-05
level=debug ts=2020-08-06T11:04:43.219Z caller=collector.go:165 msg="collector succeeded" name=uname duration_seconds=2.3819e-05
level=debug ts=2020-08-06T11:04:43.219Z caller=collector.go:165 msg="collector succeeded" name=vmstat duration_seconds=0.001534087
level=debug ts=2020-08-06T11:04:43.220Z caller=collector.go:165 msg="collector succeeded" name=udp_queues duration_seconds=0.002199194
level=debug ts=2020-08-06T11:04:43.224Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys
level=debug ts=2020-08-06T11:04:43.224Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/proc
level=debug ts=2020-08-06T11:04:43.224Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/dev
level=debug ts=2020-08-06T11:04:43.224Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/kernel/security
level=debug ts=2020-08-06T11:04:43.224Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/dev/shm
level=debug ts=2020-08-06T11:04:43.224Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/dev/pts
level=debug ts=2020-08-06T11:04:43.224Z caller=meminfo.go:53 collector=meminfo msg="Set node_mem" memInfo="unsupported value type"
level=debug ts=2020-08-06T11:04:43.219Z caller=collector.go:165 msg="collector succeeded" name=schedstat duration_seconds=0.002420785
level=debug ts=2020-08-06T11:04:43.225Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup
level=debug ts=2020-08-06T11:04:43.225Z caller=zfs_linux.go:50 collector=zfs msg="Cannot open file for reading" path=/proc/spl/kstat/zfs/fm
level=debug ts=2020-08-06T11:04:43.225Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup/systemd
level=debug ts=2020-08-06T11:04:43.225Z caller=zfs.go:71 collector=zfs err="ZFS / ZFS statistics are not available"
level=debug ts=2020-08-06T11:04:43.226Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/pstore
level=debug ts=2020-08-06T11:04:43.219Z caller=nfs_linux.go:101 collector=nfs msg="Not collecting NFS metrics" err="open /proc/net/rpc/nfs: no such file or directory"
level=debug ts=2020-08-06T11:04:43.226Z caller=collector.go:159 msg="collector returned no data" name=nfs duration_seconds=0.006779697 err="collector returned no data"
level=debug ts=2020-08-06T11:04:43.226Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup/memory
level=debug ts=2020-08-06T11:04:43.226Z caller=nfsd_linux.go:67 collector=nfsd msg="Not collecting NFSd metrics" err="open /proc/net/rpc/nfsd: no such file or directory"
level=debug ts=2020-08-06T11:04:43.226Z caller=collector.go:165 msg="collector succeeded" name=filefd duration_seconds=0.006859011
level=debug ts=2020-08-06T11:04:43.226Z caller=collector.go:159 msg="collector returned no data" name=nfsd duration_seconds=0.006870943 err="collector returned no data"
level=debug ts=2020-08-06T11:04:43.226Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup/net_cls,net_prio
level=debug ts=2020-08-06T11:04:43.226Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup/freezer
level=debug ts=2020-08-06T11:04:43.226Z caller=collector.go:165 msg="collector succeeded" name=bcache duration_seconds=0.007072053
level=debug ts=2020-08-06T11:04:43.226Z caller=collector.go:165 msg="collector succeeded" name=mdadm duration_seconds=0.007030342
level=debug ts=2020-08-06T11:04:43.226Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup/pids
level=debug ts=2020-08-06T11:04:43.226Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup/devices
level=debug ts=2020-08-06T11:04:43.226Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup/perf_event
level=debug ts=2020-08-06T11:04:43.226Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup/hugetlb
level=debug ts=2020-08-06T11:04:43.226Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup/cpu,cpuacct
level=debug ts=2020-08-06T11:04:43.225Z caller=pressure_linux.go:88 collector=pressure msg="collecting statistics for resource" resource=cpu
level=debug ts=2020-08-06T11:04:43.228Z caller=collector.go:165 msg="collector succeeded" name=entropy duration_seconds=0.008681555
level=debug ts=2020-08-06T11:04:43.228Z caller=pressure_linux.go:91 collector=pressure msg="pressure information is unavailable, you need a Linux kernel >= 4.20 and/or CONFIG_PSI enabled for your kernel"
level=debug ts=2020-08-06T11:04:43.228Z caller=collector.go:165 msg="collector succeeded" name=pressure duration_seconds=0.008992396
level=debug ts=2020-08-06T11:04:43.225Z caller=collector.go:165 msg="collector succeeded" name=timex duration_seconds=0.006536503
level=debug ts=2020-08-06T11:04:43.228Z caller=collector.go:165 msg="collector succeeded" name=xfs duration_seconds=0.012090668
level=debug ts=2020-08-06T11:04:43.225Z caller=collector.go:165 msg="collector succeeded" name=meminfo duration_seconds=0.007017089
level=debug ts=2020-08-06T11:04:43.226Z caller=zfs_linux.go:50 collector=zfs msg="Cannot open file for reading" path=/proc/spl/kstat/zfs/vdev_mirror_stats
level=debug ts=2020-08-06T11:04:43.227Z caller=bonding_linux.go:63 collector=bonding msg="Not collecting bonding, file does not exist" file=/sys/class/net
level=debug ts=2020-08-06T11:04:43.228Z caller=collector.go:165 msg="collector succeeded" name=netdev duration_seconds=0.008927686
level=debug ts=2020-08-06T11:04:43.228Z caller=collector.go:159 msg="collector returned no data" name=bonding duration_seconds=0.008950221 err="collector returned no data"
level=debug ts=2020-08-06T11:04:43.228Z caller=zfs.go:71 collector=zfs err="ZFS / ZFS statistics are not available"
level=debug ts=2020-08-06T11:04:43.228Z caller=zfs_linux.go:50 collector=zfs msg="Cannot open file for reading" path=/proc/spl/kstat/zfs/abdstats
level=debug ts=2020-08-06T11:04:43.228Z caller=collector.go:165 msg="collector succeeded" name=netstat duration_seconds=0.011077357
level=debug ts=2020-08-06T11:04:43.228Z caller=zfs.go:71 collector=zfs err="ZFS / ZFS statistics are not available"
level=debug ts=2020-08-06T11:04:43.228Z caller=collector.go:165 msg="collector succeeded" name=arp duration_seconds=0.009064494
level=debug ts=2020-08-06T11:04:43.228Z caller=zfs_linux.go:50 collector=zfs msg="Cannot open file for reading" path=/proc/spl/kstat/zfs/dbuf_stats
level=debug ts=2020-08-06T11:04:43.228Z caller=zfs.go:71 collector=zfs err="ZFS / ZFS statistics are not available"
level=debug ts=2020-08-06T11:04:43.226Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup/blkio
level=debug ts=2020-08-06T11:04:43.228Z caller=zfs_linux.go:50 collector=zfs msg="Cannot open file for reading" path=/proc/spl/kstat/zfs/dmu_tx
level=debug ts=2020-08-06T11:04:43.228Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/fs/cgroup/cpuset
level=debug ts=2020-08-06T11:04:43.228Z caller=zfs.go:71 collector=zfs err="ZFS / ZFS statistics are not available"
level=debug ts=2020-08-06T11:04:43.228Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/kernel/config
level=debug ts=2020-08-06T11:04:43.228Z caller=zfs_linux.go:50 collector=zfs msg="Cannot open file for reading" path=/proc/spl/kstat/zfs/xuio_stats
level=debug ts=2020-08-06T11:04:43.228Z caller=loadavg.go:54 collector=loadavg msg="return load" index=0 load=1.59
level=debug ts=2020-08-06T11:04:43.228Z caller=loadavg.go:54 collector=loadavg msg="return load" index=1 load=1.47
level=debug ts=2020-08-06T11:04:43.228Z caller=loadavg.go:54 collector=loadavg msg="return load" index=2 load=1.53
level=debug ts=2020-08-06T11:04:43.228Z caller=collector.go:165 msg="collector succeeded" name=edac duration_seconds=0.009353715
level=debug ts=2020-08-06T11:04:43.228Z caller=collector.go:165 msg="collector succeeded" name=loadavg duration_seconds=0.00934061
level=debug ts=2020-08-06T11:04:43.228Z caller=zfs.go:71 collector=zfs err="ZFS / ZFS statistics are not available"
level=debug ts=2020-08-06T11:04:43.219Z caller=collector.go:165 msg="collector succeeded" name=btrfs duration_seconds=0.000350148
level=debug ts=2020-08-06T11:04:43.228Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/proc/sys/fs/binfmt_misc
level=debug ts=2020-08-06T11:04:43.229Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/dev/mqueue
level=debug ts=2020-08-06T11:04:43.229Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/sys/kernel/debug
level=debug ts=2020-08-06T11:04:43.229Z caller=zfs_linux.go:50 collector=zfs msg="Cannot open file for reading" path=/proc/spl/kstat/zfs/zfetchstats
level=debug ts=2020-08-06T11:04:43.229Z caller=zfs.go:71 collector=zfs err="ZFS / ZFS statistics are not available"
level=debug ts=2020-08-06T11:04:43.229Z caller=zfs_linux.go:50 collector=zfs msg="Cannot open file for reading" path=/proc/spl/kstat/zfs/zil
level=debug ts=2020-08-06T11:04:43.229Z caller=zfs.go:71 collector=zfs err="ZFS / ZFS statistics are not available"
level=debug ts=2020-08-06T11:04:43.229Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/dev/hugepages
level=debug ts=2020-08-06T11:04:43.229Z caller=zfs_linux.go:50 collector=zfs msg="Cannot open file for reading" path=/proc/spl/kstat/zfs/arcstats
level=debug ts=2020-08-06T11:04:43.229Z caller=zfs.go:71 collector=zfs err="ZFS / ZFS statistics are not available"
level=debug ts=2020-08-06T11:04:43.229Z caller=zfs_linux.go:50 collector=zfs msg="Cannot open file for reading" path=/proc/spl/kstat/zfs/dnodestats
level=debug ts=2020-08-06T11:04:43.229Z caller=zfs.go:71 collector=zfs err="ZFS / ZFS statistics are not available"
level=debug ts=2020-08-06T11:04:43.229Z caller=zfs_linux.go:50 collector=zfs msg="Cannot open file for reading" path=/proc/spl/kstat/zfs/vdev_cache_stats
level=debug ts=2020-08-06T11:04:43.229Z caller=zfs.go:71 collector=zfs err="ZFS / ZFS statistics are not available"
level=debug ts=2020-08-06T11:04:43.229Z caller=filesystem_linux.go:57 collector=filesystem msg="Ignoring fs" type=rpc_pipefs
level=debug ts=2020-08-06T11:04:43.229Z caller=filesystem_linux.go:53 collector=filesystem msg="Ignoring mount point" mountpoint=/proc/xen
level=debug ts=2020-08-06T11:04:43.229Z caller=collector.go:165 msg="collector succeeded" name=zfs duration_seconds=0.01000883
level=debug ts=2020-08-06T11:04:43.229Z caller=collector.go:165 msg="collector succeeded" name=filesystem duration_seconds=0.011677991
level=debug ts=2020-08-06T11:04:43.229Z caller=cpu_linux.go:168 collector=cpu msg="CPU is missing core_throttle_count" cpu=/sys/devices/system/cpu/cpu0
level=debug ts=2020-08-06T11:04:43.229Z caller=cpu_linux.go:178 collector=cpu msg="CPU is missing package_throttle_count" cpu=/sys/devices/system/cpu/cpu0
level=debug ts=2020-08-06T11:04:43.230Z caller=cpu_linux.go:168 collector=cpu msg="CPU is missing core_throttle_count" cpu=/sys/devices/system/cpu/cpu1
level=debug ts=2020-08-06T11:04:43.230Z caller=cpu_linux.go:178 collector=cpu msg="CPU is missing package_throttle_count" cpu=/sys/devices/system/cpu/cpu1
level=debug ts=2020-08-06T11:04:43.230Z caller=collector.go:165 msg="collector succeeded" name=stat duration_seconds=0.010905654
level=debug ts=2020-08-06T11:04:43.230Z caller=cpu_linux.go:168 collector=cpu msg="CPU is missing core_throttle_count" cpu=/sys/devices/system/cpu/cpu10
level=debug ts=2020-08-06T11:04:43.230Z caller=cpu_linux.go:178 collector=cpu msg="CPU is missing package_throttle_count" cpu=/sys/devices/system/cpu/cpu10
level=debug ts=2020-08-06T11:04:43.230Z caller=cpu_linux.go:168 collector=cpu msg="CPU is missing core_throttle_count" cpu=/sys/devices/system/cpu/cpu11
level=debug ts=2020-08-06T11:04:43.230Z caller=cpu_linux.go:178 collector=cpu msg="CPU is missing package_throttle_count" cpu=/sys/devices/system/cpu/cpu11
level=debug ts=2020-08-06T11:04:43.231Z caller=cpu_linux.go:168 collector=cpu msg="CPU is missing core_throttle_count" cpu=/sys/devices/system/cpu/cpu12
panic: "node_rapl_package-15_joules_total" is not a valid metric name
goroutine 91 [running]:
github.com/prometheus/client_golang/prometheus.MustNewConstMetric(...)
/app/vendor/github.com/prometheus/client_golang/prometheus/value.go:106
github.com/prometheus/node_exporter/collector.(*raplCollector).Update(0xc000115e10, 0xc000184360, 0x11aa540, 0x0)
/app/collector/rapl_linux.go:69 +0x48d
github.com/prometheus/node_exporter/collector.execute(0xbc5700, 0x4, 0xccb540, 0xc000115e10, 0xc000184360, 0xccade0, 0xc0002981e0)
/app/collector/collector.go:153 +0x84
github.com/prometheus/node_exporter/collector.NodeCollector.Collect.func1(0xc000184360, 0xc0002983f0, 0xccade0, 0xc0002981e0, 0xc000190b40, 0xbc5700, 0x4, 0xccb540, 0xc000115e10)
/app/collector/collector.go:144 +0x6d
created by github.com/prometheus/node_exporter/collector.NodeCollector.Collect
/app/collector/collector.go:143 +0x133