The kernel CPU usage is not correct

[echarles]

The host CPU is correctly show, but the kernel CPU is not. This is also mentioned in jupyter-server/jupyter-resource-usage#107

Based on a few experiments with psutil, it looks like the issue is in psutil or how we use psutil.

>>> metric_value = getattr(all_processes[0], 'memory_info')()
>>> getattr(metric_value, 'rss')
11075584
>>> metric_value
pmem(rss=11075584, vms=4399271936, pfaults=2881, pageins=1)
>>> metric_value = getattr(all_processes[0], 'cpu_percent')()
>>> metric_value
0.0
>>> dir(all_processes[0])
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_create_time', '_exe', '_exitcode', '_gone', '_hash', '_ident', '_init', '_last_proc_cpu_times', '_last_sys_cpu_times', '_lock', '_name', '_pid', '_pid_reused', '_ppid', '_proc', '_send_signal', 'as_dict', 'children', 'cmdline', 'connections', 'cpu_percent', 'cpu_times', 'create_time', 'cwd', 'environ', 'exe', 'gids', 'is_running', 'kill', 'memory_full_info', 'memory_info', 'memory_info_ex', 'memory_percent', 'name', 'nice', 'num_ctx_switches', 'num_fds', 'num_threads', 'oneshot', 'open_files', 'parent', 'parents', 'pid', 'ppid', 'resume', 'send_signal', 'status', 'suspend', 'terminal', 'terminate', 'threads', 'uids', 'username', 'wait']
>>> metric_value = getattr(all_processes[0], 'cpu_percent')()
>>> metric_value
0.0
>>> 

[mlucool]

CPU is relative to the last call:

current_process = psutil.Process()
current_process.cpu_percent()
y = 0
for x in range(1000000):
    y += x
current_process.cpu_percent()

See this and this.

[echarles]

ipython/ipykernel#937 is opened to fix this issue. The kernel CPU returns a correct value (e.g. 99%)

[mlucool]

Great! Can we add a % after the 99.9. We should also add this for the Percent in the host virtual memory. Percent may not be clear as to what it means - I think we mean percent used?

Also, the HOST CPU section maybe should show cpu_count as the percentage I believe is spread across all cores so it's a bit hard to understand 8.8 percent for the host and 99.9 for the process. I am not sure if we want logical or not here. I suspect the default of logical=True is what we mean.

Can you confirm if this data is only asked for when the widget is visible on the page? I.e. if this causes issues for some reason, the user can just hide the widget and it won't request anything.

[echarles]

Great! Can we add a % after the 99.9.

Sure, will do.

We should also add this for the Percent in the host virtual memory. Percent may not be clear as to what it means - I think we mean percent used?

Percent used as label makes more sense. We can also add % after the number.

Also, the HOST CPU section maybe should show cpu_count as the percentage I believe is spread across all cores so it's a bit hard to understand 8.8 percent for the host and 99.9 for the process.

Makes me think my laptop has 16 cores which sounds inline with the shown figures :) The title could be Host CPU (all cores)

I am not sure if we want logical or not here. I suspect the default of logical=True is what we mean.

Sorry, I don't understand. Is Logical an option in a psutil method?

Can you confirm if this data is only asked for when the widget is visible on the page? I.e. if this causes issues for some reason, the user can just hide the widget and it won't request anything.

If the widget is hidden, the data fetching will still occur. Actually it is even less sub-optimal. All kernels are fetched on a scheduled basis. If this is needed, I can think on how to only fetch the selected notebook notebook/kernel, and not fetch if the panel is hidden.

[mlucool]

Sorry, I don't understand. Is Logical an option in a psutil method?

In the count, yes - https://psutil.readthedocs.io/en/latest/#psutil.cpu_count

If this is needed, I can think on how to only fetch the selected notebook notebook/kernel, and not fetch if the panel is hidden.

This sounds much safer

[echarles]

This depends on ipython/ipykernel#948

[echarles]

Closing as fixed by ipython/ipykernel#948