removing patched module left kpatch patch module stuck

[joe-lawrence]

patch.txt

With the attached kpatch, I didn't have hardware suitable to successfully load the aesni_intel driver. A modprobe of that driver did engage kpatch though, and that left the kpatch patch module loaded, but in an unloadable state (as well as dropping a WARN):

% kpatch load kpatch-klp.ko 

[21388.055840] kpatch: delaying patch of unloaded module 'aesni_intel'
[21388.073346] kpatch: loaded patch module 'kpatch_klp'

% modprobe aesni_intel
modprobe: ERROR: could not insert 'aesni_intel': No such device

[21394.026565] kpatch: patching newly loaded module 'aesni_intel'

% lsmod | grep aesni_intel        # no matches

% kpatch unload kpatch-klp
disabling patch module: kpatch_klp
/usr/local/sbin/kpatch: line 210: echo: write error: Invalid argument
kpatch: can't disable kpatch_klp

[21411.922922] kpatch: can't remove ftrace filter at address 0xffffffffa0671ab0
[21411.929970] ------------[ cut here ]------------
[21411.934615] WARNING: at /root/kpatch/kmod/core/core.c:721 kpatch_unlink_object+0x70/0xa0 [kpatch]()
[21411.943653] can't unregister ftrace for address 0xffffffffa0671ab0
[21412.021367]  bnx2 mdio dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kpatch_klp]
[21412.028310] CPU: 2 PID: 2286 Comm: kpatch Tainted: GF    U    IO--------------   3.10.0-200.el7.x86_64 #1
[21412.037868] Hardware name: Dell Inc. PowerEdge R410/0N051F, BIOS 1.11.0 07/20/2012
[21412.045430]  ffff8802f3633da8 00000000cc6b74f7 ffff8802f3633d60 ffffffff81603cd3
[21412.052863]  ffff8802f3633d98 ffffffff8106e231 ffff8802f34a6480 ffff8802f34a6618
[21412.060295]  ffff8802f34a6600 00000000ffffffea 0000000000000008 ffff8802f3633e00
[21412.067728] Call Trace:
[21412.070178]  [<ffffffff81603cd3>] dump_stack+0x19/0x1b
[21412.075317]  [<ffffffff8106e231>] warn_slowpath_common+0x61/0x80
[21412.081321]  [<ffffffff8106e2ac>] warn_slowpath_fmt+0x5c/0x80
[21412.087064]  [<ffffffffa0652f40>] kpatch_unlink_object+0x70/0xa0 [kpatch]
[21412.093848]  [<ffffffffa06537b8>] kpatch_unregister+0x1a8/0x270 [kpatch]
[21412.100544]  [<ffffffffa03970b4>] patch_enabled_store+0x54/0xb0 [kpatch_klp]
[21412.107591]  [<ffffffff812d5fdf>] kobj_attr_store+0xf/0x20
[21412.113074]  [<ffffffff8123c236>] sysfs_write_file+0xc6/0x140
[21412.118820]  [<ffffffff811c61dd>] vfs_write+0xbd/0x1e0
[21412.123954]  [<ffffffff811c6c28>] SyS_write+0x58/0xb0
[21412.129005]  [<ffffffff81613ce9>] system_call_fastpath+0x16/0x1b
[21412.135009] ---[ end trace cc33c1e1d7715c72 ]---

% lsmod | grep kpatch
kpatch_klp             14046  1 
kpatch                 52025  1 kpatch_klp

[flaming-toast]

Ah, good catch. This is because the kpatch core module notifier doesn't take into account GOING modules. But (for reasons that are currently eluding me) this also means that we don't take into account modules who are GOING because module initialization failed...

[euspectre]

By the way, I hit the same issue when I tried to load a patch for kvm_intel module and then tried to load kvm_intel in a VM. Not extremely critical issue, but still.

[joe-lawrence]

This bug can crash the machine if there is a pre or post unpatch callback defined for the problematic target module. See the testing details in PR #816.