From bc4229b6cc3e3aef3d27d0fa211c99f2692345fe Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:35:33 +0800 Subject: [PATCH 01/16] compiler.h: specify correct attribute for .rodata..c_jump_table commit c5b1184decc819756ae549ba54c63b6790c4ddfd upstream. Currently, there is an assembler message when generating kernel/bpf/core.o under CONFIG_OBJTOOL with LoongArch compiler toolchain: Warning: setting incorrect section attributes for .rodata..c_jump_table This is because the section ".rodata..c_jump_table" should be readonly, but there is a "W" (writable) part of the flags: $ readelf -S kernel/bpf/core.o | grep -A 1 "rodata..c" [34] .rodata..c_j[...] PROGBITS 0000000000000000 0000d2e0 0000000000000800 0000000000000000 WA 0 0 8 There is no above issue on x86 due to the generated section flag is only "A" (allocatable). In order to silence the warning on LoongArch, specify the attribute like ".rodata..c_jump_table,\"a\",@progbits #" explicitly, then the section attribute of ".rodata..c_jump_table" must be readonly in the kernel/bpf/core.o file. Before: $ objdump -h kernel/bpf/core.o | grep -A 1 "rodata..c" 21 .rodata..c_jump_table 00000800 0000000000000000 0000000000000000 0000d2e0 2**3 CONTENTS, ALLOC, LOAD, RELOC, DATA After: $ objdump -h kernel/bpf/core.o | grep -A 1 "rodata..c" 21 .rodata..c_jump_table 00000800 0000000000000000 0000000000000000 0000d2e0 2**3 CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA By the way, AFAICT, maybe the root cause is related with the different compiler behavior of various archs, so to some extent this change is a workaround for LoongArch, and also there is no effect for x86 which is the only port supported by objtool before LoongArch with this patch. Link: https://lkml.kernel.org/r/20240924062710.1243-1-yangtiezhu@loongson.cn Signed-off-by: Tiezhu Yang Cc: Josh Poimboeuf Cc: Peter Zijlstra Cc: [6.9+] Signed-off-by: Andrew Morton Signed-off-by: Binbin Zhou --- include/linux/compiler.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/compiler.h b/include/linux/compiler.h index 0517d344baa05..db140f106f3df 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -133,7 +133,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, #define annotate_unreachable() __annotate_unreachable(__COUNTER__) /* Annotate a C jump table to allow objtool to follow the code flow */ -#define __annotate_jump_table __section(".rodata..c_jump_table") +#define __annotate_jump_table __section(".rodata..c_jump_table,\"a\",@progbits #") #else /* !CONFIG_OBJTOOL */ #define annotate_reachable() From c21344a5468c5dddc4d2c0cbc81ed892fc22180c Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:36:32 +0800 Subject: [PATCH 02/16] objtool: Handle frame pointer related instructions commit da5b2ad1c2f18834cb1ce429e2e5a5cf5cbdf21b upstream. After commit a0f7085f6a63 ("LoongArch: Add RANDOMIZE_KSTACK_OFFSET support"), there are three new instructions "addi.d $fp, $sp, 32", "sub.d $sp, $sp, $t0" and "addi.d $sp, $fp, -32" for the secondary stack in do_syscall(), then there is a objtool warning "return with modified stack frame" and no handle_syscall() which is the previous frame of do_syscall() in the call trace when executing the command "echo l > /proc/sysrq-trigger". objdump shows something like this: 0000000000000000 : 0: 02ff8063 addi.d $sp, $sp, -32 4: 29c04076 st.d $fp, $sp, 16 8: 29c02077 st.d $s0, $sp, 8 c: 29c06061 st.d $ra, $sp, 24 10: 02c08076 addi.d $fp, $sp, 32 ... 74: 0011b063 sub.d $sp, $sp, $t0 ... a8: 4c000181 jirl $ra, $t0, 0 ... dc: 02ff82c3 addi.d $sp, $fp, -32 e0: 28c06061 ld.d $ra, $sp, 24 e4: 28c04076 ld.d $fp, $sp, 16 e8: 28c02077 ld.d $s0, $sp, 8 ec: 02c08063 addi.d $sp, $sp, 32 f0: 4c000020 jirl $zero, $ra, 0 The instruction "sub.d $sp, $sp, $t0" changes the stack bottom and the new stack size is a random value, in order to find the return address of do_syscall() which is stored in the original stack frame after executing "jirl $ra, $t0, 0", it should use fp which points to the original stack top. At the beginning, the thought is tended to decode the secondary stack instruction "sub.d $sp, $sp, $t0" and set it as a label, then check this label for the two frame pointer instructions to change the cfa base and cfa offset during the period of secondary stack in update_cfi_state(). This is valid for GCC but invalid for Clang due to there are different secondary stack instructions for ClangBuiltLinux on LoongArch, something like this: 0000000000000000 : ... 88: 00119064 sub.d $a0, $sp, $a0 8c: 00150083 or $sp, $a0, $zero ... Actually, it equals to a single instruction "sub.d $sp, $sp, $a0", but there is no proper condition to check it as a label like GCC, and so the beginning thought is not a good way. Essentially, there are two special frame pointer instructions which are "addi.d $fp, $sp, imm" and "addi.d $sp, $fp, imm", the first one points fp to the original stack top and the second one restores the original stack bottom from fp. Based on the above analysis, in order to avoid adding an arch-specific update_cfi_state(), we just add a member "frame_pointer" in the "struct symbol" as a label to avoid affecting the current normal case, then set it as true only if there is "addi.d $sp, $fp, imm". The last is to check this label for the two frame pointer instructions to change the cfa base and cfa offset in update_cfi_state(). Tested with the following two configs: (1) CONFIG_RANDOMIZE_KSTACK_OFFSET=y && CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT=n (2) CONFIG_RANDOMIZE_KSTACK_OFFSET=y && CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT=y By the way, there is no effect for x86 with this patch, tested on the x86 machine with Fedora 40 system. Cc: stable@vger.kernel.org # 6.9+ Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- tools/objtool/arch/loongarch/decode.c | 11 ++++++++++- tools/objtool/check.c | 23 ++++++++++++++++++++--- tools/objtool/include/objtool/elf.h | 1 + 3 files changed, 31 insertions(+), 4 deletions(-) diff --git a/tools/objtool/arch/loongarch/decode.c b/tools/objtool/arch/loongarch/decode.c index aee479d2191c2..69b66994f2a15 100644 --- a/tools/objtool/arch/loongarch/decode.c +++ b/tools/objtool/arch/loongarch/decode.c @@ -122,7 +122,7 @@ static bool decode_insn_reg2i12_fomat(union loongarch_instruction inst, switch (inst.reg2i12_format.opcode) { case addid_op: if ((inst.reg2i12_format.rd == CFI_SP) || (inst.reg2i12_format.rj == CFI_SP)) { - /* addi.d sp,sp,si12 or addi.d fp,sp,si12 */ + /* addi.d sp,sp,si12 or addi.d fp,sp,si12 or addi.d sp,fp,si12 */ insn->immediate = sign_extend64(inst.reg2i12_format.immediate, 11); ADD_OP(op) { op->src.type = OP_SRC_ADD; @@ -132,6 +132,15 @@ static bool decode_insn_reg2i12_fomat(union loongarch_instruction inst, op->dest.reg = inst.reg2i12_format.rd; } } + if ((inst.reg2i12_format.rd == CFI_SP) && (inst.reg2i12_format.rj == CFI_FP)) { + /* addi.d sp,fp,si12 */ + struct symbol *func = find_func_containing(insn->sec, insn->offset); + + if (!func) + return false; + + func->frame_pointer = true; + } break; case ldd_op: if (inst.reg2i12_format.rj == CFI_SP) { diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 02090727e9b17..6e9dd6d6db37a 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -2975,10 +2975,27 @@ static int update_cfi_state(struct instruction *insn, break; } - if (op->dest.reg == CFI_SP && op->src.reg == CFI_BP) { + if (op->dest.reg == CFI_BP && op->src.reg == CFI_SP && + insn->sym->frame_pointer) { + /* addi.d fp,sp,imm on LoongArch */ + if (cfa->base == CFI_SP && cfa->offset == op->src.offset) { + cfa->base = CFI_BP; + cfa->offset = 0; + } + break; + } - /* lea disp(%rbp), %rsp */ - cfi->stack_size = -(op->src.offset + regs[CFI_BP].offset); + if (op->dest.reg == CFI_SP && op->src.reg == CFI_BP) { + /* addi.d sp,fp,imm on LoongArch */ + if (cfa->base == CFI_BP && cfa->offset == 0) { + if (insn->sym->frame_pointer) { + cfa->base = CFI_SP; + cfa->offset = -op->src.offset; + } + } else { + /* lea disp(%rbp), %rsp */ + cfi->stack_size = -(op->src.offset + regs[CFI_BP].offset); + } break; } diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h index 2b8a69de4db87..d7e815c2fd156 100644 --- a/tools/objtool/include/objtool/elf.h +++ b/tools/objtool/include/objtool/elf.h @@ -68,6 +68,7 @@ struct symbol { u8 warned : 1; u8 embedded_insn : 1; u8 local_label : 1; + u8 frame_pointer : 1; struct list_head pv_target; struct reloc *relocs; }; From 20a0c2129322d13da900e898f297939ee795edf0 Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:37:03 +0800 Subject: [PATCH 03/16] LoongArch: Add ifdefs to fix LSX and LASX related warnings commit 80376323e2b6a4559f86b2b4d864848ac25cb054 upstream. There exist some warnings when building kernel if CONFIG_CPU_HAS_LBT is set but CONFIG_CPU_HAS_LSX and CONFIG_CPU_HAS_LASX are not set. In this case, there are no definitions of _restore_lsx & _restore_lasx in fpu.S, just add some ifdefs to fix these warnings. AS arch/loongarch/kernel/fpu.o arch/loongarch/kernel/fpu.o: warning: objtool: unexpected relocation symbol type in .rela.discard.func_stack_frame_non_standard: 0 arch/loongarch/kernel/fpu.o: warning: objtool: unexpected relocation symbol type in .rela.discard.func_stack_frame_non_standard: 0 Cc: stable@vger.kernel.org # 6.9+ Fixes: cb8a2ef0848c ("LoongArch: Add ORC stack unwinder support") Reported-by: kernel test robot Closes: https://lore.kernel.org/oe-kbuild-all/202408120955.qls5oNQY-lkp@intel.com/ Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- arch/loongarch/kernel/fpu.S | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/loongarch/kernel/fpu.S b/arch/loongarch/kernel/fpu.S index 69a85f2479fba..6ab640101457c 100644 --- a/arch/loongarch/kernel/fpu.S +++ b/arch/loongarch/kernel/fpu.S @@ -530,6 +530,10 @@ SYM_FUNC_END(_restore_lasx_context) #ifdef CONFIG_CPU_HAS_LBT STACK_FRAME_NON_STANDARD _restore_fp +#ifdef CONFIG_CPU_HAS_LSX STACK_FRAME_NON_STANDARD _restore_lsx +#endif +#ifdef CONFIG_CPU_HAS_LASX STACK_FRAME_NON_STANDARD _restore_lasx #endif +#endif From 61f58c140f948396237c4e5429f464a49338cb14 Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:37:28 +0800 Subject: [PATCH 04/16] LoongArch: Enable objtool for Clang commit b8468bd92ae19939d4844899fa05147888732519 upstream. For now, it can enable objtool for Clang, just remove !CC_IS_CLANG for HAVE_OBJTOOL in arch/loongarch/Kconfig. Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- arch/loongarch/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 50a3d95117598..98b0d5838f552 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -135,7 +135,7 @@ config LOONGARCH select HAVE_KVM select HAVE_MOD_ARCH_SPECIFIC select HAVE_NMI - select HAVE_OBJTOOL if AS_HAS_EXPLICIT_RELOCS && AS_HAS_THIN_ADD_SUB && !CC_IS_CLANG + select HAVE_OBJTOOL if AS_HAS_EXPLICIT_RELOCS && AS_HAS_THIN_ADD_SUB select HAVE_PCI select HAVE_PERF_EVENTS select HAVE_PERF_REGS From 190fd7cb3e9c86ab7aba512a15ffdf032d8aa164 Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:37:57 +0800 Subject: [PATCH 05/16] LoongArch: Set AS_HAS_THIN_ADD_SUB as y if AS_IS_LLVM commit a7e0837724562ea8c1d869dd1a5cb1119ef651c3 upstream. When building kernel with "make CC=clang defconfig", LLVM Assembler is used due to LLVM_IAS=0 is not specified, then AS_HAS_THIN_ADD_SUB is not set, thus objtool can not be built after enable it for Clang. config AS_HAS_THIN_ADD_SUB is to check whether -mthin-add-sub option is available to know R_LARCH_{32,64}_PCREL are supported for GNU Assembler, there is no such an option for LLVM Assembler. The minimal version of Clang is 18 for building LoongArch kernel, and Clang >= 17 has already supported R_LARCH_{32,64}_PCREL, that is to say, there is no need to depend on AS_HAS_THIN_ADD_SUB for Clang, so just set AS_HAS_THIN_ADD_SUB as y if AS_IS_LLVM. Fixes: 120dd4118e58 ("LoongArch: Only allow OBJTOOL & ORC unwinder if toolchain supports -mthin-add-sub") Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- arch/loongarch/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 98b0d5838f552..e54e87783b4c3 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -262,7 +262,7 @@ config AS_HAS_FCSR_CLASS def_bool $(as-instr,movfcsr2gr \$t0$(comma)\$fcsr0) config AS_HAS_THIN_ADD_SUB - def_bool $(cc-option,-Wa$(comma)-mthin-add-sub) + def_bool $(cc-option,-Wa$(comma)-mthin-add-sub) || AS_IS_LLVM config AS_HAS_LSX_EXTENSION def_bool $(as-instr,vld \$vr0$(comma)\$a0$(comma)0) From d4dfe70d7fca95e71bb64a9c25c1384f39e7ad2d Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:38:21 +0800 Subject: [PATCH 06/16] objtool: Handle various symbol types of rodata In the relocation section ".rela.rodata" of each .o file compiled with LoongArch toolchain, there are various symbol types such as STT_NOTYPE, STT_OBJECT, STT_FUNC in addition to the usual STT_SECTION, it needs to use reloc symbol offset instead of reloc addend to find the destination instruction in find_jump_table() and add_jump_table(). This is preparation for later patch on LoongArch, there is no effect for the other archs with this patch. Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- tools/objtool/check.c | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 6e9dd6d6db37a..0e264df708765 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -2011,6 +2011,7 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn, unsigned int prev_offset = 0; struct reloc *reloc = table; struct alternative *alt; + unsigned long offset; /* * Each @reloc is a switch table relocation which points to the target @@ -2026,12 +2027,19 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn, if (prev_offset && reloc_offset(reloc) != prev_offset + 8) break; + if (reloc->sym->type == STT_SECTION) { + /* Addend field in the relocation entry associated with the symbol */ + offset = reloc_addend(reloc); + } else { + /* The address of the symbol in the relocation entry */ + offset = reloc->sym->offset; + } + /* Detect function pointers from contiguous objects: */ - if (reloc->sym->sec == pfunc->sec && - reloc_addend(reloc) == pfunc->offset) + if (reloc->sym->sec == pfunc->sec && offset == pfunc->offset) break; - dest_insn = find_insn(file, reloc->sym->sec, reloc_addend(reloc)); + dest_insn = find_insn(file, reloc->sym->sec, offset); if (!dest_insn) break; @@ -2069,6 +2077,7 @@ static struct reloc *find_jump_table(struct objtool_file *file, { struct reloc *table_reloc; struct instruction *dest_insn, *orig_insn = insn; + unsigned long offset; /* * Backward search using the @first_jump_src links, these help avoid @@ -2092,7 +2101,16 @@ static struct reloc *find_jump_table(struct objtool_file *file, table_reloc = arch_find_switch_table(file, insn); if (!table_reloc) continue; - dest_insn = find_insn(file, table_reloc->sym->sec, reloc_addend(table_reloc)); + + if (table_reloc->sym->type == STT_SECTION) { + /* Addend field in the relocation entry associated with the symbol */ + offset = reloc_addend(table_reloc); + } else { + /* The address of the symbol in the relocation entry */ + offset = table_reloc->sym->offset; + } + + dest_insn = find_insn(file, table_reloc->sym->sec, offset); if (!dest_insn || !insn_func(dest_insn) || insn_func(dest_insn)->pfunc != func) continue; From db1750c1eebbdabc1c9e79a2441dcba6b10c8c9a Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:38:45 +0800 Subject: [PATCH 07/16] objtool: Handle special cases of dead end insn There are some "unreachable instruction" objtool warnings when compiling with Clang on LoongArch, this is because the "break" instruction is set as dead end due to its type is INSN_BUG in decode_instructions() at the beginning, and it does not set insn->dead_end of the "break" instruction as false after checking ".rela.discard.reachable" in add_dead_ends(), so the next instruction of "break" is marked as unreachable. Actually, it can find the reachable instruction after parsing the section ".rela.discard.reachable", in some cases, the "break" instruction may not be the first previous instruction with scheduling by Machine Instruction Scheduler of LLVM, it should find more times and then set insn->dead_end of the "break" instruction as false. This is preparation for later patch on LoongArch, there is no effect for the other archs with this patch. Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- tools/objtool/check.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 0e264df708765..1a5aea7879362 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -22,6 +22,10 @@ #include #include +#ifndef EM_LOONGARCH +#define EM_LOONGARCH 258 +#endif + struct alternative { struct alternative *next; struct instruction *insn; @@ -661,6 +665,18 @@ static int add_dead_ends(struct objtool_file *file) } insn->dead_end = false; + + /* Handle the special cases compiled with Clang on LoongArch */ + if (file->elf->ehdr.e_machine == EM_LOONGARCH && + reloc->sym->type == STT_SECTION) { + while (insn && insn_func(insn)) { + insn = prev_insn_same_sym(file, insn); + if (insn && insn->dead_end) { + insn->dead_end = false; + break; + } + } + } } return 0; From 1e875b4af154b311dbfc3db1edea9871e69644c5 Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:39:07 +0800 Subject: [PATCH 08/16] objtool: Handle different entry size of rodata In the most cases, the entry size of rodata is 8 bytes because the relocation type is 64 bit, but when compiling with Clang on LoongArch, there exists 32 bit relocation type, the entry size of rodata should be 4 bytes in this case. This is preparation for later patch on LoongArch, there is no effect for the other archs with this patch. Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- tools/objtool/check.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 1a5aea7879362..915a00328221f 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -26,6 +26,10 @@ #define EM_LOONGARCH 258 #endif +#ifndef R_LARCH_32_PCREL +#define R_LARCH_32_PCREL 99 +#endif + struct alternative { struct alternative *next; struct instruction *insn; @@ -2028,6 +2032,7 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn, struct reloc *reloc = table; struct alternative *alt; unsigned long offset; + unsigned long rodata_entry_size; /* * Each @reloc is a switch table relocation which points to the target @@ -2039,8 +2044,15 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn, if (reloc != table && reloc == next_table) break; + /* Handle the special cases compiled with Clang on LoongArch */ + if (file->elf->ehdr.e_machine == EM_LOONGARCH && + reloc_type(reloc) == R_LARCH_32_PCREL) + rodata_entry_size = 4; + else + rodata_entry_size = 8; + /* Make sure the table entries are consecutive: */ - if (prev_offset && reloc_offset(reloc) != prev_offset + 8) + if (prev_offset && reloc_offset(reloc) != prev_offset + rodata_entry_size) break; if (reloc->sym->type == STT_SECTION) { From 4044f63d88cba1202e0a27c6ce31e19387f430fd Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:39:36 +0800 Subject: [PATCH 09/16] objtool: Handle PC relative relocation type When compiling with Clang on LoongArch, there exists 32 bit PC relative relocation type, it needs to get the offset with "S + A - PC" according to the spec of "ELF for the LoongArch Architecture". This is preparation for later patch on LoongArch, there is no effect for the other archs with this patch. Link: https://github.com/loongson/la-abi-specs/blob/release/laelf.adoc Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- tools/objtool/check.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 915a00328221f..43bc5949b4d26 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -2058,6 +2058,11 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn, if (reloc->sym->type == STT_SECTION) { /* Addend field in the relocation entry associated with the symbol */ offset = reloc_addend(reloc); + /* Handle the special cases compiled with Clang on LoongArch */ + if (file->elf->ehdr.e_machine == EM_LOONGARCH && + reloc_type(reloc) == R_LARCH_32_PCREL) + offset = reloc->sym->offset + reloc_addend(reloc) - + (reloc_offset(reloc) - reloc_offset(table)); } else { /* The address of the symbol in the relocation entry */ offset = reloc->sym->offset; From c2b02f57bd5d6207978bcbcf4d7898320c2fac0f Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:40:11 +0800 Subject: [PATCH 10/16] objtool: Handle unreachable entry of rodata When compiling with Clang on LoongArch, there exists unreachable entry of rodata which points to a position after the function return instruction, this is generated by compiler to fill the non-existent switch case, just skip the entry when parsing the relocation section of rodata. This is preparation for later patch on LoongArch, there is no effect for the other archs with this patch. Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- tools/objtool/check.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 43bc5949b4d26..a57d910ee3e85 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -2076,6 +2076,13 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn, if (!dest_insn) break; + /* Handle the special cases compiled with Clang on LoongArch */ + if (file->elf->ehdr.e_machine == EM_LOONGARCH && reloc->sym->type == STT_SECTION && + (!insn_func(dest_insn) || insn_func(dest_insn)->pfunc != pfunc)) { + prev_offset = reloc_offset(reloc); + continue; + } + /* Make sure the destination is in the same function: */ if (!insn_func(dest_insn) || insn_func(dest_insn)->pfunc != pfunc) break; From 2dc8aee32df383de1f2cb25142b146172c1a4309 Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:40:38 +0800 Subject: [PATCH 11/16] objtool: Handle unsorted table offset of rodata When compiling with Clang on LoongArch, there are unsorted table offset of rodata if there exist many jump tables, it will get the wrong table end and find the wrong jump destination instructions in add_jump_table(), so it needs to check the table size of rodata to break the process when parsing the relocation section of rodata. This is preparation for later patch on LoongArch, there is no effect for the other archs with this patch. Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- tools/objtool/check.c | 7 +++++++ tools/objtool/include/objtool/check.h | 1 + 2 files changed, 8 insertions(+) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index a57d910ee3e85..da74e73cb339e 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -2033,6 +2033,7 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn, struct alternative *alt; unsigned long offset; unsigned long rodata_entry_size; + unsigned long rodata_table_size = insn->table_size; /* * Each @reloc is a switch table relocation which points to the target @@ -2044,6 +2045,12 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn, if (reloc != table && reloc == next_table) break; + /* Handle the special cases compiled with Clang on LoongArch */ + if (file->elf->ehdr.e_machine == EM_LOONGARCH && + reloc->sym->type == STT_SECTION && reloc != table && + reloc_offset(reloc) == reloc_offset(table) + rodata_table_size) + break; + /* Handle the special cases compiled with Clang on LoongArch */ if (file->elf->ehdr.e_machine == EM_LOONGARCH && reloc_type(reloc) == R_LARCH_32_PCREL) diff --git a/tools/objtool/include/objtool/check.h b/tools/objtool/include/objtool/check.h index daa46f1f0965a..11aafcc7b0c7e 100644 --- a/tools/objtool/include/objtool/check.h +++ b/tools/objtool/include/objtool/check.h @@ -46,6 +46,7 @@ struct instruction { struct section *sec; unsigned long offset; unsigned long immediate; + unsigned long table_size; u8 len; u8 prev_len; From bc1a33268fb5363b4344fc90332849da10aff1cd Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:40:59 +0800 Subject: [PATCH 12/16] objtool/LoongArch: Get each table size of rodata When compiling with Clang on LoongArch, there are unsorted table offset of rodata if there exist many jump tables, it will get the wrong table end and find the wrong jump destination instructions in add_jump_table(). Sort the rodata table offset by parsing ".rela.discard.tablejump_annotate" and then get each table size of rodata corresponded with each table jump instruction, it is used to check the table end and will break the process when parsing ".rela.rodata" to avoid getting the wrong jump destination instructions. Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- tools/objtool/arch/loongarch/special.c | 72 ++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/tools/objtool/arch/loongarch/special.c b/tools/objtool/arch/loongarch/special.c index 9bba1e9318e0b..454bd01226a40 100644 --- a/tools/objtool/arch/loongarch/special.c +++ b/tools/objtool/arch/loongarch/special.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-or-later #include +#include bool arch_support_alt_relocation(struct special_alt *special_alt, struct instruction *insn, @@ -8,6 +9,77 @@ bool arch_support_alt_relocation(struct special_alt *special_alt, return false; } +struct table_info { + struct list_head jump_info; + unsigned long insn_offset; + unsigned long rodata_offset; +}; + +static void get_rodata_table_size_by_table_annotate(struct objtool_file *file, + struct instruction *insn) +{ + struct section *rsec; + struct reloc *reloc; + struct list_head table_list; + struct table_info *orig_table; + struct table_info *next_table; + unsigned long tmp_insn_offset; + unsigned long tmp_rodata_offset; + + rsec = find_section_by_name(file->elf, ".rela.discard.tablejump_annotate"); + if (!rsec) + return; + + INIT_LIST_HEAD(&table_list); + + for_each_reloc(rsec, reloc) { + if (reloc->sym->type != STT_SECTION) + return; + + orig_table = malloc(sizeof(struct table_info)); + if (!orig_table) { + WARN("malloc failed"); + return; + } + + orig_table->insn_offset = reloc_addend(reloc); + reloc++; + orig_table->rodata_offset = reloc_addend(reloc); + + list_add_tail(&orig_table->jump_info, &table_list); + + if (reloc_idx(reloc) + 1 == sec_num_entries(rsec)) + break; + } + + list_for_each_entry(orig_table, &table_list, jump_info) { + next_table = list_next_entry(orig_table, jump_info); + list_for_each_entry_from(next_table, &table_list, jump_info) { + if (next_table->rodata_offset < orig_table->rodata_offset) { + tmp_insn_offset = next_table->insn_offset; + tmp_rodata_offset = next_table->rodata_offset; + next_table->insn_offset = orig_table->insn_offset; + next_table->rodata_offset = orig_table->rodata_offset; + orig_table->insn_offset = tmp_insn_offset; + orig_table->rodata_offset = tmp_rodata_offset; + } + } + } + + list_for_each_entry(orig_table, &table_list, jump_info) { + if (insn->offset == orig_table->insn_offset) { + next_table = list_next_entry(orig_table, jump_info); + if (&next_table->jump_info == &table_list) + break; + + while (next_table->rodata_offset == orig_table->rodata_offset) + next_table = list_next_entry(next_table, jump_info); + + insn->table_size = next_table->rodata_offset - orig_table->rodata_offset; + } + } +} + struct reloc *arch_find_switch_table(struct objtool_file *file, struct instruction *insn) { From 01b4d993557dff6e16901a7f82c8d51b0f020a18 Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:41:24 +0800 Subject: [PATCH 13/16] objtool/LoongArch: Add support for switch table The objtool program need to analysis the control flow of each object file generated by compiler toolchain, it needs to know all the locations that a branch instruction may jump into, if a jump table is used, objtool has to correlate the jump instruction with the table. On x86 (which is the only port supported by objtool before LoongArch), there is a relocation type on the jump instruction and directly points to the table. But on LoongArch, the relocation is on another kind of instruction prior to the jump instruction, and also with scheduling it is not very easy to tell the offset of that instruction from the jump instruction. Furthermore, because LoongArch has -fsection-anchors (often enabled at -O1 or above) the relocation may actually points to a section anchor instead of the table itself. The good news is that after continuous analysis and discussion, at last a GCC patch "LoongArch: Add support to annotate tablejump" and a Clang patch "[LoongArch] Add options for annotate tablejump" have been merged into the upstream mainline, the compiler changes make life much easier for switch table support of objtool on LoongArch. By now, there is an additional section ".discard.tablejump_annotate" to store the jump info as pairs of addresses, each pair contains the address of jump instruction and the address of jump table. In order to find switch table, it is easy to parse the relocation section ".rela.discard.tablejump_annotate" to get table_sec and table_offset, the rest process is somehow like x86. Link: https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=0ee028f55640 Link: https://github.com/llvm/llvm-project/commit/4c2c17756739 Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- tools/objtool/arch/loongarch/special.c | 60 +++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-) diff --git a/tools/objtool/arch/loongarch/special.c b/tools/objtool/arch/loongarch/special.c index 454bd01226a40..366517deb35b8 100644 --- a/tools/objtool/arch/loongarch/special.c +++ b/tools/objtool/arch/loongarch/special.c @@ -1,4 +1,5 @@ // SPDX-License-Identifier: GPL-2.0-or-later +#include #include #include @@ -80,8 +81,65 @@ static void get_rodata_table_size_by_table_annotate(struct objtool_file *file, } } +static struct reloc *find_reloc_by_table_annotate(struct objtool_file *file, + struct instruction *insn) +{ + struct section *rsec; + struct reloc *reloc; + unsigned long offset; + + rsec = find_section_by_name(file->elf, ".rela.discard.tablejump_annotate"); + if (!rsec) + return NULL; + + for_each_reloc(rsec, reloc) { + if (reloc->sym->sec->rodata) + continue; + + if (strcmp(insn->sec->name, reloc->sym->sec->name)) + continue; + + if (reloc->sym->type == STT_SECTION) + offset = reloc_addend(reloc); + else + offset = reloc->sym->offset; + + if (insn->offset == offset) { + get_rodata_table_size_by_table_annotate(file, insn); + reloc++; + return reloc; + } + } + + return NULL; +} + struct reloc *arch_find_switch_table(struct objtool_file *file, struct instruction *insn) { - return NULL; + struct reloc *annotate_reloc; + struct reloc *rodata_reloc; + struct section *table_sec; + unsigned long table_offset; + + annotate_reloc = find_reloc_by_table_annotate(file, insn); + if (!annotate_reloc) + return NULL; + + table_sec = annotate_reloc->sym->sec; + if (annotate_reloc->sym->type == STT_SECTION) + table_offset = reloc_addend(annotate_reloc); + else + table_offset = annotate_reloc->sym->offset; + + /* + * Each table entry has a rela associated with it. The rela + * should reference text in the same function as the original + * instruction. + */ + rodata_reloc = find_reloc_by_dest(file->elf, table_sec, table_offset); + if (!rodata_reloc) + return NULL; + + return rodata_reloc; } From abc09e63f4ab013a624cb8d571032cd7145aa3c4 Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:42:03 +0800 Subject: [PATCH 14/16] objtool/LoongArch: Add support for goto table The objtool program need to analysis the control flow of each object file generated by compiler toolchain, it needs to know all the locations that a branch instruction may jump into, if a jump table is used, objtool has to correlate the jump instruction with the table. On x86 (which is the only port supported by objtool before LoongArch), there is a relocation type on the jump instruction and directly points to the table. But on LoongArch, the relocation is on another kind of instruction prior to the jump instruction, and also with scheduling it is not very easy to tell the offset of that instruction from the jump instruction. Furthermore, because LoongArch has -fsection-anchors (often enabled at -O1 or above) the relocation may actually points to a section anchor instead of the table itself. For the jump table of switch cases, a GCC patch "LoongArch: Add support to annotate tablejump" and a Clang patch "[LoongArch] Add options for annotate tablejump" have been merged into the upstream mainline, it can parse the additional section ".discard.tablejump_annotate" which stores the jump info as pairs of addresses, each pair contains the address of jump instruction and the address of jump table. For the jump table of computed gotos, it is indeed not easy to implement in the compiler, especially if there is more than one computed goto in a function such as ___bpf_prog_run(). objdump kernel/bpf/core.o shows that there are many table jump instructions in ___bpf_prog_run(), but there are no relocations on the table jump instructions and to the table directly on LoongArch. Without the help of compiler, in order to figure out the address of goto table for the special case of ___bpf_prog_run(), since the instruction sequence is relatively single and stable, it makes sense to add a helper find_reloc_of_rodata_c_jump_table() to find the relocation which points to the section ".rodata..c_jump_table". If find_reloc_by_table_annotate() failed, it means there is no relocation info of switch table address in ".rela.discard.tablejump_annotate", then objtool may find the relocation info of goto table ".rodata..c_jump_table" with find_reloc_of_rodata_c_jump_table(). Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- tools/objtool/arch/loongarch/special.c | 28 ++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/tools/objtool/arch/loongarch/special.c b/tools/objtool/arch/loongarch/special.c index 366517deb35b8..c5f4217df3a7a 100644 --- a/tools/objtool/arch/loongarch/special.c +++ b/tools/objtool/arch/loongarch/special.c @@ -114,6 +114,27 @@ static struct reloc *find_reloc_by_table_annotate(struct objtool_file *file, return NULL; } +static struct reloc *find_reloc_of_rodata_c_jump_table(struct section *sec, + unsigned long offset) +{ + struct section *rsec; + struct reloc *reloc; + + rsec = sec->rsec; + if (!rsec) + return NULL; + + for_each_reloc(rsec, reloc) { + if (reloc_offset(reloc) > offset) + break; + + if (!strncmp(reloc->sym->sec->name, ".rodata..c_jump_table", 21)) + return reloc; + } + + return NULL; +} + struct reloc *arch_find_switch_table(struct objtool_file *file, struct instruction *insn) { @@ -123,8 +144,11 @@ struct reloc *arch_find_switch_table(struct objtool_file *file, unsigned long table_offset; annotate_reloc = find_reloc_by_table_annotate(file, insn); - if (!annotate_reloc) - return NULL; + if (!annotate_reloc) { + annotate_reloc = find_reloc_of_rodata_c_jump_table(insn->sec, insn->offset); + if (!annotate_reloc) + return NULL; + } table_sec = annotate_reloc->sym->sec; if (annotate_reloc->sym->type == STT_SECTION) From 3fe1189c57d17a661600bf8003ebd7226b0d15c7 Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 12:42:24 +0800 Subject: [PATCH 15/16] LoongArch: Enable jump table for objtool For now, it is time to remove -fno-jump-tables to enable jump table for objtool if the compiler has -mannotate-tablejump, otherwise it is better to remain -fno-jump-tables to keep compatibility with older compilers. Signed-off-by: Tiezhu Yang Signed-off-by: Huacai Chen Signed-off-by: Binbin Zhou --- arch/loongarch/Kconfig | 3 +++ arch/loongarch/Makefile | 6 +++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index e54e87783b4c3..f5348d6647ea3 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -276,6 +276,9 @@ config AS_HAS_LBT_EXTENSION config AS_HAS_LVZ_EXTENSION def_bool $(as-instr,hvcl 0) +config CC_HAS_ANNOTATE_TABLEJUMP + def_bool $(cc-option,-mannotate-tablejump) + menu "Kernel type and options" source "kernel/Kconfig.hz" diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile index e05c10b044560..b9514ef3b6507 100644 --- a/arch/loongarch/Makefile +++ b/arch/loongarch/Makefile @@ -94,7 +94,11 @@ KBUILD_AFLAGS += $(call cc-option,-mthin-add-sub) $(call cc-option,-Wa$(comma) KBUILD_CFLAGS += $(call cc-option,-mthin-add-sub) $(call cc-option,-Wa$(comma)-mthin-add-sub) ifdef CONFIG_OBJTOOL -KBUILD_CFLAGS += -fno-jump-tables +ifdef CONFIG_CC_HAS_ANNOTATE_TABLEJUMP +KBUILD_CFLAGS += $(call cc-option,-mannotate-tablejump) +else +KBUILD_CFLAGS += -fno-jump-tables # keep compatibility with older compilers +endif endif ifeq ($(CONFIG_RELOCATABLE),y) From 553ae144afbcc645dfb84a5a46d0957e2bac0805 Mon Sep 17 00:00:00 2001 From: Binbin Zhou Date: Wed, 19 Feb 2025 15:07:59 +0800 Subject: [PATCH 16/16] LoongArch: Update Loongson-3 config file Enable ORC stack unwinder. Signed-off-by: Binbin Zhou --- arch/loongarch/configs/deepin_loongarch_desktop_defconfig | 1 + arch/loongarch/configs/loongson3_defconfig | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/loongarch/configs/deepin_loongarch_desktop_defconfig b/arch/loongarch/configs/deepin_loongarch_desktop_defconfig index 721a099a1a86b..c7f4aab5e44d7 100644 --- a/arch/loongarch/configs/deepin_loongarch_desktop_defconfig +++ b/arch/loongarch/configs/deepin_loongarch_desktop_defconfig @@ -6010,3 +6010,4 @@ CONFIG_RV_MON_WIP=y CONFIG_RV_MON_WWNR=y # CONFIG_STRICT_DEVMEM is not set # CONFIG_RUNTIME_TESTING_MENU is not set +CONFIG_UNWINDER_ORC=y diff --git a/arch/loongarch/configs/loongson3_defconfig b/arch/loongarch/configs/loongson3_defconfig index 75f0caefacc5f..a9943d2c1ea42 100644 --- a/arch/loongarch/configs/loongson3_defconfig +++ b/arch/loongarch/configs/loongson3_defconfig @@ -2203,3 +2203,4 @@ CONFIG_RCU_CPU_STALL_TIMEOUT=60 # CONFIG_RCU_TRACE is not set # CONFIG_STRICT_DEVMEM is not set # CONFIG_RUNTIME_TESTING_MENU is not set +CONFIG_UNWINDER_ORC=y