Use pathbuffer reduce stack #14632

crafcat7 · 2024-11-04T09:30:25Z

Summary

Why is this change necessary?
Benefit from the lib_pathbuffer feature introduced in #12969, we can further optimize the stack usage size in the NuttX kernel.
Changes
In this change, we replaced the declarations related to xxx[PATH_MAX] in the NuttX directory with the lib_pathbuffer method

Impact

Is a new feature added?: NO
Impact on build: NO
Impact on hardware: NO, this change does not specifically target any particular hardware architectures or boards.
Impact on documentation: NO,This patch does not introduce any new features
Impact on compatibility: This implementation aims to be backward compatible. No existing functionality is expected to be broken.

Testing

Build Host(s): Linux x86
Target(s): sim/nsh

By default

LIBC_MAX_PATHBUFFER = 2
LIBC_PATHBUFFER_MALLOC = y

So when there is not enough memory on the stack, it will be requested from the heap

Summary: Modified the usage logic, mainly introduced lib_get_pathbuffer and lib_put_pathbuffer Signed-off-by: chenrun1 <chenrun1@xiaomi.com>

nuttxpr · 2024-11-04T09:38:01Z

[Experimental Bot, please feedback here]

Yes, this PR appears to meet the NuttX requirements based on the provided information. However, there are a couple of minor improvements that could be made to strengthen the testing section.

While the PR states testing was done on sim/nsh, the "Testing logs before change" and "Testing logs after change" sections are empty. Providing concrete examples of the stack usage before and after the change would significantly improve the review process. For example, showing the output of a command that displays stack usage, or even a code snippet demonstrating how the stack size was measured would be beneficial. This would provide concrete evidence of the optimization.

anchao · 2024-11-04T11:34:55Z

arch/risc-v/src/common/espressif/esp_timer.c

    }

-  snprintf(devpath, sizeof(devpath), "/dev/timer%" PRIu32, timer_id);
+  devpath = lib_get_pathbuffer();


Since CONFIG_PATH_MAX can define a smaller size. let us add a configure CONFIG_LIBC_PATHBUFFER to enable/disable lib_get_pathbuffer implement.

#ifdef CONFIG_LIBC_PATHBUFFER FAR char *lib_get_pathbuffer(void); #else # define lib_get_pathbuffer() alloca(PATH_MAX) #endif

alloca isn't ANSI or POSIX Api, so it isn't available on all possible compiler, and should disable by default.

Current code base:
https://github.com/apache/nuttx/blob/master/include/string.h#L40-L51

I think most toolchains support this feature (even MSVC has a similar implementation)
https://github.com/apache/nuttx/blob/master/include/alloca.h#L30-L42

libs/libc/locale/lib_catalog.c:197:9: warning: implicit declaration of function ‘lib_get_pathbuffer’ [-Wimplicit-function-declaration] 197 | buf = lib_get_pathbuffer(); | ^~~~~~~~~~~~~~~~~~ libs/libc/locale/lib_catalog.c:197:7: warning: assignment to ‘char *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion] 197 | buf = lib_get_pathbuffer(); | ^ libs/libc/locale/lib_catalog.c:294:13: warning: implicit declaration of function ‘lib_put_pathbuffer’ [-Wimplicit-function-declaration] 294 | lib_put_pathbuffer(buf); Signed-off-by: zhangshoukui <zhangshoukui@xiaomi.com>

crafcat7 · 2024-11-05T13:01:49Z

I submitted a fix for the redefinition problem in libc under kernel build when using ffs/ffsl/ffsll.
a9e1d1b

crafcat7 · 2024-11-06T08:22:35Z

There is a ld problem here, I found that the static always inline ffs function implementation exists in the compilation expansion, but it cannot find the builtin ffs in the final link.

I tried changing the ffs part to __builtin_ffs in the code, but it still prompted an undefined reference to `ffs'.
Similarly, using ctz / __builtin_ctz / ffsl / __builtin_ffsl / ffsll / __builtin_ffsll does not have this problem

It seems that the implementation of _builtin_ffs cannot be found.

So is there any good way to solve this problem? Or should we add more restrictions to the implementation of builtin_ffs?

riscv-none-elf-ld: /github/workspace/sources/nuttx/staging/libc.a(lib_pathbuffer.o): in function `lib_get_pathbuffer':
/github/workspace/sources/nuttx/libs/libc/misc/lib_pathbuffer.c:89:(.text.lib_get_pathbuffer+0x1c): undefined reference to `ffs'
make[1]: *** [Makefile:189: nuttx] Error 1
make: *** [tools/Unix.mk:551: nuttx] Error 2

libs/libc/misc/lib_pathbuffer.c

crafcat7 · 2024-11-10T12:24:23Z

But the same thing happens on riscv's ci: the error message "undefined reference to `ffs'" still appears.
Is there something wrong with the riscv related configuration?

====================================================================================
Configuration/Tool: ox64/spi
2024-11-09 15:15:26
------------------------------------------------------------------------------------
  Cleaning...
  Configuring...
  Building NuttX...
riscv-none-elf-ld: /github/workspace/sources/nuttx/staging/libkarch.a(bl808_irq.o): in function `ffs':
/github/workspace/sources/nuttx/include/strings.h:82:(.text.__m0ic_interrupt+0x1e): undefined reference to `ffs'
riscv-none-elf-ld: /github/workspace/sources/nuttx/include/strings.h:82:(.text.__m0ic_interrupt+0x30): undefined reference to `ffs'
make[1]: *** [Makefile:189: nuttx] Error 1
make: *** [tools/Unix.mk:551: nuttx] Error 2
make: Target 'all' not remade because of errors.
/github/workspace/sources/nuttx/tools/testbuild.sh: line 385: /github/workspace/sources/nuttx/../nuttx/nuttx.manifest: No such file or directory
  Normalize ox64/spi
HEAD detached at pull/14632/merge
Untracked files:
  (use "git add <file>..." to include in what will be committed)
	boards/risc-v/bl808/ox64/scripts/ld.script.tmp

nothing added to commit but untracked files present (use "git add" to track)

Summary: Avoid error riscv-none-elf-ld: /github/workspace/sources/nuttx/staging/libc.a(lib_pathbuffer.o): in function `lib_get_pathbuffer': /github/workspace/sources/nuttx/libs/libc/misc/lib_pathbuffer.c:89:(.text.lib_get_pathbuffer+0x1c): undefined reference to `ffs' make[1]: *** [Makefile:189: nuttx] Error 1 make: *** [tools/Unix.mk:551: nuttx] Error 2 Signed-off-by: chenrun1 <chenrun1@xiaomi.com>

crafcat7 · 2024-11-12T03:27:37Z

Change ffs in lib_getpathbuffer to ffsl that everything will be OK

lupyuen · 2024-11-12T13:18:49Z

Sorry @crafcat7 I think spresense/example_camera is failing because of this PR:

Configuration/Tool: spresense/example_camera,CONFIG_ARM_TOOLCHAIN_GNU_EABI
mount/fs_automount.c: In function 'automount_initialize':
Error: mount/fs_automount.c:816:7: error: 'return' with no value, in function returning non-void [-Werror=return-type]
  816 |       return;
      |       ^~~~~~
In file included from mount/fs_automount.c:43:
/github/workspace/sources/nuttx/include/nuttx/fs/automount.h:176:11: note: declared here
  176 | FAR void *automount_initialize(FAR const struct automount_lower_s *lower);
      |           ^~~~~~~~~~~~~~~~~~~~

https://github.com/NuttX/nuttx/actions/runs/11796808033/job/32859585604#step:7:338

xiaoxiang781216 · 2024-11-12T18:50:20Z

@lupyuen do you know why ci in this pr doesn't catch this error, but #14748 can.

lupyuen · 2024-11-12T22:13:40Z

@xiaoxiang781216 That's because the CI Check for this Complex PR doesn't include arm-01, which contains spresense/example_camera. We swapped arm-01 for arm-13 some weeks ago.

#14748 is a Simple Arm32 PR that will run CI Checks for all Arm32 Targets: arm-01 to arm14.

crafcat7 · 2024-11-13T02:11:44Z

Sorry @crafcat7 I think spresense/example_camera is failing because of this PR:

Configuration/Tool: spresense/example_camera,CONFIG_ARM_TOOLCHAIN_GNU_EABI
mount/fs_automount.c: In function 'automount_initialize':
Error: mount/fs_automount.c:816:7: error: 'return' with no value, in function returning non-void [-Werror=return-type]
  816 |       return;
      |       ^~~~~~
In file included from mount/fs_automount.c:43:
/github/workspace/sources/nuttx/include/nuttx/fs/automount.h:176:11: note: declared here
  176 | FAR void *automount_initialize(FAR const struct automount_lower_s *lower);
      |           ^~~~~~~~~~~~~~~~~~~~

https://github.com/NuttX/nuttx/actions/runs/11796808033/job/32859585604#step:7:338

Okay, I submitted a fix on the #14752

Use lib_get_pathbuffer instead of stack variables

6d41208

Summary: Modified the usage logic, mainly introduced lib_get_pathbuffer and lib_put_pathbuffer Signed-off-by: chenrun1 <chenrun1@xiaomi.com>

anchao reviewed Nov 4, 2024

View reviewed changes

crafcat7 force-pushed the use_pathbuffer_reduce_stack branch from 6bc2a12 to b96d327 Compare November 5, 2024 06:16

crafcat7 force-pushed the use_pathbuffer_reduce_stack branch from a9e1d1b to ce23865 Compare November 6, 2024 08:20

crafcat7 force-pushed the use_pathbuffer_reduce_stack branch 3 times, most recently from b2f1cf6 to d33219e Compare November 9, 2024 15:00

xiaoxiang781216 reviewed Nov 10, 2024

View reviewed changes

libs/libc/misc/lib_pathbuffer.c Show resolved Hide resolved

crafcat7 force-pushed the use_pathbuffer_reduce_stack branch from d33219e to 4590886 Compare November 10, 2024 12:20

crafcat7 force-pushed the use_pathbuffer_reduce_stack branch from 4590886 to eae82ed Compare November 10, 2024 12:44

crafcat7 force-pushed the use_pathbuffer_reduce_stack branch from eae82ed to 1315d48 Compare November 11, 2024 04:34

xiaoxiang781216 approved these changes Nov 12, 2024

View reviewed changes

xiaoxiang781216 merged commit 2b190de into apache:master Nov 12, 2024

lupyuen mentioned this pull request Nov 12, 2024

boards: migrate to SPDX identifier #14742

Merged

lupyuen mentioned this pull request Nov 12, 2024

arch_atomic : Introduce CONFIG_LIBC_ARCH_ATOMIC #14681

Merged

crafcat7 mentioned this pull request Nov 13, 2024

fs_automount:fix ci break #14752

Merged

Use pathbuffer reduce stack #14632

Use pathbuffer reduce stack #14632

Uh oh!

Conversation

crafcat7 commented Nov 4, 2024

Summary

Impact

Testing

Uh oh!

nuttxpr commented Nov 4, 2024

Uh oh!

anchao Nov 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xiaoxiang781216 Nov 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anchao Nov 5, 2024

Choose a reason for hiding this comment

Uh oh!

crafcat7 commented Nov 5, 2024

Uh oh!

crafcat7 commented Nov 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

crafcat7 commented Nov 10, 2024

Uh oh!

crafcat7 commented Nov 12, 2024

Uh oh!

lupyuen commented Nov 12, 2024

Uh oh!

xiaoxiang781216 commented Nov 12, 2024

Uh oh!

lupyuen commented Nov 12, 2024

Uh oh!

crafcat7 commented Nov 13, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

anchao Nov 4, 2024 •

edited

Loading

xiaoxiang781216 Nov 4, 2024 •

edited

Loading

crafcat7 commented Nov 6, 2024 •

edited

Loading