[MSYS2] CMake+Ninja Fix arm-none-eabi-ar: Argument list too long

[simbit18]

Summary

This issue is related to the Arm toolchain for Windows which is available for x86 host architecture only (compatible with x86_64)

Windows (mingw-w64-i686) hosted cross toolchains
AArch32 bare-metal target (arm-none-eabi)

Issue
/bin/sh: line 1: /home/nuttx/nuttxnew/tools/gcc-arm-none-eabi/bin/arm-none-eabi-ar: Argument list too long

On Windows, arm-none-eabi-ar can only accept strings up to a maximum length of 32,768 characters.

We could suppress the 32K include string limitation by setting the CMake variable CMAKE_NINJA_FORCE_RESPONSE_FILE to ON.

This is unfortunately not enough!!! ):

In the build phase this error comes out

$ cmake --build build
[2/1025] Building ASM object arch/CMakeFiles/arch.dir/arm/src/armv7-m/arm_exception.S.obj FAILED: arch/CMakeFiles/arch.dir/arm/src/armv7-m/arm_exception.S.obj /home/nuttx/nuttxnew/tools/gcc-arm-none-eabi/bin/arm-none-eabi-gcc.exe @arch/CMakeFiles/arch.dir/arm/src/armv7-m/arm_exception.S.obj.rsp -MD -MT arch/CMakeFiles/arch.dir/arm/src/armv7-m/arm_exception.S.obj -MF arch/CMakeFiles/arch.dir/arm/src/armv7-m/arm_exception.S.obj.d -o arch/CMakeFiles/arch.dir/arm/src/armv7-m/arm_exception.S.obj -c /home/nuttx/nxninja/nuttx/arch/arm/src/armv7-m/arm_exception.S C:/msys64/home/nuttx/nxninja/nuttx/arch/arm/src/armv7-m/arm_exception.S:42:10: fatal error: nuttx/config.h: No such file or directory
   42 | #include <nuttx/config.h>
      |          ^~~~~~~~~~~~~~~~
compilation terminated.

The Workround I found to solve this problem is to overwrite the responsible file flag CMAKE_${lang}_RESPONSE_FILE_FLAG with $DEFINES $INCLUDES $FLAGS

workflows/build.yml
Enable CMake+Ninja for Msys2

#14014
apache/nuttx-apps#2672

Impact

Impact on user: No changes to user-facing functionality
Impact on build: Build process remains the same

Testing

Github linux and msys2
local msys2

$ cmake -B build -DBOARD_CONFIG=nucleo-l152re:nsh -GNinja
-- Initializing NuttX
  Select HOST_WINDOWS=y
  Select WINDOWS_MSYS=y
--   CMake:  3.28.1
--   Ninja:  1.11.1
--   Board:  nucleo-l152re
--   Config: nsh
--   Appdir: /home/nuttx/nxninja/apps
-- The C compiler identification is GNU 13.2.1
-- The CXX compiler identification is GNU 13.2.1
-- The ASM compiler identification is GNU
-- Found assembler: /home/nuttx/nuttxnew/tools/gcc-arm-none-eabi/bin/arm-none-eabi-gcc.exe
-- Configuring done (16.9s)
-- Generating done (8.0s)
-- Build files have been written to: /home/nuttx/nxninja/nuttx/build


$ cmake --build build
[27/1025] Building C object arch/CMakeFiles/arch.dir/arm/src/stm32/stm32_gpio.c.obj
C:/msys64/home/nuttx/nxninja/nuttx/arch/arm/src/stm32/stm32_gpio.c:41:11: note: '#pragma message: CONFIG_STM32_USE_LEGACY_PINMAP will be deprecated migrate board.h see tools/stm32_pinmap_tool.py'
   41 | #  pragma message "CONFIG_STM32_USE_LEGACY_PINMAP will be deprecated migrate board.h see tools/stm32_pinmap_tool.py"
      |           ^~~~~~~
[1022/1025] Linking C executable nuttx
Memory region         Used Size  Region Size  %age Used
           flash:       55500 B       512 KB     10.59%
            sram:        3700 B        80 KB      4.52%
[1025/1025] Generating System.map

[nuttxpr]

[Experimental Bot, please feedback here]

This PR appears to mostly meet the NuttX requirements, but there are some areas that could be improved for clarity and completeness.

Strengths:

• Clear Problem Description: The summary clearly explains the issue with arm-none-eabi-ar argument length limitations on Windows and the subsequent compilation error related to nuttx/config.h.
• Solution Provided: The proposed workaround of overwriting CMAKE_${lang}_RESPONSE_FILE_FLAG is described.
• Testing Evidence: Testing logs and a link to GitHub Actions workflows are provided, showing successful builds on both Linux and Windows (msys2).
• References: Related NuttX issues are linked.

Areas for Improvement:

• Summary - What functional part is changed?: Be more specific than "Arm toolchain for Windows". Specify that this changes CMake configuration related to response file generation.
• Summary - How does the change exactly work?: Explain why overwriting CMAKE_${lang}_RESPONSE_FILE_FLAG with $DEFINES $INCLUDES $FLAGS resolves the issue. What is different about this approach that avoids the problem?
• Impact - on build: While you say it remains the same, the internal build process is changing. Clarify that the user-facing process is the same but the CMake configuration is modified.
• Impact - on hardware: Specify the target architecture (Arm) that this change affects. It's implicit, but stating it explicitly is helpful.
• Testing - Before Change Logs: Provide logs demonstrating the original error. This is crucial for showing the effectiveness of the fix.
• Testing - More Detail: "local msys2" isn't very descriptive. Specify the msys2 environment (e.g., MSYS2 MinGW 64-bit) and its version. Also, include the full CMake configuration command used. Consistency between local and CI testing is important. The CI shows -DBOARD_CONFIG=nucleo-l152re:nsh, but the local test doesn't mention this. Ensure they're the same.
• Consider Alternatives: Investigate why CMAKE_NINJA_FORCE_RESPONSE_FILE=ON isn't sufficient. Is there a way to address the root cause instead of working around it? This might involve changes to the NuttX build system itself rather than just the CI configuration. Even if you ultimately stick with the current workaround, documenting the investigation would strengthen the PR.

Revised Impact Section Example (Illustrative):

• Impact on user: NO. Users should not experience any changes in functionality or build procedures.
• Impact on build: YES. While the user-facing build process remains the same, this PR modifies the internal CMake configuration by overwriting CMAKE_${lang}_RESPONSE_FILE_FLAG to work around an argument length limitation with arm-none-eabi-ar on Windows.
• Impact on hardware: YES. This specifically affects builds targeting Arm architectures when using an x86 Windows host with the MSYS2 environment.
• Impact on documentation: NO. (Or YES if you update any documentation).
• ... (rest of impact section)

By addressing these points, the PR will be more comprehensive and easier for reviewers to understand and approve. Providing clear "before" and "after" examples, explaining the why behind the change, and providing more detail about the testing environment will significantly improve the quality of the PR.

[hartmannathan]

Thanks for fixing this! (I wonder if there's a command line length limit on Unix?)

[xuxin930]

@simbit18 marvelous!!!

[simbit18]

Thanks for fixing this! (I wonder if there's a command line length limit on Unix?)

@hartmannathan try this with cmake

cmake_minimum_required(VERSION 3.16)

project(ManySources C)
set(srcs "")
# Increase MAX_SRC_FILE_INDEX value as needed
set(MAX_SRC_FILE_INDEX 5000)
set(TEST_SRC_FILE_BASE_NAME "this_is_a_test_src_file_with_a_quite_lengthy_name_to_simulate_very_long_command_line_length_problems_on_host_")
foreach(i RANGE 1 ${MAX_SRC_FILE_INDEX})
  set(src "${CMAKE_CURRENT_BINARY_DIR}/src/${TEST_SRC_FILE_BASE_NAME}${i}.c")
  file(WRITE "${src}" "int s${i}(void) { return 0; }\n")
  list(APPEND srcs ${src})
endforeach()

add_library(foo ${srcs})