remove up_cpu_pause up_cpu_resume up_cpu_paused up_cpu_pausereq #13863

hujun260 · 2024-10-07T07:54:53Z

Summary

remove up_cpu_pause up_cpu_resume up_cpu_paused up_cpu_pausereq

reaeon:
To remove the "sync pause" and decouple the critical section from the dependency on enabling interrupts,
after that we need to further implement "schedlock + spinlock".
changelist
1 Modify the implementation of critical sections to no longer involve enabling interrupts or handling synchronous pause events.
2 GIC_SMP_CPUCALL attach to pause handler to remove arch interface up_cpu_paused_restore up_cpu_paused_save
3 Completely remove up_cpu_pause, up_cpu_resume, up_cpu_paused, and up_cpu_pausereq
4 change up_cpu_pause_async to up_send_cpu_sgi

Impact

arch
size nuttx
text data bss dec hex filename
271456 51066 63336 385858 5e342 nuttx

before
text data bss dec hex filename
272468 51066 63384 386918 5e766 nuttx

size -1060

Testing

Configuring NuttX and compile:
$ ./tools/configure.sh -l qemu-armv8a:nsh_smp
$ make
Running with qemu
$ qemu-system-aarch64 -cpu cortex-a53 -smp 4 -nographic \
   -machine virt,virtualization=on,gic-version=3 \
   -net none -chardev stdio,id=con,mux=on -serial chardev:con \
   -mon chardev=con,mode=readline -kernel ./nuttx

github-actions · 2024-10-07T07:55:32Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2024-10-07T07:56:25Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2024-10-07T08:54:07Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

arch/arm/src/rp2040/rp2040_cpupause.c

arch/arm/src/cxd56xx/cxd56_cpupause.c

arch/arm/include/sam34/sam4cm_irq.h

github-actions · 2024-10-07T13:09:50Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

arch/arm/src/cxd56xx/cxd56_smpcall.c

arch/arm/src/lc823450/lc823450_cpustart.c

arch/arm/src/lc823450/lc823450_smpcall.c

arch/arm/src/rp2040/rp2040_smpcall.c

arch/arm/src/sam34/sam4cm_smpcall.c

arch/sim/src/sim/sim_smpsignal.c

Signed-off-by: hujun5 <hujun5@xiaomi.com>

reason: To remove the "sync pause" and decouple the critical section from the dependency on enabling interrupts, after that we need to further implement "schedlock + spinlock". changelist 1 Modify the implementation of critical sections to no longer involve enabling interrupts or handling synchronous pause events. 2 GIC_SMP_CPUCALL attach to pause handler to remove arch interface up_cpu_paused_restore up_cpu_paused_save 3 Completely remove up_cpu_pause, up_cpu_resume, up_cpu_paused, and up_cpu_pausereq 4 change up_cpu_pause_async to up_send_cpu_sgi Signed-off-by: hujun5 <hujun5@xiaomi.com>

github-actions · 2024-10-07T14:43:44Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

Signed-off-by: hujun5 <hujun5@xiaomi.com>

github-actions · 2024-10-07T14:48:32Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

masayuki2009 · 2024-10-08T09:15:31Z

@hujun260

I noticed that ostest crashes now with rv-virt:netnsh64_smp after merging this PR.

user_main: FPU test
Starting task FPU#1
fpu_test: Started task FPU#1 at PID=29
[   10.840000] [CPU1] riscv_exception: EXCEPTION: Illegal instruction. MCAUSE: 0000000000000002, EPC: 000000008003bfc0, MTVAL: 00000000d004f453
[   10.840000] [CPU1] riscv_exception: PANIC!!! Exception = 0000000000000002
[   10.840000] [CPU1] dump_assert_info: Current Version: NuttX  12.0.0 31a3cea64a Oct  8 2024 18:13:26 risc-v
[   10.840000] [CPU1] dump_assert_info: Assertion failed panic: at file: :0 task(CPU1): FPU#1 process: FPU#1 0x8003bf64
[   10.840000] [CPU1] up_dump_register: EPC: 000000008003bfc0
[   10.840000] [CPU1] up_dump_register: A0: 0000000000000000 A1: 000000008007948c A2: 0000000000000001 A3: 0000000000000001
[   10.840000] [CPU1] up_dump_register: A4: 0000000080080560 A5: 0000000000000440 A6: 0000000000000000 A7: 0000000000000000
[   10.840000] [CPU1] up_dump_register: T0: 0000000000000000 T1: 0000000000000000 T2: 0000000000000000 T3: 0000000000000000
[   10.840000] [CPU1] up_dump_register: T4: 0000000000000000 T5: 0000000000000000 T6: 0000000000000000
[   10.840000] [CPU1] up_dump_register: S0: 0000000000000000 S1: 0000000000000001 S2: 0000000000000000 S3: 0000000000000000
[   10.840000] [CPU1] up_dump_register: S4: 0000000000000000 S5: 0000000080078a70 S6: 0000000000000000 S7: 0000000000000000
[   10.840000] [CPU1] up_dump_register: S8: 0000000000000000 S9: 0000000000000000 S10: 0000000000000000 S11: 0000000000000000
[   10.840000] [CPU1] up_dump_register: SP: 0000000080081fd0 FP: 0000000000000000 TP: 0000000000000000 RA: 000000008003bfb2
[   10.840000] [CPU1] dump_stack: User Stack:
[   10.840000] [CPU1] dump_stack:   base: 0x800818d0
[   10.840000] [CPU1] dump_stack:   size: 00002000
[   10.840000] [CPU1] dump_stack:     sp: 0x80081fd0

tmedicci · 2024-10-30T17:34:55Z

Hi @hujun260 ,

esp32s3-devkit:sta_softap eventually asserts at DEBUGASSERT(tcb->cpu == this_cpu (https://github.com/apache/incubator-nuttx/blob/a56714888868399b5148ab84f8a100505bd3e974/sched/sched/sched_removereadytorun.c#L291) when running iPerf server for a while (some minutes). Do you have any idea why? (or what should we look for to solve this problem?)

patacongo · 2024-10-30T22:42:32Z

Hi @hujun260 ,

esp32s3-devkit:sta_softap eventually asserts at DEBUGASSERT(tcb->cpu == this_cpu (https://github.com/apache/incubator-nuttx/blob/a56714888868399b5148ab84f8a100505bd3e974/sched/sched/sched_removereadytorun.c#L291) when running iPerf server for a while (some minutes). Do you have any idea why? (or what should we look for to solve this problem?)

DEBUGASSERT(tcb->cpu == this_cpu) means that the logic should be running on tcb->cpu but instead is running on some other CPU. This means that the critical section is broken and unmanaged context switches are occurring. One fix is to revert these commit(s) that remove the required SMP support from enter_critical_section().

Try it.

Removing that SMP support allows context switches to occur in the middle of a critical section and completely breaks the critical section logic. A variety of unpredictable bad results will occur occasionally.

I am not sure why that is. I suspect that this is caused beccause with this change, interrupts are no longer disabled on the other CPUs and there are probably ways that interrupt activity can force context switches.

hujun260 · 2024-10-31T00:40:47Z

Hi @hujun260 ,

esp32s3-devkit:sta_softap eventually asserts at DEBUGASSERT(tcb->cpu == this_cpu (https://github.com/apache/incubator-nuttx/blob/a56714888868399b5148ab84f8a100505bd3e974/sched/sched/sched_removereadytorun.c#L291) when running iPerf server for a while (some minutes). Do you have any idea why? (or what should we look for to solve this problem?)
@tmedicci
Please give your crash call stack? so that I can analyze it.

tmedicci · 2024-10-31T18:28:49Z

Hi @hujun260 ,
esp32s3-devkit:sta_softap eventually asserts at DEBUGASSERT(tcb->cpu == this_cpu (https://github.com/apache/incubator-nuttx/blob/a56714888868399b5148ab84f8a100505bd3e974/sched/sched/sched_removereadytorun.c#L291) when running iPerf server for a while (some minutes). Do you have any idea why? (or what should we look for to solve this problem?)
@tmedicci
Please give your crash call stack? so that I can analyze it.

The crash call stack:

nsh> iperf -s -p 11111 -i 5 -B 0.0.0.0 -t 600 &
iperf [15:100]
nsh>      IP: 0.0.0.0

 mode=tcp-server sip=0.0.0.0:11111,dip=0.0.0.0:5001, interval=5, time=600
accept: 10.0.0.2:56070

           Interval         Transfer         Bandwidth

   0.00-   5.01 sec    4268216 Bytes    6.82 Mbits/sec
   5.01-  10.02 sec    3904300 Bytes    6.23 Mbits/sec
  10.02-  15.03 sec    2544302 Bytes    4.06 Mbits/sec
  15.03-  20.04 sec    3863134 Bytes    6.17 Mbits/sec
  20.04-  25.05 sec    3993092 Bytes    6.38 Mbits/sec
  25.05-  30.06 sec    3131298 Bytes    5.00 Mbits/sec
  30.06-  35.07 sec    4182710 Bytes    6.68 Mbits/sec
  35.07-  40.08 sec    3930936 Bytes    6.28 Mbits/sec
  40.08-  45.09 sec    3357936 Bytes    5.36 Mbits/sec
  45.09-  50.10 sec    4144182 Bytes    6.62 Mbits/sec
[CPU0] dump_assert_info: Current Version: NuttX  10.4.0 d1e5b38b9b Oct 31 2024 15:18:36 xtensa
[CPU0] dump_assert_info: Assertion failed tcb->cpu == up_cpu_index(): at file: sched/sched_removereadytorun.c:291 task(CPU0): wifi process: Kernel 0x4037e4b8
[CPU0] up_dump_register:    PC: 4206ac56    PS: 00060c22
[CPU0] up_dump_register:    A0: 80379b01    A1: 3fc987e4    A2: 3fcb83e0    A3: 3fc9a9e0
[CPU0] up_dump_register:    A4: 00060c22    A5: 00000000    A6: 3fc987c0    A7: 00000001
[CPU0] up_dump_register:    A8: 3fc995b4    A9: 3fc987d4   A10: 3fc995b4   A11: 00000001
[CPU0] up_dump_register:   A12: 00000001   A13: 00060a22   A14: 00000001   A15: 00000001
[CPU0] up_dump_register:   SAR: 0000001f CAUSE: 00000000 VADDR: 00000000
[CPU0] up_dump_register:  LBEG: 400570e8  LEND: 400570f3  LCNT: 00000000
[CPU0] dump_stackinfo: IRQ Stack:
[CPU0] dump_stackinfo:   base: 0x3fc98214
[CPU0] dump_stackinfo:   size: 00002048
[CPU0] dump_stackinfo:     sp: 0x3fc987e4
[CPU0] stack_dump: 0x3fc987c4: 82047cad 3fc98804 3c010b95 3fcb6c58 000019d8 3fcb8720 deadbeef deadbeef
[CPU0] stack_dump: 0x3fc987e4: 3fc98214 3fc987e4 00000000 00000000 82045377 3fc988f4 3c010b95 00000123
[CPU0] stack_dump: 0x3fc98804: 00000000 3fcb6d04 3c017424 4037e4b8 7474754e dead0058 deadbeef deadbeef
[CPU0] stack_dump: 0x3fc98824: deadbeef dead00ef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
[CPU0] stack_dump: 0x3fc98844: deadbeef 2e303100 00302e34 deadbeef deadbeef deadbeef 3164beef 33623565
[CPU0] stack_dump: 0x3fc98864: 62396238 74634f20 20313320 34323032 3a353120 333a3831 3fc90036 ffffffff
[CPU0] stack_dump: 0x3fc98884: 00000001 deadbeef deadbeef 657478ef 0061736e deadbeef deadbeef deadbeef
[CPU0] stack_dump: 0x3fc988a4: deadbeef 3fcb6c58 3fc9a9e0 3c010c1c 3c010b95 00000123 00000001 deadbeef
[CPU0] stack_dump: 0x3fc988c4: 00000006 00060222 00000123 00000000 3c010c1c 3fc9a9e0 3c017600 4037e4b8
[CPU0] stack_dump: 0x3fc988e4: 820459d0 3fc98914 3fcc8108 00000001 3c010c1c 00000004 3fc981f4 0000001f
[CPU0] stack_dump: 0x3fc98904: 8206457d 3fc98934 3fcc8108 00000064 00040023 3fc98a14 00000001 00018044
[CPU0] stack_dump: 0x3fc98924: 82063b5c 3fc98954 3fc90930 3fcb6c58 00000001 3fcb85b0 3fcb6c58 3fcb5560
[CPU0] stack_dump: 0x3fc98944: 8204b1ae 3fc98974 00060023 00008209 00060022 3fcc8108 3fcae250 ffffffff
[CPU0] stack_dump: 0x3fc98964: 8037a00e 3fc98994 0000003e 3fcb83e0 00000001 3fc917c8 00060023 00060120
[CPU0] stack_dump: 0x3fc98984: 80379538 3fc989b4 0000003e 3fcb83e0 00000000 3fcb8610 3fcae250 fffffffc
[CPU0] stack_dump: 0x3fc989a4: 80378cc0 3fc989d4 0000003e 3fcb83e0 00000001 3fcb8540 00000000 3fcc8510
[CPU0] stack_dump: 0x3fc989c4: 40374d08 3fc989f4 00000004 3fcb83e0 3fc995b4 3fc90930 3fcb6c58 00000001
[CPU0] stack_dump: 0x3fc989e4: 00040023 3fc98a14 00000001 00018044 00000002 00000004 3fc981f4 0000001f
[CPU0] stack_dump: 0x3fc98a04: deadbeef deadbeef deadbeef 00000000 00000000 00000000 00000000 00000000
[CPU0] dump_stackinfo: User Stack:
[CPU0] dump_stackinfo:   base: 0x3fcb6d48
[CPU0] dump_stackinfo:   size: 00006616
[CPU0] dump_stackinfo:     sp: 0x3fcb8590
[CPU0] stack_dump: 0x3fcb8570: 00000000 00000000 00000000 00000000 82047e25 3fcb85b0 3fcb6c58 3fcb5560
[CPU0] stack_dump: 0x3fcb8590: 00000000 00000000 3fc90700 3fcc8860 82049644 3fcb85d0 3fcae250 ffffffff
[CPU0] stack_dump: 0x3fcb85b0: 00060120 00000000 00000000 3fcc8510 8204980c 3fcb85f0 3fcae250 00060120
[CPU0] stack_dump: 0x3fcb85d0: 3c0106e6 3fc98924 00000000 00000001 8204983d 3fcb8610 3fcae250 fffffffc
[CPU0] stack_dump: 0x3fcb85f0: 00000000 00060122 00000000 00000000 8204938c 3fcb8630 3fcae250 3fcc8510
[CPU0] stack_dump: 0x3fcb8610: 00000000 00000000 00000000 00000001 82076c00 3fcb8650 3fcc8510 00060120
[CPU0] stack_dump: 0x3fcb8630: 420757e4 3fca17ac 00000000 3fcb84e0 8037cc68 3fcb8670 3fcc8510 3fcc8526
[CPU0] stack_dump: 0x3fcb8650: 8037956a 3fc989c4 3fc995b4 3fcb84e0 820c5338 3fcb8690 00000005 00000001
[CPU0] stack_dump: 0x3fcb8670: 00000001 ffffefff 3fcb6bd0 00000001 8037e580 3fcb86b0 00000000 00000000
[CPU0] stack_dump: 0x3fcb8690: 3fcef940 42076bf4 00000001 3fcef954 82044700 3fcb86d0 3fcef940 3fcef904
[CPU0] stack_dump: 0x3fcb86b0: 00000001 00000001 00000000 3fcc8510 00000000 3fcb8700 00000002 4037e4b8
[CPU0] stack_dump: 0x3fcb86d0: 00000010 00000000 00000000 00000000 00000000 4037e577 3fc90700 000000b1
[CPU0] stack_dump: 0x3fcb86f0: 00000000 3fcb8720 00000000 00000000 00050020 00000000 00000000 00000000
[CPU0] stack_dump: 0x3fcb8710: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[CPU0] dump_fatal_info: Dump CPU1: RUNNING
[CPU0] dump_tasks:    PID GROUP   CPU PRI POLICY   TYPE    NPX STATE   EVENT      SIGMASK          STACKBASE  STACKSIZE      USED   FILLED    COMMAND
[CPU0] dump_tasks:   ----   ---     0 --- -------- ------- --- ------- ---------- ---------------- 0x3fc98214      2048      1292    63.0%    irq
[CPU0] dump_tasks:   ----   ---     1 --- -------- ------- --- ------- ---------- ---------------- 0x3fc98a14      2048        76     3.7%    irq
[CPU0] dump_task:       0     0     0   0 FIFO     Kthread -   Assigned           0000000000000000 0x3fcaa280      3056       672    21.9%    CPU0 IDLE
[CPU0] dump_task:       1     0     1   0 FIFO     Kthread -   Assigned           0000000000000000 0x3fcae420      3056       864    28.2%    CPU1 IDLE
[CPU0] dump_task:       2     0     1 100 RR       Kthread -   Assigned           0000000000000000 0x3fcaf260      4032      1168    28.9%    lpwork 0x3fcaae70 0x3fcaae94
[CPU0] dump_task:       3     3     0 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fcb06a0      8144      1952    23.9%    nsh_main
[CPU0] dump_task:       4     0     0 255 RR       Kthread -   Waiting Semaphore  0000000000000000 0x3fcb2a30       704       560    79.5%    spiflash_op 0x3fcb253c
[CPU0] dump_task:       5     0     1 255 RR       Kthread -   Waiting Semaphore  0000000000000000 0x3fcb2e08       696       560    80.4%!   spiflash_op 0x3fcb253c
[CPU0] dump_task:       6     0     1 223 RR       Kthread -   Assigned           0000000000000000 0x3fcb5660      4048       912    22.5%    rt_timer
[CPU0] dump_task:       7     0     0 253 RR       Kthread -   Running            0000000000000000 0x3fcb6d48      6616      2288    34.5%    wifi
[CPU0] dump_task:      24    15     1 100 RR       pthread -   Running            0000000000000000 0x3fcc97b0      4080      1744    42.7%    iperf_traffic 0x4206dcc0 0x3fcc0e00
[CPU0] dump_task:      25    15     0 100 RR       pthread -   Waiting Signal     0000000000000000 0x3fcca7b8      4072      1440    35.3%    iperf_report 0x4206e1a8 0x3fcc0e00
[CPU0] dump_task:      11    11     1 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fcc2fb8      4040      1200    29.7%    DHCPD_daemon wlan1
[CPU0] dump_task:      15    15     1 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fcbffd8      3976      1008    25.3%    iperf -s -p 11111 -i 5 -B 0.0.0.0 -t 600

The firmware:
nuttx.zip

tmedicci · 2024-10-31T20:09:29Z

Hi @hujun260 ,
esp32s3-devkit:sta_softap eventually asserts at DEBUGASSERT(tcb->cpu == this_cpu (https://github.com/apache/incubator-nuttx/blob/a56714888868399b5148ab84f8a100505bd3e974/sched/sched/sched_removereadytorun.c#L291) when running iPerf server for a while (some minutes). Do you have any idea why? (or what should we look for to solve this problem?)

DEBUGASSERT(tcb->cpu == this_cpu) means that the logic should be running on tcb->cpu but instead is running on some other CPU. This means that the critical section is broken and unmanaged context switches are occurring. One fix is to revert these commit(s) that remove the required SMP support from enter_critical_section().

Try it.

Removing that SMP support allows context switches to occur in the middle of a critical section and completely breaks the critical section logic. A variety of unpredictable bad results will occur occasionally.

I am not sure why that is. I suspect that this is caused beccause with this change, interrupts are no longer disabled on the other CPUs and there are probably ways that interrupt activity can force context switches.

Thanks for your comment, Greg. The assertion started triggering after these commits were added. Checking out to the revision just before them being merged works as expected

patacongo · 2024-10-31T21:48:00Z

@hujun260 are you planning to fix this? Or should we revert the change? I breaks SMP on some platforms. Perhaps @tmedicci could help verify any changes?

hujun260 · 2024-11-01T01:39:29Z

Hi @hujun260 ,
esp32s3-devkit:sta_softap eventually asserts at DEBUGASSERT(tcb->cpu == this_cpu (https://github.com/apache/incubator-nuttx/blob/a56714888868399b5148ab84f8a100505bd3e974/sched/sched/sched_removereadytorun.c#L291) when running iPerf server for a while (some minutes). Do you have any idea why? (or what should we look for to solve this problem?)
@tmedicci
Please give your crash call stack? so that I can analyze it.

The crash call stack:

nsh> iperf -s -p 11111 -i 5 -B 0.0.0.0 -t 600 &
iperf [15:100]
nsh>      IP: 0.0.0.0

 mode=tcp-server sip=0.0.0.0:11111,dip=0.0.0.0:5001, interval=5, time=600
accept: 10.0.0.2:56070

           Interval         Transfer         Bandwidth

   0.00-   5.01 sec    4268216 Bytes    6.82 Mbits/sec
   5.01-  10.02 sec    3904300 Bytes    6.23 Mbits/sec
  10.02-  15.03 sec    2544302 Bytes    4.06 Mbits/sec
  15.03-  20.04 sec    3863134 Bytes    6.17 Mbits/sec
  20.04-  25.05 sec    3993092 Bytes    6.38 Mbits/sec
  25.05-  30.06 sec    3131298 Bytes    5.00 Mbits/sec
  30.06-  35.07 sec    4182710 Bytes    6.68 Mbits/sec
  35.07-  40.08 sec    3930936 Bytes    6.28 Mbits/sec
  40.08-  45.09 sec    3357936 Bytes    5.36 Mbits/sec
  45.09-  50.10 sec    4144182 Bytes    6.62 Mbits/sec
[CPU0] dump_assert_info: Current Version: NuttX  10.4.0 d1e5b38b9b Oct 31 2024 15:18:36 xtensa
[CPU0] dump_assert_info: Assertion failed tcb->cpu == up_cpu_index(): at file: sched/sched_removereadytorun.c:291 task(CPU0): wifi process: Kernel 0x4037e4b8
[CPU0] up_dump_register:    PC: 4206ac56    PS: 00060c22
[CPU0] up_dump_register:    A0: 80379b01    A1: 3fc987e4    A2: 3fcb83e0    A3: 3fc9a9e0
[CPU0] up_dump_register:    A4: 00060c22    A5: 00000000    A6: 3fc987c0    A7: 00000001
[CPU0] up_dump_register:    A8: 3fc995b4    A9: 3fc987d4   A10: 3fc995b4   A11: 00000001
[CPU0] up_dump_register:   A12: 00000001   A13: 00060a22   A14: 00000001   A15: 00000001
[CPU0] up_dump_register:   SAR: 0000001f CAUSE: 00000000 VADDR: 00000000
[CPU0] up_dump_register:  LBEG: 400570e8  LEND: 400570f3  LCNT: 00000000
[CPU0] dump_stackinfo: IRQ Stack:
[CPU0] dump_stackinfo:   base: 0x3fc98214
[CPU0] dump_stackinfo:   size: 00002048
[CPU0] dump_stackinfo:     sp: 0x3fc987e4
[CPU0] stack_dump: 0x3fc987c4: 82047cad 3fc98804 3c010b95 3fcb6c58 000019d8 3fcb8720 deadbeef deadbeef
[CPU0] stack_dump: 0x3fc987e4: 3fc98214 3fc987e4 00000000 00000000 82045377 3fc988f4 3c010b95 00000123
[CPU0] stack_dump: 0x3fc98804: 00000000 3fcb6d04 3c017424 4037e4b8 7474754e dead0058 deadbeef deadbeef
[CPU0] stack_dump: 0x3fc98824: deadbeef dead00ef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
[CPU0] stack_dump: 0x3fc98844: deadbeef 2e303100 00302e34 deadbeef deadbeef deadbeef 3164beef 33623565
[CPU0] stack_dump: 0x3fc98864: 62396238 74634f20 20313320 34323032 3a353120 333a3831 3fc90036 ffffffff
[CPU0] stack_dump: 0x3fc98884: 00000001 deadbeef deadbeef 657478ef 0061736e deadbeef deadbeef deadbeef
[CPU0] stack_dump: 0x3fc988a4: deadbeef 3fcb6c58 3fc9a9e0 3c010c1c 3c010b95 00000123 00000001 deadbeef
[CPU0] stack_dump: 0x3fc988c4: 00000006 00060222 00000123 00000000 3c010c1c 3fc9a9e0 3c017600 4037e4b8
[CPU0] stack_dump: 0x3fc988e4: 820459d0 3fc98914 3fcc8108 00000001 3c010c1c 00000004 3fc981f4 0000001f
[CPU0] stack_dump: 0x3fc98904: 8206457d 3fc98934 3fcc8108 00000064 00040023 3fc98a14 00000001 00018044
[CPU0] stack_dump: 0x3fc98924: 82063b5c 3fc98954 3fc90930 3fcb6c58 00000001 3fcb85b0 3fcb6c58 3fcb5560
[CPU0] stack_dump: 0x3fc98944: 8204b1ae 3fc98974 00060023 00008209 00060022 3fcc8108 3fcae250 ffffffff
[CPU0] stack_dump: 0x3fc98964: 8037a00e 3fc98994 0000003e 3fcb83e0 00000001 3fc917c8 00060023 00060120
[CPU0] stack_dump: 0x3fc98984: 80379538 3fc989b4 0000003e 3fcb83e0 00000000 3fcb8610 3fcae250 fffffffc
[CPU0] stack_dump: 0x3fc989a4: 80378cc0 3fc989d4 0000003e 3fcb83e0 00000001 3fcb8540 00000000 3fcc8510
[CPU0] stack_dump: 0x3fc989c4: 40374d08 3fc989f4 00000004 3fcb83e0 3fc995b4 3fc90930 3fcb6c58 00000001
[CPU0] stack_dump: 0x3fc989e4: 00040023 3fc98a14 00000001 00018044 00000002 00000004 3fc981f4 0000001f
[CPU0] stack_dump: 0x3fc98a04: deadbeef deadbeef deadbeef 00000000 00000000 00000000 00000000 00000000
[CPU0] dump_stackinfo: User Stack:
[CPU0] dump_stackinfo:   base: 0x3fcb6d48
[CPU0] dump_stackinfo:   size: 00006616
[CPU0] dump_stackinfo:     sp: 0x3fcb8590
[CPU0] stack_dump: 0x3fcb8570: 00000000 00000000 00000000 00000000 82047e25 3fcb85b0 3fcb6c58 3fcb5560
[CPU0] stack_dump: 0x3fcb8590: 00000000 00000000 3fc90700 3fcc8860 82049644 3fcb85d0 3fcae250 ffffffff
[CPU0] stack_dump: 0x3fcb85b0: 00060120 00000000 00000000 3fcc8510 8204980c 3fcb85f0 3fcae250 00060120
[CPU0] stack_dump: 0x3fcb85d0: 3c0106e6 3fc98924 00000000 00000001 8204983d 3fcb8610 3fcae250 fffffffc
[CPU0] stack_dump: 0x3fcb85f0: 00000000 00060122 00000000 00000000 8204938c 3fcb8630 3fcae250 3fcc8510
[CPU0] stack_dump: 0x3fcb8610: 00000000 00000000 00000000 00000001 82076c00 3fcb8650 3fcc8510 00060120
[CPU0] stack_dump: 0x3fcb8630: 420757e4 3fca17ac 00000000 3fcb84e0 8037cc68 3fcb8670 3fcc8510 3fcc8526
[CPU0] stack_dump: 0x3fcb8650: 8037956a 3fc989c4 3fc995b4 3fcb84e0 820c5338 3fcb8690 00000005 00000001
[CPU0] stack_dump: 0x3fcb8670: 00000001 ffffefff 3fcb6bd0 00000001 8037e580 3fcb86b0 00000000 00000000
[CPU0] stack_dump: 0x3fcb8690: 3fcef940 42076bf4 00000001 3fcef954 82044700 3fcb86d0 3fcef940 3fcef904
[CPU0] stack_dump: 0x3fcb86b0: 00000001 00000001 00000000 3fcc8510 00000000 3fcb8700 00000002 4037e4b8
[CPU0] stack_dump: 0x3fcb86d0: 00000010 00000000 00000000 00000000 00000000 4037e577 3fc90700 000000b1
[CPU0] stack_dump: 0x3fcb86f0: 00000000 3fcb8720 00000000 00000000 00050020 00000000 00000000 00000000
[CPU0] stack_dump: 0x3fcb8710: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[CPU0] dump_fatal_info: Dump CPU1: RUNNING
[CPU0] dump_tasks:    PID GROUP   CPU PRI POLICY   TYPE    NPX STATE   EVENT      SIGMASK          STACKBASE  STACKSIZE      USED   FILLED    COMMAND
[CPU0] dump_tasks:   ----   ---     0 --- -------- ------- --- ------- ---------- ---------------- 0x3fc98214      2048      1292    63.0%    irq
[CPU0] dump_tasks:   ----   ---     1 --- -------- ------- --- ------- ---------- ---------------- 0x3fc98a14      2048        76     3.7%    irq
[CPU0] dump_task:       0     0     0   0 FIFO     Kthread -   Assigned           0000000000000000 0x3fcaa280      3056       672    21.9%    CPU0 IDLE
[CPU0] dump_task:       1     0     1   0 FIFO     Kthread -   Assigned           0000000000000000 0x3fcae420      3056       864    28.2%    CPU1 IDLE
[CPU0] dump_task:       2     0     1 100 RR       Kthread -   Assigned           0000000000000000 0x3fcaf260      4032      1168    28.9%    lpwork 0x3fcaae70 0x3fcaae94
[CPU0] dump_task:       3     3     0 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fcb06a0      8144      1952    23.9%    nsh_main
[CPU0] dump_task:       4     0     0 255 RR       Kthread -   Waiting Semaphore  0000000000000000 0x3fcb2a30       704       560    79.5%    spiflash_op 0x3fcb253c
[CPU0] dump_task:       5     0     1 255 RR       Kthread -   Waiting Semaphore  0000000000000000 0x3fcb2e08       696       560    80.4%!   spiflash_op 0x3fcb253c
[CPU0] dump_task:       6     0     1 223 RR       Kthread -   Assigned           0000000000000000 0x3fcb5660      4048       912    22.5%    rt_timer
[CPU0] dump_task:       7     0     0 253 RR       Kthread -   Running            0000000000000000 0x3fcb6d48      6616      2288    34.5%    wifi
[CPU0] dump_task:      24    15     1 100 RR       pthread -   Running            0000000000000000 0x3fcc97b0      4080      1744    42.7%    iperf_traffic 0x4206dcc0 0x3fcc0e00
[CPU0] dump_task:      25    15     0 100 RR       pthread -   Waiting Signal     0000000000000000 0x3fcca7b8      4072      1440    35.3%    iperf_report 0x4206e1a8 0x3fcc0e00
[CPU0] dump_task:      11    11     1 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fcc2fb8      4040      1200    29.7%    DHCPD_daemon wlan1
[CPU0] dump_task:      15    15     1 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fcbffd8      3976      1008    25.3%    iperf -s -p 11111 -i 5 -B 0.0.0.0 -t 600

The firmware: nuttx.zip
@tmedicci
you could add following config to print crash backtrace when trigger assert

CONFIG_SCHED_BACKTRACE=y
CONFIG_SYSTEM_DUMPSTACK=y
CONFIG_ALLSYMS=y
CONFIG_ARCH_HAVE_BACKTRACE=y

i just want to know call stack like this
for example:
#0 nxsched_remove_readytorun (tcb=tcb@entry=0x40452150) at sched/sched_removereadytorun.c:289
#1 0x00000000402ee484 in nxtask_reset_task (tcb=tcb@entry=0x40452150, remove=true) at task/task_restart.c:120
#2 0x00000000402ee68c in nxtask_restart (pid=8) at task/task_restart.c:260
#3 task_restart (pid=pid@entry=8) at task/task_restart.c:313
#4 0x00000000402e4148 in restart_test () at restart.c:199
#5 0x00000000402e2728 in user_main (argc=argc@entry=5, argv=argv@entry=0x4044e150) at ostest_main.c:361
#6 0x00000000402c33f0 in nxtask_startup (entrypt=0x402e253c <user_main>, argc=argc@entry=5, argv=0x4044e150) at sched/task_startup.c:72
#7 0x00000000402bd8a8 in nxtask_start () at task/task_start.c:116
#8 0x0000000000000000 in ?? ()

hujun260 · 2024-11-01T03:58:04Z

compile:
make distclean -j12;./tools/configure.sh -E esp32s3-devkit:sta_softap;make -j12
Program burning:
make flash ESPTOOL_PORT=/dev/ttyUSB0 ESPTOOL_BINDIR=/home/hujun5/work/ssd/esp-bins

then
minicom -D /dev/ttyUSB0 -b 115200

i can start the server. But how can I reproduce crash?

tmedicci · 2024-11-01T13:20:33Z

you could add following config to print crash backtrace when trigger assert

CONFIG_SCHED_BACKTRACE=y
CONFIG_SYSTEM_DUMPSTACK=y
CONFIG_ALLSYMS=y
CONFIG_ARCH_HAVE_BACKTRACE=y

See below.

i can start the server. But how can I reproduce crash?

Use the device as a Wi-Fi AP and connect your PC to it. Run the server on the device and the client on PC:

nsh> 
nsh> ifup wlan1
ifup wlan1...OK
nsh> dhcpd_start wlan1
nsh> wapi psk wlan1 mypasswd 3
nsh> wapi essid wlan1 myssid 1
nsh> iperf -s -p 11111 -i 5 -B 0.0.0.0 -t 600 &
iperf [12:100]
nsh>      IP: 0.0.0.0

 mode=tcp-server sip=0.0.0.0:11111,dip=0.0.0.0:5001, interval=5, time=600
accept: 10.0.0.2:34022

           Interval         Transfer         Bandwidth

   0.00-   5.02 sec    2549304 Bytes    4.06 Mbits/sec
[CPU0] dump_assert_info: Current Version: NuttX  10.4.0 31a3cea64a Nov  1 2024 09:57:23 xtensa
[CPU0] dump_assert_info: Assertion failed tcb->cpu == up_cpu_index(): at file: sched/sched_removereadytorun.c:293 tas8
[CPU0] up_dump_register:    PC: 4206a1ee    PS: 00060722
[CPU0] up_dump_register:    A0: 80379b89    A1: 3fc987e8    A2: 3fcc09d0    A3: 3fc9a9f0
[CPU0] up_dump_register:    A4: 00060722    A5: 00000000    A6: 3fc987c4    A7: 00000001
[CPU0] up_dump_register:    A8: 3fc995b8    A9: 3fc987d8   A10: 3fc995b8   A11: 00000001
[CPU0] up_dump_register:   A12: 00000001   A13: 00060522   A14: 00000001   A15: 00000001
[CPU0] up_dump_register:   SAR: 0000001f CAUSE: 00000000 VADDR: 00000000
[CPU0] up_dump_register:  LBEG: 400570e8  LEND: 400570f3  LCNT: 00000000
[CPU0] dump_stack: IRQ Stack:
[CPU0] dump_stack:   base: 0x3fc98218
[CPU0] dump_stack:   size: 00002048
[CPU0] dump_stack:     sp: 0x3fc987e8
[CPU0] stack_dump: 0x3fc987c8: 8204791d 3fc98808 3c014818 3fcbf1a8 000019d8 3fcc0c70 3c014818 3fcbf1a8
[CPU0] stack_dump: 0x3fc987e8: 3fc98218 3fc987e8 00000000 00000000 82044f83 3fc988f8 3c014818 00000125
[CPU0] stack_dump: 0x3fc98808: 00000000 3fcbf254 3c01adaf 4037e548 7474754e dead0058 deadbeef deadbeef
[CPU0] stack_dump: 0x3fc98828: deadbeef dead00ef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
[CPU0] stack_dump: 0x3fc98848: deadbeef 2e303100 00302e34 deadbeef deadbeef deadbeef 3133beef 65633361
[CPU0] stack_dump: 0x3fc98868: 61343661 766f4e20 20312020 34323032 3a393020 323a3735 3fc90033 ffffffff
[CPU0] stack_dump: 0x3fc98888: 00000001 deadbeef deadbeef 657478ef 0061736e deadbeef deadbeef deadbeef
[CPU0] stack_dump: 0x3fc988a8: 00000003 3fcbf1a8 3fc9a9f0 3c01489f 3c014818 00000125 00000002 deadbeef
[CPU0] stack_dump: 0x3fc988c8: 00000006 00060d22 00000125 00000000 3c01489f 3fc9a9f0 3c01af87 4037e548
[CPU0] stack_dump: 0x3fc988e8: 820455cd 3fc98918 3fcb76f0 00000001 3c01489f 00000004 3fc981f8 0000001f
[CPU0] stack_dump: 0x3fc98908: 82063ee5 3fc98938 3fcb76f0 00000064 420a255c 3fc98a18 00000001 3fcc0b80
[CPU0] stack_dump: 0x3fc98928: 82063598 3fc98958 3fc90930 3fcbf1a8 00000001 3fcc0be0 3fcd36b8 ffffffe7
[CPU0] stack_dump: 0x3fc98948: 8204adca 3fc98978 00060023 0000820a 00060022 3fcb76f0 3fcd36b8 420a0b94
[CPU0] stack_dump: 0x3fc98968: 8037a09e 3fc98998 0000003e 3fcc09d0 00000001 3fc917c8 00060023 00000000
[CPU0] stack_dump: 0x3fc98988: 8037959c 3fc989b8 0000003e 3fcc09d0 00000000 3fcc0b00 00000000 3fca17dc
[CPU0] stack_dump: 0x3fc989a8: 80378d24 3fc989d8 0000003e 3fcc09d0 00000006 00000051 00000006 3fca77c0
[CPU0] stack_dump: 0x3fc989c8: 40374d2d 3fc989f8 00000004 3fcc09d0 3fc995b8 3fc90930 3fcbf1a8 3fccc6a4
[CPU0] stack_dump: 0x3fc989e8: 420a255c 3fc98a18 00000001 3fcc0b80 00000002 00000004 3fc981f8 0000001f
[CPU0] stack_dump: 0x3fc98a08: 820a0bb0 3fcc0b80 00000000 00000000 00000000 00000000 00000000 00000000
[CPU0] dump_stack: User Stack:
[CPU0] dump_stack:   base: 0x3fcbf298
[CPU0] dump_stack:   size: 00006616
[CPU0] dump_stack:     sp: 0x3fcc0b80
[CPU0] stack_dump: 0x3fcc0b60: 3fca83b4 0000001a 000005ea 00000c26 8037ffe8 3fcc0be0 3fcd36b8 ffffffe7
[CPU0] stack_dump: 0x3fcc0b80: 3bb3b668 000069b0 3fcbf148 00000003 3fcc77f0 00000612 00000001 00000088
[CPU0] stack_dump: 0x3fcc0ba0: 00000001 3fcd3752 ffffffe7 00000000 00000001 3fccc6a0 42075d00 3fca9f7c
[CPU0] stack_dump: 0x3fcc0bc0: 3fccca58 00000008 00000000 3fcccda8 8037e616 3fcc0c00 3fcd36b8 420a0b94
[CPU0] stack_dump: 0x3fcc0be0: 00000000 3fcef944 3fcbebe8 00000002 82044518 3fcc0c20 3fcef940 3fcef904
[CPU0] stack_dump: 0x3fcc0c00: 00000010 3fcef920 3fcef940 3fcc2bac 00000000 3fcc0c50 00000002 4037e548
[CPU0] stack_dump: 0x3fcc0c20: 00000011 00000000 00000000 00000000 00000000 4037e613 3fc90700 000000b3
[CPU0] stack_dump: 0x3fcc0c40: 00000000 3fcc0c70 00000000 00000000 00050020 00000000 00000000 00000000
[CPU0] stack_dump: 0x3fcc0c60: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[CPU0] sched_dumpstack: [ 7] [<0x42066191>] sched_dumpstack+0x15/0x50
[CPU0] sched_dumpstack: [ 7] [<0x403798d1>] dump_running_task+0xed/0xf0
[CPU0] sched_dumpstack: [ 7] [<0x40379c91>] _assert+0x289/0x3dc
[CPU0] sched_dumpstack: [ 7] [<0x4204791d>] __assert+0x11/0x18
[CPU0] sched_dumpstack: [ 7] [<0x42044f83>] nxsched_remove_readytorun+0x4b/0xc0
[CPU0] sched_dumpstack: [ 7] [<0x420455cd>] nxsched_reprioritize_rtr+0xd/0x34
[CPU0] sched_dumpstack: [ 7] [<0x42063ee5>] nxsched_process_roundrobin+0x6d/0x90
[CPU0] sched_dumpstack: [ 7] [<0x42063598>] nxsched_process_timer+0x44/0x5c
[CPU0] sched_dumpstack: [ 7] [<0x4204adca>] systimer_isr+0x16/0x1c
[CPU0] sched_dumpstack: [ 7] [<0x4037a09e>] irq_dispatch+0x2a/0x2c
[CPU0] sched_dumpstack: [ 7] [<0x4037959c>] xtensa_irq_dispatch+0x54/0x94
[CPU0] sched_dumpstack: [ 7] [<0x40378d24>] xtensa_int_decode+0x8c/0xa4
[CPU0] sched_dumpstack: [ 7] [<0x40374d2d>] _xtensa_level1_handler+0x79/0x9c
[CPU0] sched_dumpstack: [ 7] [<0x420a255c>] hostap_input+0x214/0xbfc
[CPU0] sched_dumpstack: [ 7] [<0x420a0bb0>] ap_rx_cb+0x1c/0x34
[CPU0] sched_dumpstack: [ 7] [<0x4037ffe8>] ppRxPkt+0xf0/0x128
[CPU0] sched_dumpstack: [ 7] [<0x420a0bb0>] ap_rx_cb+0x1c/0x34
[CPU0] dump_fatal_info: Dump CPU1: PAUSED
[CPU0] up_dump_register:    PC: 42069d76    PS: 00060430
[CPU0] up_dump_register:    A0: 82055aa8    A1: 3fcb8630    A2: 00000001    A3: 00000000
[CPU0] up_dump_register:    A4: 00000001    A5: 00060c22    A6: 3fca77d8    A7: 00000000
[CPU0] up_dump_register:    A8: 82069d6d    A9: 3fcb8610   A10: 0000006c   A11: 3fcb8630
[CPU0] up_dump_register:   A12: 3fca18c4   A13: 3fca77d8   A14: 00000001   A15: 3fca6004
[CPU0] up_dump_register:   SAR: 0000001f CAUSE: 40056fe7 VADDR: 00000000
[CPU0] up_dump_register:  LBEG: 40056fc5  LEND: 40056fe7  LCNT: ffffffff
[CPU0] dump_stack: User Stack:
[CPU0] dump_stack:   base: 0x3fcb7800
[CPU0] dump_stack:   size: 00004032
[CPU0] dump_stack:     sp: 0x3fcb8630
[CPU0] stack_dump: 0x3fcb8610: 000000d6 350c5280 00000001 00000024 82055b0c 3fcb8660 3fca1860 3fc998b8
[CPU0] stack_dump: 0x3fcb8630: 0000006c 3fcb8670 3fca8a18 00000000 82075d0c 3fcb8670 3fcbeff0 3fcbeff8
[CPU0] stack_dump: 0x3fcb8650: 8205567a 3fcb8680 3fc998b8 00004064 00000014 3fc991c8 3fc995bc 3fcbe950
[CPU0] stack_dump: 0x3fcb8670: 82072170 3fcb86a0 3fca1860 3fc998b8 3fca1860 3fca1860 3fc90938 00000000
[CPU0] stack_dump: 0x3fcb8690: 820718c0 3fcb86d0 3fca1860 3fc998b8 820718c0 3fcb86d0 3fca1860 3fc998b8
[CPU0] stack_dump: 0x3fcb86b0: 00000010 3fca1860 00000064 00060e22 82071971 3fcb86f0 3fca1860 420c1e04
[CPU0] stack_dump: 0x3fcb86d0: 3fca1860 00060c20 00060c20 3fcef940 82074c4c 3fcb8710 3fca1860 420750dc
[CPU0] stack_dump: 0x3fcb86f0: 3fc998b8 00000000 00000000 00000002 82074c90 3fcb8730 3fca17c0 3fca1860
[CPU0] stack_dump: 0x3fcb8710: 3fca18c4 00000000 fffffff2 3fca24a4 8204369a 3fcb8750 3fca17c0 3fc917d8
[CPU0] stack_dump: 0x3fcb8730: 3fca18c4 3fca77d8 00000001 3fca6004 82044518 3fcb8770 3fcb3434 00000000
[CPU0] stack_dump: 0x3fcb8750: 00000000 00060c22 3fca77d8 00000000 00000000 3fcb87a0 00000002 42043654
[CPU0] stack_dump: 0x3fcb8770: 00000001 00000000 00000000 00000000 00060c20 3fcb3458 3fca17c0 42074c70
[CPU0] stack_dump: 0x3fcb8790: 00000000 3fcb87c0 00000000 00000000 00050020 00000000 00000000 00000000
[CPU0] stack_dump: 0x3fcb87b0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 2] [<0x42055aa8>] tcp_get_recvwindow+0x1c/0x58
[CPU0] sched_dumpstack: [ 2] [<0x42055b0c>] tcp_should_send_recvwindow+0x28/0x50
[CPU0] sched_dumpstack: [ 2] [<0x4205567a>] tcp_appsend+0xe/0xe4
[CPU0] sched_dumpstack: [ 2] [<0x42072170>] tcp_poll+0x78/0x7c
[CPU0] sched_dumpstack: [ 2] [<0x420718c0>] devif_poll_connections+0x2c/0xac
[CPU0] sched_dumpstack: [ 2] [<0x42071971>] devif_poll+0x31/0x6c
[CPU0] sched_dumpstack: [ 2] [<0x42074c4c>] wlan_dopoll+0x2c/0x50
[CPU0] sched_dumpstack: [ 2] [<0x42074c90>] wlan_txavail_work+0x20/0x28
[CPU0] sched_dumpstack: [ 2] [<0x4204369a>] work_thread+0x46/0xb8
[CPU0] sched_dumpstack: [ 2] [<0x42044518>] nxtask_start+0x64/0x7c
[CPU0] sched_dumpstack: [ 2] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 2] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] dump_tasks:    PID GROUP   CPU PRI POLICY   TYPE    NPX STATE   EVENT      SIGMASK          STACKBASE  STACKSID
[CPU0] dump_tasks:   ----   ---     0 --- -------- ------- --- ------- ---------- ---------------- 0x3fc98218      20q
[CPU0] dump_tasks:   ----   ---     1 --- -------- ------- --- ------- ---------- ---------------- 0x3fc98a18      20q
[CPU0] dump_task:       0     0     0   0 FIFO     Kthread -   Assigned           0000000000000000 0x3fcaa150      30E
[CPU0] dump_task:       1     0     1   0 FIFO     Kthread -   Assigned           0000000000000000 0x3fcb69c0      30E
[CPU0] dump_task:       2     0     1 100 RR       Kthread -   Running            0000000000000000 0x3fcb7800      408
[CPU0] dump_task:       3     3     0 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fcb8c40      81n
[CPU0] dump_task:       4     0     0 255 RR       Kthread -   Waiting Semaphore  0000000000000000 0x3fcbafa8       6c
[CPU0] dump_task:       5     0     1 255 RR       Kthread -   Waiting Semaphore  0000000000000000 0x3fcbb380       7c
[CPU0] dump_task:       6     0     1 223 RR       Kthread -   Waiting Semaphore  0000000000000000 0x3fcbdbd8      40r
[CPU0] dump_task:       7     0     0 253 RR       Kthread -   Running            0000000000000000 0x3fcbf298      66i
[CPU0] dump_task:       9     9     1 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fcc9fb0      401
[CPU0] dump_task:      12    12     1 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fccd710      390
[CPU0] dump_task:      13    12     1 100 RR       pthread -   Assigned           0000000000000000 0x3fccb6a8      400
[CPU0] dump_task:      14    12     0 100 RR       pthread -   Waiting Signal     0000000000000000 0x3fcd26c0      400
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 0] [<0x42042ea2>] nx_start+0x2ea/0x2f0
[CPU0] sched_dumpstack: [ 0] [<0x403750ec>] __esp32s3_start+0x70/0x74
[CPU0] sched_dumpstack: [ 0] [<0x403751aa>] __start+0xba/0xbc
[CPU0] sched_dumpstack: [ 0] [<0x40045c04>] rom_phy_dig_reg_backup+0x3f7fc/0x36dd78
[CPU0] sched_dumpstack: [ 0] [<0x40043ab9>] rom_phy_dig_reg_backup+0x3d6b1/0x36dd78
[CPU0] sched_dumpstack: [ 0] [<0x40034c48>] rom_phy_dig_reg_backup+0x2e840/0x36dd78
[CPU0] sched_dumpstack: [ 0] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 1] [<0x42042f2a>] nx_idle_trampoline+0x1e/0x24
[CPU0] sched_dumpstack: [ 1] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 1] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 2] [<0x42055aa8>] tcp_get_recvwindow+0x1c/0x58
[CPU0] sched_dumpstack: [ 2] [<0x42055b0c>] tcp_should_send_recvwindow+0x28/0x50
[CPU0] sched_dumpstack: [ 2] [<0x4205567a>] tcp_appsend+0xe/0xe4
[CPU0] sched_dumpstack: [ 2] [<0x42072170>] tcp_poll+0x78/0x7c
[CPU0] sched_dumpstack: [ 2] [<0x420718c0>] devif_poll_connections+0x2c/0xac
[CPU0] sched_dumpstack: [ 2] [<0x42071971>] devif_poll+0x31/0x6c
[CPU0] sched_dumpstack: [ 2] [<0x42074c4c>] wlan_dopoll+0x2c/0x50
[CPU0] sched_dumpstack: [ 2] [<0x42074c90>] wlan_txavail_work+0x20/0x28
[CPU0] sched_dumpstack: [ 2] [<0x4204369a>] work_thread+0x46/0xb8
[CPU0] sched_dumpstack: [ 2] [<0x42044518>] nxtask_start+0x64/0x7c
[CPU0] sched_dumpstack: [ 2] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 2] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 3] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [ 3] [<0x42064c82>] uart_read+0x1b2/0x224
[CPU0] sched_dumpstack: [ 3] [<0x420589a6>] file_read+0x32/0x38
[CPU0] sched_dumpstack: [ 3] [<0x420589ca>] nx_read+0x1e/0x2c
[CPU0] sched_dumpstack: [ 3] [<0x420589e8>] read+0x10/0x24
[CPU0] sched_dumpstack: [ 3] [<0x4204c29d>] readline_getc+0x21/0x3c
[CPU0] sched_dumpstack: [ 3] [<0x4204fdc0>] readline_common+0x58/0x100
[CPU0] sched_dumpstack: [ 3] [<0x4204c2da>] readline_fd+0x22/0x28
[CPU0] sched_dumpstack: [ 3] [<0x4204be59>] nsh_session+0xe5/0x124
[CPU0] sched_dumpstack: [ 3] [<0x4204bcfe>] nsh_consolemain+0x2a/0x34
[CPU0] sched_dumpstack: [ 3] [<0x4204bcc1>] nsh_main+0x31/0x44
[CPU0] sched_dumpstack: [ 3] [<0x42047cf1>] nxtask_startup+0x21/0x28
[CPU0] sched_dumpstack: [ 3] [<0x42044528>] nxtask_start+0x74/0x7c
[CPU0] sched_dumpstack: [ 3] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 3] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 4] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [ 4] [<0x4037920a>] spi_flash_op_block_task+0x56/0x98
[CPU0] sched_dumpstack: [ 4] [<0x42044518>] nxtask_start+0x64/0x7c
[CPU0] sched_dumpstack: [ 4] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 4] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 5] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [ 5] [<0x4037920a>] spi_flash_op_block_task+0x56/0x98
[CPU0] sched_dumpstack: [ 5] [<0x42044518>] nxtask_start+0x64/0x7c
[CPU0] sched_dumpstack: [ 5] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 5] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 6] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [ 6] [<0x42043e61>] nxsem_wait_uninterruptible+0xd/0x14
[CPU0] sched_dumpstack: [ 6] [<0x4204b8dc>] rt_timer_thread+0xc/0xc0
[CPU0] sched_dumpstack: [ 6] [<0x42044518>] nxtask_start+0x64/0x7c
[CPU0] sched_dumpstack: [ 6] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 6] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 7] [<0x42066191>] sched_dumpstack+0x15/0x50
[CPU0] sched_dumpstack: [ 7] [<0x403797e0>] dump_backtrace+0xc/0x10
[CPU0] sched_dumpstack: [ 7] [<0x42063289>] nxsched_foreach+0x29/0x48
[CPU0] sched_dumpstack: [ 7] [<0x40379da1>] _assert+0x399/0x3dc
[CPU0] sched_dumpstack: [ 7] [<0x4204791d>] __assert+0x11/0x18
[CPU0] sched_dumpstack: [ 7] [<0x42044f83>] nxsched_remove_readytorun+0x4b/0xc0
[CPU0] sched_dumpstack: [ 7] [<0x420455cd>] nxsched_reprioritize_rtr+0xd/0x34
[CPU0] sched_dumpstack: [ 7] [<0x42063ee5>] nxsched_process_roundrobin+0x6d/0x90
[CPU0] sched_dumpstack: [ 7] [<0x42063598>] nxsched_process_timer+0x44/0x5c
[CPU0] sched_dumpstack: [ 7] [<0x4204adca>] systimer_isr+0x16/0x1c
[CPU0] sched_dumpstack: [ 7] [<0x4037a09e>] irq_dispatch+0x2a/0x2c
[CPU0] sched_dumpstack: [ 7] [<0x4037959c>] xtensa_irq_dispatch+0x54/0x94
[CPU0] sched_dumpstack: [ 7] [<0x40378d24>] xtensa_int_decode+0x8c/0xa4
[CPU0] sched_dumpstack: [ 7] [<0x40374d2d>] _xtensa_level1_handler+0x79/0x9c
[CPU0] sched_dumpstack: [ 7] [<0x420a255c>] hostap_input+0x214/0xbfc
[CPU0] sched_dumpstack: [ 7] [<0x420a0bb0>] ap_rx_cb+0x1c/0x34
[CPU0] sched_dumpstack: [ 7] [<0x420a0bb0>] ap_rx_cb+0x1c/0x34
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 9] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [ 9] [<0x42056478>] _net_timedwait+0x40/0x5c
[CPU0] sched_dumpstack: [ 9] [<0x420564a4>] net_sem_timedwait+0x10/0x14
[CPU0] sched_dumpstack: [ 9] [<0x420705cc>] psock_udp_recvfrom+0x174/0x1ac
[CPU0] sched_dumpstack: [ 9] [<0x4206e896>] inet_recvmsg+0x52/0x58
[CPU0] sched_dumpstack: [ 9] [<0x420546e5>] psock_recvmsg+0x49/0x68
[CPU0] sched_dumpstack: [ 9] [<0x420545d0>] psock_recvfrom+0x30/0x40
[CPU0] sched_dumpstack: [ 9] [<0x42054606>] recvfrom+0x26/0x44
[CPU0] sched_dumpstack: [ 9] [<0x42054599>] recv+0x15/0x1c
[CPU0] sched_dumpstack: [ 9] [<0x4206a805>] dhcpd_run+0xe9/0x60c
[CPU0] sched_dumpstack: [ 9] [<0x4206ad34>] dhcpd_task_run+0xc/0x10
[CPU0] sched_dumpstack: [ 9] [<0x42047cf1>] nxtask_startup+0x21/0x28
[CPU0] sched_dumpstack: [ 9] [<0x42044528>] nxtask_start+0x74/0x7c
[CPU0] sched_dumpstack: [ 9] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 9] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [12] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [12] [<0x42043e61>] nxsem_wait_uninterruptible+0xd/0x14
[CPU0] sched_dumpstack: [12] [<0x42063145>] pthread_join+0x91/0xb0
[CPU0] sched_dumpstack: [12] [<0x4206da50>] iperf_start+0xe4/0x12c
[CPU0] sched_dumpstack: [12] [<0x4206b191>] iperf_main+0x359/0x368
[CPU0] sched_dumpstack: [12] [<0x42047cf1>] nxtask_startup+0x21/0x28
[CPU0] sched_dumpstack: [12] [<0x42044528>] nxtask_start+0x74/0x7c
[CPU0] sched_dumpstack: [12] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [12] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [13] [<0x4205785d>] fs_putfilep+0xb5/0xc8
[CPU0] sched_dumpstack: [13] [<0x42054610>] recvfrom+0x30/0x44
[CPU0] sched_dumpstack: [13] [<0x42054599>] recv+0x15/0x1c
[CPU0] sched_dumpstack: [13] [<0x4206d449>] iperf_tcp_server+0xd5/0x13c
[CPU0] sched_dumpstack: [13] [<0x4206d291>] iperf_run_server$isra$0+0x75/0x78
[CPU0] sched_dumpstack: [13] [<0x4206d2e4>] iperf_task_traffic+0x50/0x88
[CPU0] sched_dumpstack: [13] [<0x420660f8>] pthread_startup+0x18/0x20
[CPU0] sched_dumpstack: [13] [<0x420732f4>] pthread_start+0x58/0x64
[CPU0] sched_dumpstack: [13] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [13] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [14] [<0x42047724>] nxsig_timedwait+0x150/0x218
[CPU0] sched_dumpstack: [14] [<0x4204726d>] nxsig_nanosleep+0x4d/0xc4
[CPU0] sched_dumpstack: [14] [<0x42047360>] clock_nanosleep+0x7c/0x88
[CPU0] sched_dumpstack: [14] [<0x42068430>] sleep+0x1c/0x34
[CPU0] sched_dumpstack: [14] [<0x4206d7d2>] iperf_report_task+0x56/0x1f0
[CPU0] sched_dumpstack: [14] [<0x420660f8>] pthread_startup+0x18/0x20
[CPU0] sched_dumpstack: [14] [<0x420732f4>] pthread_start+0x58/0x64
[CPU0] sched_dumpstack: [14] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [14] [<0x40000000>] Unknown+0x40000000/0x40001c5

The client (PC):

iperf -c 10.0.0.1 -p 11111 -i 1 -t 600

Update firmware archives:
nuttx.zip

hujun260 · 2024-11-01T13:44:52Z

you could add following config to print crash backtrace when trigger assert
CONFIG_SCHED_BACKTRACE=y
CONFIG_SYSTEM_DUMPSTACK=y
CONFIG_ALLSYMS=y
CONFIG_ARCH_HAVE_BACKTRACE=y

See below.

i can start the server. But how can I reproduce crash?

Use the device as a Wi-Fi AP and connect your PC to it. Run the server on the device and the client on PC:

nsh> 
nsh> ifup wlan1
ifup wlan1...OK
nsh> dhcpd_start wlan1
nsh> wapi psk wlan1 mypasswd 3
nsh> wapi essid wlan1 myssid 1
nsh> iperf -s -p 11111 -i 5 -B 0.0.0.0 -t 600 &
iperf [12:100]
nsh>      IP: 0.0.0.0

 mode=tcp-server sip=0.0.0.0:11111,dip=0.0.0.0:5001, interval=5, time=600
accept: 10.0.0.2:34022

           Interval         Transfer         Bandwidth

   0.00-   5.02 sec    2549304 Bytes    4.06 Mbits/sec
[CPU0] dump_assert_info: Current Version: NuttX  10.4.0 31a3cea64a Nov  1 2024 09:57:23 xtensa
[CPU0] dump_assert_info: Assertion failed tcb->cpu == up_cpu_index(): at file: sched/sched_removereadytorun.c:293 tas8
[CPU0] up_dump_register:    PC: 4206a1ee    PS: 00060722
[CPU0] up_dump_register:    A0: 80379b89    A1: 3fc987e8    A2: 3fcc09d0    A3: 3fc9a9f0
[CPU0] up_dump_register:    A4: 00060722    A5: 00000000    A6: 3fc987c4    A7: 00000001
[CPU0] up_dump_register:    A8: 3fc995b8    A9: 3fc987d8   A10: 3fc995b8   A11: 00000001
[CPU0] up_dump_register:   A12: 00000001   A13: 00060522   A14: 00000001   A15: 00000001
[CPU0] up_dump_register:   SAR: 0000001f CAUSE: 00000000 VADDR: 00000000
[CPU0] up_dump_register:  LBEG: 400570e8  LEND: 400570f3  LCNT: 00000000
[CPU0] dump_stack: IRQ Stack:
[CPU0] dump_stack:   base: 0x3fc98218
[CPU0] dump_stack:   size: 00002048
[CPU0] dump_stack:     sp: 0x3fc987e8
[CPU0] stack_dump: 0x3fc987c8: 8204791d 3fc98808 3c014818 3fcbf1a8 000019d8 3fcc0c70 3c014818 3fcbf1a8
[CPU0] stack_dump: 0x3fc987e8: 3fc98218 3fc987e8 00000000 00000000 82044f83 3fc988f8 3c014818 00000125
[CPU0] stack_dump: 0x3fc98808: 00000000 3fcbf254 3c01adaf 4037e548 7474754e dead0058 deadbeef deadbeef
[CPU0] stack_dump: 0x3fc98828: deadbeef dead00ef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
[CPU0] stack_dump: 0x3fc98848: deadbeef 2e303100 00302e34 deadbeef deadbeef deadbeef 3133beef 65633361
[CPU0] stack_dump: 0x3fc98868: 61343661 766f4e20 20312020 34323032 3a393020 323a3735 3fc90033 ffffffff
[CPU0] stack_dump: 0x3fc98888: 00000001 deadbeef deadbeef 657478ef 0061736e deadbeef deadbeef deadbeef
[CPU0] stack_dump: 0x3fc988a8: 00000003 3fcbf1a8 3fc9a9f0 3c01489f 3c014818 00000125 00000002 deadbeef
[CPU0] stack_dump: 0x3fc988c8: 00000006 00060d22 00000125 00000000 3c01489f 3fc9a9f0 3c01af87 4037e548
[CPU0] stack_dump: 0x3fc988e8: 820455cd 3fc98918 3fcb76f0 00000001 3c01489f 00000004 3fc981f8 0000001f
[CPU0] stack_dump: 0x3fc98908: 82063ee5 3fc98938 3fcb76f0 00000064 420a255c 3fc98a18 00000001 3fcc0b80
[CPU0] stack_dump: 0x3fc98928: 82063598 3fc98958 3fc90930 3fcbf1a8 00000001 3fcc0be0 3fcd36b8 ffffffe7
[CPU0] stack_dump: 0x3fc98948: 8204adca 3fc98978 00060023 0000820a 00060022 3fcb76f0 3fcd36b8 420a0b94
[CPU0] stack_dump: 0x3fc98968: 8037a09e 3fc98998 0000003e 3fcc09d0 00000001 3fc917c8 00060023 00000000
[CPU0] stack_dump: 0x3fc98988: 8037959c 3fc989b8 0000003e 3fcc09d0 00000000 3fcc0b00 00000000 3fca17dc
[CPU0] stack_dump: 0x3fc989a8: 80378d24 3fc989d8 0000003e 3fcc09d0 00000006 00000051 00000006 3fca77c0
[CPU0] stack_dump: 0x3fc989c8: 40374d2d 3fc989f8 00000004 3fcc09d0 3fc995b8 3fc90930 3fcbf1a8 3fccc6a4
[CPU0] stack_dump: 0x3fc989e8: 420a255c 3fc98a18 00000001 3fcc0b80 00000002 00000004 3fc981f8 0000001f
[CPU0] stack_dump: 0x3fc98a08: 820a0bb0 3fcc0b80 00000000 00000000 00000000 00000000 00000000 00000000
[CPU0] dump_stack: User Stack:
[CPU0] dump_stack:   base: 0x3fcbf298
[CPU0] dump_stack:   size: 00006616
[CPU0] dump_stack:     sp: 0x3fcc0b80
[CPU0] stack_dump: 0x3fcc0b60: 3fca83b4 0000001a 000005ea 00000c26 8037ffe8 3fcc0be0 3fcd36b8 ffffffe7
[CPU0] stack_dump: 0x3fcc0b80: 3bb3b668 000069b0 3fcbf148 00000003 3fcc77f0 00000612 00000001 00000088
[CPU0] stack_dump: 0x3fcc0ba0: 00000001 3fcd3752 ffffffe7 00000000 00000001 3fccc6a0 42075d00 3fca9f7c
[CPU0] stack_dump: 0x3fcc0bc0: 3fccca58 00000008 00000000 3fcccda8 8037e616 3fcc0c00 3fcd36b8 420a0b94
[CPU0] stack_dump: 0x3fcc0be0: 00000000 3fcef944 3fcbebe8 00000002 82044518 3fcc0c20 3fcef940 3fcef904
[CPU0] stack_dump: 0x3fcc0c00: 00000010 3fcef920 3fcef940 3fcc2bac 00000000 3fcc0c50 00000002 4037e548
[CPU0] stack_dump: 0x3fcc0c20: 00000011 00000000 00000000 00000000 00000000 4037e613 3fc90700 000000b3
[CPU0] stack_dump: 0x3fcc0c40: 00000000 3fcc0c70 00000000 00000000 00050020 00000000 00000000 00000000
[CPU0] stack_dump: 0x3fcc0c60: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[CPU0] sched_dumpstack: [ 7] [<0x42066191>] sched_dumpstack+0x15/0x50
[CPU0] sched_dumpstack: [ 7] [<0x403798d1>] dump_running_task+0xed/0xf0
[CPU0] sched_dumpstack: [ 7] [<0x40379c91>] _assert+0x289/0x3dc
[CPU0] sched_dumpstack: [ 7] [<0x4204791d>] __assert+0x11/0x18
[CPU0] sched_dumpstack: [ 7] [<0x42044f83>] nxsched_remove_readytorun+0x4b/0xc0
[CPU0] sched_dumpstack: [ 7] [<0x420455cd>] nxsched_reprioritize_rtr+0xd/0x34
[CPU0] sched_dumpstack: [ 7] [<0x42063ee5>] nxsched_process_roundrobin+0x6d/0x90
[CPU0] sched_dumpstack: [ 7] [<0x42063598>] nxsched_process_timer+0x44/0x5c
[CPU0] sched_dumpstack: [ 7] [<0x4204adca>] systimer_isr+0x16/0x1c
[CPU0] sched_dumpstack: [ 7] [<0x4037a09e>] irq_dispatch+0x2a/0x2c
[CPU0] sched_dumpstack: [ 7] [<0x4037959c>] xtensa_irq_dispatch+0x54/0x94
[CPU0] sched_dumpstack: [ 7] [<0x40378d24>] xtensa_int_decode+0x8c/0xa4
[CPU0] sched_dumpstack: [ 7] [<0x40374d2d>] _xtensa_level1_handler+0x79/0x9c
[CPU0] sched_dumpstack: [ 7] [<0x420a255c>] hostap_input+0x214/0xbfc
[CPU0] sched_dumpstack: [ 7] [<0x420a0bb0>] ap_rx_cb+0x1c/0x34
[CPU0] sched_dumpstack: [ 7] [<0x4037ffe8>] ppRxPkt+0xf0/0x128
[CPU0] sched_dumpstack: [ 7] [<0x420a0bb0>] ap_rx_cb+0x1c/0x34
[CPU0] dump_fatal_info: Dump CPU1: PAUSED
[CPU0] up_dump_register:    PC: 42069d76    PS: 00060430
[CPU0] up_dump_register:    A0: 82055aa8    A1: 3fcb8630    A2: 00000001    A3: 00000000
[CPU0] up_dump_register:    A4: 00000001    A5: 00060c22    A6: 3fca77d8    A7: 00000000
[CPU0] up_dump_register:    A8: 82069d6d    A9: 3fcb8610   A10: 0000006c   A11: 3fcb8630
[CPU0] up_dump_register:   A12: 3fca18c4   A13: 3fca77d8   A14: 00000001   A15: 3fca6004
[CPU0] up_dump_register:   SAR: 0000001f CAUSE: 40056fe7 VADDR: 00000000
[CPU0] up_dump_register:  LBEG: 40056fc5  LEND: 40056fe7  LCNT: ffffffff
[CPU0] dump_stack: User Stack:
[CPU0] dump_stack:   base: 0x3fcb7800
[CPU0] dump_stack:   size: 00004032
[CPU0] dump_stack:     sp: 0x3fcb8630
[CPU0] stack_dump: 0x3fcb8610: 000000d6 350c5280 00000001 00000024 82055b0c 3fcb8660 3fca1860 3fc998b8
[CPU0] stack_dump: 0x3fcb8630: 0000006c 3fcb8670 3fca8a18 00000000 82075d0c 3fcb8670 3fcbeff0 3fcbeff8
[CPU0] stack_dump: 0x3fcb8650: 8205567a 3fcb8680 3fc998b8 00004064 00000014 3fc991c8 3fc995bc 3fcbe950
[CPU0] stack_dump: 0x3fcb8670: 82072170 3fcb86a0 3fca1860 3fc998b8 3fca1860 3fca1860 3fc90938 00000000
[CPU0] stack_dump: 0x3fcb8690: 820718c0 3fcb86d0 3fca1860 3fc998b8 820718c0 3fcb86d0 3fca1860 3fc998b8
[CPU0] stack_dump: 0x3fcb86b0: 00000010 3fca1860 00000064 00060e22 82071971 3fcb86f0 3fca1860 420c1e04
[CPU0] stack_dump: 0x3fcb86d0: 3fca1860 00060c20 00060c20 3fcef940 82074c4c 3fcb8710 3fca1860 420750dc
[CPU0] stack_dump: 0x3fcb86f0: 3fc998b8 00000000 00000000 00000002 82074c90 3fcb8730 3fca17c0 3fca1860
[CPU0] stack_dump: 0x3fcb8710: 3fca18c4 00000000 fffffff2 3fca24a4 8204369a 3fcb8750 3fca17c0 3fc917d8
[CPU0] stack_dump: 0x3fcb8730: 3fca18c4 3fca77d8 00000001 3fca6004 82044518 3fcb8770 3fcb3434 00000000
[CPU0] stack_dump: 0x3fcb8750: 00000000 00060c22 3fca77d8 00000000 00000000 3fcb87a0 00000002 42043654
[CPU0] stack_dump: 0x3fcb8770: 00000001 00000000 00000000 00000000 00060c20 3fcb3458 3fca17c0 42074c70
[CPU0] stack_dump: 0x3fcb8790: 00000000 3fcb87c0 00000000 00000000 00050020 00000000 00000000 00000000
[CPU0] stack_dump: 0x3fcb87b0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 2] [<0x42055aa8>] tcp_get_recvwindow+0x1c/0x58
[CPU0] sched_dumpstack: [ 2] [<0x42055b0c>] tcp_should_send_recvwindow+0x28/0x50
[CPU0] sched_dumpstack: [ 2] [<0x4205567a>] tcp_appsend+0xe/0xe4
[CPU0] sched_dumpstack: [ 2] [<0x42072170>] tcp_poll+0x78/0x7c
[CPU0] sched_dumpstack: [ 2] [<0x420718c0>] devif_poll_connections+0x2c/0xac
[CPU0] sched_dumpstack: [ 2] [<0x42071971>] devif_poll+0x31/0x6c
[CPU0] sched_dumpstack: [ 2] [<0x42074c4c>] wlan_dopoll+0x2c/0x50
[CPU0] sched_dumpstack: [ 2] [<0x42074c90>] wlan_txavail_work+0x20/0x28
[CPU0] sched_dumpstack: [ 2] [<0x4204369a>] work_thread+0x46/0xb8
[CPU0] sched_dumpstack: [ 2] [<0x42044518>] nxtask_start+0x64/0x7c
[CPU0] sched_dumpstack: [ 2] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 2] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] dump_tasks:    PID GROUP   CPU PRI POLICY   TYPE    NPX STATE   EVENT      SIGMASK          STACKBASE  STACKSID
[CPU0] dump_tasks:   ----   ---     0 --- -------- ------- --- ------- ---------- ---------------- 0x3fc98218      20q
[CPU0] dump_tasks:   ----   ---     1 --- -------- ------- --- ------- ---------- ---------------- 0x3fc98a18      20q
[CPU0] dump_task:       0     0     0   0 FIFO     Kthread -   Assigned           0000000000000000 0x3fcaa150      30E
[CPU0] dump_task:       1     0     1   0 FIFO     Kthread -   Assigned           0000000000000000 0x3fcb69c0      30E
[CPU0] dump_task:       2     0     1 100 RR       Kthread -   Running            0000000000000000 0x3fcb7800      408
[CPU0] dump_task:       3     3     0 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fcb8c40      81n
[CPU0] dump_task:       4     0     0 255 RR       Kthread -   Waiting Semaphore  0000000000000000 0x3fcbafa8       6c
[CPU0] dump_task:       5     0     1 255 RR       Kthread -   Waiting Semaphore  0000000000000000 0x3fcbb380       7c
[CPU0] dump_task:       6     0     1 223 RR       Kthread -   Waiting Semaphore  0000000000000000 0x3fcbdbd8      40r
[CPU0] dump_task:       7     0     0 253 RR       Kthread -   Running            0000000000000000 0x3fcbf298      66i
[CPU0] dump_task:       9     9     1 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fcc9fb0      401
[CPU0] dump_task:      12    12     1 100 RR       Task    -   Waiting Semaphore  0000000000000000 0x3fccd710      390
[CPU0] dump_task:      13    12     1 100 RR       pthread -   Assigned           0000000000000000 0x3fccb6a8      400
[CPU0] dump_task:      14    12     0 100 RR       pthread -   Waiting Signal     0000000000000000 0x3fcd26c0      400
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 0] [<0x42042ea2>] nx_start+0x2ea/0x2f0
[CPU0] sched_dumpstack: [ 0] [<0x403750ec>] __esp32s3_start+0x70/0x74
[CPU0] sched_dumpstack: [ 0] [<0x403751aa>] __start+0xba/0xbc
[CPU0] sched_dumpstack: [ 0] [<0x40045c04>] rom_phy_dig_reg_backup+0x3f7fc/0x36dd78
[CPU0] sched_dumpstack: [ 0] [<0x40043ab9>] rom_phy_dig_reg_backup+0x3d6b1/0x36dd78
[CPU0] sched_dumpstack: [ 0] [<0x40034c48>] rom_phy_dig_reg_backup+0x2e840/0x36dd78
[CPU0] sched_dumpstack: [ 0] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 1] [<0x42042f2a>] nx_idle_trampoline+0x1e/0x24
[CPU0] sched_dumpstack: [ 1] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 1] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 2] [<0x42055aa8>] tcp_get_recvwindow+0x1c/0x58
[CPU0] sched_dumpstack: [ 2] [<0x42055b0c>] tcp_should_send_recvwindow+0x28/0x50
[CPU0] sched_dumpstack: [ 2] [<0x4205567a>] tcp_appsend+0xe/0xe4
[CPU0] sched_dumpstack: [ 2] [<0x42072170>] tcp_poll+0x78/0x7c
[CPU0] sched_dumpstack: [ 2] [<0x420718c0>] devif_poll_connections+0x2c/0xac
[CPU0] sched_dumpstack: [ 2] [<0x42071971>] devif_poll+0x31/0x6c
[CPU0] sched_dumpstack: [ 2] [<0x42074c4c>] wlan_dopoll+0x2c/0x50
[CPU0] sched_dumpstack: [ 2] [<0x42074c90>] wlan_txavail_work+0x20/0x28
[CPU0] sched_dumpstack: [ 2] [<0x4204369a>] work_thread+0x46/0xb8
[CPU0] sched_dumpstack: [ 2] [<0x42044518>] nxtask_start+0x64/0x7c
[CPU0] sched_dumpstack: [ 2] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 2] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 3] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [ 3] [<0x42064c82>] uart_read+0x1b2/0x224
[CPU0] sched_dumpstack: [ 3] [<0x420589a6>] file_read+0x32/0x38
[CPU0] sched_dumpstack: [ 3] [<0x420589ca>] nx_read+0x1e/0x2c
[CPU0] sched_dumpstack: [ 3] [<0x420589e8>] read+0x10/0x24
[CPU0] sched_dumpstack: [ 3] [<0x4204c29d>] readline_getc+0x21/0x3c
[CPU0] sched_dumpstack: [ 3] [<0x4204fdc0>] readline_common+0x58/0x100
[CPU0] sched_dumpstack: [ 3] [<0x4204c2da>] readline_fd+0x22/0x28
[CPU0] sched_dumpstack: [ 3] [<0x4204be59>] nsh_session+0xe5/0x124
[CPU0] sched_dumpstack: [ 3] [<0x4204bcfe>] nsh_consolemain+0x2a/0x34
[CPU0] sched_dumpstack: [ 3] [<0x4204bcc1>] nsh_main+0x31/0x44
[CPU0] sched_dumpstack: [ 3] [<0x42047cf1>] nxtask_startup+0x21/0x28
[CPU0] sched_dumpstack: [ 3] [<0x42044528>] nxtask_start+0x74/0x7c
[CPU0] sched_dumpstack: [ 3] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 3] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 4] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [ 4] [<0x4037920a>] spi_flash_op_block_task+0x56/0x98
[CPU0] sched_dumpstack: [ 4] [<0x42044518>] nxtask_start+0x64/0x7c
[CPU0] sched_dumpstack: [ 4] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 4] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 5] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [ 5] [<0x4037920a>] spi_flash_op_block_task+0x56/0x98
[CPU0] sched_dumpstack: [ 5] [<0x42044518>] nxtask_start+0x64/0x7c
[CPU0] sched_dumpstack: [ 5] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 5] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 6] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [ 6] [<0x42043e61>] nxsem_wait_uninterruptible+0xd/0x14
[CPU0] sched_dumpstack: [ 6] [<0x4204b8dc>] rt_timer_thread+0xc/0xc0
[CPU0] sched_dumpstack: [ 6] [<0x42044518>] nxtask_start+0x64/0x7c
[CPU0] sched_dumpstack: [ 6] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 6] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 7] [<0x42066191>] sched_dumpstack+0x15/0x50
[CPU0] sched_dumpstack: [ 7] [<0x403797e0>] dump_backtrace+0xc/0x10
[CPU0] sched_dumpstack: [ 7] [<0x42063289>] nxsched_foreach+0x29/0x48
[CPU0] sched_dumpstack: [ 7] [<0x40379da1>] _assert+0x399/0x3dc
[CPU0] sched_dumpstack: [ 7] [<0x4204791d>] __assert+0x11/0x18
[CPU0] sched_dumpstack: [ 7] [<0x42044f83>] nxsched_remove_readytorun+0x4b/0xc0
[CPU0] sched_dumpstack: [ 7] [<0x420455cd>] nxsched_reprioritize_rtr+0xd/0x34
[CPU0] sched_dumpstack: [ 7] [<0x42063ee5>] nxsched_process_roundrobin+0x6d/0x90
[CPU0] sched_dumpstack: [ 7] [<0x42063598>] nxsched_process_timer+0x44/0x5c
[CPU0] sched_dumpstack: [ 7] [<0x4204adca>] systimer_isr+0x16/0x1c
[CPU0] sched_dumpstack: [ 7] [<0x4037a09e>] irq_dispatch+0x2a/0x2c
[CPU0] sched_dumpstack: [ 7] [<0x4037959c>] xtensa_irq_dispatch+0x54/0x94
[CPU0] sched_dumpstack: [ 7] [<0x40378d24>] xtensa_int_decode+0x8c/0xa4
[CPU0] sched_dumpstack: [ 7] [<0x40374d2d>] _xtensa_level1_handler+0x79/0x9c
[CPU0] sched_dumpstack: [ 7] [<0x420a255c>] hostap_input+0x214/0xbfc
[CPU0] sched_dumpstack: [ 7] [<0x420a0bb0>] ap_rx_cb+0x1c/0x34
[CPU0] sched_dumpstack: [ 7] [<0x420a0bb0>] ap_rx_cb+0x1c/0x34
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [ 9] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [ 9] [<0x42056478>] _net_timedwait+0x40/0x5c
[CPU0] sched_dumpstack: [ 9] [<0x420564a4>] net_sem_timedwait+0x10/0x14
[CPU0] sched_dumpstack: [ 9] [<0x420705cc>] psock_udp_recvfrom+0x174/0x1ac
[CPU0] sched_dumpstack: [ 9] [<0x4206e896>] inet_recvmsg+0x52/0x58
[CPU0] sched_dumpstack: [ 9] [<0x420546e5>] psock_recvmsg+0x49/0x68
[CPU0] sched_dumpstack: [ 9] [<0x420545d0>] psock_recvfrom+0x30/0x40
[CPU0] sched_dumpstack: [ 9] [<0x42054606>] recvfrom+0x26/0x44
[CPU0] sched_dumpstack: [ 9] [<0x42054599>] recv+0x15/0x1c
[CPU0] sched_dumpstack: [ 9] [<0x4206a805>] dhcpd_run+0xe9/0x60c
[CPU0] sched_dumpstack: [ 9] [<0x4206ad34>] dhcpd_task_run+0xc/0x10
[CPU0] sched_dumpstack: [ 9] [<0x42047cf1>] nxtask_startup+0x21/0x28
[CPU0] sched_dumpstack: [ 9] [<0x42044528>] nxtask_start+0x74/0x7c
[CPU0] sched_dumpstack: [ 9] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [ 9] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [12] [<0x42043e3d>] nxsem_wait+0xf5/0x10c
[CPU0] sched_dumpstack: [12] [<0x42043e61>] nxsem_wait_uninterruptible+0xd/0x14
[CPU0] sched_dumpstack: [12] [<0x42063145>] pthread_join+0x91/0xb0
[CPU0] sched_dumpstack: [12] [<0x4206da50>] iperf_start+0xe4/0x12c
[CPU0] sched_dumpstack: [12] [<0x4206b191>] iperf_main+0x359/0x368
[CPU0] sched_dumpstack: [12] [<0x42047cf1>] nxtask_startup+0x21/0x28
[CPU0] sched_dumpstack: [12] [<0x42044528>] nxtask_start+0x74/0x7c
[CPU0] sched_dumpstack: [12] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [12] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [13] [<0x4205785d>] fs_putfilep+0xb5/0xc8
[CPU0] sched_dumpstack: [13] [<0x42054610>] recvfrom+0x30/0x44
[CPU0] sched_dumpstack: [13] [<0x42054599>] recv+0x15/0x1c
[CPU0] sched_dumpstack: [13] [<0x4206d449>] iperf_tcp_server+0xd5/0x13c
[CPU0] sched_dumpstack: [13] [<0x4206d291>] iperf_run_server$isra$0+0x75/0x78
[CPU0] sched_dumpstack: [13] [<0x4206d2e4>] iperf_task_traffic+0x50/0x88
[CPU0] sched_dumpstack: [13] [<0x420660f8>] pthread_startup+0x18/0x20
[CPU0] sched_dumpstack: [13] [<0x420732f4>] pthread_start+0x58/0x64
[CPU0] sched_dumpstack: [13] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [13] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: backtrace:
[CPU0] sched_dumpstack: [14] [<0x42047724>] nxsig_timedwait+0x150/0x218
[CPU0] sched_dumpstack: [14] [<0x4204726d>] nxsig_nanosleep+0x4d/0xc4
[CPU0] sched_dumpstack: [14] [<0x42047360>] clock_nanosleep+0x7c/0x88
[CPU0] sched_dumpstack: [14] [<0x42068430>] sleep+0x1c/0x34
[CPU0] sched_dumpstack: [14] [<0x4206d7d2>] iperf_report_task+0x56/0x1f0
[CPU0] sched_dumpstack: [14] [<0x420660f8>] pthread_startup+0x18/0x20
[CPU0] sched_dumpstack: [14] [<0x420732f4>] pthread_start+0x58/0x64
[CPU0] sched_dumpstack: [14] [<0x40000000>] Unknown+0x40000000/0x40001c50
[CPU0] sched_dumpstack: [14] [<0x40000000>] Unknown+0x40000000/0x40001c5

The client (PC):

iperf -c 10.0.0.1 -p 11111 -i 1 -t 600

Update firmware archives: nuttx.zip

Thank you very much. I roughly understand the cause and am considering a solution.

reason: If we need to handle a tcb that is running on another CPU, we need to process it through the smpcall method. This commit fixes the regression from apache#13863 Signed-off-by: hujun5 <hujun5@xiaomi.com>

hujun260 · 2024-11-03T03:10:07Z

@tmedicci
#14611
Please help me verify the fix.

reason: If we need to handle a tcb that is running on another CPU, we need to process it through the smpcall method. This commit fixes the regression from apache#13863 Signed-off-by: hujun5 <hujun5@xiaomi.com>

reason: If we need to handle a tcb that is running on another CPU, we need to process it through the smpcall method. This commit fixes the regression from #13863 Signed-off-by: hujun5 <hujun5@xiaomi.com>

reason: If we need to handle a tcb that is running on another CPU, we need to process it through the smpcall method. This commit fixes the regression from apache#13863 Signed-off-by: hujun5 <hujun5@xiaomi.com>

hujun260 force-pushed the apache_14 branch from 6e27949 to 3a35c0a Compare October 7, 2024 07:55

hujun260 force-pushed the apache_14 branch from 3a35c0a to 3bcc7d8 Compare October 7, 2024 08:53

xiaoxiang781216 requested review from anchao, masayuki2009 and patacongo October 7, 2024 09:36

xiaoxiang781216 reviewed Oct 7, 2024

View reviewed changes

arch/arm/src/rp2040/rp2040_cpupause.c Show resolved Hide resolved

arch/arm/src/cxd56xx/cxd56_cpupause.c Show resolved Hide resolved

arch/arm/include/sam34/sam4cm_irq.h Outdated Show resolved Hide resolved

arch/arm/include/sam34/sam4cm_irq.h Outdated Show resolved Hide resolved

hujun260 force-pushed the apache_14 branch from 3bcc7d8 to 53e5228 Compare October 7, 2024 13:09

xiaoxiang781216 reviewed Oct 7, 2024

View reviewed changes

hujun260 added 2 commits October 7, 2024 22:37

xtensa: Replace the implementation of up_cpu_pause

4a64cdc

Signed-off-by: hujun5 <hujun5@xiaomi.com>

hujun260 force-pushed the apache_14 branch from 53e5228 to 01d4926 Compare October 7, 2024 14:43

arch: rename xxxx_pause.c to xxxx_smpcall.c

ad8120b

Signed-off-by: hujun5 <hujun5@xiaomi.com>

hujun260 force-pushed the apache_14 branch from 01d4926 to ad8120b Compare October 7, 2024 14:47

xiaoxiang781216 approved these changes Oct 7, 2024

View reviewed changes

xiaoxiang781216 merged commit 31a3cea into apache:master Oct 8, 2024

hujun260 deleted the apache_14 branch October 10, 2024 05:32

jerpelea mentioned this pull request Oct 10, 2024

{bp-sim/arch} sim: fix sim smp boot regression #14066

Merged

hujun260 mentioned this pull request Nov 3, 2024

fix regression caused by remove sync pause #14611

Merged

sumpfralle mentioned this pull request Apr 14, 2025

[BUG] rp2040: up_cpu_pause and up_cpu_resume are referenced (these functions were removed before) #16203

Closed

1 task

remove up_cpu_pause up_cpu_resume up_cpu_paused up_cpu_pausereq #13863

remove up_cpu_pause up_cpu_resume up_cpu_paused up_cpu_pausereq #13863

Uh oh!

Conversation

hujun260 commented Oct 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Impact

Testing

Uh oh!

github-actions bot commented Oct 7, 2024

Uh oh!

github-actions bot commented Oct 7, 2024

Uh oh!

github-actions bot commented Oct 7, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 7, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 7, 2024

Uh oh!

github-actions bot commented Oct 7, 2024

Uh oh!

masayuki2009 commented Oct 8, 2024

Uh oh!

tmedicci commented Oct 30, 2024

Uh oh!

patacongo commented Oct 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hujun260 commented Oct 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tmedicci commented Oct 31, 2024

Uh oh!

tmedicci commented Oct 31, 2024

Uh oh!

patacongo commented Oct 31, 2024

Uh oh!

hujun260 commented Nov 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hujun260 commented Nov 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tmedicci commented Nov 1, 2024

Uh oh!

hujun260 commented Nov 1, 2024

Uh oh!

hujun260 commented Nov 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

hujun260 commented Oct 7, 2024 •

edited

Loading

patacongo commented Oct 30, 2024 •

edited

Loading

hujun260 commented Oct 31, 2024 •

edited

Loading

hujun260 commented Nov 1, 2024 •

edited

Loading

hujun260 commented Nov 1, 2024 •

edited

Loading