LLEXT: don't reload upon resume #10028

lyakh · 2025-05-23T15:14:49Z

Resuming after D3 currently reloads all LLEXT modules / libraries, which wastes time and can be avoided by re-using data stored in DRAM. This PR adds that functionality. It depends on zephyrproject-rtos/zephyr#90400 and also needs the Linux driver to avoid re-sending modules. @ujfalusi I used a simple diff for testing:

diff --git a/sound/soc/sof/intel/hda-loader.c b/sound/soc/sof/intel/hda-loader.c
index 49085ca7b46b..4dbf41b73e4d 100644
--- a/sound/soc/sof/intel/hda-loader.c
+++ b/sound/soc/sof/intel/hda-loader.c
@@ -573,14 +573,13 @@ int hda_dsp_ipc4_load_library(struct snd_sof_dev *sdev,
 			      struct sof_ipc4_fw_library *fw_lib, bool reload)
 {
 	struct sof_intel_hda_dev *hda = sdev->pdata->hw_pdata;
-	struct sof_ipc4_fw_data *ipc4_data = sdev->private;
 	struct hdac_ext_stream *hext_stream;
 	struct firmware stripped_firmware;
 	struct sof_ipc4_msg msg = {};
 	int ret, ret1;
 
 	/* if IMR booting is enabled and fw context is saved for D3 state, skip the loading */
-	if (reload && hda->booted_from_imr && ipc4_data->fw_context_save)
+	if (reload && hda->booted_from_imr)
 		return 0;
 
 	/* the fw_lib has been verified during loading, we can trust the validity here */

ujfalusi · 2025-05-26T06:17:53Z

@lyakh, how does the kernel knows that the booted firmware saves the library context thus no need to re-load the libraries?
What shall the kernel do (and how it might know it) if the lib manager context restore fails and it needs to re-load the libraries?

Can we send flag(s) via FW_READY for example? Or the context restore happens after the firmware has sent the FW_READY?

kv2019i

No blockers, some questions in line.

zephyr/lib/alloc.c

kv2019i · 2025-05-26T13:27:14Z

src/ipc/ipc4/handler.c

 			return IPC4_BUSY;
 		}

+		ret = llext_manager_store_to_dram();


How does this play with full CONTEXT_SAVE feature? If full SRAM is saved, then this is redundant, right?

right, need to add that

any update here ?

kv2019i · 2025-05-26T13:27:40Z

src/library_manager/llext_manager_dram.c

@@ -0,0 +1,308 @@
+// SPDX-License-Identifier: BSD-3-Clause
+//
+// Copyright(c) 2025 Intel Corporation. All rights reserved.


src/library_manager/llext_manager_dram.c

lyakh · 2025-05-26T14:35:55Z

Can we send flag(s) via FW_READY for example? Or the context restore happens after the firmware has sent the FW_READY?

@ujfalusi if this comment is correct 5e6ae48#diff-4287fd67d00ed77ae6296c02fa3a29e7f113fe39a7c488846b9e38cdb46447ffR194 then yes, in the present form restoring module context takes place before sending "ready" to the host, so it should be possible to adjust that message if needed

lgirdwood

Good stuff, in general could do with more inline comments.

src/include/sof/lib_manager.h

zephyr/lib/alloc.c

src/library_manager/llext_manager.c

src/library_manager/llext_manager_dram.c

lgirdwood · 2025-05-26T15:38:34Z

src/library_manager/llext_manager_dram.c

+	lib_manager_dram.ext_lib = *_ext_lib;
+	for (i = 0, n_lib = 0, n_mod = 0, n_llext = 0, n_sect = 0, n_sym = 0;
+	     i < ARRAY_SIZE(_ext_lib->desc); i++)
+		if (_ext_lib->desc[i]) {


if (!) continue ? reduces the indentation.

lgirdwood · 2025-05-26T15:39:18Z

src/library_manager/llext_manager_dram.c

+			n_lib++;
+			n_mod += _ext_lib->desc[i]->n_mod;
+			for (k = 0; k < _ext_lib->desc[i]->n_mod; k++)
+				if (_ext_lib->desc[i]->mod[k].ebl) {


ditto if(!) continue

src/library_manager/llext_manager_dram.c

lgirdwood · 2025-05-26T15:44:27Z

src/library_manager/llext_manager_dram.c

+
+	/* Also flatten dependency lists */
+	int ret = llext_relink_dependency(lib_manager_dram.llext, n_llext);
+


newline needed ?

@lgirdwood unfortunately yes, otherwise something (checkpatch?) will complain, that it wants to have a blank line between declarations and code

I ignore it now - it can be removed, this stuff is caught in review now.

I ignore it now - it can be removed, this stuff is caught in review now.

@lgirdwood sorry, which review? Some tool is now complaining about these blank lines?

no I mean us human reviewers. I pattern match the if() {block} together without the new line. new line here makes it look like another unrelated block to the if condition.

src/library_manager/llext_manager_dram.c

lyakh · 2025-06-02T12:52:23Z

thesofproject/linux#5430 is merged, this can now be tested, now waiting for zephyrproject-rtos/zephyr#90400

lgirdwood

LGTM, some opens but lets get the Zephyr part merged 1st.

lgirdwood · 2025-06-09T14:43:19Z

src/ipc/ipc4/handler.c

 			return IPC4_BUSY;
 		}

+		ret = llext_manager_store_to_dram();


any update here ?

lyakh · 2025-06-11T08:31:50Z

zephyrproject-rtos/zephyr#90400 got merged, now this becomes mergeable too

The authentication context, used by the library manager to check module signature, is fully reinitialised and released for each loaded module, no need to store it permanently. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

L3 heap allocates memory in DRAM. Usually this is done to preserve contents over DSP reset. This patch adds a method to do that. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

lib_manager.h uses some types without including respective headers or declaring them. Fix two such omissions. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

If module registration fails, lib_manager_load_library() will jump to "cleanup" where it will attempt to check whether a module is an LLEXT but in such cases the "mod" pointer can be NULL, which will cause an exception. Check the error code before checking the module type. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

llext_get_section_info() shouldn't fail when called with a valid section number, but it's still cleaner to check its success and skip the section in case of an error. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

When failing llext_manager_link_single() should return a negative error code, not 0. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Updpate Zephyr to include 4de0c9abc0e SoC: Intel: ADSP: ACE30: add .imrdata to MMU definitions 5f4177b47c9 llext: add context save and restore d8195c05c78 llext: rename _llext_list to llext_list Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

lyakh · 2025-06-12T10:14:28Z

for the record. Having enabled partial restore shortens resume time as follows: before enabling

[  392.474926] kernel: snd_sof:snd_sof_boot_dsp_firmware: sof-audio-pci-intel-ptl 0000:00:1f.3: Booting DSP firmware
...
[  392.639208] kernel: snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-ptl 0000:00:1f.3: ipc rx      : 0x1b080000|0x0: GLB_NOTIFICATION|FW_READY

i.e. 164ms, after:

[ 2519.523002] kernel: snd_sof:snd_sof_boot_dsp_firmware: sof-audio-pci-intel-ptl 0000:00:1f.3: Booting DSP firmware
...
[ 2519.598833] kernel: snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-ptl 0000:00:1f.3: ipc rx      : 0x1b088000|0x0: GLB_NOTIFICATION|FW_READY

i.e. 76ms

lgirdwood

One open.

lgirdwood · 2025-06-12T11:46:57Z

app/boards/intel_adsp_ace30_ptl.conf

 CONFIG_SYS_CLOCK_TICKS_PER_SEC=12000

 # Zephyr / power settings
-CONFIG_ADSP_IMR_CONTEXT_SAVE=y


do we need a Kconfig to select partial context save ?

do we need a Kconfig to select partial context save ?

@lgirdwood currently we have the following configuration possibilities:

no LLEXT - obviously LLEXT selective context saving isn't used

LLEXT and full context saving - LLEXT selective context saving isn't used

otherwise - if LLEXT is enabled and full context saving isn't enabled, then LLEXT selective context saving is used.

So you'd like to also be able to select
4. LLEXT but no context saving - LLEXT reloading when booting

I didn't add a Kconfig option for this because why would anyone want to disable this? It almost looks like a fix instead of a new feature - we have LLEXT modules in DRAM, reloading them instead of reusing sounds more like a bug than a viable option to me. But if needed, sure, we can make this configurable. Also, there's a Zephyr option that this depends on: CONFIG_LLEXT_EXPERIMENTAL - without it this will fail, but it might to fail completely, haven't checked that

@lgirdwood with the new version one can disable CONFIG_LLEXT_EXPERIMENTAL and return to the old behaviour - reloading libraries on each resume

Currently when resuming from D3 we lose the complete LLEXT context, which forces SOF to reload all the libraries, which costs time. This isn't necessary, because the libraries are stored in DRAM, which preserves its contents across DSP reset. Instead of reloading save LLEXT context before power down and reload it when resuming. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

ranj063 · 2025-06-26T18:54:48Z

app/boards/intel_adsp_ace15_mtpm.conf

 CONFIG_HEAP_MEM_POOL_SIZE=8192
 CONFIG_LLEXT=y
 CONFIG_LLEXT_STORAGE_WRITABLE=y
+CONFIG_LLEXT_EXPERIMENTAL=y


why call it experimental? Perhaps LLEXT_SAVE_RESTORE makes better sense?

@ranj063 we don't, Zephyr does. This option has recently been added to Zephyr to support our partial LLEXT restore, but the way how we added it there isn't perfectly clean. It's hardly usable by other LLEXT users, it's almost SOF-specific, therefore it's been decided to mark it "experimental" until a proper universal solution replaces it.

kv2019i · 2025-07-02T07:11:19Z

@lyakh Have you checked the PM-with-audio fail in https://sof-ci.01.org/sofpr/PR10028/build13381/devicetest/index.html ? Other tests (that we have available now) seem good.

kv2019i · 2025-07-02T10:25:42Z

The check-suspend-resume-with-playback-5.sh fail is suspicious, let's rerun the test. This PR should not have any impact, but the Zephyr update could bring in some new problem. https://sof-ci.01.org/sofpr/PR10028/build13381/devicetest/index.html?model=PTLH_RVP_NOCODEC&testcase=check-suspend-resume-with-playback-5

kv2019i · 2025-07-02T10:25:57Z

SOFCI TEST

kv2019i · 2025-07-02T12:12:17Z

check-suspend-resume-with-playback-5.sh fail not related to this PR. It has been seen with baseline (albeit with low repro rate) and the rerun of tests today with this PR didn't hit the failure. Proceeding with merge.

kv2019i reviewed May 26, 2025

View reviewed changes

lyakh force-pushed the llext branch from d83c108 to e502265 Compare May 26, 2025 14:29

lgirdwood reviewed May 26, 2025

View reviewed changes

lyakh force-pushed the llext branch from e502265 to e8f87d1 Compare June 2, 2025 12:50

lyakh force-pushed the llext branch from e8f87d1 to a511a61 Compare June 2, 2025 13:22

lyakh mentioned this pull request Jun 3, 2025

llext: add context save and restore zephyrproject-rtos/zephyr#90400

Merged

lgirdwood reviewed Jun 9, 2025

View reviewed changes

src/ipc/ipc4/handler.c

return IPC4_BUSY;

}

ret = llext_manager_store_to_dram();

Copy link

Member

lgirdwood Jun 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any update here ?

lyakh force-pushed the llext branch from 131ec5f to 1cf72d1 Compare June 11, 2025 08:28

lyakh marked this pull request as ready for review June 11, 2025 08:28

lyakh requested review from RanderWang, abonislawski, dabekjakub, dbaluta, iuliana-prodan, lbetlej, marcinszkudlinski, mmaka1, pblaszko and plbossart as code owners June 11, 2025 08:28

lyakh force-pushed the llext branch from 1cf72d1 to df1a570 Compare June 11, 2025 08:39

lyakh added 6 commits June 11, 2025 11:55

heap: enable saving L3 heap over DSP reset

f8f3af9

L3 heap allocates memory in DRAM. Usually this is done to preserve contents over DSP reset. This patch adds a method to do that. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

library-manager: fix compilation errors in certain builds

91783f3

lib_manager.h uses some types without including respective headers or declaring them. Fix two such omissions. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

llext-manager: fix an error case in llext_manager_link_single()

6a4e0f4

When failing llext_manager_link_single() should return a negative error code, not 0. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

lgirdwood approved these changes Jun 11, 2025

View reviewed changes

lyakh force-pushed the llext branch 2 times, most recently from b17e512 to 46afc5a Compare June 12, 2025 06:59

lgirdwood reviewed Jun 12, 2025

View reviewed changes

lyakh force-pushed the llext branch from 46afc5a to cf23faf Compare June 13, 2025 08:22

lyakh force-pushed the llext branch from cf23faf to 1ebde97 Compare June 25, 2025 13:06

lgirdwood approved these changes Jun 25, 2025

View reviewed changes

ranj063 reviewed Jun 26, 2025

View reviewed changes

ranj063 mentioned this pull request Jun 26, 2025

[DNM]cpu: fix cpu_disable_core for primary core #10036

Closed

kv2019i mentioned this pull request Jul 1, 2025

app: add os_linux_overlay.conf #10084

Merged

kv2019i approved these changes Jul 2, 2025

View reviewed changes

kv2019i merged commit 60e7a0a into thesofproject:main Jul 2, 2025
23 of 48 checks passed

lyakh deleted the llext branch July 2, 2025 12:27


		/* Also flatten dependency lists */
		int ret = llext_relink_dependency(lib_manager_dram.llext, n_llext);

LLEXT: don't reload upon resume #10028

LLEXT: don't reload upon resume #10028

Uh oh!

Conversation

lyakh commented May 23, 2025

Uh oh!

ujfalusi commented May 26, 2025

Uh oh!

kv2019i left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lyakh commented May 26, 2025

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lyakh commented Jun 2, 2025

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lyakh commented Jun 11, 2025

Uh oh!

lyakh commented Jun 12, 2025

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lyakh Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kv2019i commented Jul 2, 2025

Uh oh!

kv2019i commented Jul 2, 2025 • edited by lyakh Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kv2019i commented Jul 2, 2025

lyakh Jun 12, 2025 •

edited

Loading

kv2019i commented Jul 2, 2025 •

edited by lyakh

Loading