library manager: initial implementation #5796

pjdobrowolski · 2022-05-10T10:46:35Z

Library manager is part of loading external libraries responsible of
handling data and placing it in proper order and places.

After receiving IPC4 load library command

reads manifest
after module_id choses module to load and verify it (WIP)
loads manifest
allocs propper memory size
transfer module from IMR to HPSRAM (WIP)

Signed-off-by: Stelter, Jaroslaw jaroslaw.stelter@intel.com
Signed-off-by: Dobrowolski, PawelX pawelx.dobrowolski@intel.com

sofci · 2022-05-10T10:48:04Z

Can one of the admins verify this patch?

reply test this please to run this test once

pjdobrowolski · 2022-05-10T10:51:44Z

test this please

gkbldcig · 2022-05-10T11:18:55Z

Can one of the admins verify this patch?

src/include/sof/library_manager.h

src/library_manager/library_manager.c

marc-hb · 2022-05-10T19:16:08Z

test this please

lgirdwood · 2022-05-11T13:39:00Z

@pjdobrowolski pls check your inbox, invite sent to autorun the CI.

lgirdwood

Some questions

How are we testing this today ? and how should we test in CI
Does this version support the dynamic linking or just the dynamic loading today ?

src/include/sof/library_manager.h

src/library_manager/library_manager.c

dbaluta · 2022-05-11T14:34:09Z

@pjdobrowolski @lgirdwood aside from having this quick merged what stops us from implementing this feature in zephyr?

marc-hb · 2022-05-11T17:14:45Z

Most checkpatch warnings at https://sof-ci.01.org/sofpr/PR5796/build327/checkpatch/ seem valid (and easy to fix)

jxstelter · 2022-05-12T07:19:10Z

Some questions

How are we testing this today ? and how should we test in CI

This PR is only small part responsible for loading library binary. More changes are required to have something testable.

Does this version support the dynamic linking or just the dynamic loading today ?

We support now dynamic loading of library containing 3rd party module binary. Currently available libraries in IADK (Windows close source FW) are not supporting dynamic linking, however it could be supported in the future.

marc-hb

More changes are required to have something testable.

Maybe that's why it's still a draft?

src/library_manager/Kconfig

src/library_manager/library_manager.c

src/include/sof/library_manager.h

src/library_manager/library_manager.c

src/library_manager/Kconfig

src/library_manager/lib_manager.c

src/library_manager/lib_notification.c

src/include/sof/lib_manager.h

src/library_manager/lib_manager.c

src/library_manager/lib_notification.c

lyakh · 2022-08-31T09:00:11Z

src/library_manager/lib_notification.c

your .lib_notif_pool is somewhat... unusual. Usually you'd just have a list head in the higher level object (in ext_lib in this case) to which you then link elements as you allocate them. Instead you have a pointer to lib_notif_pool there and once you allocate the first element and attach it there, you then link further allocated elements to the first one. That makes the list head instance in that object both - a list head and a list element. list_empty() now means not 0 elements, but 1... All that can be made to work of course but just is a bit unusual and one has to remember about this when working with this code.

So what would you suggest. That might be refactored for sure but I think that main goal is to port that feature from legacy and integrate. Many mechanisms used here are borrowed from older firmware and for sure they can be improved. We can do that later if it isn't so wrong.

src/library_manager/lib_notification.c

src/include/ipc4/module.h

src/ipc/ipc4/helper.c

pjdobrowolski · 2022-08-31T10:00:56Z

SOFCI TEST

lgirdwood · 2022-08-31T10:40:23Z

@lrudyX @wszypelt can you check CI, not expecting this to fail where it did. Looks like a copy issue ? Thanks.

src/library_manager/lib_notification.c

src/library_manager/lib_manager.c

wszypelt · 2022-08-31T11:09:32Z

@lgirdwood yea...
TGL was disabled, I fixed it and ran the tests again.

Library manager is part of loading external libraries responsible of handling data and placing it in proper order and places. This feature is using module adapter API for loadable native and external libraries. After receiving IPC4 load library command - reads manifest - after module_id choses module to load and verify it (WIP) - loads manifest - allocs proper memory size - transfer module from external MEMORY to HPSRAM (WIP) Signed-off-by: Stelter, Jaroslaw <jaroslaw.stelter@intel.com> Signed-off-by: Dobrowolski, PawelX <pawelx.dobrowolski@intel.com>

Commit adds ipc which contains information need to load external libraries from HOST and required actions to make it possible using library manager functionality. Signed-off-by: Stelter, Jaroslaw <jaroslaw.stelter@intel.com> Signed-off-by: Dobrowolski, PawelX <pawelx.dobrowolski@intel.com>

Adding function responsible for loading library action after IPC4 message. Signed-off-by: Stelter, Jaroslaw <jaroslaw.stelter@intel.com> Signed-off-by: Dobrowolski, PawelX <pawelx.dobrowolski@intel.com>

Adding for ipc4_get_comp_drv check if module is loadable and for active library manager functionality register it dynamically. Signed-off-by: Stelter, Jaroslaw <jaroslaw.stelter@intel.com> Signed-off-by: Dobrowolski, PawelX <pawelx.dobrowolski@intel.com>

…red. System service functionality used by 3rd party modules expects that IPC data buffer will be filled with data during notification preparation. To avoid additional data copy, provide message buffer directly to module. This requires that ipc_msg_send() function will not fail if source data pointer will be equal to destination one. Signed-off-by: Stelter, Jaroslaw <jaroslaw.stelter@intel.com> Signed-off-by: Dobrowolski, PawelX <pawelx.dobrowolski@intel.com>

pjdobrowolski · 2022-08-31T11:46:42Z

rebased, resolved, ready to merge?

lgirdwood · 2022-08-31T12:44:30Z

rebased, resolved, ready to merge?

Good to merge as soon as CI finishes.

wszypelt · 2022-08-31T12:59:11Z

SOFCI TEST

lyakh · 2022-09-01T05:53:31Z

src/library_manager/lib_notification.c

+		msg = ipc_msg_init(header, SRAM_OUTBOX_SIZE);
+		if (!msg)
+			rfree(msg_pool_elem);
+			return NULL;


You need braces here

So reopen PR and will fix it.

dbaluta reviewed May 10, 2022

View reviewed changes

src/include/sof/library_manager.h Outdated Show resolved Hide resolved

dbaluta reviewed May 10, 2022

View reviewed changes

src/include/sof/library_manager.h Outdated Show resolved Hide resolved

dbaluta reviewed May 10, 2022

View reviewed changes

src/include/sof/library_manager.h Outdated Show resolved Hide resolved

dbaluta reviewed May 10, 2022

View reviewed changes

src/library_manager/library_manager.c Outdated Show resolved Hide resolved

pjdobrowolski force-pushed the library_manager branch from 5971bea to 24832db Compare May 11, 2022 13:41

lgirdwood reviewed May 11, 2022

View reviewed changes

pjdobrowolski force-pushed the library_manager branch 2 times, most recently from ea4d814 to 8865e18 Compare May 11, 2022 14:30

pjdobrowolski force-pushed the library_manager branch from 8865e18 to 2c72666 Compare May 11, 2022 14:37

jxstelter closed this May 12, 2022

pjdobrowolski reopened this May 12, 2022

marc-hb reviewed May 12, 2022

View reviewed changes

src/library_manager/Kconfig Outdated Show resolved Hide resolved

src/library_manager/library_manager.c Outdated Show resolved Hide resolved

dbaluta reviewed May 13, 2022

View reviewed changes

src/include/sof/library_manager.h Outdated Show resolved Hide resolved

src/include/sof/library_manager.h Outdated Show resolved Hide resolved

src/include/sof/library_manager.h Outdated Show resolved Hide resolved

src/library_manager/library_manager.c Outdated Show resolved Hide resolved

jxstelter requested a review from mwasko May 13, 2022 10:25

pjdobrowolski force-pushed the library_manager branch 5 times, most recently from ccec0d4 to e440d45 Compare May 17, 2022 13:05

ranj063 reviewed May 17, 2022

View reviewed changes

dbaluta reviewed May 18, 2022

View reviewed changes

pjdobrowolski force-pushed the library_manager branch from e440d45 to 377bee8 Compare May 18, 2022 12:32

pjdobrowolski force-pushed the library_manager branch 7 times, most recently from 6ebba7e to f1d3cb9 Compare August 31, 2022 08:52

lyakh reviewed Aug 31, 2022

View reviewed changes

pjdobrowolski force-pushed the library_manager branch from f1d3cb9 to 5895bd4 Compare August 31, 2022 09:27

marcinszkudlinski approved these changes Aug 31, 2022

View reviewed changes

lyakh reviewed Aug 31, 2022

View reviewed changes

lgirdwood approved these changes Aug 31, 2022

View reviewed changes

pjdobrowolski force-pushed the library_manager branch from 5895bd4 to 16f135a Compare August 31, 2022 11:08

pjdobrowolski force-pushed the library_manager branch 2 times, most recently from 7d787e1 to a601214 Compare August 31, 2022 11:33

pjdobrowolski added 5 commits August 31, 2022 13:35

ipc4: library manager: handling load library function

e596280

Adding function responsible for loading library action after IPC4 message. Signed-off-by: Stelter, Jaroslaw <jaroslaw.stelter@intel.com> Signed-off-by: Dobrowolski, PawelX <pawelx.dobrowolski@intel.com>

pjdobrowolski force-pushed the library_manager branch from a601214 to 3a68e90 Compare August 31, 2022 11:40

lgirdwood merged commit adaebb3 into thesofproject:main Aug 31, 2022

lyakh reviewed Sep 1, 2022

View reviewed changes

marc-hb mentioned this pull request Sep 1, 2022

[SKIP CI] Minor docker-run.sh fixes #6238

Merged

library manager: initial implementation #5796

library manager: initial implementation #5796

Uh oh!

Conversation

pjdobrowolski commented May 10, 2022

Uh oh!

sofci commented May 10, 2022

Uh oh!

pjdobrowolski commented May 10, 2022

Uh oh!

gkbldcig commented May 10, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

marc-hb commented May 10, 2022

Uh oh!

lgirdwood commented May 11, 2022

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dbaluta commented May 11, 2022

Uh oh!

marc-hb commented May 11, 2022

Uh oh!

jxstelter commented May 12, 2022 • edited by marc-hb Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

marc-hb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lyakh Aug 31, 2022

Choose a reason for hiding this comment

Uh oh!

pjdobrowolski Aug 31, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pjdobrowolski commented Aug 31, 2022

Uh oh!

lgirdwood commented Aug 31, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jxstelter commented May 12, 2022 •

edited by marc-hb

Loading