[RFC] tools: add python blob generator by juimonen · Pull Request #2536 · thesofproject/sof

juimonen · 2020-03-12T14:26:33Z

Provide a general tool for creating binary blobs for sof process
components.

Signed-off-by: Jaska Uimonen jaska.uimonen@intel.com

juimonen · 2020-03-12T14:27:18Z

just something as my python skills are not that good... based on python's "struct" module.

lgirdwood

We already have a cmd line tool that prepends ABI for coefficient blobs. I want to kill the mux/demux blobs, it should be using regular ALSA enum controls.

juimonen · 2020-03-12T16:51:56Z

@lgirdwood, so this tool is generating the whole blob for you (not just abi header). So it is not for fir/iir where you need to use complex math design tools for getting the filter coeffs, unless you are very good doing math in your head :). It is more for those components where you build the blob from multiple structs having quite simple parameters in the end.

I'm not sure the mux/demux we have is usable with enums only. It is more complicated than mux/demux envisaged in asoc.

Anyway, @Keyon and @mmaka1 are suggesting to move totally to blobs in processing components:
https://github.com/orgs/thesofproject/teams/sof-developers/discussions/32, we would need someone else than me commenting on that.

Also, having this tool around would not close any options, but you could actually create the mux/demux blob (which was never possible before in public).

lgirdwood · 2020-03-12T17:15:40Z

@juimonen I want rid of any blob that is used for runtime config unless it's loading coefficients

plbossart · 2020-03-12T20:21:45Z

Even for coefficients we should have a header and a syntax that identifies blob type/versions/etc. There's nothing worse than a random set of bytes, it's unmanageable.

juimonen · 2020-03-12T20:34:28Z

@plbossart so all the blobs we are sending have sof_abi_hdr with component specific data after it. That has abi versions etc. If you are talking about for example fir/iir specific structs, they are in sof includes and removed from kernel uapi. So if you'd like to use those in user space software, you need to somehow include sof headers.

Anyway I made this tool to help someone to create a blob, as it is requested from me all the time and as there is currently nothing more public than @singalsuo eq matlab scripts, and they are specific to eq.

I'm not trying to promote use of blobs here, I will try to modify the demux out of blob (and var array), the intent of you and @lgirdwood is now pretty clear to me. I've been just little bit hesitant to touch the demux on fw side as it is not written by me. But it seems nothing happens unless I try to do it myself :) people are busy...

lgirdwood · 2020-03-20T13:23:22Z

I've been just little bit hesitant to touch the demux on fw side as it is not written by me. But it seems nothing happens unless I try to do it myself :) people are busy...

Please feel free to touch anything you like on FW side. :-)

xiulipan · 2020-04-08T09:56:20Z

@juimonen Would you mind add one more feature here to have another option to create a simple snd_ctl_tlv wrapper outside sof headers?

https://github.com/alsa-project/alsa-lib/blob/fb48ad9e4f6b84fd4ade689bd79e3a3c37d3e034/include/sound/uapi/asound.h#L1057-L1061

struct snd_ctl_tlv {
	unsigned int numid;	/* control element numeric identification */
	unsigned int length;	/* in bytes aligned to 4 */
	unsigned int tlv[0];	/* first TLV */
};

This will help to make the blob directly usable with alsaucm with cset-tlv

juimonen · 2020-04-08T12:02:09Z

@xiulipan sure will do!

juimonen · 2020-04-08T12:12:04Z

@xiulipan so you want this to replace the sof header, or as extra around sof header + blob?

juimonen · 2020-04-08T12:22:29Z

@xiulipan and btw where is this cset-tlv? Can't seem to find it? is it some chromium specific thing?

xiulipan · 2020-04-09T07:43:33Z

@juimonen From my test using byte blob with UCM, I found the binary accepted by SOF should be looks like

snd_ctl_tlv Header
sof blob Header
blob binary

So I would like to have another option to append a snd_ctl_tlv header.

The cset-tlv is something from alsaucm.
https://github.com/alsa-project/alsa-lib/blob/fb48ad9e4f6b84fd4ade689bd79e3a3c37d3e034/src/ucm/parser.c#L537-L545

		if (strcmp(cmd, "cset-tlv") == 0) {
			curr->type = SEQUENCE_ELEMENT_TYPE_CSET_TLV;
			err = parse_string(n, &curr->data.cset);
			if (err < 0) {
				uc_error("error: cset-tlv requires a string!");
				return err;
			}
			continue;
		}

Provide a general tool for creating binary blobs for sof process components. Signed-off-by: Jaska Uimonen <jaska.uimonen@intel.com>

monocodus

This is autogenerated code-style review, new suggestions: 16

monocodus · 2020-04-09T12:46:46Z

tools/blobgen/sof_gen_blob.py

+
+items_per_line = 8
+
+def conv_and_get_type(val):


Suggested change

def conv_and_get_type(val):

def conv_and_get_type(val):

This comment was generated by: AutoPEP8

monocodus · 2020-04-09T12:46:46Z

tools/blobgen/sof_gen_blob.py

+        except ValueError:
+            pass
+
+parser = argparse.ArgumentParser(description='Print C struct as hex blob for alsa conf.', epilog=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)


Suggested change

parser = argparse.ArgumentParser(description='Print C struct as hex blob for alsa conf.', epilog=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)

This comment was generated by: AutoPEP8

monocodus · 2020-04-09T12:46:47Z

tools/blobgen/sof_gen_blob.py

+            pass
+
+parser = argparse.ArgumentParser(description='Print C struct as hex blob for alsa conf.', epilog=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+parser.add_argument("--abiver", "-a", type=int, nargs=3, help='sof abi version major minor patch')


Suggested change

parser.add_argument("--abiver", "-a", type=int, nargs=3, help='sof abi version major minor patch')

parser = argparse.ArgumentParser(description='Print C struct as hex blob for alsa conf.',

This comment was generated by: AutoPEP8

monocodus · 2020-04-09T12:46:47Z

tools/blobgen/sof_gen_blob.py

+
+parser = argparse.ArgumentParser(description='Print C struct as hex blob for alsa conf.', epilog=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+parser.add_argument("--abiver", "-a", type=int, nargs=3, help='sof abi version major minor patch')
+parser.add_argument('--controlid', "-c", type=int, nargs=1, help='define controlid and wrap the blob around snd_ctl_tlv header')


Suggested change

parser.add_argument('--controlid', "-c", type=int, nargs=1, help='define controlid and wrap the blob around snd_ctl_tlv header')

epilog=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)

This comment was generated by: AutoPEP8

monocodus · 2020-04-09T12:46:47Z

tools/blobgen/sof_gen_blob.py

+parser = argparse.ArgumentParser(description='Print C struct as hex blob for alsa conf.', epilog=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+parser.add_argument("--abiver", "-a", type=int, nargs=3, help='sof abi version major minor patch')
+parser.add_argument('--controlid', "-c", type=int, nargs=1, help='define controlid and wrap the blob around snd_ctl_tlv header')
+parser.add_argument("--members", "-m", nargs='+', help='char values specifying the types of the struct members')


Suggested change

parser.add_argument("--members", "-m", nargs='+', help='char values specifying the types of the struct members')

parser.add_argument("--abiver", "-a", type=int, nargs=3,

help='sof abi version major minor patch')

parser.add_argument('--controlid', "-c", type=int, nargs=1,

help='define controlid and wrap the blob around snd_ctl_tlv header')

parser.add_argument("--members", "-m", nargs='+',

help='char values specifying the types of the struct members')

This comment was generated by: AutoPEP8

monocodus · 2020-04-09T12:46:49Z

tools/blobgen/sof_gen_blob.py

+if args.controlid:
+    topo_hdr_extra = struct.pack(topo_ctl_members, args.controlid[0], size_extra)
+
+#pack header


Suggested change

#pack header

# pack header

This comment was generated by: AutoPEP8

monocodus · 2020-04-09T12:46:49Z

tools/blobgen/sof_gen_blob.py

+#pack header
+topo_hdr = struct.pack(topo_hdr_members, magic, comp_type, size, abi, reserved)
+
+#pack payload


Suggested change

#pack payload

# pack payload

This comment was generated by: AutoPEP8

monocodus · 2020-04-09T12:46:49Z

tools/blobgen/sof_gen_blob.py

+#pack payload
+topo_data = struct.pack(topo_data_members, *(values_t))
+
+#full blob


Suggested change

#full blob

# full blob

This comment was generated by: AutoPEP8

monocodus · 2020-04-09T12:46:49Z

tools/blobgen/sof_gen_blob.py

+else:
+    topo_full = topo_hdr + topo_data
+
+#for sof topology m4


Suggested change

#for sof topology m4

# for sof topology m4

This comment was generated by: AutoPEP8

monocodus · 2020-04-09T12:46:50Z

tools/blobgen/sof_gen_blob.py

+
+print("'\n")
+
+#for sof ctl tool


Suggested change

#for sof ctl tool

# for sof ctl tool

This comment was generated by: AutoPEP8

juimonen · 2020-04-09T12:47:42Z

@xiuli new version updated. with specifying argument -c "control_id", the tool should create snd_ctl_tlv around the blob.

lgirdwood

What's the difference between this and the C version ?

paulstelian97 · 2021-03-29T09:02:05Z

Please resubmit with "main" as PR base branch.

juimonen requested review from dbaluta, jajanusz, lbetlej, lgirdwood, mmaka1, plbossart and tlauda as code owners March 12, 2020 14:26

lgirdwood reviewed Mar 12, 2020

View reviewed changes

juimonen mentioned this pull request Mar 13, 2020

[BUG]DEMUX comp not work normally with channel 4 to 2 with format conversion #2533

Closed

juimonen force-pushed the blobgen branch from c1e6a0a to 1a64fcf Compare March 20, 2020 10:43

juimonen mentioned this pull request Mar 31, 2020

[RFC] add mux/demux documentation thesofproject/sof-docs#203

Merged

tools: add python blob generator

b35b897

Provide a general tool for creating binary blobs for sof process components. Signed-off-by: Jaska Uimonen <jaska.uimonen@intel.com>

juimonen force-pushed the blobgen branch from 1a64fcf to b35b897 Compare April 9, 2020 12:46

monocodus bot suggested changes Apr 9, 2020

View reviewed changes

lgirdwood reviewed Apr 10, 2020

View reviewed changes

juimonen mentioned this pull request Apr 15, 2020

[RFC] make sof_abi_hdr optional in binary configs thesofproject/linux#2002

Closed

lgirdwood closed this Mar 28, 2021

lgirdwood deleted the branch thesofproject:master March 28, 2021 13:44

	parser.add_argument("--abiver", "-a", type=int, nargs=3, help='sof abi version major minor patch')
	parser = argparse.ArgumentParser(description='Print C struct as hex blob for alsa conf.',

	parser.add_argument('--controlid', "-c", type=int, nargs=1, help='define controlid and wrap the blob around snd_ctl_tlv header')
	epilog=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)

Conversation

juimonen commented Mar 12, 2020

Uh oh!

juimonen commented Mar 12, 2020

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

juimonen commented Mar 12, 2020

Uh oh!

lgirdwood commented Mar 12, 2020

Uh oh!

plbossart commented Mar 12, 2020

Uh oh!

juimonen commented Mar 12, 2020

Uh oh!

lgirdwood commented Mar 20, 2020

Uh oh!

xiulipan commented Apr 8, 2020

Uh oh!

juimonen commented Apr 8, 2020

Uh oh!

juimonen commented Apr 8, 2020

Uh oh!

juimonen commented Apr 8, 2020

Uh oh!

xiulipan commented Apr 9, 2020

Uh oh!

monocodus bot left a comment

Choose a reason for hiding this comment

Uh oh!

monocodus bot Apr 9, 2020

Choose a reason for hiding this comment

Uh oh!

monocodus bot Apr 9, 2020

Choose a reason for hiding this comment

Uh oh!

monocodus bot Apr 9, 2020

Choose a reason for hiding this comment

Uh oh!

monocodus bot Apr 9, 2020

Choose a reason for hiding this comment

Uh oh!

monocodus bot Apr 9, 2020

Choose a reason for hiding this comment

Uh oh!

monocodus bot Apr 9, 2020

Choose a reason for hiding this comment

Uh oh!

monocodus bot Apr 9, 2020

Choose a reason for hiding this comment

Uh oh!

monocodus bot Apr 9, 2020

Choose a reason for hiding this comment

Uh oh!

monocodus bot Apr 9, 2020

Choose a reason for hiding this comment

Uh oh!

monocodus bot Apr 9, 2020

Choose a reason for hiding this comment

Uh oh!

juimonen commented Apr 9, 2020

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

paulstelian97 commented Mar 29, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants