Switching all zone IPS operations to be fully UTF-8

[Toasterson]

No description provided.

[Toasterson]

@citrus-it Here the PR to make all operations UTF-8. @olbohlen Did you change any other occurances of LC_ALL=C other then these here?

[danmcd]

Probably not a blocker for this, but the recent discovery of source/naming attacks that exploit bidirectional (bidi) or homonym-character attacks may be something to consider in future pkg5 updates, now that it will support UTF.

[Toasterson]

Well not if we have manifest signatures and catalog signatures properly implemented.
We rebuild all manifests on the buildeserver so any attack in the source would show there. I would need to think more in detail about it, but many parts of the IPS infrastructure have had support for UTF-8 before. This is just a leftover. I think we can handle this issue with properly implemented secure packaging. And not trusting too much what comes through git :)

[olbohlen]

lgtm, I haven't changed anything else

[AndWac]

Is it sufficient to only fix shell scripts or do we also need to fix Python code like this:
run_cmd(args, podir, updenv={"LC_ALL": "C"}, ignerr=True)
or
updenv={"LC_ALL": "C", "XGETTEXT": "/usr/gnu/bin/xgettext"})
?

[Toasterson]

Shell scripts are sufficient, as it is the python based action and catalog decoders that fail.