autogenerate NOTICE.BINARY from NOTICE and licenses.yaml#8306
autogenerate NOTICE.BINARY from NOTICE and licenses.yaml#8306clintropolis merged 10 commits intoapache:masterfrom
Conversation
…yaml and NOTICE to generate NOTICE.BINARY at distribution time
clintropolis
added
Apache
| import yaml | ||
|
|
||
|
|
||
| outfile = sys.stdout |
There was a problem hiding this comment.
When is the value sys.stdout used? out_path is a required positional command-line arg, so this variable should always get initialized to that value. If sys.stdout is not used there, then perhaps print_error should be printing to stdout instead?
| outfile = sys.stdout | ||
|
|
||
|
|
||
| moduleHeaderLine = "############" |
There was a problem hiding this comment.
FYI, you can do something like: moduleHeaderLine = "#" * 12
| print(string, file=sys.stderr) | ||
|
|
||
| def print_jar(name, version, notice): | ||
| print_outfile("{} {}-{}.jar {}".format(dependencyHeaderLine, name, version, dependencyHeaderLine)) |
There was a problem hiding this comment.
Consider using an f-string instead of string.format(): https://www.python.org/dev/peps/pep-0498/#code-equivalence
print_outfile(f"{dependencyHeaderLine} {name}-{version}.jar {dependencyHeaderLine}")| # Print Apache license first. | ||
| print_outfile(source_notice) | ||
| with open(dependences_yaml) as registry_file: | ||
| dependencies_list = list(yaml.load_all(registry_file)) |
There was a problem hiding this comment.
Does it make sense to use a set instead of a list? Using a set may also be beneficial for the membership test below (O(1) versus O(n)).
| modules_map = {} | ||
| for dependency in dependencies_list: | ||
| if 'notice' in dependency or 'notices' in dependency: | ||
| if dependency['module'] not in modules_map: |
There was a problem hiding this comment.
Consider using defaultdict for this pattern: https://docs.python.org/3.7/library/collections.html#defaultdict-examples
from collections import defaultdict
modules_map = defaultdict(list)
for dependency in dependencies_list:
if 'notice' in dependency or 'notices' in dependency:
modules_map[dependency['module']].append(dependency)| generate_notice(source_notice, dependencies_yaml) | ||
|
|
||
| except KeyboardInterrupt: | ||
| print('Interrupted, closing.') No newline at end of file |
There was a problem hiding this comment.
I'm guessing the weird character showing up here is because of a missing newline at the end of the file?
| source_notice = apache_notice_file.read() | ||
| dependencies_yaml = args.license_yaml | ||
|
|
||
| with open(args.out_path, "w") as outfile: |
There was a problem hiding this comment.
FYI, if it's desired for this to fail if the file already exists, then the "x" mode can be used.
Also, this line has the side-effect of changing the behavior of print_outfile() which is a bit hidden as it goes through the outfile global variable.
|
|
||
|
|
||
| def generate_notice(source_notice, dependences_yaml): | ||
| # Generate NOTICE.BINARY file |
There was a problem hiding this comment.
This comment is redundant with the message string on the next line.
| def print_outfile(string): | ||
| print(string, file=outfile) | ||
|
|
||
| def print_error(string): |
There was a problem hiding this comment.
This name is a bit misleading as it's used to print informational messages instead of error messages
…ependencies for avro and kerberos in licenses.yaml
3 participants
Description
This PR migrates binary
NOTICEentries to live inlicenses.yaml, and adds a new script,docs/_bin/generate-notice-binary.pythat useslicenses.yamlandNOTICEto generateNOTICE.BINARYat distribution time. Adapted fromgenerate-license.py.This should make maintaining the notice file a bit easier, but this PR doesn't include anything to automatically check that everything that needs a notice has one in here, so we will still need to manually make sure the
noticeornoticesare set for a dependency for now.Additionally, all scripts related to performing releases have been moved into
distribution/binfromdocs/_bin.