Embed templates inside dd agent binary

[prognant]

What does this PR do?

It embeds templates files into the agent binary at compilation time.

Motivation

It makes the agent binary able to run agent status without requiring
to the original template file.

Additional Notes

It relies on go-bindata that has been added to the dependencies.

[codecov]

Codecov Report

Merging #4782 into master will increase coverage by 0.06%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #4782      +/-   ##
==========================================
+ Coverage   52.32%   52.39%   +0.06%     
==========================================
  Files         709      709              
  Lines       51004    50947      -57     
==========================================
+ Hits        26689    26692       +3     
+ Misses      22735    22676      -59     
+ Partials     1580     1579       -1

Flag	Coverage Δ
#linux	53.63% <0%> (+0.05%)	⬆️
#windows	52.79% <0%> (+0.08%)	⬆️

Impacted Files	Coverage Δ
pkg/status/render.go	0% <0%> (-1.24%)	⬇️
pkg/collector/scheduler/scheduler.go	84.66% <0%> (+0.61%)	⬆️
pkg/logs/auditor/auditor.go	71.83% <0%> (+0.7%)	⬆️
pkg/secrets/secrets.go	69.84% <0%> (+2.38%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6515895...f3c70d2. Read the comment docs.

[truthbk]

This looks very clean to me! Just a comment on the go-bindata tool we'll be using: we should pin it.

[truthbk]

I was wondering if agent:generate wasn't a better invoke target, but I realized, also from your changes, that generate may be required by other tasks in the main collection too, so I guess putting it here is the sensible thing to do.

[truthbk]

Thank you! Nice job!

[KSerrania]

LGTM

[KSerrania]

Nit for later: the name feels a bit too generic for what this function does: for instance, it's difficult to understand without going into the specifics of the function that the:

def build(...):
  ...
  # Generate go source from template
  generate(ctx)
  ...

line in tasks/agent.py means we apply go generate on ./pkg/status.

Moreover, we use go generate in some other places (such as tasks/cluster_agent.py:

datadog-agent/tasks/cluster_agent.py

Lines 66 to 67 in 527af5e

	cmd = "go generate -tags '{build_tags}' {repo_path}/cmd/cluster-agent"
	ctx.run(cmd.format(build_tags=" ".join(build_tags), repo_path=REPO_PATH), env=env)

, tasks/dogstatsd.py:

datadog-agent/tasks/dogstatsd.py

Lines 95 to 96 in 527af5e

	cmd = "go generate {}/cmd/dogstatsd"
	ctx.run(cmd.format(REPO_PATH), env=env)

, and tasks/trace_agent.py:

datadog-agent/tasks/trace_agent.py

Lines 75 to 76 in 527af5e

	ctx.run("go generate {REPO_PATH}/pkg/trace/info".format(**args), env=env)
	ctx.run(cmd.format(**args), env=env)

), so we could use this function in these cases too (with some tweaks to take different targets and options).

I'd either:

• Not change anything, but show explicitly that this function renders only what's specified in GO_GENERATE_TARGETS (eg. by adding a comment around its invocation in tasks/agent.py and tasks/cluster_agent.py) so that someone reading the build tasks understands what's happening without having to read the internal function's behavior),
• Make this function more generic by accepting a list of targets (and maybe a list of options if needed)

[prognant]

I agree that it will make sense at some point to write a full-featured wrapper for go generate. In-between I took option #1 and added more meaningful comments.

[bits-bot]

All committers have signed the CLA.

[albertvaka]

I wonder if we could also do this for the html files we use in the agent launch-gui command? I guess not because the embedded browser component needs to be able to find them?

[prognant]

IMHO that would worth trying but in the meantime we have a windows issue likely related to this change. I will try to follow up on that topic.