KAFKA-7828: Add ExternalCommandWorker to Trogdor

[cmccabe]

Allow the Trogdor agent to execute external commands. The agent communicates with the external commands via stdin, stdout, and stderr. Based on a patch by Xi Yang xi@confluent.io

[mumrah]

Any thoughts on using a library like commons-exec or zt-exec?

[cmccabe]

A lot of the rationale for the commons-exec library seems to have gone away with the newer JDKs. For example, it mentions that "in J2SE 1.1-1.4 there is not support for... [setting] environment variables". This is not an issue any more since the JDK's ProcessBuilder now allows setting up the environment. This might be one reason why the last published release of the commons-exec library was from 2014 and mentions Java 1.5.

zt-exec seems a little more active (it had a release less than a year ago) but still doesn't seem to really solve for the things that I want. Again, a lot of the stuff it mentions is already in the standard java library. For example-- "Redirecting stderr to stdout"-- Java's ProcessBuilder can now do this easily. And so on.

In general, most of the complexity in this patch is about dealing with event streams coming into and out of the process. That would be the same even if a different library were used. The JDK's library is the best maintained of any of these libraries, so I think it's the clear choice.

The features that I really want that are missing from the the JDK's library are:

1. The ability to put the child process' stdout into line-buffered mode by default (Java defaults it to fully buffered)
2. The ability to send signals to any children of the subprocess we're running.

#1 is not really a big deal since the subprocess can just flush its stdout as needed, or set line-buffered mode on its own. Also, I don't think any of the other libraries have this.

#2 is something we could probably hack up by running a shell command rather than using Process#destroy. But it's probably not really needed unless people want to run bash scripts through this interface (bash loves spawning tons of subprocesses.) For running python or C, it's not really needed. And again, I doubt any other library provides something like this.

[mumrah]

Bunch of small stuff

[mumrah]

Thanks for the explanation @cmccabe. Left a review with several small comments/questions.

[mumrah]

LGTM