API to call out shell scripts & functions from within Golang

[cardil]

Summary

Write an API to enable calling out shell scripts or functions from within hack or test Golang code.

Why?

We would like to write more code in Golang, that now written in shell scripts. That involves environment orchestration, test frameworks, build tools and hacks. To make that happen we need a bridge solution which let us call out working shell scripts, and not force us to rewrite everything at once.

Such proposals were previously discussed on Productivity WG knative/test-infra#2418

There are, also other use cases that can benefit from this. Like, new upgrade test framework in which we would like to install either latest stable release or current git HEAD version. Those are now written as Bash shell functions. Such shell call outs can be seen implemented in knative/eventing#4519 test/upgrade/installation/git_head.go L25-L37 Implementing that in knative/pkg is better idea then doing that for eventing, serving, contrib... etc.

How?

We should write an API that will enable us to simple call out shell scripts or just ad-hoc shell functions. It should have a logic to easily let users call out shell files relative to a given Golang source file, so they can invoke project or vendored scripts.

Invoking that script should result in STDOUT and STDERR streamed to Golang process os.Stdout and os.Stderr.

It would be nice to prefix those streams with some labels to make that output more readable.

[cardil]

/assign

[dprotaso]

To make that happen we need a bridge solution which let us call out working shell scripts, and not force us to rewrite everything at once.

A bridge too far. I think this approach is messy and it would be easier to piecemeal some shell logic into small go executables.

[cardil]

Did you seen knative/eventing#4519? To implement that new framework I need to invoke:

install_head
install_channel_crds
install_mt_broker
install_sugar

to install eventing HEAD version and:

install_latest_release

to install latest released version.

Or rewrite all that code.

I believe rewriting that code should be a gradual process, instead of big bang. That rewrite shouldn't be required to merge knative/eventing#4519.

[dprotaso]

That's a top down approach - I'm suggesting going bottom up

[n3wscott]

I do not think we should call shell scripts from go.

[coryrc]

shouldn't put any Go code in hack, defeats the purpose of having created it in the first place

[cardil]

This was discussed at productivity WG. We intend to have this ONLY as a bridge solution, until it is required.