Refactored instrumented; Added profiling; better error handling

[bwplotka]

NOTE: This breaks API, but it should be now more consistent, easier to use and easier to extend.

I tried different approaches, but I found that splitting extensions to different packages (e.g profiling and monitoring) and simplifying will work the best. Let me know what you think!

Essentially this replaces definition like this:

return e2e.NewInstrumentedRunnable(env, name).WithPorts(map[string]int{AccessPortName: 2379, "metrics": 9000}, "metrics").Init(
		e2e.StartOptions{
			Image:     o.image,
			Command:   e2e.NewCommand("/usr/local/bin/etcd", "--listen-client-urls=http://0.0.0.0:2379", "--advertise-client-urls=http://0.0.0.0:2379", "--listen-metrics-urls=http://0.0.0.0:9000", "--log-level=error"),
			Image: o.image,
			Command: e2e.NewCommand(
				"/usr/local/bin/etcd",
				"--listen-client-urls=http://0.0.0.0:2379",
				"--advertise-client-urls=http://0.0.0.0:2379",
				"--listen-metrics-urls=http://0.0.0.0:9000",
				"--log-level=error",
			),
			Readiness: e2e.NewHTTPReadinessProbe("metrics", "/health", 200, 204),
		},
)

with

return e2emon.AsInstrumented(env.Runnable(name).WithPorts(map[string]int{AccessPortName: 2379, "metrics": 9000}).Init(
		e2e.StartOptions{
			Image:     o.image,
			Command:   e2e.NewCommand("/usr/local/bin/etcd", "--listen-client-urls=http://0.0.0.0:2379", "--advertise-client-urls=http://0.0.0.0:2379", "--listen-metrics-urls=http://0.0.0.0:9000", "--log-level=error"),
			Image: o.image,
			Command: e2e.NewCommand(
				"/usr/local/bin/etcd",
				"--listen-client-urls=http://0.0.0.0:2379",
				"--advertise-client-urls=http://0.0.0.0:2379",
				"--listen-metrics-urls=http://0.0.0.0:9000",
				"--log-level=error",
			),
			Readiness: e2e.NewHTTPReadinessProbe("metrics", "/health", 200, 204),
		},
	)
), "metrics")

Not a big deal from usage, but much more cleaner when we start adding other extension to runnables like profiling.

[saswatamcode]

Thanks! This looks awesome! 🌟

One minor comment.

[jessicalins]

cool refactor, I think it makes sense to split in different packages according with different responsibilities, as you did here :)
regarding breaking the current API - is being backwards compatible something we would pay attention to? or once this is merged and a new tag is created, those changes will be communicated in the release?
I left some suggestions

[bwplotka]

regarding breaking the current API - is being backwards compatible something we would pay attention to? or once this is merged and a new tag is created, those changes will be communicated in the release?

Correct, plus we are not v1.x, yet so we can change things.

[yeya24]

What's the replacement for this now? Trying to upgrade the version on the Thanos side

[bwplotka]

It's InstrumentedRunnable{Runnable: e2e.NewError... (:

[yeya24]

Removing this breaks the meta monitoring E2E test so probably we need to add it back.