Add Prometheus /metrics handler and some basic metrics

[DrJosh9000]

Description

More ways to look at what an agent is doing! Here are some metrics geared towards monitoring long-lived agents: agent workers running, pings, jobs, and log chunk uploads, plus all the default Go runtime metrics that the library creates.

Agent-specific metrics:

• buildkite_agent_jobs_ended_total
• buildkite_agent_jobs_started_total
• buildkite_agent_logs_bytes_uploaded_total
• buildkite_agent_logs_bytes_uploads_errored_total
• buildkite_agent_logs_chunk_uploads_errored_total
• buildkite_agent_logs_chunks_uploaded_total
• buildkite_agent_logs_upload_duration_seconds_total
• buildkite_agent_pings_actions_total
• buildkite_agent_pings_duration_seconds_total
• buildkite_agent_pings_errors_total
• buildkite_agent_pings_sent_total
• buildkite_agent_pings_wait_duration_seconds_total
• buildkite_agent_workers_ended_total
• buildkite_agent_workers_started_total

Context

https://linear.app/buildkite/issue/PB-920

Changes

• Attach a promhttp.Handler() to the health-check server mux at the standard path, /metrics.
• Define some basic metrics
• Add data to the metrics

Testing

• Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
• Code is formatted (with go fmt ./...)
• A bit of manual testing

Disclosures / Credits

I enjoy this stuff.

[zhming0]

Looks good to me 👍🏿