Enable NatssChannel control plane metrics

[marcobebway]

Helps knative/eventing#1693.

Proposed Changes

• Enable NatssChannel control plane metrics.

Release Note

 the natsschannel-controller and the natsschannel-dispatcher are scrapable

[knative-prow-robot]

Hi @marcobebway. Thanks for your PR.

I'm waiting for a knative member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Harwayne]

Looks good to me, but I'll leave it to @nachocano to LGTM.

[nachocano]

/ok-to-test

[nachocano]

I think you are not using the controller stats_reporter from pkg...
The controller uses the stats_reporter from natsschannel, this one https://github.com/knative/eventing-contrib/blob/master/natss/pkg/reconciler/stats_reporter.go
So you are reporting natsschannel_ready_count and natsschannel_ready_latency...
Can you update the doc?

[marcobebway]

We are using the controller impl in the pkg which in turn uses the stats_reporter in the pkg:

• https://github.com/marcobebway/eventing-contrib/blob/master/natss/cmd/channel_controller/main.go#L35
• https://github.com/marcobebway/eventing-contrib/blob/master/natss/cmd/channel_controller/main.go#L95

[nachocano]

Ohh my bad, you are right.
But check that we also create the other stats_reporter here:

eventing-contrib/natss/pkg/reconciler/reconciler.go

Line 141 in ad4569b

statsReporter := opt.StatsReporter

which reports ready counts and latencies as well.

I think I'd rather leave this part of the documentation as:
Access NatssChannel controller metrics http://localhost:9091/metrics.
and not mention a specific line in the source code (https://github.com/knative/pkg/blob/master/controller/stats_reporter.go#L35-L37), as that might change, and the documentation will quickly become stale.

Do you agree?

[marcobebway]

Yes, I agreed. I removed it from the markdown file.

Also, I verified that we can see the following metrics:

knative/pkg/controller/stats_reporter.go

• work_queue_depth
• reconcile_count
• reconcile_latency

knative/eventing-contrib/natss/pkg/reconciler/stats_reporter.go

• natsschannel_ready_count
• natsschannel_ready_latency

Click for more details

# HELP natsschannel_controller_natsschannel_ready_count Number of NatssChannel that became ready
# TYPE natsschannel_controller_natsschannel_ready_count counter
natsschannel_controller_natsschannel_ready_count{key="default/my-test-channel",reconciler="natss-ch-controller"} 1
natsschannel_controller_natsschannel_ready_count{key="test-natss/natss-kn2-ingress",reconciler="natss-ch-controller"} 1
natsschannel_controller_natsschannel_ready_count{key="test-natss/natss-kn2-trigger",reconciler="natss-ch-controller"} 1
# HELP natsschannel_controller_natsschannel_ready_latency Time it takes for a NatssChannel to become ready since created
# TYPE natsschannel_controller_natsschannel_ready_latency gauge
natsschannel_controller_natsschannel_ready_latency{key="default/my-test-channel",reconciler="natss-ch-controller"} 494
natsschannel_controller_natsschannel_ready_latency{key="test-natss/natss-kn2-ingress",reconciler="natss-ch-controller"} 804
natsschannel_controller_natsschannel_ready_latency{key="test-natss/natss-kn2-trigger",reconciler="natss-ch-controller"} 381
# HELP natsschannel_controller_reconcile_count Number of reconcile operations
# TYPE natsschannel_controller_reconcile_count counter
natsschannel_controller_reconcile_count{key="default/my-test-channel",reconciler="NatssChannels",success="true"} 3
natsschannel_controller_reconcile_count{key="test-natss/natss-kn2-ingress",reconciler="NatssChannels",success="true"} 5
natsschannel_controller_reconcile_count{key="test-natss/natss-kn2-trigger",reconciler="NatssChannels",success="true"} 3
# HELP natsschannel_controller_reconcile_latency Latency of reconcile operations
# TYPE natsschannel_controller_reconcile_latency histogram
natsschannel_controller_reconcile_latency_bucket{key="default/my-test-channel",reconciler="NatssChannels",success="true",le="10"} 2
natsschannel_controller_reconcile_latency_bucket{key="default/my-test-channel",reconciler="NatssChannels",success="true",le="100"} 3
natsschannel_controller_reconcile_latency_bucket{key="default/my-test-channel",reconciler="NatssChannels",success="true",le="1000"} 3
natsschannel_controller_reconcile_latency_bucket{key="default/my-test-channel",reconciler="NatssChannels",success="true",le="10000"} 3
natsschannel_controller_reconcile_latency_bucket{key="default/my-test-channel",reconciler="NatssChannels",success="true",le="30000"} 3
natsschannel_controller_reconcile_latency_bucket{key="default/my-test-channel",reconciler="NatssChannels",success="true",le="60000"} 3
natsschannel_controller_reconcile_latency_bucket{key="default/my-test-channel",reconciler="NatssChannels",success="true",le="+Inf"} 3
natsschannel_controller_reconcile_latency_sum{key="default/my-test-channel",reconciler="NatssChannels",success="true"} 33
natsschannel_controller_reconcile_latency_count{key="default/my-test-channel",reconciler="NatssChannels",success="true"} 3
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-ingress",reconciler="NatssChannels",success="true",le="10"} 4
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-ingress",reconciler="NatssChannels",success="true",le="100"} 5
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-ingress",reconciler="NatssChannels",success="true",le="1000"} 5
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-ingress",reconciler="NatssChannels",success="true",le="10000"} 5
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-ingress",reconciler="NatssChannels",success="true",le="30000"} 5
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-ingress",reconciler="NatssChannels",success="true",le="60000"} 5
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-ingress",reconciler="NatssChannels",success="true",le="+Inf"} 5
natsschannel_controller_reconcile_latency_sum{key="test-natss/natss-kn2-ingress",reconciler="NatssChannels",success="true"} 49.99999999999999
natsschannel_controller_reconcile_latency_count{key="test-natss/natss-kn2-ingress",reconciler="NatssChannels",success="true"} 5
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-trigger",reconciler="NatssChannels",success="true",le="10"} 2
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-trigger",reconciler="NatssChannels",success="true",le="100"} 3
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-trigger",reconciler="NatssChannels",success="true",le="1000"} 3
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-trigger",reconciler="NatssChannels",success="true",le="10000"} 3
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-trigger",reconciler="NatssChannels",success="true",le="30000"} 3
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-trigger",reconciler="NatssChannels",success="true",le="60000"} 3
natsschannel_controller_reconcile_latency_bucket{key="test-natss/natss-kn2-trigger",reconciler="NatssChannels",success="true",le="+Inf"} 3
natsschannel_controller_reconcile_latency_sum{key="test-natss/natss-kn2-trigger",reconciler="NatssChannels",success="true"} 27.999999999999996
natsschannel_controller_reconcile_latency_count{key="test-natss/natss-kn2-trigger",reconciler="NatssChannels",success="true"} 3
# HELP natsschannel_controller_work_queue_depth Depth of the work queue
# TYPE natsschannel_controller_work_queue_depth gauge
natsschannel_controller_work_queue_depth{reconciler="NatssChannels"} 0

[nachocano]

the dispatchers currently do not report stuff. But I think we can add this for now, and then see what metrics we want for all channel dispatchers...

[nachocano]

my bad, reporting at least what the pkg controller stats_reporter does...

[nachocano]

Thanks for doing this @marcobebway
Just a small comment on documentation and I think it's ready to be merged!

[nachocano]

Seems that is complaining regarding the md file.

natss/docs/metrics.md:20: MD007 Unordered list indentation
natss/docs/metrics.md:39: MD007 Unordered list indentation
natss/docs/metrics.md:5: MD012 Multiple consecutive blank lines
natss/docs/metrics.md:71: MD012 Multiple consecutive blank lines
natss/docs/metrics.md:54: MD022 Headers should be surrounded by blank lines
natss/docs/metrics.md:9: MD031 Fenced code blocks should be surrounded by blank lines
natss/docs/metrics.md:14: MD031 Fenced code blocks should be surrounded by blank lines
natss/docs/metrics.md:55: MD031 Fenced code blocks should be surrounded by blank lines
natss/docs/metrics.md:62: MD031 Fenced code blocks should be surrounded by blank lines
natss/docs/metrics.md:75: MD031 Fenced code blocks should be surrounded by blank lines
natss/docs/metrics.md:83: MD031 Fenced code blocks should be surrounded by blank lines
natss/docs/metrics.md:69: MD034 Bare URL used
natss/docs/metrics.md:93: MD034 Bare URL used

[nachocano]

/lgtm
@Harwayne for approval

[nachocano]

actually, we need @Abd4llA for approval

[marcobebway]

/assign @Abd4llA

[devguyio]

/approve

[knative-prow-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Abd4llA, marcobebway

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• natss/OWNERS [Abd4llA]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[devguyio]

/lgtm