Skip to content

Ruler API and ObjectStore Backend#2269

Merged
jtlisi merged 11 commits intocortexproject:masterfrom
grafana:20191122_ruler_with_api
Mar 20, 2020
Merged

Ruler API and ObjectStore Backend#2269
jtlisi merged 11 commits intocortexproject:masterfrom
grafana:20191122_ruler_with_api

Conversation

@jtlisi
Copy link
Contributor

@jtlisi jtlisi commented Mar 12, 2020 •
edited
Loading

API Changes

This PR adds an API to the ruler service based on this design document (https://docs.google.com/document/d/1gNKiC5L_zWQ4GGgtdhJJ6FnaZXyJJBvdGTUmF0wgEFg/edit?usp=sharing)

Object Storage Changes

  • It creates a generic storage client for rules based on our chunk.ObjectClient interface
  • The delimiter was made a configurable option for the object storage clients to make this possible. The delimiter is set at the construction of the object storage client

Integration Tests

  • A ruler integration test was added that ensures a rule group can be set, is running in a Prometheus manager, and can be deleted
  • A ruler address was added to the integration test cortex client

Which issue(s) this PR fixes:
Related #1547 #1244

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md

jtlisi added 4 commits March 12, 2020 12:54
@jtlisi
add ruler API backed by object storage
6223db8 
Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>
@jtlisi
simplify and comment ruler storage api
5d19596 
Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>
@jtlisi
wrape calls to ruler with auth middleware
572f9b3 
Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>
@jtlisi
use saner metrics for ruler integration test
a5c0063 
Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>
jtlisi added 4 commits March 12, 2020 16:03
@jtlisi
remove omitempty from configs
7d34690 
Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>
@jtlisi
add clarifying comments for ruler integration tests and reorder metri…
42fa09c 
…c checks

Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>
@jtlisi
update changelog
5cbecc7 
Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>
@jtlisi
update docs
fda86bf 
Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>
@jtlisi jtlisi mentioned this pull request Mar 16, 2020
Closed
pstibrany
pstibrany reviewed Mar 17, 2020
View reviewed changes
Copy link
Contributor

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work.
I'd like to see changes to remove some code duplication in api.go, and unit test or two for escaping namespaces and groups in the request paths, to make sure it's not broken by accident in the future, but overall 👍

jtlisi and others added 2 commits March 18, 2020 12:10
@jtlisi @pstibrany
Update pkg/ruler/api.go
f1d626f 
Co-Authored-By: Peter Štibraný <peter.stibrany@grafana.com>
Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>
@jtlisi
refactor per PR comments
5d88330 
Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>
@jtlisi jtlisi force-pushed the 20191122_ruler_with_api branch from f0c09ba to 5d88330 Compare March 18, 2020 18:12
pstibrany
pstibrany reviewed Mar 19, 2020
View reviewed changes
Copy link
Contributor

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvements! There are few missing returns in methods, my other comments are mostly nitpicking.

pkg/ruler/api.go
// RegisterRoutes registers the ruler API HTTP routes with the provided Router.
func (r *Ruler) RegisterRoutes(router *mux.Router) {
func (r *Ruler) RegisterRoutes(router *mux.Router, middleware middleware.Interface) {
router = router.UseEncodedPath()
Copy link
Contributor

@pstibrany pstibrany Mar 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One possible way to test it would be to have muxVars global variable, that defaults to mux.Vars but could be set to your own function in tests.

@jtlisi
Copy link
Contributor Author

jtlisi commented Mar 19, 2020

One possible way to test it would be to have muxVars global variable, that defaults to mux.Vars but could be set to your own function in tests.

One way would be injecting them into the ctx of the request that is passed to the function. But I feel like it's more important to have integration tests for this behavior since the interplay with the gorilla router is so important.

@jtlisi
fix API per PR comments
0876dd6 
Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>
pstibrany
pstibrany approved these changes Mar 20, 2020
View reviewed changes
Copy link
Contributor

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing my feedback!

@jtlisi jtlisi merged commit a32919e into cortexproject:master Mar 20, 2020
@jtlisi jtlisi mentioned this pull request Mar 25, 2020
Closed
@jtlisi jtlisi deleted the 20191122_ruler_with_api branch July 21, 2020 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@pstibrany pstibrany pstibrany approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

size/XL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jtlisi @pstibrany