Add a metric on the number of uploaded files#457
Conversation
This commit adds a new Prometheus metric to track how much files we uploaed to S3 or added to the database. This will be useful to track the usefulness of parallel S3 upload once we land it.
|
Can we also track the number of failed uploads? That way we can also try to warn if we're failing more than 50% of uploads or so. I'm also not quite sure -- it might be that we'd need some data which would allow us to associate these with a particular build, i.e., to allow us to say "It used to take 5 minutes at the 85 percentile to upload documentation for a crate, it now takes 2." -- I think if we can't make such a statement this metric isn't quite succeeding :) |
I don't think that's useful, as we panic if a single file fails to upload more than 3 times so it would get incremented only one time. The crate should be marked as failed in that case, and we already track those.
I'm not really sure how it's possible to express that in prometheus. The only way I can think of is to add a label to the metric for each crate, but prometheus doesn't recommend that as each label is stored and processed separately. We'll still probably be able to glance at the improvements by looking at the graphs, as there is a natural gap (while the docs are building) between crates. |
|
Makes sense to both points. I wasn't sure if there was some good way to do it, sounds like no :/ r=me then |
|
Deployed this. |
This commit adds a new Prometheus metric to track how much files we uploaed to S3 or added to the database. This will be useful to track the usefulness of parallel S3 upload once we land it.
cc #393
r? @Mark-Simulacrum