Skip to content

[Enhancement] Convert metric to Json format #3635

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
chaoyli merged 5 commits into from
May 27, 2020
Merged

[Enhancement] Convert metric to Json format #3635

chaoyli merged 5 commits into from
May 27, 2020

Conversation

@chaoyli
Copy link
Contributor

@chaoyli chaoyli commented May 19, 2020

Add a JSON format for existing metrics like this.

{
    "tags":
    {
        "metric":"thread_pool",
        "name":"thrift-server-pool",
        "type":"active_thread_num"
    },
    "unit":"number",
    "value":3
}

I add a new JsonMetricVisitor to handle the convertion.
It's not to modify existing PrometheusMetricVisitor and SimpleCoreMetricVisitor.
Also I add a unit item to indicate the metric better.

@chaoyli chaoyli self-assigned this May 19, 2020
@chaoyli chaoyli changed the title [Feature] Convert metric to Json format [Enhancement] Convert metric to Json format May 19, 2020
@chaoyli
Copy link
Contributor Author

chaoyli commented May 19, 2020

#3636

morningman
morningman requested changes May 19, 2020
View reviewed changes
@morningman
Copy link
Contributor

morningman commented May 19, 2020

Why we need a json format metric output?

@chaoyli
Copy link
Contributor Author

chaoyli commented May 21, 2020 •
edited
Loading

Why we need a json format metric output?

It's for incorporting another Monitor Tools easily. Json will be a good choice for this demand.

morningman
morningman reviewed May 21, 2020
View reviewed changes
@morningman
Copy link
Contributor

morningman commented May 21, 2020

Why we need a json format metric output?

It's for incorporting another Monitor Tools easily. Json will be a good choice for this demand.

If we adopt this json form of expansion each new line, will it result in a very large result set?
Which monitoring system uses Json?

@chaoyli
Copy link
Contributor Author

chaoyli commented May 25, 2020 •
edited
Loading

Why we need a json format metric output?

It's for incorporating another Monitor Tools easily. Json will be a good choice for this demand.

If we adopt this json form of expansion each new line, will it result in a very large result set?
Which monitoring system uses Json?

I have test it, It will not larger than a query profile.
open-falcon opensourced by Xiaomi is use JSON to collect metric.
https://github.com/open-falcon/falcon-plus

morningman
morningman reviewed May 25, 2020
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, except for some little problem.
And please fix UT.

chaoyli added 4 commits May 26, 2020 20:37
@chaoyli
[Feature] Convert metric to Json format
8befd9b 
Add a JSON format for existing metrics like this.
```
{
    "tags":
    {
        "metric":"thread_pool",
        "name":"thrift-server-pool",
        "type":"active_thread_num"
    },
    "unit":"number",
    "value":3
}
```
I add a new JsonMetricVisitor to handle the convertion.
It's not to modify existing PrometheusMetricVisitor and SimpleCoreMetricVisitor.
Also I add a unit item to indicate the metric better.
@chaoyli
[Feature] Convert metric to Json format
b5d2f75 
Add a JSON format for existing metrics like this.
```
{
    "tags":
    {
        "metric":"thread_pool",
        "name":"thrift-server-pool",
        "type":"active_thread_num"
    },
    "unit":"number",
    "value":3
}
```
I add a new JsonMetricVisitor to handle the convertion.
It's not to modify existing PrometheusMetricVisitor and SimpleCoreMetricVisitor.
Also I add a unit item to indicate the metric better.

I also add cloning tablet into statistic divided by database.
@chaoyli
[Feature] Change some function name to better understanding
f178894 
And also use white space to replace newline in audit.log
@chaoyli
Use rapidjson instead
cf25ffb
@chaoyli
Copy link
Contributor Author

chaoyli commented May 26, 2020 •
edited
Loading

LGTM, except for some little problem.
And please fix UT.

I have fixed the UT

morningman
morningman approved these changes May 26, 2020
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman added the approved Indicates a PR has been approved by one committer. label May 26, 2020
@chaoyli chaoyli merged commit 1cc78fe into apache:master May 27, 2020
@morningman morningman mentioned this pull request May 27, 2020
Closed
baidu-doris pushed a commit to baidu-doris/incubator-doris that referenced this pull request Jun 17, 2020
@chaoyli
[Enhancement] Convert metric to Json format (apache#3635)
b1d6b8f 
Add a JSON format for existing metrics like this.
```
{
    "tags":
    {
        "metric":"thread_pool",
        "name":"thrift-server-pool",
        "type":"active_thread_num"
    },
    "unit":"number",
    "value":3
}
```
I add a new JsonMetricVisitor to handle the transformation.
It's not to modify existing PrometheusMetricVisitor and SimpleCoreMetricVisitor.
Also I add
1.  A unit item to indicate the metric better
2. Cloning tablet statistics divided by database.
3. Use white space to replace newline in audit.log
acelyc111 pushed a commit to acelyc111/incubator-doris that referenced this pull request Jan 20, 2021
@chaoyli @acelyc111
[Enhancement] Convert metric to Json format (apache#3635)
6f7bf60 
Add a JSON format for existing metrics like this.
```
{
    "tags":
    {
        "metric":"thread_pool",
        "name":"thrift-server-pool",
        "type":"active_thread_num"
    },
    "unit":"number",
    "value":3
}
```
I add a new JsonMetricVisitor to handle the transformation.
It's not to modify existing PrometheusMetricVisitor and SimpleCoreMetricVisitor.
Also I add
1.  A unit item to indicate the metric better 
2. Cloning tablet statistics divided by database.
3. Use white space to replace newline in audit.log
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

Assignees

@chaoyli chaoyli

Labels

approved Indicates a PR has been approved by one committer. kind/improvement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@chaoyli @morningman