Skip to content

[WIP]add postgresql plugin agent sql param show#3618

Closed
ghost wants to merge 14 commits intomasterfrom
unknown repository
Closed

[WIP]add postgresql plugin agent sql param show#3618
ghost wants to merge 14 commits intomasterfrom
unknown repository

Conversation

@ghost
Copy link
Copy Markdown

@ghost ghost commented Oct 14, 2019

Please answer these questions before submitting pull request

  • Why submit this pull request?

  • Bug fix

  • New feature provided

  • Improve performance

  • Related issues


Bug fix

  • Bug description.

  • How to fix?


New feature or improvement

  • Describe the details and related test reports.

@wu-sheng
Copy link
Copy Markdown
Member

I didn't see rebase :) Still a lot of commits.

@wu-sheng wu-sheng added agent Language agent related. feature New feature plugin Plugin for agent or collector. Be used to extend the capabilities of default implementor. labels Oct 14, 2019
@wu-sheng
Copy link
Copy Markdown
Member

You should learn from the existing case(https://github.com/apache/skywalking/tree/master/test/plugin/scenarios/solrj-7.x-scenario), and move the following two into the main repo.

  1. https://github.com/SkyAPMTest/agent-auto-integration-testcases/tree/master/postgresql-scenario
  2. https://github.com/SkyAPMTest/agent-auto-integration-testcases/tree/master/postgresql-9.4-scenario

You could ping me on QQ, wechat or mail, as we don't have the document(we are working on that) for the new plugin test development guide.

@wu-sheng
Copy link
Copy Markdown
Member

The old test includes 55 cases, https://github.com/SkyAPMTest/agent-integration-test-report#postgresql. Last time, we passed. This should be easy work to move them in.

@wu-sheng wu-sheng added the TBD To be decided later, need more discussion or input. label Oct 14, 2019
@ghost
Copy link
Copy Markdown
Author

ghost commented Oct 14, 2019

@wu-sheng ok,thanks

@wu-sheng
Copy link
Copy Markdown
Member

Any update about the postgre test PR?

@ghost
Copy link
Copy Markdown
Author

ghost commented Oct 17, 2019

WIP

Copy link
Copy Markdown
Member

@kezhenxu94 kezhenxu94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't looked carefully yet, but just point out some obvious issues, and remember to add the config item into the documentations

* If set to true, the parameters of the sql (typically {@link java.sql.PreparedStatement}) would be
* collected.
*/
public static boolean TRACE_SQL_PARAMETERS = true;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default value should be false due to possible performance issue

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How much impact on performance?

Comment on lines +55 to +61
if (Config.Plugin.PostgreSQL.TRACE_SQL_PARAMETERS) {
final Set<String> setters = ignorable ? PS_IGNORABLE_SETTERS : PS_SETTERS;
for (String setter : setters) {
matcher = matcher.or(named(setter));
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simply collapse this to line 48

        if (Config.Plugin.MySQL.TRACE_SQL_PARAMETERS || Config.Plugin.PostgreSQL.TRACE_SQL_PARAMETERS) {
            final Set<String> setters = ignorable ? PS_IGNORABLE_SETTERS : PS_SETTERS;
            for (String setter : setters) {
                matcher = matcher.or(named(setter));
             }
         }

Comment on lines +81 to +84
while ((findPos = sql.indexOf("?", startPos)) > 0) {
resultSql.append(sql.substring(startPos, findPos));
resultSql.append("'");
resultSql.append(parameters[index++]);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unsafe to parse the SQL using ?, are there possibilities when the sql includes ? itself in the value, not a parameter placeholder?

select * from news_table where author=? and title like '%?', I know it's a little tricky, just to remind the risk here

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point is not to try to parse the SQL, store the parameters in another tag

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parsing SQL is the next generation after ShardingShpere ( @tristaZero and @terrymanu ) provides the real SQL Parser. We don't do this by ourselves.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wu-sheng @kezhenxu94 ACK,i will store the parameters in another tag.

@wu-sheng
Copy link
Copy Markdown
Member

Please fix the conflict. And we could continue this now.

@wu-sheng wu-sheng removed the TBD To be decided later, need more discussion or input. label Oct 20, 2019
@wu-sheng
Copy link
Copy Markdown
Member

This should continue.

@ghost
Copy link
Copy Markdown
Author

ghost commented Oct 23, 2019

ok

@ghost ghost changed the title add postgresql plugin agent sql param show [WIP]add postgresql plugin agent sql param show Oct 23, 2019
@wu-sheng
Copy link
Copy Markdown
Member

Please resolve the conflict.

@ghost
Copy link
Copy Markdown
Author

ghost commented Oct 23, 2019

refer #3695

@wu-sheng
Copy link
Copy Markdown
Member

Should close this one?

@kezhenxu94
Copy link
Copy Markdown
Member

replaced by #3695

@kezhenxu94 kezhenxu94 closed this Oct 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Language agent related. feature New feature plugin Plugin for agent or collector. Be used to extend the capabilities of default implementor.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants