Skip to content

Conversation

@alamb
Copy link
Contributor

@alamb alamb commented Jun 17, 2024

Which issue does this PR close?

Closes #.

Rationale for this change

While profiling code for for CSE copy removal (#10835) and noticed that 5% of the time was spent in Column::from_qualified_name (see screen shot)

Screenshot 2024-06-15 at 2 58 38 PM

When I looked into this a bit more, it seems to be copying the string twice on the error path which can be avoided.

I doubt this will matter much in practice but I also think the code is cleaner this way.

What changes are included in this PR?

Only make a string when needed

Are these changes tested?

Are there any user-facing changes?

/// Treats the name as a SQL identifier. For example
/// `foo.BAR` would be parsed to a reference to relation `foo`, column name `bar` (lower case)
/// where `"foo.BAR"` would be parsed to a reference to column named `foo.BAR`
pub fn from_qualified_name(flat_name: impl Into<String>) -> Self {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could make this significantly better for the common case (non error) by changing the signature to take &str but that would be a breaking change, and I think we should only do it if benchmarking shows it helps.

Most of the time is actually taken parsing the identifier

@alamb alamb force-pushed the alamb/less_copy_in_df_schema branch from 1a6048d to 5d3ec72 Compare June 17, 2024 10:28
Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @alamb I had this from_qualified_name on my radar!

@comphead comphead merged commit e1cfb48 into apache:main Jun 17, 2024
@alamb alamb deleted the alamb/less_copy_in_df_schema branch June 17, 2024 17:58
findepi pushed a commit to findepi/datafusion that referenced this pull request Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants