Conversation
|
Hi, @clintropolis! I think thrift is the last extension which use |
clintropolis
left a comment
clintropolis
left a comment
There was a problem hiding this comment.
very sorry for the delayed review, the changes overall lgtm 👍
| |flattenSpec| JSON Object |Define a [`flattenSpec`](#flattenspec) to extract nested values from a Parquet file. Note that only 'path' expression are supported ('jq' is unavailable).| no (default will auto-discover 'root' level properties) | | ||
| | binaryAsString | Boolean | Specifies if the bytes parquet column which is not logically marked as a string or enum type should be treated as a UTF-8 encoded string. | no (default = false) | | ||
|
|
||
| ### Thrift Stream |
There was a problem hiding this comment.
hmm, I'm not sure we have any other 'contrib' extensions described in this section, it might be best if this lives in https://github.com/apache/druid/blob/master/docs/development/extensions-contrib/thrift.md for now. On the other hand, thrift i think is the only data format that isn't a core extension (maybe in the future we should just consider adding integration tests and making it a core extension?), so maybe it is ok to be here. @techdocsmith do you have any thoughts?
Also, looking closer at the code, I guess this might also work with batch ingestion too since the deserializer detects the format based on the bytes given to it, though I haven't personally used this extension or tested this scenario. I'll see if I can find some time to pull your branch and test it out
There was a problem hiding this comment.
+1 for @clintropolis suggestion to keep the doc in /docs/development/extensions-contrib/thrift.md until the extension is made core.
| <dependency> | ||
| <groupId>com.fasterxml.jackson.core</groupId> | ||
| <artifactId>jackson-core</artifactId> | ||
| <version>2.10.2</version> |
There was a problem hiding this comment.
i think versions on a lot of these should be already defined in the top level pom (the dependency checker in travis sometimes suggests more than is necessary to fix the issue)
|
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If you think that's incorrect or this pull request should instead be reviewed, please simply write any comment. Even if closed, you can still revive the PR at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions. |
|
@bananaaggle - can you resolve these conflicts? it will be great to get rid of the dependency on |
|
This issue is no longer marked as stale. |
|
This pull request has been marked as stale due to 60 days of inactivity. |
|
This pull request/issue has been closed due to lack of activity. If you think that |
4 participants
Because of deprecated of parseSpec, I develop ThriftInputFormat for new interface, which supports stream ingestion for data encoded by Thrift.
This PR has: