Web console: Do not put __time in the dimensions list#11085
Web console: Do not put __time in the dimensions list#11085vogievetsky merged 2 commits intoapache:masterfrom
Conversation
| { | ||
| type: 'expression', | ||
| name: 'barPage', | ||
| expression: "concat('foo' + page)", |
There was a problem hiding this comment.
heh, barPage concats 'foo'?
There was a problem hiding this comment.
This test is also verifying that running this function on transforms that have the same expression does not does not cause an asynchronous desynchronization of the DOM rendering thread with the memory management subsystem thereby causing memory liquefaction, which will cause a memory leek and water damage the browser.
There was a problem hiding this comment.
good catch! we don't want our Druid bits to get too corroded from any water spillover into the query request packets, else we might one day wake up to find all our code has been converted to rust.
|
Could you add an end-to-end test too? Manipulations of |
|
I'll see what I can do |
| /* page */ 'Talk:Oswald Tilghman', | ||
| /* regionIsoCode */ 'null', | ||
| /* regionName */ 'null', | ||
| /* time */ '2015-09-12T00:46:58.771Z', |
There was a problem hiding this comment.
I updated the e2e test and realized/was faced with this interesting side-effect:
If you specify the time column via the timestampSpec then that column gets excluded from the dimension auto-detection system. If instead you use a transform to construct a __time dimension then (obviously) the component dimensions do not get removed from the auto detection. In this case I set __time to be timestamp_parse("time") + 1 and now clicking though all the defaults yields an extra field, the original time column.
There was a problem hiding this comment.
Does this cause a problem? It's documented behavior (item 3): https://druid.apache.org/docs/latest/ingestion/index.html#inclusions-and-exclusions
There was a problem hiding this comment.
No, it is not a problem. Just something that took me by surprise. Simply because I did not put 1 + 1 together. Maybe there should be some docs of this inside the web console.
3 participants
A change made in #10267 to forbid
__timein the dimensions list has exposed the fact that the transform step of the data loader did just that.