Description
For each historical node, we have to configure the path and size for segment storage as below
druid.segmentCache.locations=[{"path":"/disk1/var/druid/segment-cache","maxSize":450000000000},"path":"/disk2/var/druid/segment-cache","maxSize":450000000000}]
druid.server.maxSize=900000000000
At the first glance of these configurations, does anyone know how many bytes are there?
In most of cases, the maxSize is hundreds of GB or TB, but current unit of maxSize is BYTE, which means the length of the value can be very long.
It's very hard to tell how many bytes we have configured for each disk at the first glance of these configs. Taking above number 450000000000 as an example, we have to count digit one by one to know it's 450GB. We have to set the value very carefully every time we add a new disk. It's painful.
So my suggestion here is to allow user to set maxSize in a simpler way: allow them to specify the unit of the value, just like how we set the heap memory size of JVM.
We could config it as below:
druid.segmentCache.locations=[
{"path":"/disk1/var/druid/segment-cache","maxSize":"450G"},
{"path":"/disk2/druid/var/druid/segment-cache","maxSize":"1T"},
{"path":"/disk3/druid/var/druid/segment-cache","maxSize":"500M"},
{"path":"/disk4/druid/var/druid/segment-cache","maxSize":500000000},
druid.server.maxSize="1.5T"
In summary, the value of size could be:
- a number in bytes(current way)
- a string number with a character suffix representing unit of the number
- suffix can be M/G/T, case insensitive. (M is 1,000,000 instead of 1,048,576)
- no need to support Kilobyte
Description
For each historical node, we have to configure the path and size for segment storage as below
At the first glance of these configurations, does anyone know how many bytes are there?
In most of cases, the
maxSizeis hundreds of GB or TB, but current unit ofmaxSizeis BYTE, which means the length of the value can be very long.It's very hard to tell how many bytes we have configured for each disk at the first glance of these configs. Taking above number
450000000000as an example, we have to count digit one by one to know it's450GB. We have to set the value very carefully every time we add a new disk. It's painful.So my suggestion here is to allow user to set
maxSizein a simpler way: allow them to specify the unit of the value, just like how we set the heap memory size of JVM.We could config it as below:
In summary, the value of size could be: