Skip to content

allow users to specify the unit of disk size #9998

@FrankChen021

Description

@FrankChen021

Description

For each historical node, we have to configure the path and size for segment storage as below

druid.segmentCache.locations=[{"path":"/disk1/var/druid/segment-cache","maxSize":450000000000},"path":"/disk2/var/druid/segment-cache","maxSize":450000000000}]
druid.server.maxSize=900000000000

At the first glance of these configurations, does anyone know how many bytes are there?

In most of cases, the maxSize is hundreds of GB or TB, but current unit of maxSize is BYTE, which means the length of the value can be very long.

It's very hard to tell how many bytes we have configured for each disk at the first glance of these configs. Taking above number 450000000000 as an example, we have to count digit one by one to know it's 450GB. We have to set the value very carefully every time we add a new disk. It's painful.

So my suggestion here is to allow user to set maxSize in a simpler way: allow them to specify the unit of the value, just like how we set the heap memory size of JVM.

We could config it as below:

druid.segmentCache.locations=[
{"path":"/disk1/var/druid/segment-cache","maxSize":"450G"}, 
{"path":"/disk2/druid/var/druid/segment-cache","maxSize":"1T"},
{"path":"/disk3/druid/var/druid/segment-cache","maxSize":"500M"},
{"path":"/disk4/druid/var/druid/segment-cache","maxSize":500000000},
druid.server.maxSize="1.5T"

In summary, the value of size could be:

  1. a number in bytes(current way)
  2. a string number with a character suffix representing unit of the number
    1. suffix can be M/G/T, case insensitive. (M is 1,000,000 instead of 1,048,576)
    2. no need to support Kilobyte

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions