Skip to content

when does druid delete file from basePersist,I have so many file in realtime basePersist dir #1933

@492066199

Description

@492066199

I set segmentGranularity to "HOUR",use the timezone utc and not the local,there are so many file in realtime basePersist dir,and little file in localStorage.this below is the file list.
basePersist dir (in datasource dir, the same data source)
drwxr-xr-x 45 root root 4096 Nov 7 04:16 2015-11-06T08:00:00.000Z_2015-11-06T09:00:00.000Z
drwxr-xr-x 46 root root 4096 Nov 7 02:42 2015-11-06T09:00:00.000Z_2015-11-06T10:00:00.000Z
drwxr-xr-x 49 root root 4096 Nov 7 01:02 2015-11-06T10:00:00.000Z_2015-11-06T11:00:00.000Z
drwxr-xr-x 56 root root 4096 Nov 6 21:01 2015-11-06T12:00:00.000Z_2015-11-06T13:00:00.000Z
drwxr-xr-x 62 root root 4096 Nov 7 10:20 2015-11-06T13:00:00.000Z_2015-11-06T14:00:00.000Z
drwxr-xr-x 66 root root 4096 Nov 7 07:14 2015-11-06T14:00:00.000Z_2015-11-06T15:00:00.000Z
drwxr-xr-x 64 root root 4096 Nov 7 00:19 2015-11-06T15:00:00.000Z_2015-11-06T16:00:00.000Z
drwxr-xr-x 41 root root 4096 Nov 7 10:20 2015-11-06T16:00:00.000Z_2015-11-06T17:00:00.000Z
drwxr-xr-x 21 root root 4096 Nov 7 02:05 2015-11-06T17:00:00.000Z_2015-11-06T18:00:00.000Z
drwxr-xr-x 11 root root 4096 Nov 7 03:09 2015-11-06T18:00:00.000Z_2015-11-06T19:00:00.000Z
drwxr-xr-x 7 root root 4096 Nov 7 04:15 2015-11-06T19:00:00.000Z_2015-11-06T20:00:00.000Z
drwxr-xr-x 6 root root 4096 Nov 7 05:17 2015-11-06T20:00:00.000Z_2015-11-06T21:00:00.000Z
drwxr-xr-x 7 root root 4096 Nov 7 06:10 2015-11-06T21:00:00.000Z_2015-11-06T22:00:00.000Z
drwxr-xr-x 12 root root 4096 Nov 7 07:03 2015-11-06T22:00:00.000Z_2015-11-06T23:00:00.000Z
drwxr-xr-x 24 root root 4096 Nov 7 08:02 2015-11-06T23:00:00.000Z_2015-11-07T00:00:00.000Z
drwxr-xr-x 34 root root 4096 Nov 7 09:01 2015-11-07T00:00:00.000Z_2015-11-07T01:00:00.000Z
drwxr-xr-x 41 root root 4096 Nov 7 10:01 2015-11-07T01:00:00.000Z_2015-11-07T02:00:00.000Z
drwxr-xr-x 16 root root 4096 Nov 7 10:20 2015-11-07T02:00:00.000Z_2015-11-07T03:00:00.000Z
localStorage dir (in the same data source)
drwxr-xr-x 3 root root 4096 Nov 6 15:56 2015-11-06T05:00:00.000Z_2015-11-06T06:00:00.000Z
drwxr-xr-x 3 root root 4096 Nov 6 17:15 2015-11-06T06:00:00.000Z_2015-11-06T07:00:00.000Z
drwxr-xr-x 3 root root 4096 Nov 6 18:41 2015-11-06T07:00:00.000Z_2015-11-06T08:00:00.000Z
drwxr-xr-x 3 root root 4096 Nov 7 04:14 2015-11-06T08:00:00.000Z_2015-11-06T09:00:00.000Z
drwxr-xr-x 3 root root 4096 Nov 7 02:41 2015-11-06T09:00:00.000Z_2015-11-06T10:00:00.000Z
drwxr-xr-x 3 root root 4096 Nov 7 01:00 2015-11-06T10:00:00.000Z_2015-11-06T11:00:00.000Z
drwxr-xr-x 3 root root 4096 Nov 6 22:58 2015-11-06T11:00:00.000Z_2015-11-06T12:00:00.000Z
drwxr-xr-x 3 root root 4096 Nov 7 10:18 2015-11-06T13:00:00.000Z_2015-11-06T14:00:00.000Z
drwxr-xr-x 3 root root 4096 Nov 7 07:12 2015-11-06T14:00:00.000Z_2015-11-06T15:00:00.000Z
I can search the data well,is there something wrong with druid.
thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions