clean up intermediate_pushes directory for LocalDataSegmentPusher

[dclim]

intermediateDirFor() used to create intermediate directories like:

/druid/localStorage/intermediate_pushes/myDatasource/2016-05-14T00:00:00.000Z_2016-05-15T00:00:00.000Z/2018-01-30T07:22:02.796Z/0.73253533-1fd0-4787-9240-d01f3faad752

with only the innermost directory being cleaned up (0.73253533-1fd0-4787-9240-d01f3faad752), leaving lots of empty directories on the filesystem.

Switched to a flat /druid/localStorage/intermediate_pushes/{UUID} which is removed on completion.

[himanshug]

👍

however, just wanted to note for the record:
another alternative would be to clean all the other directories as well which is possible given that we generate the intermediate path here and know all the components in the path.

I usually like paths to give information, current path gives some information about what indexing task would've caused to create those files etc. For example if the directories are not cleaned up, then you would have some idea about which tasks might have created them and what went wrong. In this particular case, it isn't really important though because temp dir creation/deletion happens so closely. hence, I'm not as strongly opinionated on above alternative.