Improve megatron dataset preprocessing script and update docs#918

kevalmorabia97 merged 1 commit intomainNVIDIA/Model-Optimizer:mainfrom

kmorabia/megatron-data-preprocess-cleanupNVIDIA/Model-Optimizer:kmorabia/megatron-data-preprocess-cleanupCopy head branch name to clipboard