BigFileSorter: Auto assing block size based in the input size

[MarcosMeli]

Check the size of the input file and assign a block size

[theAshenfire]

Hi Marcos, I'd love to take on this task for you! Is there a specific requirement on how the block size is determined, or samples i can base the decision on?

[borowszky]

is this still open? Would like to contribute

[mcavigelli]

Yes, it is still open, Matthias

[PentaPinguino]

Hello,

I believe this issue has been solved in 2016 in commit 028a2a4

FileHelpers/FileHelpers/Sort/BigFileSorter.T.cs

Lines 138 to 151 in ecc07e9

	/// <summary>
	/// Sort a file from one filename to another filename
	/// </summary>
	/// <remarks>Handles very large files</remarks>
	/// <param name="sourceFile">File to read in</param>
	/// <param name="destinationFile">File to write out</param>
	public void Sort(string sourceFile, string destinationFile)
	{
	if (AutoSetBlockSize) BlockFileSizeInBytes = BlockSizeFromFileSize(sourceFile);
	var parts = new List<string>();
	var engine = SplitAndSortParts(sourceFile, parts);
	var queues = CreateQueues(parts);
	MergeTheChunks(queues, destinationFile, engine.HeaderText, engine.FooterText);
	}

FileHelpers/FileHelpers/Sort/BigFileSorter.T.cs

Lines 218 to 239 in ecc07e9

	/// <summary>
	/// Returns a block size based on the source file size
	/// </summary>
	/// <param name="sourceFile">Source file path</param>
	/// <returns></returns>
	private int BlockSizeFromFileSize(string sourceFile) {
	int blockSize = DefaultBlockSize;
	var fi = new FileInfo(sourceFile);
	var fileSize = fi.Length;
	var initialBlockSize = (int)(fileSize / AutoBlockSizeRatio);
	if (initialBlockSize >= MinBlockSize && initialBlockSize <= MaxBufferSize)
	blockSize = initialBlockSize;
	else if (initialBlockSize < MinBlockSize)
	blockSize = MinBlockSize;
	else
	blockSize = MaxBufferSize;
	return blockSize;
	}

[kowalkos]

is it still open?