[C++] CSV stream reader has no progress indication

[asfimport]

The StreamingReader API does not provide a way to detect progress of the stream. It would be useful to have a methods to expose number of bytes read as well as total number of bytes in the csv file so that progress can be calculated.

Reporter: Nate Clark / @n3world
Assignee: Nate Clark / @n3world

Related issues:

• [Python] csv.read_csv progress bar (relates to)

PRs and other links:

• GitHub Pull Request #10509

_{Note: This issue was originally created as ARROW-12996. Please see the migration documentation for further details.}

[asfimport]

Weston Pace / @westonpace:
If your input is a file (or some other source that you can know the size ahead of time) then before you create the streaming reader you can grab the size of the input.  Between that and the batch size you should know exactly how many record batches to expect correct?  Why do you need to add the info to the streaming reader?

[asfimport]

Nate Clark / @n3world:
Yes the user can grab the size before hand so that was more of a convenience method. The batches returned by the streamer are not equal to the batch size of the reader because the end of a row will not always align with the end of the data. The user could estimate the progress relatively closely as long as the size of rows are not really large like multiple megabytes. The progress indicator would just be a more precise implementation.

 

[asfimport]

Antoine Pitrou / @pitrou:
Issue resolved by pull request 10509
#10509