Skip to content

[R] RecordBatchStreamWriter api #20327

@asfimport

Description

@asfimport

To support the "Writing and Reading Streams" section of the vignette, perhaps we should rely more on the RecordBatchStreamWriter class and less the write_record_batch function.

We should be able to write code resembling the python api :

batch <- ... 
sink <- buffer_output_stream()
writer <- record_batch_stream_writer(sink, batch$schema())
writer$write_batch()
writer$close()
sink$getvalue()

Most of the code is there, but we need to add

  • RecordBatchStreamWriter$write_batch() : write a record batch to the stream. We already have RecordBatchStreamWriter$WriteRecordBatch

  • RecordBatchStreamWriter$close() : not sure why it is lower case close() in python but upper case in C++. We already have RecordBatchWriter$Close()

  • BufferOutputStream$getvalue() : we already have BufferOutputStream$Finish()

    Currently the constructor for a BufferOutputStream is buffer_output_stream(), perhaps we can align with python and make it BufferOutputStream, that would not clash with the arrow::BufferOutputStream class because of the namespacing.

Reporter: Romain Francois / @romainfrancois
Assignee: Romain Francois / @romainfrancois

PRs and other links:

Note: This issue was originally created as ARROW-3842. Please see the migration documentation for further details.

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions