[R] Version 1.0.1 breaks data.frame attributes when reading file written by 2.0.0

[asfimport]

How to reproduce

• Create a data frame:

  
  df <- data.frame(col1 = 1:100)
  

• Write it to parquet file using apache 2.0.0. The demo uses R 3.6 but same happens if you use R 4.0

• Read the parquet file using apache 1.0.1. I only tried that in R 3.6

  Expected

  The data frame is the same as it was before:

  
  structure(list(col1 = 1:100), row.names = c(NA, 100L), class = "data.frame")
  

  Actual

  The data frame has lost some information:

  
  structure(list(1:100), class = "data.frame")
  

  Demo

  I'm not sure what the easiest way is to put up a demo project for this, since you need to switch between arrow installations. But I've created this docker based demo:

  https://github.com/fdlk/arrow2/

Reporter: Fleur Kelpin
Assignee: Jonathan Keane / @jonkeane

PRs and other links:

• GitHub Pull Request #9107
• GitHub Pull Request #9118

_{Note: This issue was originally created as ARROW-10623. Please see the migration documentation for further details.}

[asfimport]

Neal Richardson / @nealrichardson:
The problem is that arrow 2.0 knows how to preserve data.frame attributes in Arrow schema metadata but arrow 1.0 did not--it only preserved column attributes. Unfortunately the code in arrow 1.0 is seeing the new metadata written by 2.0 and erases the existing data.frame attributes but doesn't know how to restore correctly the new ones.

A workaround for now would be to drop the metadata before converting to a data.frame, like:

# Using arrow 1.0.x
tab <- read_parquet("file-written-by-arrow-2-0-0.parquet", as_data_frame = FALSE)
tab$metadata$r <- NULL
df <- as.data.frame(tab)

[asfimport]

Neal Richardson / @nealrichardson:
Issue resolved by pull request 9118
#9118