Skip to content

[R] glimpse() errors if there is a UDF #20371

@asfimport

Description

@asfimport

Using the example from ARROW-17437:

register_scalar_function(
  "test", 
  function(context, x) paste(x, collapse=","), 
  utf8(), 
  utf8(), 
  auto_convert=TRUE
)

Table$create(x = c("a", "b", "c")) |>
  transmute(test(x)) |>
  glimpse()

# Table (query)
# 3 rows x 1 columns
# Error in `dplyr::collect()`:
# ! NotImplemented: Call to R (resolve scalar user-defined function output data type) from a non-R thread from an unsupported context
# Run `rlang::last_error()` to see where the error occurred.

A variety of things could fix this:

  • Supporting UDFs in any query (I think there's a draft PR open for this)
  • The limit operator (FetchNode?) so that head() is handled in the ExecPlan and we don't need to use the RecordBatchReader workaround to get it efficiently (also PR in the works)
  • Worse case, error more informatively

Reporter: Neal Richardson / @nealrichardson

Related issues:

Note: This issue was originally created as ARROW-17438. Please see the migration documentation for further details.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions