Error from diffdf::diffdf related to deparse(substitute())

[slager]

diffdf::diffdf can throw a confusing error depending on the exact way its arguments are constructed. This appears to be caused by the code's use of non-standard evaluation in trying to reconstruct argument names from its arguments, e.g., deparse(substitute(compare)).

library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
library(diffdf)

# throws error

diffdf(
    as_tibble(mtcars),
    as_tibble(mtcars) %>%
      filter(cyl == 8) %>%
      dplyr::select(
        -mpg,
        -carb
      )
  )
#> Error in `tibble()`:
#> ! Tibble columns must have compatible sizes.
#> • Size 4: Existing data.
#> • Size 5: Column `COMP`.
#> ℹ Only values of size one are recycled.

# does not throw error

mtcars2 <- as_tibble(mtcars)
mtcars3 <- mtcars2 %>%
  filter(cyl == 8) %>%
  dplyr::select(
    -mpg,
    -carb
  )

diffdf(
  mtcars2,
  mtcars3
)
#> Warning in diffdf(mtcars2, mtcars3): 
#> There are rows in BASE that are not in COMPARE !!
#> There are columns in BASE that are not in COMPARE !!
#> Not all Values Compared Equal
#> Differences found between the objects!
#> 
#> Summary of BASE and COMPARE
#>   ==================================================================
#>     PROPERTY             BASE                       COMP            
#>   ------------------------------------------------------------------
#>       Name              mtcars2                    mtcars3          
#>      Class     "tbl_df, tbl, data.frame"  "tbl_df, tbl, data.frame" 
#>     Rows(#)               32                         14             
#>    Columns(#)             11                          9             
#>   ------------------------------------------------------------------
#> 
#> 
#> There are rows in BASE that are not in COMPARE !!
#> First 10 of 18 rows are shown in table below
#>   ===============
#>    ..ROWNUMBER.. 
#>   ---------------
#>         15       
#>         16       
#>         17       
#>         18       
#>         19       
#>         20       
#>         21       
#>         22       
#>         23       
#>         24       
#>   ---------------
#> 
#> 
#> There are columns in BASE that are not in COMPARE !!
#>   =========
#>    COLUMNS 
#>   ---------
#>      mpg   
#>     carb   
#>   ---------
#> 
#> 
#> Not all Values Compared Equal
#>   =============================
#>    Variable  No of Differences 
#>   -----------------------------
#>      cyl             9         
#>      disp           14         
#>       hp            14         
#>      drat           14         
#>       wt            14         
#>      qsec           14         
#>       vs             7         
#>       am             5         
#>      gear            9         
#>   -----------------------------
#> 
#> 
#>   ========================================
#>    VARIABLE  ..ROWNUMBER..  BASE  COMPARE 
#>   ----------------------------------------
#>      cyl           1         6       8    
#>      cyl           2         6       8    
#>      cyl           3         4       8    
#>      cyl           4         6       8    
#>      cyl           6         6       8    
#>      cyl           8         4       8    
#>      cyl           9         4       8    
#>      cyl          10         6       8    
#>      cyl          11         6       8    
#>   ----------------------------------------
#> 
#> 
#> First 10 of 14 rows are shown in table below
#>   =========================================
#>    VARIABLE  ..ROWNUMBER..  BASE   COMPARE 
#>   -----------------------------------------
#>      disp          1        160.0   360.0  
#>      disp          2        160.0   360.0  
#>      disp          3        108.0   275.8  
#>      disp          4        258.0   275.8  
#>      disp          5        360.0   275.8  
#>      disp          6        225.0   472.0  
#>      disp          7        360.0   460.0  
#>      disp          8        146.7   440.0  
#>      disp          9        140.8   318.0  
#>      disp         10        167.6   304.0  
#>   -----------------------------------------
#> 
#> 
#> First 10 of 14 rows are shown in table below
#>   ========================================
#>    VARIABLE  ..ROWNUMBER..  BASE  COMPARE 
#>   ----------------------------------------
#>       hp           1        110     175   
#>       hp           2        110     245   
#>       hp           3         93     180   
#>       hp           4        110     180   
#>       hp           5        175     180   
#>       hp           6        105     205   
#>       hp           7        245     215   
#>       hp           8         62     230   
#>       hp           9         95     150   
#>       hp          10        123     150   
#>   ----------------------------------------
#> 
#> 
#> First 10 of 14 rows are shown in table below
#>   ========================================
#>    VARIABLE  ..ROWNUMBER..  BASE  COMPARE 
#>   ----------------------------------------
#>      drat          1        3.90   3.15   
#>      drat          2        3.90   3.21   
#>      drat          3        3.85   3.07   
#>      drat          4        3.08   3.07   
#>      drat          5        3.15   3.07   
#>      drat          6        2.76   2.93   
#>      drat          7        3.21   3.00   
#>      drat          8        3.69   3.23   
#>      drat          9        3.92   2.76   
#>      drat         10        3.92   3.15   
#>   ----------------------------------------
#> 
#> 
#> First 10 of 14 rows are shown in table below
#>   =========================================
#>    VARIABLE  ..ROWNUMBER..  BASE   COMPARE 
#>   -----------------------------------------
#>       wt           1        2.620   3.440  
#>       wt           2        2.875   3.570  
#>       wt           3        2.320   4.070  
#>       wt           4        3.215   3.730  
#>       wt           5        3.440   3.780  
#>       wt           6        3.460   5.250  
#>       wt           7        3.570   5.424  
#>       wt           8        3.190   5.345  
#>       wt           9        3.150   3.520  
#>       wt          10        3.440   3.435  
#>   -----------------------------------------
#> 
#> 
#> First 10 of 14 rows are shown in table below
#>   =========================================
#>    VARIABLE  ..ROWNUMBER..  BASE   COMPARE 
#>   -----------------------------------------
#>      qsec          1        16.46   17.02  
#>      qsec          2        17.02   15.84  
#>      qsec          3        18.61   17.40  
#>      qsec          4        19.44   17.60  
#>      qsec          5        17.02   18.00  
#>      qsec          6        20.22   17.98  
#>      qsec          7        15.84   17.82  
#>      qsec          8        20.00   17.42  
#>      qsec          9        22.90   16.87  
#>      qsec         10        18.30   17.30  
#>   -----------------------------------------
#> 
#> 
#>   ========================================
#>    VARIABLE  ..ROWNUMBER..  BASE  COMPARE 
#>   ----------------------------------------
#>       vs           3         1       0    
#>       vs           4         1       0    
#>       vs           6         1       0    
#>       vs           8         1       0    
#>       vs           9         1       0    
#>       vs          10         1       0    
#>       vs          11         1       0    
#>   ----------------------------------------
#> 
#> 
#>   ========================================
#>    VARIABLE  ..ROWNUMBER..  BASE  COMPARE 
#>   ----------------------------------------
#>       am           1         1       0    
#>       am           2         1       0    
#>       am           3         1       0    
#>       am          13         0       1    
#>       am          14         0       1    
#>   ----------------------------------------
#> 
#> 
#>   ========================================
#>    VARIABLE  ..ROWNUMBER..  BASE  COMPARE 
#>   ----------------------------------------
#>      gear          1         4       3    
#>      gear          2         4       3    
#>      gear          3         4       3    
#>      gear          8         4       3    
#>      gear          9         4       3    
#>      gear         10         4       3    
#>      gear         11         4       3    
#>      gear         13         3       5    
#>      gear         14         3       5    
#>   ----------------------------------------

> sessionInfo()
R version 4.4.2 (2024-10-31)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 24.04.2 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.12.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: America/Los_Angeles
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] diffdf_1.1.1 dplyr_1.1.4 

loaded via a namespace (and not attached):
 [1] assertthat_0.2.1  R6_2.6.1          tidyselect_1.2.1  magrittr_2.0.3    glue_1.8.0        tibble_3.2.1      pkgconfig_2.0.3  
 [8] generics_0.1.4    lifecycle_1.0.4   cli_3.6.5         vctrs_0.6.5       withr_3.0.2       compiler_4.4.2    rstudioapi_0.17.1
[15] tools_4.4.2       pillar_1.10.2     rlang_1.1.6      

[kieranjmartin]

Thanks @slager yes you have identified what is causing the error; deparse(substitute()) returns a name of length 2, which causes the error. I suspect it is the pipe which is causing this problem; if you replace the magrittr pipe with the native pipe the code runs without error

diffdf(
  as_tibble(mtcars),
  as_tibble(mtcars) |>
    filter(cyl == 8) |>
    dplyr::select(
      -mpg,
      -carb
    )
)

Nonetheless I think we need to add a handler for this case, as there may be other ways for deparse to return a vector of length greater than 2; I can generate the same error with this (slightly odd, but valid) code

diffdf(
  as_tibble(mtcars),
  {x <- as_tibble(mtcars) |>
    filter(cyl == 8) |>
    dplyr::select(
      -mpg,
      -carb
    ) 
  x
  }
)

[slager]

Yes, the example code can be changed in various ways to make the error disappear, but it will probably be fragile and dependent on the way the argument is constructed as long as deparse(substitute()) is involved.

It looks like the deparsing is only used to construct the Name: field in the data summary. I have a local version of diffdf that just omits this Name: field from the summary. This resolves my reprex and still passes all the other diffdf unit tests. If you think this is a worthwhile approach I'd be happy to submit it as a PR.

[gowerc]

For reference this appears to have nothing to do with the pipes, it is purely due to the length of the generated string. Looks like by default deparse has an argument width.cutoff = 60L which basically cuts the returned string onto a new line if it exceeds that limit.

Tbh the variable title gets super unwieldy when the name is too long so I think I will address this by reducing the length limit to something like 30 characters and then use a default "<expression>" for when it breaks that limit.

Happy to take alternative suggestions if you don't like / agree with that approach though.

EDIT - Interestingly it only breaks if there is white space available to break on such as during an expression, if you have a single variable name it will return it as a single string no matter how long it is. Thankfully (though I forgot) we already have a truncation system in place for long variable names so this is fine.