read_csvy doesn't read the primary example on csvy.org

[MichaelChirico]

Not sure if the schema is canonical and has changed, but read_csvy assumes there's a headline field fields, which is now nested in resources[[1]]$schema$fields.

C/P of file currently on the front page of csvy.org

---
name: my-dataset
resources:
- order: 1
  schema:
    fields:
    - name: var1
      type: string
    - name: var2
      type: integer
    - name: var3
      type: number
  dialect:
    csvddfVersion: 1.0
    delimiter: ","
    doubleQuote: false
    lineTerminator: "\r\n"
    quoteChar: "\""
    skipInitialSpace: true
    header: true
---
var1,var2,var3
A,1,2.0
B,3,4.3

Current output of read_csvy (CRAN):

read_csvy('~/data.table/inst/tests/test.csvy')
#   var1 var2 var3
# 1    A    1  2.0
# 2    B    3  4.3

Warning message:
In check_metadata(y, out) :
Metadata is missing for variables listed in data: var1, var2, var3

[leeper]

The spec changed. I've updated this. Thanks for reporting it!

[MichaelChirico]

@leeper thanks. could you point me to the spec? I tried finding it but ended up in a maze of circular links...

[leeper]

File level: https://frictionlessdata.io/specs/tabular-data-package/

Dataset and variable level: https://frictionlessdata.io/specs/table-schema/

[jrovegno]

@leeper Please do not update the package with the new specs. I am not clear if for the community of users of csvy, it is worthwhile that in the format several dataset can be implemented in a single file.

For simplicity originally csvy was thought to contain only one dataset with its metada.

Then it occurred to me that we could be "CSVY is a simple container of a Tabular Data Package", but that may make the format even more incompatible with the existing CSV.

I would like to know your opinion about it.

[leeper]

@jrovegno I don't see a problem with following the spec except that I don't think I would ever encourage creating a single file with multiple tables in it. So, I'd say the new spec looks good except I would limit it to one table per file. As such, I think you could use the spec as currently stated on the webpage AND a simplified version that just contains the dialect element and the fields sub-element of resources.

[jrovegno]

@leeper Great idea, can you make a PR with the changes in the specs of CSVY?

[MichaelChirico]

Also FYI that we're trying to improve fread (and later, fwrite) to handle csvy through the yaml argument; progress here:

Rdatatable/data.table#2656