Begin recording site-level meta-data for TERRA experimental sites

[dlebauer]

• record information that fits within the ICASA standard (e.g. Add example data formatted in ICASA standards #6) in a spreadsheet or json blob
• Data beyond scope of ICASA standards in something like a 'native' format - that can be automatically generated, synced when updated, and parsed
  • site layout including soils (in geojson (?))
  • managements in csv file (site, management type, level, date)
    • fertilization, tillage, irrigation, etc.
  • subdaily meteorological data (datetime, site, variable1, variable2, ...)
  • cultivars
  • experimental design (location of plants, plots)
  • precision planter output
• Insert into BETYdb

[JeffWhiteAZ]

China_WH_AgMIP_all_data_V0.6.xlsx
The spreadsheet is a work in progress for a set of wheat experiments at 5 locations. It mostly follows the ICASA data definitions and divides data into groups corresponding to those used in the standards.
The paper by Reginato explains the agronomy of the experiments.
Reginato1988WW response to water nitrogen.pdf
We will start organizing data from the three wheat experiments using a similar approach. Right now, we're scrambling to assemble the seed and prepare the field for planting next week.

[dlebauer]

@JeffWhiteAZ Thank you for sending these. Do you think it will be possible to send the metadata for experiments at MAC in json format, or is there a way to translate between json and xlsx?

[JeffWhiteAZ]

Hi David,
Cheryl can send sample JSON from another project, but we are not yet set up to interconvert from the spreadsheet layout that I used.

• Jeff

[dlebauer]

@chporter do you have scripts to process multiple metadata formats? Or to convert something familiar to field researchers (excel) into machine readable text?

[chporter]

I'm attaching a ZIP file (changed to txt for upload) containing a sample json file with two treatments of a maize experiment from Iowa and the associated AgMIP metadata in a csv file. Keep in mind that AgMIp currently considers metadata to be "searchable data". As we start migrating the AgMIP format to NARDN (Nat'l Ag Research Data Network), we will also adopt the NAL metadata, which is based on the Dublin Core.

We have scripts to take data from spreadsheet templates and put into AgMIP json data formats. As data are added to the database, the metadata are extracted from the data. When data are downloaded from the database, the csv format metadata table is created and downloaded with data.

So in answer to your question, we do not have scripts to process multiple metadata formats. But we do have scripts to convert spreadsheet data into machine readable text.

IowaMaize.TXT

[JeffWhiteAZ]

Hi Cheryl,
Thanks for providing the example scripts. Heidi and I are closing in on releasing the China wheat dataset. After that, I can think about the UF SCA, the new proposal, livestock, etc.
The gantry is almost running and should be planted with wheat this week.
Are you available for Skype sometime soon?

• Jeff

[dlebauer]

@chporter thanks for those; I've posted them among our data samples here: https://uofi.box.com/s/8nm7azvgtjeort8hmg4c91j1orwvuvjh

Can you link to the scripts that you use to convert from spreadsheets to json?

[chporter]

@dlebauer - The repo for the csv -> json translator is here:
https://github.com/agmip/translator-generic-csv

with some additional documentation here: http://research.agmip.org/display/dev/Spreadsheet+import#Spreadsheetimport-Links