Skip to content

Input data validation checks - Process-related #136

@ahawkes

Description

@ahawkes

Here is an initial list of input data validation checks for process-related input files, organised by input data CSV file.

PROCESSES

  • No duplicate ID (already implemented)

PROCESS_PARAMETERS

PROCESS_FLOWS

  • process_id must exist in processes.csv file (duplicates are allowed) (already implemented)
  • commodity_id must exist in commodities.csv (Make ProcessFlow::commodity_id into Rc<Commodity> #166)
  • flow_type can be "fixed" or "flexible" (defaults to "fixed" if not supplied) only (already implemented)

PROCESS_PACS

  • process_id must exist in processes.csv file (duplicates are allowed) (already implemented)
  • Any identified PAC must exist in commodities.csv (i.e. PAC is the commodity_id) (process.rs: Check all PACs are valid commodity IDs #164)
  • every process must have at least one PAC (already implemented - empty CSV files are not allowed)
  • if more than one PAC is identified for one process, all PACs must be either all output flows or all input flows (i.e. can't be a combination of inputs and outputs). Important not to skip this check! (Check that all PACs for a process are inputs or outputs #165)

PROCESS_AVAILABILITIES

  • process_id must exist in processes.csv file (duplicates are allowed) (already implemented)
  • limit_type must be "fx", "lo" or "up" (already implemented)
  • time_slice must be a time slice (a season.time_of_day, whole season, or annual level)
  • value must be between 0 and 1 (already implemented)

PROCESS_REGIONS

  • process_id must exist in processes.csv file (duplicates are allowed as process can be available in several regions) (already implemented)
  • region_id must exist in regions.csv (Check process region IDs are valid #138)

ASSET (putting this here as it's mainly process-related) (all should be done with #63)

  • process_id must exist in processes.csv file (duplicates are allowed as assets of the same process type can be in different regions, or in the same region with different commission_year or different owner agents)
  • region_id must exist in regions.csv file and process_regions.csv must have an entry with that process_id / region_id
  • agent_id must exist in agents.csv file
  • commission_year must be on or before the 1st milestone year
  • decommission_year must be after the 1st milestone year

GENERAL PROCESS-RELATED CONSISTENCY CHECKS

Metadata

Metadata

Assignees

No one assigned

    Labels

    umbrellaAn umbrella issue

    Type

    No type

    Projects

    Status

    ✅ Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions