Skip to content

Refine evaluation process #4

@cyyeh

Description

@cyyeh

In the current evaluation process, there are some issues for further refinement

  • reproducibility: there is no dvc.lock related file to make sure the pipelines are reproducible in evaluation.(we can directly reference this to spider-benchmark repo; however, if it's not that important as of now, we can skip this right now.)
  • evaluation report:
    • adding cost/latency
    • adding pipeline metadata so that we can easily understand what parameters cause the differences among different reports

Metadata

Metadata

Assignees

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions