Skip to content

Extract commonality between datasets and experiments #232

@jonc125

Description

@jonc125

Part of #203. Preparatory to adding FittingResultVersion, extract the parts it will share in common with Dataset and ExperimentVersion; similarly for their views.

  • Create a mixin class core.models.FileCollectionMixin used by Dataset and ExperimentVersion
    • Properties abs_path, archive_name, archive_path, files
    • Methods open_file, mkdir
    • Remove Dataset.is_visible_to_user - only used by tests!
    • Where implementations differ, raise NotImplementedError in the mixin version
  • Update datasets views to use the namespace trick, and inherit from/reuse them in experiments
  • Consider refactoring the JSON views (including for entities) as part of this issue?

"Collection of files" type views:

  • DatasetCreateView - user creation of dataset - only in datasets
    • DatasetAddFilesView
    • DatasetFileUploadView
  • NewExperimentView - submission of expt run to backend - only in experiments
    • ExperimentCallbackView - backend upload of expt result (Prediction)
  • DatasetListView - only in datasets
    • ExperimentsView gives a matrix rather than list; there is no corresponding list view for expts
  • ExperimentVersionListView (no equivalent for datasets)
  • DatasetView == ExperimentVersionView
    • DatasetJsonView == ExperimentVersionJsonView (but with a few different fields, and the download URLs take different arguments!)
    • DatasetFileDownloadView == ExperimentFileDownloadView (dataset version has extra error handling - use that!)
    • DatasetArchiveView == ExperimentVersionArchiveView
  • DatasetDeleteView == ExperimentDeleteView
    • Same as ExperimentVersionDeleteView but different success_url
  • ExperimentComparisonView (not implemented for datasets yet)
    • ExperimentComparisonJsonView

Fields returned by JSON views are listed below. I'm not yet sure where extracted common methods for these would sensibly go. There might also have to be different versions for the repository and archive backed versions of file views. So possibly we should add methods to FileCollection and Entity?

  • EntityVersionJsonView
    • returns {version: }
    • version keys: id (sha), entityId (DB id), author (full_name), parsedOk, visibility, created (timestamp), name (entity.name), version (sha), files (strips manifest & metadata.rdf), commitMessage, numFiles, planned_experiments, url (:version), download_url (:entity_archive), change_url (:change_visibility)
    • file keys: id (blob.name), name (=id), created, filetype (get_file_type()), size, url (:file_download)
  • EntityComparisonJsonView
    • returns {getEntityInfos: {entities: [versions]}}
    • version keys: id (sha), entityId (DB id) [unused?], author (full_name), parsedOk (False) [unused], visibility [unused?], created (timestamp), name (entity.name), version (sha), files (strips manifest & metadata.rdf), commitMessage, numFiles
    • file keys: id (blob.name) [unused?], name (=id), author (full_name), created, filetype (get_file_type()), size, url (:file_download)
  • DatasetJsonView
    • returns {version: }
    • version keys: id (DB id), author, parsedOk (False), visibility, created, name, files, numFiles, download_url (datasets:archive)
    • file keys: id (.name), name, author, created, filetype, masterFile (bool), size, url (:file_download)
  • ExperimentVersionJsonView
    • returns {version: }
    • version keys: as DatasetJsonView PLUS status, experimentId, version (=id)
    • file keys: as DatasetJsonView
  • ExperimentComparisonJsonView
    • returns {getEntityInfos: {entities: [versions]}}
    • version keys: as DatasetJsonView PLUS status, experimentId, versionId (=id), url (:version), modelName, protoName, modelVersion, protoVersion, runNumber
    • file keys: as DatasetJsonView

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions