Create a list of test procedures before releases

[BryonLewis]

Develop a standardize list of testing procedures that should be done before release to make sure we can catch small bugs and issues that occur.

[subdavis]

Likewise, we should have standard example data that we use during these tests.

• a CSV with detections
• An image set of frames in a video
• An image set of non-video frames
• An actual video in each format (h264, mpeg4, avi, mpv, mpg)
• Some tiff or sgi imagery that requires transcoding.
• A few examples of KPF

Data ingest

• For each reasonable combination of data, verify upload and transcoding happen as appropriate and that the dataset will open in the viewer.
• Verify that uploading several combinations of data at once (imagery and videos) works.
• Try to upload more data while existing data is in the process of uploading
• Verify that delete works for all resources

Annotation UI

• TODO: tasks we should run in the UI

Pipeline Running

• For all appropriate combinations of data above, verify that appropriate pipelines can be run.
• Verify that the pipeline results look correct in the viewer

Export

• Verify that export works for each data set
• Verify that export works even after pipelines have been run
• Ideally, compare the exported files against a known checksum.