Enable environment tab and file scan from files in environment

[bobbai00]

This PR introduces the GUI of environment and some fixes to previous dataset features. For the backend of environment, see #2434

After introducing the environment, the way of uploading data and scanning data using workflow is presented in this blog. For more specific information, there is a demo video.

Features

• View the Environment information at the workspace
  

• Add dataset to the current environment
  

• Preview Data File in Dataset of environment
  

• Scan Files that are in the datasets
  

Implementation Details

The changes on the ScanSourceOperatorDesc

Previously, the source file is located by its absolute path and scanned into the workflow. Now, since all the files are within the dataset and managed by JGit, its physical file may not be directly available. Therefore, couple of changes are made regarding the way that source operator scans the file.

1. In the source operator descriptor: ScanSourceOpDesc
   A new member variable is added:

  @JsonIgnore
  var filePath: Option[String] = None

// new
  @JsonIgnore
  var datasetFileDesc: Option[DatasetFileDesc] = None

class DatasetFileDesc contains the softlink to the file in the dataset, and has utilities to read the file as stream/tempraory file.

datasetFileDesc will be initialized when setContext is called:

    if (getContext.userId.isDefined) {
      val environmentEid = WorkflowResource.getEnvironmentEidOfWorkflow(
        UInteger.valueOf(workflowContext.workflowId.id)
      )
      // if user system is defined, a datasetFileDesc will be initialized, which is the handle of reading file from the dataset
      datasetFileDesc = Some(
        getEnvironmentDatasetFilePathAndVersion(getContext.userId.get, environmentEid, fileName.get)
      )
    }

2. For each source operator executor, i.e. CSVScanSourceExec

A new parameter is added in the constructor:

class CSVScanSourceOpExec private[csv] (
    filePath: String,
    datasetFileDesc: DatasetFileDesc,

If datasetFileDesc is set non-null(i.e. user system is enabled), when creating the input stream reader, the stream will be created using datasetFileDesc.fileInputStream:

  // this function create the input stream accordingly:
  // - if filePath is set, create the stream from the file
  // - if fileDesc is set, create the stream via JGit call
  def createInputStream(filePath: String, fileDesc: DatasetFileDesc): InputStream = {
    if (filePath != null && fileDesc != null) {
      throw new RuntimeException(
        "File Path and Dataset File Descriptor cannot present at the same time."
      )
    }
    if (filePath != null) {
      new FileInputStream(filePath)
    } else {
      // create stream from dataset file desc
      fileDesc.fileInputStream()
    }
  }

[aglinxinyuan]

The environment is not visible on my local test.

[aglinxinyuan]

LGTM!