Add Environment WebServer APIs

[bobbai00]

This PR introduces the APIs of Environment to the Web server. It depends on the dataset feature, #2413 and #2391 .

Designs

1. Each workflow will have a unique environment when the workflow is being created and persisted.
2. Environment currently store the datasets and the versions visible to the workflow.
3. When a workflow is executed, the environment id will be recorded as the part of workflow_execution record.
4. When using the source scan operator, workflow can ONLY scan the files in the datasets dictated by its environment.

DB Schema Changes

Three new Tables are added:

CREATE TABLE IF NOT EXISTS environment
(
    `eid`              INT UNSIGNED AUTO_INCREMENT NOT NULL,
    `owner_uid`        INT UNSIGNED NOT NULL,
    `name`			   VARCHAR(128) NOT NULL DEFAULT 'Untitled Environment',
    `description`      VARCHAR(1000),
    `creation_time`    TIMESTAMP    NOT NULL DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (`eid`),
    FOREIGN KEY (`owner_uid`) REFERENCES `user` (`uid`) ON DELETE CASCADE
) ENGINE = INNODB;

CREATE TABLE IF NOT EXISTS environment_of_workflow
(
    `eid`              INT UNSIGNED NOT NULL,
    `wid`              INT UNSIGNED NOT NULL,
    PRIMARY KEY (`eid`, `wid`),
    FOREIGN KEY (`wid`) REFERENCES `workflow` (`wid`) ON DELETE CASCADE,
    FOREIGN KEY (`eid`) REFERENCES `environment` (`eid`) ON DELETE CASCADE
) ENGINE = INNODB;

CREATE TABLE IF NOT EXISTS dataset_of_environment
(
    `did`                   INT UNSIGNED NOT NULL,
    `eid`                   INT UNSIGNED NOT NULL,
    `dvid`                  INT UNSIGNED NOT NULL,
    PRIMARY KEY (`did`, `eid`),
    FOREIGN KEY (`eid`) REFERENCES `environment` (`eid`) ON DELETE CASCADE,
    FOREIGN KEY (`dvid`) REFERENCES `dataset_version` (`dvid`) ON DELETE CASCADE
) ENGINE = INNODB;

• environment is the table for storing the environment info.
• environment_of_workflow maintains the which environment the workflow is within. CURRENTLY, WORKFLOW is 1-to-1 correspondence to environment.
• dataset_of_environment records which dataset(s) are visible to the workflow that is using this environment.

New Column is added to the workflow_executions:

-- Add the `environment_eid` column to the `workflow_executions` table
ALTER TABLE workflow_executions
ADD COLUMN `environment_eid` INT UNSIGNED;

-- Add the foreign key constraint for `environment_eid`
ALTER TABLE workflow_executions
ADD CONSTRAINT fk_environment_eid
FOREIGN KEY (`environment_eid`) REFERENCES environment(`eid`) ON DELETE SET NULL;

New column environment_eid is used to record which the environment is used for that workflow execution.

New APIs

Several APIs related to environment is added.

    POST    /api/environment/create    // create the environment
    POST    /api/environment/delete    // delete the environment
    GET     /api/environment/{eid}        // get the environment info by eid
    GET     /api/environment/{eid}/dataset/list // list all datasets(the datasetID, datasetVersionID) of the environment
    POST    /api/environment/{eid}/dataset/add // add dataset to the environment
    GET     /api/environment/{eid}/dataset/list/details   // list the details info of the datasets in the environment
    POST    /api/environment/{eid}/dataset/remove  // remove dataset from the environment
    GET     /api/environment/{eid}/files/{query:.*}  // for file auto complete, retrieve the file path matching the query

Existing API Updates

I changed the implementation of persistWorkflow in WorkflowResource. Specifically, a environment will be created if the workflow has no corresponding environment when persisting, the code snippet is:

    val wid = workflow.getWid
    // check if the runtime environment of this workflow exists, if not, create one
    if (!doesWorkflowHaveEnvironment(context, wid)) {
      // create an environment, and associate this environment to this workflow
      val createdEnvironment = createEnvironment(
        context,
        uid,
        "Environment of Workflow #%d %s".format(wid.intValue(), workflow.getName),
        "Runtime Environment of Workflow #%d %s".format(wid.intValue(), workflow.getName)
      )

      environmentOfWorkflowDao.insert(new EnvironmentOfWorkflow(createdEnvironment.getEid, wid))
    }

[aglinxinyuan]

LGTM!