Recursive inline saving

[jvdzwaan]

The result of saving a workflow with a subworkflow with inline steps has ugly ids and outputSources:

  align-texts-wf:
    run:
      cwlVersion: v1.0
      class: Workflow
      requirements:
      - class: ScatterFeatureRequirement
      inputs:
      - type:
        - 'null'
        - string
        id: _:align-texts-wf#align_c
      - type:
        - 'null'
        - string
        id: _:align-texts-wf#align_m
      - type:
          type: array
          items: File
        id: _:align-texts-wf#gs
      - type:
          type: array
          items: File
        id: _:align-texts-wf#ocr
      outputs:
      - type:
          items: File
          type: array
        outputSource: file:///home/jvdzwaan/code/ocr/cwl/align-texts-wf.cwl#char-align/out_file
        id: _:align-texts-wf#alignments
      - type: File
        outputSource: file:///home/jvdzwaan/code/ocr/cwl/align-texts-wf.cwl#merge-json-1/merged
        id: _:align-texts-wf#changes
      - type: File
        outputSource: file:///home/jvdzwaan/code/ocr/cwl/align-texts-wf.cwl#merge-json/merged
        id: _:align-texts-wf#metadata

Also the problem of duplicate ids arises if a subworkflow is added multiple times (related to #82).

[LourensVeen]

Unfortunately, cwltool will insert URLs if you don't supply a URI yourself. The _: thing is a JSON-LD local identifier, which is accepted by cwltool and not replaced, and at least avoids having absolute file paths in there.

I agree that it's ugly though :).

[jvdzwaan]

Actually, the problem isn't so much that it is ugly (although that definitely is a problem!), it also doesn't work:

../../code/ocr/cwl/vudnc-preprocess.cwl:38:1: checking field `steps`
../../code/ocr/cwl/vudnc-preprocess.cwl:374:3:   checking object `../../code/ocr/cwl/vudnc-preprocess.cwl#align-texts-wf`
../../code/ocr/cwl/vudnc-preprocess.cwl:409:7:     checking field `steps`
../../code/ocr/cwl/vudnc-preprocess.cwl:410:9:       checking object `../../code/ocr/cwl/align-texts-wf.cwl#align`
../../code/ocr/cwl/vudnc-preprocess.cwl:443:9:         checking field `in`
../../code/ocr/cwl/vudnc-preprocess.cwl:444:11:           checking object `../../code/ocr/cwl/align-texts-wf.cwl#align/file1`
../../code/ocr/cwl/vudnc-preprocess.cwl:444:11:             Field `source` references unknown identifier `file:///home/jvdzwaan/code/ocr/cwl/align-texts-wf.cwl#ocr`, tried file:///home/jvdzwaan/code/ocr/cwl/align-texts-wf.cwl#file:///home/jvdzwaan/code/ocr/cwl/align-texts-wf.cwl#ocr

[jvdzwaan]

It seems that removing all the source fields (in the subworkflow) fixes the errors!

[jvdzwaan]

In the documentation, we recommend not using inline saving if you use subworkflows: http://scriptcwl.readthedocs.io/en/stable/saving_workflows.html#embedding-steps-in-the-workflow