Skip to content

Add memoryRequirements to tools #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
simleo merged 2 commits into from
May 12, 2023
Merged

Add memoryRequirements to tools #28

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
80cd0fc
add memoryRequirements to tools
simleo May 11, 2023
e0be07c
use correct symbol for mebibytes
simleo May 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions cwl/grepsort/greptool.cwl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
class: CommandLineTool
cwlVersion: v1.0

hints:
- class: ResourceRequirement
ramMin: 64

baseCommand: ["bash", "-c"]

inputs:
Expand Down
7 changes: 7 additions & 0 deletions src/runcrate/convert.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -341,6 +341,13 @@ def add_tool(self, crate, workflow, cwl_tool):
properties["@type"] = ["SoftwareSourceCode", "ComputationalWorkflow", "HowTo"]
else:
properties["@type"] = "SoftwareApplication"
if hasattr(cwl_tool, "hints") and cwl_tool.hints:
hints_map = {_["class"]: _ for _ in cwl_tool.hints}
rreq = hints_map.get("ResourceRequirement")
if rreq:
ramMin = rreq.get("ramMin")
if ramMin:
properties["memoryRequirements"] = f"{int(ramMin)} MiB"
tool = crate.add(ContextEntity(crate, tool_id, properties=properties))
tool["input"] = self.add_params(crate, cwl_tool.inputs)
tool["output"] = self.add_params(crate, cwl_tool.outputs)
Expand Down
6 changes: 3 additions & 3 deletions tests/data/grepsort-run-1/bag-info.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Bag-Software-Agent: cwltool 3.1.20220802125926
Bag-Software-Agent: cwltool 3.1.20230425144158
BagIt-Profile-Identifier: https://w3id.org/ro/bagit/profile
Bagging-Date: 2022-12-09
Bagging-Date: 2023-05-11
External-Description: Research Object of CWL workflow run
External-Identifier: arcp://uuid,e442628c-e95b-48f5-b5c2-ae2f266a7fbe/
External-Identifier: arcp://uuid,a59b2ee0-e2cf-4f8c-b6ed-193fb588c154/
Payload-Oxum: 192.4
20 changes: 0 additions & 20 deletions tests/data/grepsort-run-1/metadata/logs/engine.c8915798-6b30-42c9-a29f-83ad8866310d.txt
View file
Open in desktop

This file was deleted.

20 changes: 20 additions & 0 deletions tests/data/grepsort-run-1/metadata/logs/engine.f66f5014-d8d8-4544-bc78-ab0ac7cb1e36.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[2023-05-11T12:54:09,213.033438Z] [cwltool] /tmp/venv/bin/cwltool --provenance ro grepsort.cwl params.yml
[2023-05-11T12:54:09,214.917898Z] Resolved 'grepsort.cwl' to 'file:///home/simleo/git/runcrate/cwl/grepsort/grepsort.cwl'
[2023-05-11T12:54:09,810.973883Z] [provenance] Adding to RO file:///home/simleo/git/runcrate/cwl/grepsort/lines.txt
[2023-05-11T12:54:09,811.661243Z] [provenance] Adding to RO file:///home/simleo/git/runcrate/cwl/grepsort/lines.aux
[2023-05-11T12:54:09,814.031839Z] [workflow ] start
[2023-05-11T12:54:09,814.396143Z] [workflow ] starting step grep
[2023-05-11T12:54:09,814.627886Z] [step grep] start
[2023-05-11T12:54:09,817.400694Z] [job grep] /tmp/_i2f82au$ bash \
-c \
'grep -f /tmp/26d8jeyl/stgc839385b-f723-4ab7-a7bc-a8c62ddb6ed7/lines.aux /tmp/26d8jeyl/stgc839385b-f723-4ab7-a7bc-a8c62ddb6ed7/lines.txt >grep_out.txt'
[2023-05-11T12:54:09,825.176477Z] [job grep] completed success
[2023-05-11T12:54:09,825.467348Z] [step grep] completed success
[2023-05-11T12:54:09,825.929165Z] [workflow ] starting step sorted
[2023-05-11T12:54:09,826.179743Z] [step sorted] start
[2023-05-11T12:54:09,828.734159Z] [job sorted] /tmp/0yah5kc_$ sort \
/tmp/e1qxjn80/stgf144cefc-21a6-4547-a687-9617dde38166/grep_out.txt > /tmp/0yah5kc_/sort_out.txt
[2023-05-11T12:54:09,834.231853Z] [job sorted] completed success
[2023-05-11T12:54:09,834.487200Z] [step sorted] completed success
[2023-05-11T12:54:09,834.689617Z] [workflow ] completed success
[2023-05-11T12:54:09,899.184704Z] Final process status is success
Loading