Attempting to use renku workflow iterate -p toil with SLURM results in a crash.
Additionally, it causes subsequent runs of renku log in the project to error.
Using the same workflow:
renku workflow execute works with SLURM
renku workflow iterate works with -p toil without SLURM
To Reproduce
Steps to reproduce the behavior:
0. renku init
- Create workflow:
renku -S run --name demo1 -- echo 'demo1' > demo1.out
- Run iterate on SLURM through toil:
TOIL_SLURM_ARGS="-p parallel -A free --export=ALL" renku -S workflow iterate \
-m 'parameter-1=[1, 2, 3]' \
-m output-2='demo1_iter{iter_index}.out' \
-p toil \
-c conf.yaml \
demo1
Expected behavior
The workflow runs 3 times, creating 3 output files demo1_{0,1,2}.
Observed behaviour
The command crashes with the following traceback:
Traceback (most recent call last):
File "[...]/site-packages/renku/cli/exception_handler.py", line 126, in main
result = super().main(*args, **kwargs)
File "[...]/site-packages/renku/cli/exception_handler.py", line 92, in main
return super().main(*args, **kwargs)
File "[...]/site-packages/click/core.py", line 1062, in main
rv = self.invoke(ctx)
File "[...]/site-packages/click/core.py", line 1668, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "[...]/site-packages/click/core.py", line 1668, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "[...]/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "[...]/site-packages/click/core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "[...]/site-packages/renku/cli/workflow.py", line 1148, in iterate
iterate_workflow_command().with_communicator(communicator).build().execute(
File "[...]/site-packages/renku/core/management/command_builder/command.py", line 242, in execute
output = context["click_context"].invoke(self._operation, *args, **kwargs)
File "[...]/site-packages/click/core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "[...]/site-packages/renku/core/commands/workflow.py", line 784, in _iterate_workflow
execute_workflow(dag=graph.workflow_graph, command_name="iterate", provider=provider, config=config)
File "[...]/site-packages/inject/__init__.py", line 342, in injection_wrapper
return sync_func(*args, **kwargs)
File "[...]/site-packages/renku/core/commands/workflow.py", line 463, in execute_workflow
execute(dag=dag, basedir=client.path, provider=provider, config=config)
File "[...]/site-packages/renku/core/plugins/provider.py", line 83, in execute
return executor(dag=dag, basedir=basedir, config=config)
File "[...]/site-packages/pluggy/_hooks.py", line 265, in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File "[...]/site-packages/pluggy/_manager.py", line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File "[...]/site-packages/pluggy/_callers.py", line 60, in _multicall
return outcome.get_result()
File "[...]/site-packages/pluggy/_result.py", line 60, in get_result
raise ex[1].with_traceback(ex[2])
File "[...]/site-packages/pluggy/_callers.py", line 39, in _multicall
res = hook_impl.function(*args)
File "[...]/site-packages/renku/core/management/workflow/providers/toil.py", line 227, in workflow_execute
[setattr(options, k, v) for k, v in config.items()]
AttributeError: 'str' object has no attribute 'items'
Renku version: 1.0.1
OS: Linux (#1 SMP Thu Jul 8 00:48:33 UTC 2021)
Python: 3.9.7
After this crash, running renku log also causes the following error:
Traceback (most recent call last):
File "[...]/site-packages/inject/__init__.py", line 342, in injection_wrapper
return sync_func(*args, **kwargs)
File "[...]/site-packages/renku/core/commands/log.py", line 38, in _log
log_entries = [LogViewModel.from_activity(a) for a in activities]
File "[...]/site-packages/renku/core/commands/log.py", line 38, in <listcomp>
log_entries = [LogViewModel.from_activity(a) for a in activities]
File "[...]/site-packages/renku/core/commands/view_model/log.py", line 74, in from_activity
description=" ".join(activity.plan_with_values.to_argv()),
TypeError: sequence item 1: expected str instance, int found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "[...]/site-packages/renku/cli/exception_handler.py", line 126, in main
result = super().main(*args, **kwargs)
File "[...]/site-packages/renku/cli/exception_handler.py", line 92, in main
return super().main(*args, **kwargs)
File "[...]/site-packages/click/core.py", line 1062, in main
rv = self.invoke(ctx)
File "[...]/site-packages/click/core.py", line 1668, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "[...]/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "[...]/site-packages/click/core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "[...]/site-packages/renku/cli/log.py", line 53, in log
result = log_command().with_database().build().execute(workflows_only=workflows).output
File "[...]/site-packages/renku/core/management/command_builder/command.py", line 242, in execute
output = context["click_context"].invoke(self._operation, *args, **kwargs)
File "[...]/site-packages/click/core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "[...]/site-packages/inject/__init__.py", line 344, in injection_wrapper
raise ConstructorTypeError(func, previous_error)
inject.ConstructorTypeError: <function _log at 0x2b95c7b65ca0> raised an error: sequence item 1: expected str instance, int found
Attempting to use
renku workflow iterate -p toilwith SLURM results in a crash.Additionally, it causes subsequent runs of
renku login the project to error.Using the same workflow:
renku workflow executeworks with SLURMrenku workflow iterateworks with-p toilwithout SLURMTo Reproduce
Steps to reproduce the behavior:
0.
renku initExpected behavior
The workflow runs 3 times, creating 3 output files
demo1_{0,1,2}.Observed behaviour
The command crashes with the following traceback:
Renku version: 1.0.1
OS: Linux (#1 SMP Thu Jul 8 00:48:33 UTC 2021)
Python: 3.9.7
After this crash, running
renku logalso causes the following error: