Dockerfile to Singularity recipe conversion error

[tengwenxuan]

I'm trying spython recipe Dockerfile Singularity to convert our existing (working) Dockerfiles, but ran into some wrongly converted recipe:

A Dockerfile containing the following:

WORKDIR /mydir
COPY mydir/CMakeLists.txt mydir/build mydir/README ./

is wrongly converted as the following Singularity recipe:

%files
mydir/CMakeLists.txt mydir/build
mydir/CMakeLists.txt mydir/README
mydir/CMakeLists.txt ./

cd /mydir

spython version: 0.0.45 (btw: spython --version doesn't work as expected)
singularity --version
2.6.0-dist

[vsoch]

hey @tengwenxuan !

For the second, the correct usage is spython version

For the first, could you please show me the expected output? I'm not familiar with that edge case version of the copy. Thanks!

[tengwenxuan]

Thanks for your prompt reply. I'm not very familiar with Singularity syntax yet, but I think it should be
%files
mydir/build mydir/build
mydir/README mydir/README
mydir/CMakeLists.txt mydir/CMakeLists.txt

or probably the following should work too:
...
mydir/CMakeLists.txt ./
mydir/README

[vsoch]

Oh I think I see! So in other words, the COPY command is correctly described as:

COPY <src> <src> <src> <dest>

with traditional usage as

COPY <src> <dest>

and spython is incorrectly doing

COPY <src> <dest> <dest>

Is that correct?

[vsoch]

okay when this finishes testing please give it a whirl! It should be a fairly easy fix to switch the two around. #74 Let me know if there are any issues, and if not, we can merge and release.

[tengwenxuan]

Thanks! I didn't run the program but the fix looks correct to me. Any suggestion about how to try out the non-released fix?

[vsoch]

Sure thing! The easiest thing to do would be to clone the branch and install that version, and make sure to remove previous versions first. Something like:

pip uninstall spython
# do the above lines multiple times until it tells you no longer installed, in case you have more than one

Then clone the branch

git clone -b fix/coy-src-dest https://www.github.com/singularityhub/singularity-cli.git
cd singularity-cli
python setup.py install

Then check to make sure the version is right

$ spython --version
0.0.46

Let me know how it goes!

[tengwenxuan]

Sorry for the delay. I notice that the latest release is already 0.0.47, Does it already contain the fix? Thanks!

[vsoch]

Nope, #74 is not merged yet! You should test the PR (the version will need to be updated, and I'll do that) before we confirm the fix is good to merge. The reason the version is bumped is because other PRs came (and were tested) before this one, but no problemo at all! This fix will be allocated to the next version.

[tengwenxuan]

I see. Just tested the code from fix/copy-src-dest, it works as expected and fixed the issue. Thanks!

Btw, I notice that the conversion regroupes all the COPY actions into one %files section at the beginning, I'm not sure if this is a requirement/constraint of Singularity recipe but this may cause some problem because Dockerfile has an execution order of cmds.

[vsoch]

Yes, Singularity recipes just have one "block" for files, and the files are copied at the beginning (before the %post) section. Does that answer your question? If so, I can go ahead and update the PR and merge to master.

[tengwenxuan]

Yes well answered. Thanks for the prompt replies.

[vsoch]

Released: https://pypi.org/project/spython/0.0.48/