adding stashcp2 #17
adding stashcp2 #17
Conversation
|
Why is the |
|
You're right, it shouldn't be here... lemme try to re do this one |
djw8605
left a comment
djw8605
left a comment
There was a problem hiding this comment.
Some changes required. Proper testing with docker images, and using python functions instead of creating new processes and calling other python scripts.
bin/stashcp2/tests/test.sh
Outdated
| @@ -0,0 +1,4 @@ | |||
| python stashcp.py --debug user/taburaad/public/2gb_file.tar . | |||
There was a problem hiding this comment.
I don't think this will work since xrdcp is not installed by default at travis-ci. I think you will need something like what we do with the HTCondor-CE, where we start a CentOS docker image.
- Setup a docker image in the travis-ci.
- Create the docker image.
- Finally, run the tests
There was a problem hiding this comment.
Do I need to change anything in this travis.yml to set up my docker image? I understand the general idea but am figuring out the specifics of how to configure the docker image through your CentOS example.
There was a problem hiding this comment.
You should be able to copy most of the HTCondor-CE's travis.yml. And probably copy all of the setup_tests.sh file.
bin/stashcp2/stashcp.py
Outdated
| else: #Pull from origin | ||
| print "2nd try failed on %s, pulling from origin" % cache | ||
| cache="root://stash.osgconnect.net" | ||
| command = "python ./timeout.py -t "+str(TIMEOUT)+ " -f "+sourceFile + " -d "+str(DIFF)+" -s "+str(fileSize)+" -x "+str(xrdargs)+" -c "+cache+" -z "+destination |
There was a problem hiding this comment.
Why are you calling another python script as a process. Can't you just create a module, class, or function that does the time and takes all the necessary arguments?
bin/stashcp2/stashcp.py
Outdated
| args,opts=parser.parse_args() | ||
|
|
||
| def find_closest(): | ||
| closest=subprocess.Popen(['./get_best_stashcache.py', '0'], stdout=subprocess.PIPE) |
There was a problem hiding this comment.
Same, why are you calling another python script? Create a module, class, or function to do this!
bin/stashcp2/stashcp.py
Outdated
| xrdfs = subprocess.Popen(["xrdfs", "root://stash.osgconnect.net", "stat", sourceFile], stdout=subprocess.PIPE).communicate()[0] | ||
| fileSize=int(re.findall(r"Size: \d+",xrdfs)[0].split(": ")[1]) | ||
| cache=find_closest() | ||
| command = "python ./timeout.py -t "+str(TIMEOUT)+ " -f "+sourceFile + " -d "+str(DIFF)+" -s "+str(fileSize)+" -x "+str(xrdargs)+" -c "+cache+" -z "+destination |
There was a problem hiding this comment.
Same, call the module, class, or function.
|
Ok - should have this in a day or two. |
|
Hi @Tony044, do you consider this pull request done? |
|
It still needs a working test_inside_docker.sh script, which I'm not sure how to configure. After that I would consider it done. |
|
Superceded by #18 |
2 participants
No description provided.