Skip to content

CONCORE for Windows enabled #50

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Rahuljagwani wants to merge 17 commits into from
Closed

CONCORE for Windows enabled #50

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
e63c7de
CORS enabled
Rahuljagwani Feb 19, 2023
9b46d89
Merge branch 'ControlCore-Project:main' into main
Rahuljagwani Feb 19, 2023
43f44a3
Merge branch 'ControlCore-Project:main' into main
Rahuljagwani Feb 20, 2023
c39f076
Merge branch 'ControlCore-Project:main' into main
Rahuljagwani Feb 20, 2023
2988d24
Concore for windows enabled
Rahuljagwani Feb 20, 2023
4fb1706
Merge branch 'main' of https://github.com/Rahuljagwani/concore
Rahuljagwani Feb 20, 2023
4bbd5e1
test.py updated for testing
Rahuljagwani Feb 20, 2023
9042a11
unnecessary file removed
Rahuljagwani Feb 20, 2023
ef10dc3
apikey added to urls in main.py
Rahuljagwani Feb 20, 2023
b30493c
URLs changed in test.py
Rahuljagwani Feb 20, 2023
409aa3b
mkconcore modified
Rahuljagwani Feb 20, 2023
16cdebb
test.py FileList in Uppercase
Rahuljagwani Feb 20, 2023
f3b7289
case senstive changes in test.py for LINUX
Rahuljagwani Feb 20, 2023
38ea218
apikey resolved mkconcore reverted
Rahuljagwani Feb 21, 2023
1fe53f6
main.py uploadfiles correction
Rahuljagwani Feb 21, 2023
6552dfe
unnecessary file removed
Rahuljagwani Feb 21, 2023
0d3ef01
mkconcore reverted to org
Rahuljagwani Feb 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 47 additions & 20 deletions fri/server/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
from flask import Flask, request, jsonify, send_file, send_from_directory
from werkzeug.utils import secure_filename
import os
import subprocess
from subprocess import call
from pathlib import Path
import json
import subprocess
import platform
from flask_cors import CORS, cross_origin

cur_path = os.path.dirname(os.path.abspath(__file__))
Expand All @@ -21,8 +22,10 @@
@app.route('/upload/<dir>', methods=['POST'])
def upload(dir):
apikey = request.args.get('apikey')
Copy link
Member

@pradeeban pradeeban Feb 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually intentional. Although not properly implemented yet, the rationale is to have a Kong API gateway (you can see it in the Dockerfile) to sit in front of the FRI.

So, based on an API key, a folder is created. In a server deployment, many users will access FRI with an API key. So, you can have a folder name "test" and I also can have the same folder name. With the API key in place, such parallel server execution (which we do not use much at this point) is allowed.

Pls do not remove this (from here or from test.py). But if you say there are methods that do not have this apikey in place, you should rather add the API key in place for those methods.

That way, for now, we can hard code "xyz" or whatever as an API key and eventually use an actual API key with the Kong - for the server deployment.

Copy link
Contributor Author

@Rahuljagwani Rahuljagwani Feb 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually intentional. Although not properly implemented yet, the rationale is to have a Kong API gateway (you can see it in the Dockerfile) to sit in front of the FRI.

So, based on an API key, a folder is created. In a server deployment, many users will access FRI with an API key. So, you can have a folder name "test" and I also can have the same folder name. With the API key in place, such parallel server execution (which we do not use much at this point) is allowed.

Pls do not remove this (from here or from test.py). But if you say there are methods that do not have this apikey in place, you should rather add the API key in place for those methods.

That way, for now, we can hard code "xyz" or whatever as an API key and eventually use an actual API key with the Kong - for the server deployment.

Understood @pradeeban I will be adding them. The sole purpose to remove them was at some paths only the dir_name was there without API key, but you answered that also. Now I will be adding them performing all tests and revert back to you

Copy link
Member

@pradeeban pradeeban Feb 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, yes.

The fix is to either remove them altogether (which you did) or add them uniformly (which is a more future-proof method - because, in case of a missing API Key, we can always have an empty string or a default string. So nothing really breaks that way, whether we have Kong or not).

dirname = secure_filename(dir) + "_" + apikey

if(apikey == None):
dirname = secure_filename(dir)
else:
dirname = secure_filename(dir) + "_" + apikey
if 'files[]' not in request.files:
resp = jsonify({'message': 'No file in the request'})
resp.status_code = 400
Expand Down Expand Up @@ -66,26 +69,38 @@ def upload(dir):
def build(dir):
graphml_file = request.args.get('fetch')
apikey = request.args.get('apikey')
dirname = secure_filename(dir) + "_" + apikey
if(apikey == None):
dirname = secure_filename(dir)
else:
dirname = secure_filename(dir) + "_" + apikey
makestudy_dir = dirname + "/" + graphml_file #for makestudy
dir_path = os.path.abspath(os.path.join(concore_path, graphml_file)) #path for ./build
if not os.path.exists(dir_path):
proc = call(["./makestudy", makestudy_dir], cwd=concore_path)
if(platform.uname()[0]=='Windows'):
proc= call(["makestudy", makestudy_dir], shell=True, cwd=concore_path)
else:
proc = call(["./makestudy", makestudy_dir], cwd=concore_path)
if(proc == 0):
resp = jsonify({'message': 'Directory successfully created'})
resp.status_code = 201
else:
resp = jsonify({'message': 'There is an Error'})
resp.status_code = 500
call(["./build"], cwd=dir_path)
resp.status_code = 500
if(platform.uname()[0]=='Windows'):
call(["build"], cwd=dir_path, shell=True)
else:
call(["./build"], cwd=dir_path)
return resp


@app.route('/debug/<dir>', methods=['POST'])
def debug(dir):
dir = secure_filename(dir)
dir_path = os.path.abspath(os.path.join(concore_path, dir))
proc = call(["./debug"], cwd=dir_path)
dir_name = secure_filename(dir)
dir_path = os.path.abspath(os.path.join(concore_path, dir_name))
if(platform.uname()[0]=='Windows'):
proc=call(["debug"],shell=True, cwd=dir_path)
else:
proc = call(["./debug"], cwd=dir_path)
if(proc == 0):
resp = jsonify({'message': 'Close the pop window after obtaining result'})
resp.status_code = 201
Expand All @@ -98,9 +113,12 @@ def debug(dir):

@app.route('/run/<dir>', methods=['POST'])
def run(dir):
dir = secure_filename(dir)
dir_path = os.path.abspath(os.path.join(concore_path, dir))
proc = call(["./run"], cwd=dir_path)
dir_name = secure_filename(dir)
dir_path = os.path.abspath(os.path.join(concore_path, dir_name))
if(platform.uname()[0]=='Windows'):
proc=call(["run"],shell=True, cwd=dir_path)
else:
proc = call(["./run"], cwd=dir_path)
if(proc == 0):
resp = jsonify({'message': 'result prepared'})
resp.status_code = 201
Expand All @@ -112,9 +130,12 @@ def run(dir):

@app.route('/stop/<dir>', methods=['POST'])
def stop(dir):
dir = secure_filename(dir)
dir_path = os.path.abspath(os.path.join(concore_path, dir))
proc = call(["./stop"], cwd=dir_path)
dir_name = secure_filename(dir)
dir_path = os.path.abspath(os.path.join(concore_path, dir_name))
if(platform.uname()[0]=='Windows'):
proc=call(["stop"],shell=True, cwd=dir_path)
else:
proc = call(["./stop"], cwd=dir_path)
if(proc == 0):
resp = jsonify({'message': 'resources cleaned'})
resp.status_code = 201
Expand All @@ -127,9 +148,12 @@ def stop(dir):

@app.route('/clear/<dir>', methods=['POST'])
def clear(dir):
dir = secure_filename(dir)
dir_path = os.path.abspath(os.path.join(concore_path, dir))
proc = call(["./clear"], cwd=dir_path)
dir_name = secure_filename(dir)
dir_path = os.path.abspath(os.path.join(concore_path, dir_name))
if(platform.uname()[0]=='Windows'):
proc=call(["clear"],shell=True, cwd=dir_path)
else:
proc = call(["./clear"], cwd=dir_path)
if(proc == 0):
resp = jsonify({'message': 'result deleted'})
resp.status_code = 201
Expand Down Expand Up @@ -161,7 +185,10 @@ def download(dir):
@app.route('/destroy/<dir>', methods=['DELETE'])
def destroy(dir):
dir = secure_filename(dir)
proc = call(["./destroy", dir], cwd=concore_path)
if(platform.uname()[0]=='Windows'):
proc=call(["destroy"],shell=True, cwd=concore_path)
else:
proc = call(["./destroy"], cwd=concore_path)
if(proc == 0):
resp = jsonify({'message': 'Successfuly deleted Dirctory'})
resp.status_code = 201
Expand Down
44 changes: 22 additions & 22 deletions fri/test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,37 +23,37 @@ def build(dir, graphml, apikey):
print(response.text)

# function to debug
def debug(graphml):
url = "http://127.0.0.1:5000/debug/"+graphml
def debug(graphml, apikey):
url = "http://127.0.0.1:5000/debug/"+graphml+"?"+"apikey="+apikey
response = requests.request("POST", url)
print(response.text)

# function to test run() method.
def run(graphml):
url = "http://127.0.0.1:5000/run/"+graphml
def run(graphml, apikey):
url = "http://127.0.0.1:5000/run/"+graphml+"?"+"apikey="+apikey
response = requests.request("POST", url)
print(response.text)

def clear(graphml):
url = "http://127.0.0.1:5000/clear/"+graphml
def clear(graphml, apikey):
url = "http://127.0.0.1:5000/clear/"+graphml+"?"+"apikey="+apikey
response = requests.request("POST", url)
print(response.text)

def stop(graphml):
url = "http://127.0.0.1:5000/stop/"+graphml
def stop(graphml, apikey):
url = "http://127.0.0.1:5000/stop/"+graphml+"?"+"apikey="+apikey
response = requests.request("POST", url)
print(response.text)


#function to destroy dir.
def destroy(dir):
url = "http://127.0.0.1:5000/destroy/" + dir
def destroy(dir, apikey):
url = "http://127.0.0.1:5000/destroy/" + dir+"?"+"apikey="+apikey
response = requests.request("DELETE", url)

print(response.text)

def getFilesList(dir, sub_dir = ""):
url = "http://127.0.0.1:5000/getFilesList/" + dir + "?"+"fetch="+sub_dir
def getFilesList(apikey, dir, sub_dir = ""):
Copy link
Member

@pradeeban pradeeban Feb 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In your other methods, apikey was the last argument. Please keep the consistency and leave it as the last argument.

Copy link
Contributor Author

@Rahuljagwani Rahuljagwani Feb 21, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In your other methods, apikey was the last argument. Please keep the consistency and leave it as the last argument.

So one problem is there in doing so as non-default arguments cannot be followed by default arguments. There are few fixes of this problem to maintain consistency:

  1. Make apikey a default argument : There is a small issue with this as if someone calls getFilesList from test.py and not want to pass sub_dir but only apikey. In that case apikey will go into sub_dir. But I think no one will majorly call function as test.py is for testing purpose.

  2. By function overloading : 4 (3 new + existing 1) different functions have to be created .

  3. Make 'apikey' argument as first argument in all functions.

I will implement one of them after getting conformation.
Thanks

Copy link
Member

@pradeeban pradeeban Feb 21, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, as long as you keep the changes to FRI and ensure your follow-up changes do not break any of the progress that you have made in this integration, it is cool. We can always go incrementally.

url = "http://127.0.0.1:5000/getFilesList/" + dir + "?"+"fetch="+sub_dir+"&"+"apikey="+apikey
response = requests.request("POST", url)
print(response.text)

Expand All @@ -63,8 +63,8 @@ def openJupyter():
print(response.text)

# function to test download() method.
def download(dir, subDir, fileName ):
url = "http://127.0.0.1:5000/download/"+dir+"?"+"fetchDir="+subDir+"&"+"fetch="+ fileName
def download(dir, subDir, fileName , apikey ):
url = "http://127.0.0.1:5000/download/"+dir+"?"+"fetchDir="+subDir+"&"+"fetch="+ fileName+"&"+"apikey="+apikey
urllib.request.urlretrieve(url, fileName)

# file list to be uploaded
Expand All @@ -90,18 +90,18 @@ def download(dir, subDir, fileName ):
time.sleep(6)
method = input("methods - 1 for debug, 0 for run :")
if method == "1":
debug("sample")
debug("sample", "xyz")
else:
run("sample")
run("sample", "xyz")
time.sleep(2)
stop("sample")
stop("sample", "xyz")
time.sleep(2)
getFilesList("sample", "cu")
getFilesList("sample", "pym")
getFilesList("xyz", "sample", "CU")
getFilesList("xyz","sample", "PYM")
time.sleep(5)
download("sample", "cu", "u")
clear("sample")
destroy("sample")
download("sample", "CU", "u", "xyz")
clear("sample", "xyz")
destroy("sample", "xyz")
openJupyter()


1 change: 0 additions & 1 deletion mkconcore.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -921,4 +921,3 @@
os.chmod(outdir+"/maxtime",stat.S_IRWXU)
os.chmod(outdir+"/params",stat.S_IRWXU)
os.chmod(outdir+"/unlock",stat.S_IRWXU)