Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
164 changes: 0 additions & 164 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,167 +57,3 @@ http://www.opensource.org/licenses/apache2.0.php
url = {https://www.sciencedirect.com/science/article/pii/S2666389922001465}
}
```

## Running tests

```
# Non "end to end tests"
$ docker-compose exec django py.test -m "not e2e"

# "End to end tests" (a shell script to launch a selenium docker container)
$ ./run_selenium_tests.sh

# If you are on Mac OSX it is easy to watch these tests, no need to install
# anything just do:
$ open vnc://0.0.0.0:5900

# And login with password "secret"
```

## Example competitions

The repo comes with a couple examples that are used during tests:

### v2 test data
```
src/tests/functional/test_files/submission.zip
src/tests/functional/test_files/competition.zip
```
### v1.5 legacy test data
```
src/tests/functional/test_files/submission15.zip
src/tests/functional/test_files/competition15.zip
```

### Other Codalab Competition examples
https://github.com/codalab/competition-examples/tree/master/v2/


## Building compute worker

To build the normal image:

```bash
docker build -t codalab/competitions-v2-compute-worker:latest -f Dockerfile.compute_worker .
```

To build the GPU version:
```bash
docker build -t codalab/competitions-v2-compute-worker:nvidia -f Dockerfile.compute_worker_gpu .
```

Updating the image

```bash
docker push codalab/competitions-v2-compute-worker
```


# Worker setup

```bash
# install docker
$ curl https://get.docker.com | sudo sh
$ sudo usermod -aG docker $USER

# >>> reconnect <<<
```

## Start CPU worker

Make a file `.env` and put this in it:
```
# Queue URL
BROKER_URL=<desired broker url>

# Location to store submissions/cache -- absolute path!
HOST_DIRECTORY=/your/path/to/codabench/storage

# If SSL is enabled, then uncomment the following line
#BROKER_USE_SSL=True
```

NOTE `/your/path/to/codabench` -- this path needs to be volumed into `/codabench` on the worker, as you can
see below.

```bash
$ docker run \
-v /your/path/to/codabench/storage:/codabench \
-v /var/run/docker.sock:/var/run/docker.sock \
-d \
--env-file .env \
--restart unless-stopped \
--log-opt max-size=50m \
--log-opt max-file=3 \
codalab/competitions-v2-compute-worker:latest
```


## Start GPU worker

[nvidia installation instructions](https://github.com/NVIDIA/nvidia-docker#quickstart)

```bash
$ nvidia-docker run \
-v /your/path/to/codabench/storage:/codabench \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /var/lib/nvidia-docker/nvidia-docker.sock:/var/lib/nvidia-docker/nvidia-docker.sock \
-d \
--env-file .env \
--restart unless-stopped \
--log-opt max-size=50m \
--log-opt max-file=3 \
codalab/competitions-v2-compute-worker:nvidia
```

# Worker management

Outside of docker containers install [Fabric](http://fabfile.org/) like so:

```bash
pip install fab-classic==1.17.0
```

Create a `server_config.yaml` in the root of this repository using:
```
cp server_config_sample.yaml server_config.yaml
```

Below is an example `server_config.yaml` that defines 2 roles `comp-gpu` and `comp-cpu`,
one with gpu style workers (`is_gpu` and the nvidia `docker_image`) and one with cpu style workers

```yaml
comp-gpu:
hosts:
- ubuntu@12.34.56.78
- ubuntu@12.34.56.79
broker_url: pyamqp://user:pass@host:port/vhost-gpu
is_gpu: true
docker_image: codalab/competitions-v2-compute-worker:nvidia

comp-cpu:
hosts:
- ubuntu@12.34.56.80
broker_url: pyamqp://user:pass@host:port/vhost-cpu
is_gpu: false
docker_image: codalab/competitions-v2-compute-worker:latest
```

You can of course create your own `docker_image` and specify it here.

You can execute commands against a role:

```bash
❯ fab -R comp-gpu status
..
[ubuntu@12.34.56.78] out: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
[ubuntu@12.34.56.78] out: 1d318268bee1 codalab/competitions-v2-compute-worker:nvidia "/bin/sh -c 'celery …" 2 hours ago Up 2 hours hardcore_greider
..

❯ fab -R comp-gpu update
..
(updates workers)
```

See available commands with `fab -l`

2 changes: 1 addition & 1 deletion src/apps/queues/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ def __str__(self):
def broker_url(self):
# host = Site.objects.get_current().domain
if self.owner:
return f"pyamqp://{self.owner.rabbitmq_username}:{self.owner.rabbitmq_password}@{settings.RABBITMQ_HOST}:{settings.RABBITMQ_PORT}/{self.vhost}"
return f"pyamqp://{self.owner.rabbitmq_username}:{self.owner.rabbitmq_password}@{settings.DOMAIN_NAME}:{settings.RABBITMQ_PORT}/{self.vhost}"

def delete(self, *args, **kwargs):
try:
Expand Down
2 changes: 2 additions & 0 deletions src/settings/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
SITE_ID = 1

SITE_DOMAIN = os.environ.get('SITE_DOMAIN', 'http://localhost')
DOMAIN_NAME = os.environ.get('DOMAIN_NAME', 'localhost').split(':')[0]

THIRD_PARTY_APPS = (
'django_su', # Must come before django.contrib.admin
Expand Down Expand Up @@ -105,6 +106,7 @@
USE_TZ = True
SECRET_KEY = os.environ.get("SECRET_KEY", '(*0&74%ihg0ui+400+@%2pe92_c)x@w2m%6s(jhs^)dc$&&g93')
LOGIN_REDIRECT_URL = '/'
LOGOUT_REDIRECT_URL = '/'


# =============================================================================
Expand Down
12 changes: 6 additions & 6 deletions src/static/riot/competitions/detail/submission_manager.tag
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@
<tr show="{!loading}" each="{ submission, index in filter_children(submissions) }"
onclick="{ submission_clicked.bind(this, submission) }" class="submission_row">
<td if="{opts.admin}">
<div class="ui checkbox" onclick="{submission_checked.bind(this)}">
<div class="ui checkbox" onclick="{on_submission_checked.bind(this)}">
<input type="checkbox" name="{submission.id}">
<label></label>
</div>
Expand Down Expand Up @@ -399,12 +399,12 @@
}
}

self.on_submission_checked = function(event){
event.stopPropagation()
self.submission_checked()
}

self.submission_checked = function () {
if (typeof(event) === "object" ){
// We can't stop upon page load as there is no "event" (button click).
// We can when we we check the checkboxes as that is an "event".
event.stopPropagation()
}
let inputs = $(self.refs.submission_table).find('input')
let checked_boxes = inputs.not(':first').filter('input:checked')
let unchecked_boxes = inputs.not(':first').filter('input:not(:checked)')
Expand Down
22 changes: 0 additions & 22 deletions src/templates/registration/logged_out.html

This file was deleted.