Issue with automatic testing: CircleCi tests failing #691
Description
Automated tests are failing: https://app.circleci.com/pipelines/github/codalab/codabench/810/workflows/fb18741d-4081-4784-b034-56c41c2a638b/jobs/2472
From this commit 6aa1f60 the tests now build so we can see the actual errors.
Summary:
=========================== short test summary info ============================
FAILED src/tests/functional/test_submissions.py::TestSubmissions::test_v15_submission_end_to_end
FAILED src/tests/functional/test_submissions.py::TestSubmissions::test_v18_submission_end_to_end
FAILED src/tests/functional/test_submissions.py::TestSubmissions::test_v2_submission_end_to_end
============= 3 failed, 5 passed, 8 warnings in 400.20s (0:06:40) ==============Full logs:
#!/bin/bash -eo pipefail
docker-compose -f docker-compose.yml -f docker-compose.selenium.yml exec django py.test src/tests/functional/ -m e2e
WARNING: Some services (site_worker) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
^@^@============================= test session starts ==============================
platform linux -- Python 3.8.13, pytest-6.2.1, py-1.11.0, pluggy-0.13.1
django: settings: settings.test (from option)
rootdir: /app, configfile: setup.cfg
plugins: celery-4.2.1, Faker-13.3.3, django-4.1.0, pythonpath-0.7.3
collecting ... collected 8 items
src/tests/functional/test_competitions.py .... [ 50%]
src/tests/functional/test_login.py . [ 62%]
src/tests/functional/test_submissions.py FFF [100%]
=================================== FAILURES ===================================
________________ TestSubmissions.test_v15_submission_end_to_end ________________
self = <tests.functional.test_submissions.TestSubmissions testMethod=test_v15_submission_end_to_end>
def test_v15_submission_end_to_end(self):
> self._run_submission_and_add_to_leaderboard('competition_15.zip', 'submission_15.zip', '*** prediction_score', has_solutions=False, timeout=2000)
src/tests/functional/test_submissions.py:74:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/tests/functional/test_submissions.py:54: in _run_submission_and_add_to_leaderboard
assert self.find('submission-manager#user-submission-table table tbody tr:nth-of-type(1) td:nth-of-type(2)').text == submission_zip_path
src/tests/utils.py:126: in find
raise e
src/tests/utils.py:123: in find
return wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, selector)))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <selenium.webdriver.support.wait.WebDriverWait (session="5aaf197b-cb90-4afa-bf5d-a7cdeef884aa")>
method = <selenium.webdriver.support.expected_conditions.presence_of_element_located object at 0x7f6c30e6d490>
message = ''
def until(self, method, message=''):
"""Calls the method provided with the driver as an argument until the \
return value is not False."""
screen = None
stacktrace = None
end_time = time.time() + self._timeout
while True:
try:
value = method(self._driver)
if value:
return value
except self._ignored_exceptions as exc:
screen = getattr(exc, 'screen', None)
stacktrace = getattr(exc, 'stacktrace', None)
time.sleep(self._poll)
if time.time() > end_time:
break
> raise TimeoutException(message, screen, stacktrace)
E selenium.common.exceptions.TimeoutException: Message:
/usr/local/lib/python3.8/site-packages/selenium/webdriver/support/wait.py:80: TimeoutException
----------------------------- Captured stderr call -----------------------------
HTTP/2 support not enabled (install the http2 and tls Twisted extras)
Configuring endpoint tcp:port=36475:interface=172.18.0.11
Listening on TCP address 172.18.0.11:36475
Starting unpack with status pk = 4
Download competition bundle: dataset/2022-03-25-1648227758/0c69fcc62853/competition_15.zip
The YAML version is: 1
Checking if path is directory: /tmp/tmph0tqmr3l/ingestion_program.zip
Checking if path is directory: /tmp/tmph0tqmr3l/evaluation_data.zip
Checking if path is directory: /tmp/tmph0tqmr3l/scoring_program
Zipping it up because it is directory, saving it to: scoring_program.zip
New zip file path = /tmp/scoring_program.zip
Checking if path is directory: /tmp/tmph0tqmr3l/evaluation_data.zip
Checking if path is directory: /tmp/tmph0tqmr3l/ingestion_program.zip
Checking if path is directory: /tmp/tmph0tqmr3l/evaluation_data.zip
Checking if path is directory: /tmp/tmph0tqmr3l/scoring_program
Zipping it up because it is directory, saving it to: scoring_program.zip
New zip file path = /tmp/scoring_program.zip
Checking if path is directory: /tmp/tmph0tqmr3l/evaluation_data.zip
Competition saved!
Task competitions.tasks.unpack_competition[9d9867d7-d681-45fc-8a21-c02496c15d66] succeeded in 0.6977351440000348s: None
Task data for submission id = 390
{'user_pk': 1049, 'submissions_api_url': 'http://django:36475/api', 'secret': UUID('68011253-9b44-4c6d-95ef-d84420042e72'), 'docker_image': 'codalab/codalab-legacy:py3', 'execution_time_limit': 300, 'id': 390, 'is_scoring': False, 'prediction_result': 'http://172.17.0.1:9000/private/prediction_result/2022-03-25-1648227765/7f0242709aef/prediction_result.zip?AWSAccessKeyId=testkey&Signature=We3ZSa%2Fg0KWODWml%2FIRZoy1FuRQ%3D&content-type=application%2Fzip&Expires=1648314165', 'ingestion_program': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648227759/e67b620212ba/ingestion_program.zip?AWSAccessKeyId=testkey&Signature=MZQpmt66LWYrSQxDss9Gf7UMnDY%3D&Expires=1648314165', 'input_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648227759/342c1ac00476/evaluation_data.zip?AWSAccessKeyId=testkey&Signature=v0VaqyUWYNdUp67VPZHqw3DLVEI%3D&Expires=1648314165', 'ingestion_only_during_scoring': False, 'program_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648227765/94835bff8fb3/submission_15.zip?AWSAccessKeyId=testkey&Signature=Su3eFYKrihRIcYC467nqgy2akqI%3D&Expires=1648314165', 'prediction_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227765/95b54b707235/prediction_stdout.txt?AWSAccessKeyId=testkey&Signature=SQY2P%2BXa7L7d%2FE%2FR0FIU3e8Gg2I%3D&content-type=application%2Fzip&Expires=1648314165', 'prediction_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227765/daff51f887cd/prediction_stderr.txt?AWSAccessKeyId=testkey&Signature=NDuO42gtNq9LFA%2BACGLrf2QoVBw%3D&content-type=application%2Fzip&Expires=1648314165', 'prediction_ingestion_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227765/4c4766d06e31/prediction_ingestion_stdout.txt?AWSAccessKeyId=testkey&Signature=Gxj11ZjAbsxYZg%2B9C2OrmyxEHA0%3D&content-type=application%2Fzip&Expires=1648314165', 'prediction_ingestion_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227765/76d1e2b4e346/prediction_ingestion_stderr.txt?AWSAccessKeyId=testkey&Signature=BnxRl2DbPgC04b3Uh3vf9FwyIPg%3D&content-type=application%2Fzip&Expires=1648314165'}
Task competitions.tasks._run_submission[8b67e1a0-02d0-40c7-8c8e-2a632add0f0f] succeeded in 0.306283146999931s: None
Task data for submission id = 390
{'user_pk': 1049, 'submissions_api_url': 'http://django:36475/api', 'secret': UUID('68011253-9b44-4c6d-95ef-d84420042e72'), 'docker_image': 'codalab/codalab-legacy:py3', 'execution_time_limit': 300, 'id': 390, 'is_scoring': True, 'detailed_results_url': 'http://172.17.0.1:9000/private/detailed_result/2022-03-25-1648227765/b993a7aecb27/detailed_results.html?AWSAccessKeyId=testkey&Signature=ceBJvIkC1HM8%2Fct4OzdX9i9thnM%3D&Expires=1648314291', 'prediction_result': 'http://172.17.0.1:9000/private/prediction_result/2022-03-25-1648227765/7f0242709aef/prediction_result.zip?AWSAccessKeyId=testkey&Signature=cDCYaLXXyvFvTE3bayUDQqiz77M%3D&Expires=1648314291', 'scoring_result': 'http://172.17.0.1:9000/private/scoring_result/2022-03-25-1648227765/b4d991f4ac1c/scoring_result.zip?AWSAccessKeyId=testkey&Signature=8BLKqdGx8dEBEblEURLsU2m8A6A%3D&content-type=application%2Fzip&Expires=1648314291', 'reference_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648227759/8d6837bbd95b/evaluation_data.zip?AWSAccessKeyId=testkey&Signature=qmWoOTU4mqvyV%2B0lzOQNVARjFbQ%3D&Expires=1648314291', 'ingestion_only_during_scoring': False, 'program_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648227759/72075c8a0966/scoring_program.zip?AWSAccessKeyId=testkey&Signature=ERn0H3b1hiN1bD%2FbdAKH4C%2B3Xls%3D&Expires=1648314291', 'scoring_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227891/2518b11d7635/scoring_stdout.txt?AWSAccessKeyId=testkey&Signature=H715ezGKRtkjb2RN4gynPx%2FpDb4%3D&content-type=application%2Fzip&Expires=1648314291', 'scoring_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227891/c3b54f2df452/scoring_stderr.txt?AWSAccessKeyId=testkey&Signature=p81Aym%2B6rkmQaouP0awbVUOloyU%3D&content-type=application%2Fzip&Expires=1648314291', 'scoring_ingestion_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227891/cb6d4e94ce6c/scoring_ingestion_stdout.txt?AWSAccessKeyId=testkey&Signature=eSQMzW8kQmZ0qts0%2FHrC88fhN8s%3D&content-type=application%2Fzip&Expires=1648314291', 'scoring_ingestion_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227891/25684e4eb41f/scoring_ingestion_stderr.txt?AWSAccessKeyId=testkey&Signature=V9EwlR13VTrVXow32NJe4AQJAB4%3D&content-type=application%2Fzip&Expires=1648314291'}
Task competitions.tasks._run_submission[5a224230-81ce-434c-a5ae-387c6bf42431] succeeded in 0.2897826119999536s: None
________________ TestSubmissions.test_v18_submission_end_to_end ________________
self = <tests.functional.test_submissions.TestSubmissions testMethod=test_v18_submission_end_to_end>
def test_v18_submission_end_to_end(self):
> self._run_submission_and_add_to_leaderboard('competition_18.zip', 'submission_18.zip', 'results', has_solutions=False)
src/tests/functional/test_submissions.py:77:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/tests/functional/test_submissions.py:54: in _run_submission_and_add_to_leaderboard
assert self.find('submission-manager#user-submission-table table tbody tr:nth-of-type(1) td:nth-of-type(2)').text == submission_zip_path
src/tests/utils.py:126: in find
raise e
src/tests/utils.py:123: in find
return wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, selector)))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <selenium.webdriver.support.wait.WebDriverWait (session="5aaf197b-cb90-4afa-bf5d-a7cdeef884aa")>
method = <selenium.webdriver.support.expected_conditions.presence_of_element_located object at 0x7f6c30e157f0>
message = ''
def until(self, method, message=''):
"""Calls the method provided with the driver as an argument until the \
return value is not False."""
screen = None
stacktrace = None
end_time = time.time() + self._timeout
while True:
try:
value = method(self._driver)
if value:
return value
except self._ignored_exceptions as exc:
screen = getattr(exc, 'screen', None)
stacktrace = getattr(exc, 'stacktrace', None)
time.sleep(self._poll)
if time.time() > end_time:
break
> raise TimeoutException(message, screen, stacktrace)
E selenium.common.exceptions.TimeoutException: Message:
/usr/local/lib/python3.8/site-packages/selenium/webdriver/support/wait.py:80: TimeoutException
----------------------------- Captured stderr call -----------------------------
HTTP/2 support not enabled (install the http2 and tls Twisted extras)
Configuring endpoint tcp:port=36475:interface=172.18.0.11
Listening on TCP address 172.18.0.11:36475
Starting unpack with status pk = 5
Download competition bundle: dataset/2022-03-25-1648227912/bec766802dde/competition_18.zip
The YAML version is: 1
Checking if path is directory: /tmp/tmpsbnwkxll/ingestion_program_2/
Zipping it up because it is directory, saving it to: ingestion_program_2.zip
New zip file path = /tmp/tmpsbnwkxll/ingestion_program_2.zip
Checking if path is directory: /tmp/tmpsbnwkxll/input_data_2.zip
Checking if path is directory: /tmp/tmpsbnwkxll/scoring_program_2/
Zipping it up because it is directory, saving it to: scoring_program_2.zip
New zip file path = /tmp/tmpsbnwkxll/scoring_program_2.zip
Checking if path is directory: /tmp/tmpsbnwkxll/reference_data_2.zip
Checking if path is directory: /tmp/tmpsbnwkxll/ingestion_program_2/
Zipping it up because it is directory, saving it to: ingestion_program_2.zip
New zip file path = /tmp/tmpsbnwkxll/ingestion_program_2.zip
Checking if path is directory: /tmp/tmpsbnwkxll/input_data_3.zip
Checking if path is directory: /tmp/tmpsbnwkxll/scoring_program_2/
Zipping it up because it is directory, saving it to: scoring_program_2.zip
New zip file path = /tmp/tmpsbnwkxll/scoring_program_2.zip
Checking if path is directory: /tmp/tmpsbnwkxll/reference_data_3.zip
Competition saved!
Task competitions.tasks.unpack_competition[943bff73-d7f3-4e91-b6a4-2624d61a52cc] succeeded in 0.7433373200000233s: None
Task data for submission id = 391
{'user_pk': 1050, 'submissions_api_url': 'http://django:36475/api', 'secret': UUID('6df587e7-8f6d-4f2e-8f02-d6ddf98b8ded'), 'docker_image': 'vergilgxw/autotable:v2', 'execution_time_limit': 600, 'id': 391, 'is_scoring': False, 'prediction_result': 'http://172.17.0.1:9000/private/prediction_result/2022-03-25-1648227918/ebbbfd6fc857/prediction_result.zip?AWSAccessKeyId=testkey&Signature=ec3FRS4uYA9fXD9Q97enKK%2FW52M%3D&content-type=application%2Fzip&Expires=1648314318', 'ingestion_program': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648227912/064e7f07066c/ingestion_program_2.zip?AWSAccessKeyId=testkey&Signature=7YFTILe2sWVlbS5yJw%2B9hglOt%2Fc%3D&Expires=1648314318', 'input_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648227912/65c2ebc9a169/input_data_3.zip?AWSAccessKeyId=testkey&Signature=8ERTHS3P3eJs1uBcznRvhElrmCg%3D&Expires=1648314318', 'ingestion_only_during_scoring': False, 'program_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648227918/b90ec4f2d50e/submission_18.zip?AWSAccessKeyId=testkey&Signature=gZk7UZjzBof5Vjcv1CU%2FDBzRaH8%3D&Expires=1648314318', 'prediction_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227918/1d5db74d82ef/prediction_stdout.txt?AWSAccessKeyId=testkey&Signature=gWyRJ9kDRveBwwi9SOhyMLZX%2BOY%3D&content-type=application%2Fzip&Expires=1648314318', 'prediction_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227918/d6636924d9d5/prediction_stderr.txt?AWSAccessKeyId=testkey&Signature=g2VSYxOr0aK%2FYft1EQiWAM%2BP4uA%3D&content-type=application%2Fzip&Expires=1648314318', 'prediction_ingestion_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227918/fa95b4189d52/prediction_ingestion_stdout.txt?AWSAccessKeyId=testkey&Signature=BRjZSmU%2BuliU%2B%2FK0h70OFTo4NhI%3D&content-type=application%2Fzip&Expires=1648314318', 'prediction_ingestion_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648227918/99ea3e4a812c/prediction_ingestion_stderr.txt?AWSAccessKeyId=testkey&Signature=iY4dRXqKghjUAefIgQ8clw1TXAI%3D&content-type=application%2Fzip&Expires=1648314318'}
Task competitions.tasks._run_submission[afa2e972-19b8-4f9f-afd1-2ad05ddabc2d] succeeded in 0.24058851299992057s: None
Task data for submission id = 391
{'user_pk': 1050, 'submissions_api_url': 'http://django:36475/api', 'secret': UUID('6df587e7-8f6d-4f2e-8f02-d6ddf98b8ded'), 'docker_image': 'vergilgxw/autotable:v2', 'execution_time_limit': 600, 'id': 391, 'is_scoring': True, 'prediction_result': 'http://172.17.0.1:9000/private/prediction_result/2022-03-25-1648227918/ebbbfd6fc857/prediction_result.zip?AWSAccessKeyId=testkey&Signature=Kj57znL4dRGchiF%2F7CQupouJyKo%3D&Expires=1648314443', 'scoring_result': 'http://172.17.0.1:9000/private/scoring_result/2022-03-25-1648227918/9070d0451099/scoring_result.zip?AWSAccessKeyId=testkey&Signature=122y%2FmM5bjbqPcVOwmGygn9wqYg%3D&content-type=application%2Fzip&Expires=1648314443', 'reference_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648227912/0c09f04a6ec1/reference_data_3.zip?AWSAccessKeyId=testkey&Signature=oTGa0oKEXQZ4bzKSLzdZg5hN39k%3D&Expires=1648314443', 'ingestion_only_during_scoring': False, 'program_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648227912/4d64350e5d5f/scoring_program_2.zip?AWSAccessKeyId=testkey&Signature=ACg5J6mhf5DOFd2SQD9DXMPL%2BYs%3D&Expires=1648314443', 'scoring_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228043/196fd5622281/scoring_stdout.txt?AWSAccessKeyId=testkey&Signature=19V9w8ubpqlaLwievNG6Td9vafY%3D&content-type=application%2Fzip&Expires=1648314443', 'scoring_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228043/15b28d1722da/scoring_stderr.txt?AWSAccessKeyId=testkey&Signature=mDbzdzbFlZ447ePmBxC0%2FALIPi4%3D&content-type=application%2Fzip&Expires=1648314443', 'scoring_ingestion_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228043/02e591bbfa50/scoring_ingestion_stdout.txt?AWSAccessKeyId=testkey&Signature=cGPeWj8IR96SvrAoCmG8WOTn6vY%3D&content-type=application%2Fzip&Expires=1648314443', 'scoring_ingestion_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228043/7109c66f5065/scoring_ingestion_stderr.txt?AWSAccessKeyId=testkey&Signature=LIyyScLBRuJBh4G9I%2FsAzXggtaI%3D&content-type=application%2Fzip&Expires=1648314443'}
Task competitions.tasks._run_submission[2bfffa67-ae6e-4a88-b888-4f9b5edd0206] succeeded in 0.19657969200000025s: None
________________ TestSubmissions.test_v2_submission_end_to_end _________________
self = <tests.functional.test_submissions.TestSubmissions testMethod=test_v2_submission_end_to_end>
def test_v2_submission_end_to_end(self):
> self._run_submission_and_add_to_leaderboard('competition.zip', 'submission.zip', 'Scores')
src/tests/functional/test_submissions.py:80:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/tests/functional/test_submissions.py:54: in _run_submission_and_add_to_leaderboard
assert self.find('submission-manager#user-submission-table table tbody tr:nth-of-type(1) td:nth-of-type(2)').text == submission_zip_path
src/tests/utils.py:126: in find
raise e
src/tests/utils.py:123: in find
return wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, selector)))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <selenium.webdriver.support.wait.WebDriverWait (session="5aaf197b-cb90-4afa-bf5d-a7cdeef884aa")>
method = <selenium.webdriver.support.expected_conditions.presence_of_element_located object at 0x7f6c30ec9400>
message = ''
def until(self, method, message=''):
"""Calls the method provided with the driver as an argument until the \
return value is not False."""
screen = None
stacktrace = None
end_time = time.time() + self._timeout
while True:
try:
value = method(self._driver)
if value:
return value
except self._ignored_exceptions as exc:
screen = getattr(exc, 'screen', None)
stacktrace = getattr(exc, 'stacktrace', None)
time.sleep(self._poll)
if time.time() > end_time:
break
> raise TimeoutException(message, screen, stacktrace)
E selenium.common.exceptions.TimeoutException: Message:
/usr/local/lib/python3.8/site-packages/selenium/webdriver/support/wait.py:80: TimeoutException
----------------------------- Captured stderr call -----------------------------
HTTP/2 support not enabled (install the http2 and tls Twisted extras)
Configuring endpoint tcp:port=36475:interface=172.18.0.11
Listening on TCP address 172.18.0.11:36475
Starting unpack with status pk = 6
Download competition bundle: dataset/2022-03-25-1648228061/96dea957c858/competition.zip
The YAML version is: 2
Checking if path is directory: /tmp/tmp91s1o_7u/ingestion_program/
Zipping it up because it is directory, saving it to: ingestion_program.zip
New zip file path = /tmp/tmp91s1o_7u/ingestion_program.zip
Checking if path is directory: /tmp/tmp91s1o_7u/dev_phase/input_data/
Zipping it up because it is directory, saving it to: input_data.zip
New zip file path = /tmp/tmp91s1o_7u/dev_phase/input_data.zip
Checking if path is directory: /tmp/tmp91s1o_7u/scoring_program/
Zipping it up because it is directory, saving it to: scoring_program.zip
New zip file path = /tmp/tmp91s1o_7u/scoring_program.zip
Checking if path is directory: /tmp/tmp91s1o_7u/dev_phase/reference_data/
Zipping it up because it is directory, saving it to: reference_data.zip
New zip file path = /tmp/tmp91s1o_7u/dev_phase/reference_data.zip
Checking if path is directory: /tmp/tmp91s1o_7u/ingestion_program/
Zipping it up because it is directory, saving it to: ingestion_program.zip
New zip file path = /tmp/tmp91s1o_7u/ingestion_program.zip
Checking if path is directory: /tmp/tmp91s1o_7u/final_phase/input_data/
Zipping it up because it is directory, saving it to: input_data.zip
New zip file path = /tmp/tmp91s1o_7u/final_phase/input_data.zip
Checking if path is directory: /tmp/tmp91s1o_7u/scoring_program/
Zipping it up because it is directory, saving it to: scoring_program.zip
New zip file path = /tmp/tmp91s1o_7u/scoring_program.zip
Checking if path is directory: /tmp/tmp91s1o_7u/final_phase/reference_data/
Zipping it up because it is directory, saving it to: reference_data.zip
New zip file path = /tmp/tmp91s1o_7u/final_phase/reference_data.zip
Checking if path is directory: /tmp/tmp91s1o_7u/solution.zip
Competition saved!
Task competitions.tasks.unpack_competition[528c28e7-5a1c-4d06-87c1-4934d75a86a6] succeeded in 0.506170222000037s: None
Task data for submission id = 392
{'user_pk': 1051, 'submissions_api_url': 'http://django:36475/api', 'secret': UUID('d4796b99-6b92-413c-a615-6aa634cc1934'), 'docker_image': 'codalab/codalab-legacy:py3', 'execution_time_limit': 600, 'id': 392, 'is_scoring': False, 'prediction_result': 'http://172.17.0.1:9000/private/prediction_result/2022-03-25-1648228067/d8f087ac8c8a/prediction_result.zip?AWSAccessKeyId=testkey&Signature=9ZwHh%2BO35n9cSDScWEgHRE7pFoM%3D&content-type=application%2Fzip&Expires=1648314467', 'ingestion_program': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648228061/e3b7420df88a/ingestion_program.zip?AWSAccessKeyId=testkey&Signature=mph9eQQdy%2FqokutAftklKNmH394%3D&Expires=1648314467', 'input_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648228061/ab156e7f1357/input_data.zip?AWSAccessKeyId=testkey&Signature=7oZgd9%2B0NWe2ecFFj1XSD53Vr2Y%3D&Expires=1648314467', 'ingestion_only_during_scoring': False, 'program_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648228067/a40755d234ff/submission.zip?AWSAccessKeyId=testkey&Signature=Cpqk4BBXNUziAtyAGGpt69iYOA4%3D&Expires=1648314467', 'prediction_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228067/e1381d6ca7b1/prediction_stdout.txt?AWSAccessKeyId=testkey&Signature=G%2FFUhG3tJQgFXPHmBlkgNOF4H34%3D&content-type=application%2Fzip&Expires=1648314467', 'prediction_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228067/9026cd32c470/prediction_stderr.txt?AWSAccessKeyId=testkey&Signature=7a3Q7HCSKrM4KTl4uQJd3GtWBeo%3D&content-type=application%2Fzip&Expires=1648314467', 'prediction_ingestion_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228067/c4b331c2bc77/prediction_ingestion_stdout.txt?AWSAccessKeyId=testkey&Signature=115dqtX3dXzmzx%2BJoIizetLBvJg%3D&content-type=application%2Fzip&Expires=1648314467', 'prediction_ingestion_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228067/73d2bd931363/prediction_ingestion_stderr.txt?AWSAccessKeyId=testkey&Signature=uQieyLfWdXjxKrkVbeWlmVGnhME%3D&content-type=application%2Fzip&Expires=1648314467'}
Task competitions.tasks._run_submission[38503f77-bb2a-457b-b4ae-cf0eaea746e4] succeeded in 0.24798858700000892s: None
Task data for submission id = 392
{'user_pk': 1051, 'submissions_api_url': 'http://django:36475/api', 'secret': UUID('d4796b99-6b92-413c-a615-6aa634cc1934'), 'docker_image': 'codalab/codalab-legacy:py3', 'execution_time_limit': 600, 'id': 392, 'is_scoring': True, 'prediction_result': 'http://172.17.0.1:9000/private/prediction_result/2022-03-25-1648228067/d8f087ac8c8a/prediction_result.zip?AWSAccessKeyId=testkey&Signature=Ufn3sQ5gNqHAxvv5C4Mz1NG3K84%3D&Expires=1648314471', 'scoring_result': 'http://172.17.0.1:9000/private/scoring_result/2022-03-25-1648228067/f64ee80e7a40/scoring_result.zip?AWSAccessKeyId=testkey&Signature=AnxzpLHD8ltqivAeBZ%2BPTuR8JmY%3D&content-type=application%2Fzip&Expires=1648314471', 'reference_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648228061/668c26dda715/reference_data.zip?AWSAccessKeyId=testkey&Signature=grO5%2Fd5ZfeuwtFM1p%2FFNx8fRy38%3D&Expires=1648314471', 'ingestion_only_during_scoring': False, 'program_data': 'http://172.17.0.1:9000/private/dataset/2022-03-25-1648228061/0957fd1698d7/scoring_program.zip?AWSAccessKeyId=testkey&Signature=A7M9RW5Z0%2BV%2BYuPVQJ7Sg0HCHZI%3D&Expires=1648314471', 'scoring_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228071/b2af2fe2ed85/scoring_stdout.txt?AWSAccessKeyId=testkey&Signature=LLgvGT1OfbTBu6wefwo0fUGXYkg%3D&content-type=application%2Fzip&Expires=1648314471', 'scoring_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228071/0d68990f566a/scoring_stderr.txt?AWSAccessKeyId=testkey&Signature=qigviE9UWMo%2FwkrFOtIxGAv8jMc%3D&content-type=application%2Fzip&Expires=1648314471', 'scoring_ingestion_stdout': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228071/14fed17e4570/scoring_ingestion_stdout.txt?AWSAccessKeyId=testkey&Signature=bheiwPx6Oo28AX8SLdfuxLiWWUI%3D&content-type=application%2Fzip&Expires=1648314471', 'scoring_ingestion_stderr': 'http://172.17.0.1:9000/private/submission_details/2022-03-25-1648228071/ae80a7ab75ea/scoring_ingestion_stderr.txt?AWSAccessKeyId=testkey&Signature=qixkyhYPNoR8PG%2FvAiQKSKJ3VSU%3D&content-type=application%2Fzip&Expires=1648314471'}
Task competitions.tasks._run_submission[6b2044f6-cde2-49be-987a-fffe90a2935f] succeeded in 0.13600534599981984s: None
=============================== warnings summary ===============================
../usr/local/lib/python3.8/site-packages/storages/backends/s3boto3.py:238
../usr/local/lib/python3.8/site-packages/storages/backends/s3boto3.py:238
src/tests/functional/test_competitions.py::TestCompetitions::test_manual_competition_creation
src/tests/functional/test_competitions.py::TestCompetitions::test_manual_competition_creation
src/tests/functional/test_competitions.py::TestCompetitions::test_manual_competition_creation
src/tests/functional/test_competitions.py::TestCompetitions::test_manual_competition_creation
src/tests/functional/test_competitions.py::TestCompetitions::test_manual_competition_creation
/usr/local/lib/python3.8/site-packages/storages/backends/s3boto3.py:238: DeprecationWarning: The bucket argument of S3Boto3Storage is deprecated. Use argument bucket_name or setting AWS_STORAGE_BUCKET_NAME instead. The bucket argument will be removed in version 2.0.
warnings.warn(
src/tests/utils.py:73
/app/src/tests/utils.py:73: PytestUnknownMarkWarning: Unknown pytest.mark.e2e - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
@pytest.mark.e2e
-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED src/tests/functional/test_submissions.py::TestSubmissions::test_v15_submission_end_to_end
FAILED src/tests/functional/test_submissions.py::TestSubmissions::test_v18_submission_end_to_end
FAILED src/tests/functional/test_submissions.py::TestSubmissions::test_v2_submission_end_to_end
============= 3 failed, 5 passed, 8 warnings in 400.20s (0:06:40) ==============
Exited with code exit status 1
CircleCI received exit code 1