Skip to content
This repository was archived by the owner on Mar 21, 2024. It is now read-only.

Commit 9c8ef64

Browse files
authored
BUG: Fix broken deployments + delicate envs (#83)
* 📌 Replace environment files with simple reqs * 👷 Update enviornmeny.yml for CI tests * 👷 Make PR tests nightly too * 📝 Update documentation
1 parent c326084 commit 9c8ef64

File tree

6 files changed

+24
-245
lines changed

6 files changed

+24
-245
lines changed
Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
1-
name: PR Tests
1+
name: Build, Test and Deploy
22

33
on:
44
pull_request:
55
branches: [ main ]
6+
schedule:
7+
- cron: '0 0 * * *'
68

79
jobs:
810
linux-build-and-test:

README.md

Lines changed: 14 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,14 @@
11
# Introduction
22

3-
InnerEye-Inference is a AppService webapp in python to run inference on medical imaging models trained with the [InnerEye-DeepLearning toolkit](https://github.com/microsoft/InnerEye-Inference).
3+
InnerEye-Inference is an App Service webapp in python to run inference on medical imaging models trained with the [InnerEye-DeepLearning toolkit](https://github.com/microsoft/InnerEye-Inference).
44

5-
You can also integrate this with DICOM using the [InnerEye-Gateway](https://github.com/microsoft/InnerEye-Gateway)
5+
You can also integrate this with DICOM using the [InnerEye-Gateway](https://github.com/microsoft/InnerEye-Gateway).
66

77
## Getting Started
88

99
### Operating System
1010

11-
If developing or using this tool locally, we highly recommend using [Ubuntu 20.04](https://releases.ubuntu.com/20.04/) as your operating system. This is for two reasons:
12-
13-
- The Azure App Service base image will be Ubuntu. By developing locally in Ubuntu you can guarantee maximum repeatibility between local and cloud behaviour.
14-
- The provided `environment.yml` contains locked secondary package dependencies for Ubuntu **only**. We cannot guarantee identical functionality to our dev environments if you use other operating systems.
11+
If developing or using this tool locally, we highly recommend using [Ubuntu 20.04](https://releases.ubuntu.com/20.04/) as your operating system. This is as the Azure App Service base image will be Ubuntu. By developing locally in Ubuntu you can guarantee maximum repeatibility between local and cloud behaviour.
1512

1613
For windows users this is easily done through [Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/install).
1714

@@ -24,31 +21,25 @@ and run it.
2421

2522
Note that in order to create the Conda environment you will need to have build tools installed on your machine. If you are running Windows, they should be already installed with Conda distribution.
2623

27-
You can install build tools on Ubuntu (and Debian-based distributions) by running
28-
`sudo apt-get install build-essential`.
29-
If you are running CentOS/RHEL distributions, you can install the build tools by running
30-
`yum install gcc gcc-c++ kernel-devel make`.
24+
You can install build tools on Ubuntu (and Debian-based distributions) by running:
3125

32-
#### Ubuntu Users
26+
```shell
27+
sudo apt-get install build-essential
28+
```
3329

34-
Start the `conda` prompt for your platform. In that prompt, navigate to your repository root and run
30+
If you are running CentOS/RHEL distributions, you can install the build tools by running:
3531

36-
```console
37-
conda env create --file environment.yml
38-
conda activate inference
32+
```shell
33+
yum install gcc gcc-c++ kernel-devel make
3934
```
4035

41-
#### Other Operating Sytem Users
42-
43-
Start the `conda` prompt for your platform. In that prompt, navigate to your repository root and run
36+
Start the `conda` prompt for your platform. In that prompt, navigate to your repository root and run:
4437

4538
```console
46-
conda env create --file primary_deps.yml
39+
conda env create --file environment.yml
4740
conda activate inference
4841
```
4942

50-
This will build an environment from the primary package dependencies only. This means that the packages and versions that the primary packages depend upon will be different from those defined in `environment.yml` and as such may behave differently to the application when [deployed to Azure](#running-flask-app-in-azure).
51-
5243
### Configuration
5344

5445
Add this script with name set_environment.sh to set your env variables. This can be executed in Linux. The code will read the file if the environment variables are not present.
@@ -185,16 +176,9 @@ If you would like to reproduce the automatic deployment of the service for testi
185176

186177
During inference the image data zip file is copied to the IMAGE_DATA_FOLDER in the AzureML workspace's DATASTORE_NAME datastore. At the end of inference the copied image data zip file is overwritten with a simple line of text. At present we cannot delete these. If you would like these overwritten files removed from your datastore you can [add a policy](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-lifecycle-management-concepts?tabs=azure-portal) to delete items from the datastore after a period of time. We recommend 7 days.
187178

188-
## Changing Primary Dependencies
189-
190-
1. Make your desired changes in `primary_deps.yml`. Make sure your package name and version are correct.
191-
1. To create a new environment and a valid `environment.yml`, run the following command:
192-
193-
```shell
194-
bash -i create_and_lock_environment.sh
195-
```
179+
## Changing Dependencies
196180

197-
1. Voila! You will now have a new conda environment with your desired primary package versions, as well as a new `environment.yml` which can be ingested by AzureML to create a copy of your local environment.
181+
The Azure App Service will use the packages specified in `requirements.txt` to create the python virtual environment in which the flask app is run. The `environment.yml` is used for local environments only. Therefore if you want to change the packages your app service has access to, you must update `requirements.txt`.
198182

199183
## Help and Bug Reporting
200184

create_and_lock_environment.sh

Lines changed: 0 additions & 49 deletions
This file was deleted.

environment.yml

Lines changed: 3 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,6 @@
1-
# WARNING - DO NOT EDIT THIS FILE MANUALLY
2-
# Please refer to the environment documentation for instructions on how to create a new version of this file: https://github.com/microsoft/InnerEye-DeepLearning/blob/main/docs/environment.md
31
name: inference
4-
channels:
5-
- defaults
62
dependencies:
7-
- _libgcc_mutex=0.1=main
8-
- _openmp_mutex=5.1=1_gnu
9-
- ca-certificates=2022.4.26=h06a4308_0
10-
- certifi=2022.6.15=py37h06a4308_0
11-
- libedit=3.1.20210910=h7f8727e_0
12-
- libffi=3.2.1=hf484d3e_1007
13-
- libgcc-ng=11.2.0=h1234567_1
14-
- libgomp=11.2.0=h1234567_1
15-
- libstdcxx-ng=11.2.0=h1234567_1
16-
- ncurses=6.3=h7f8727e_2
17-
- openssl=1.1.1p=h5eee18b_0
18-
- pip=20.1.1=py37_1
19-
- python=3.7.3
20-
- readline=7.0=h7b6447c_5
21-
- setuptools=61.2.0=py37h06a4308_0
22-
- sqlite=3.33.0=h62c20be_0
23-
- tk=8.6.12=h1ccaba5_0
24-
- wheel=0.37.1=pyhd3eb1b0_0
25-
- xz=5.2.5=h7f8727e_1
26-
- zlib=1.2.12=h7f8727e_2
3+
- python=3.7.15
4+
- pip
275
- pip:
28-
- -r file:requirements.txt
6+
- -r requirements.txt

primary_deps.yml

Lines changed: 0 additions & 23 deletions
This file was deleted.

requirements.txt

Lines changed: 4 additions & 117 deletions
Original file line numberDiff line numberDiff line change
@@ -1,126 +1,13 @@
1-
absl-py==1.1.0
2-
adal==1.2.7
3-
applicationinsights==0.11.10
4-
argcomplete==2.0.0
5-
attrs==21.4.0
6-
azure-common==1.1.28
7-
azure-core==1.22.1
8-
azure-graphrbac==0.61.1
9-
azure-identity==1.7.0
10-
azure-mgmt-authorization==2.0.0
11-
azure-mgmt-containerregistry==9.1.0
12-
azure-mgmt-core==1.3.1
13-
azure-mgmt-keyvault==9.3.0
14-
azure-mgmt-resource==20.1.0
15-
azure-mgmt-storage==19.1.0
16-
azureml-automl-core==1.41.1.post1
17-
azureml-core==1.41.0.post3
18-
azureml-dataprep==3.1.4
19-
azureml-dataprep-native==38.0.0
20-
azureml-dataprep-rslex==2.5.5
21-
azureml-dataset-runtime==1.41.0
22-
azureml-pipeline==1.41.0
23-
azureml-pipeline-core==1.41.0
24-
azureml-pipeline-steps==1.41.0
251
azureml-sdk==1.41.0
26-
azureml-telemetry==1.41.0
27-
azureml-tensorboard==1.43.0
28-
azureml-train-automl-client==1.41.0
29-
azureml-train-core==1.41.0
30-
azureml-train-restclients-hyperdrive==1.41.0
31-
backports-tempfile==1.0
32-
backports-weakref==1.0.post1
33-
bcrypt==3.2.2
34-
cachetools==5.2.0
35-
cffi==1.15.0
36-
charset-normalizer==2.0.12
37-
click==8.1.3
38-
cloudpickle==2.1.0
39-
conda-merge==0.2.0
40-
contextlib2==21.6.0
41-
coverage==6.4.1
42-
cryptography==36.0.2
43-
distro==1.7.0
44-
docker==5.0.3
45-
dotnetcore2==2.1.23
462
flake8==3.8.4
47-
flask==2.1.1
483
flask-injector==0.13.0
49-
fusepy==3.0.1
50-
google-auth==2.9.0
51-
google-auth-oauthlib==0.4.6
52-
grpcio==1.47.0
4+
Flask==2.1.1
535
hi-ml-azure==0.2.1
54-
humanfriendly==10.0
55-
idna==3.3
56-
importlib-metadata==4.12.0
57-
iniconfig==1.1.1
58-
injector==0.20.0
59-
isodate==0.6.1
60-
itsdangerous==2.1.2
61-
jeepney==0.8.0
62-
jinja2==3.1.2
63-
jmespath==0.10.0
64-
jsonpickle==2.2.0
65-
knack==0.9.0
66-
markdown==3.3.7
67-
markupsafe==2.1.1
68-
mccabe==0.6.1
696
memory-tempfile==2.2.3
70-
more-itertools==8.13.0
71-
msal==1.18.0
72-
msal-extensions==0.3.1
737
msrest==0.6.21
74-
msrestazure==0.6.4
758
mypy==0.790
76-
mypy-extensions==0.4.3
77-
ndg-httpsclient==0.5.1
78-
numpy==1.21.6
79-
oauthlib==3.2.0
80-
packaging==21.3
81-
pandas==1.3.5
82-
param==1.12.2
83-
paramiko==2.11.0
84-
pathspec==0.9.0
85-
pkginfo==1.8.3
86-
pluggy==0.13.1
87-
portalocker==2.4.0
88-
protobuf==3.19.4
89-
py==1.11.0
90-
pyarrow==3.0.0
91-
pyasn1==0.4.8
92-
pyasn1-modules==0.2.8
93-
pycodestyle==2.6.0
94-
pycparser==2.21
959
pydicom==2.1.2
96-
pyflakes==2.2.0
97-
pygments==2.12.0
98-
pyjwt==2.4.0
99-
pynacl==1.5.0
100-
pyopenssl==22.0.0
101-
pyparsing==3.0.9
102-
pysocks==1.7.1
103-
pytest==6.0.1
10410
pytest-cov==2.10.1
105-
python-dateutil==2.8.2
106-
pytz==2022.1
107-
pyyaml==6.0
108-
requests==2.28.0
109-
requests-oauthlib==1.3.1
110-
rsa==4.8
111-
ruamel-yaml==0.17.21
112-
ruamel-yaml-clib==0.2.6
113-
secretstorage==3.3.2
114-
simpleitk==2.0.2
115-
six==1.16.0
116-
tabulate==0.8.10
117-
tensorboard==2.9.1
118-
tensorboard-data-server==0.6.1
119-
tensorboard-plugin-wit==1.8.1
120-
toml==0.10.2
121-
typed-ast==1.4.3
122-
typing-extensions==4.2.0
123-
urllib3==1.26.7
124-
websocket-client==1.3.3
125-
werkzeug==2.1.1
126-
zipp==3.8.0
11+
pytest==6.0.1
12+
SimpleITK==2.0.2
13+
Werkzeug==2.1.1

0 commit comments

Comments
 (0)