Adding EfficientNetsB0-B7 support #1938

masadcv · 2021-04-03T23:12:09Z

Signed-off-by: masadcv muhammad.asad@kcl.ac.uk

Fixes #1090 .

Description

Adding support for original EfficientNetB0-B7 models from: https://arxiv.org/abs/1905.11946

Based on https://github.com/lukemelas/EfficientNet-PyTorch

Extended to use MONAI factories to generalise for spatial_dims in [1, 2, 3]
Can be used with pretrained weights from above repo when spatial_dims=2
Also adds a memory-efficient version of Swish activation function from: Memory Issues lukemelas/EfficientNet-PyTorch#18 (comment) which is reported to save 30% memory during training on NVIDIA GPUs
Added unittests to cover checking output shapes for EfficientnetB0-B7 implementation variations and correct loading of pretrained weights from above repo
Added unittests to cover saving/loading with Torchscript using tests from tests.utils.test_script_save()
Added a kitty test image for classification models unittesting at: ./tests/testing_data/kitty_test.jpg (image from: https://commons.wikimedia.org/wiki/File:Tabby_cat_with_blue_eyes-3336579.jpg)
Added unittests to cover loading and running from pretrained weights for correct 'kitty' classification
Fixed formatting/coding style to comply with ./runtests.sh
Updated docs by adding new modules and checking the compiled html

Status

Ready

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
Breaking change (fix or new feature that would cause existing functionality to change).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

wyli

thanks @masadcv! this looks nice I have some minor comments inline, perhaps @yiheng-wang-nv could help double-check with the network details? also please add @skip_if_quick to the unit tests if they requires downloading, e.g. https://github.com/Project-MONAI/MONAI/blob/master/tests/test_ahnet.py#L138

tests/test_efficientnet.py

monai/networks/layers/factories.py

monai/networks/nets/efficientnet.py

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

masadcv · 2021-04-07T00:36:23Z

thanks @masadcv! this looks nice I have some minor comments inline, perhaps @yiheng-wang-nv could help double-check with the network details? also please add @skip_if_quick to the unit tests if they requires downloading, e.g. https://github.com/Project-MONAI/MONAI/blob/master/tests/test_ahnet.py#L138

@wyli thank you for your comments! I have addressed all but the drop_connect_rate test. There doesn't seem to be a straightforward way to test it without exposing unnecessary arguments for the class. I have added unittests for drop_connect layer which covers a number of cases. The unittests with pretrained models now use @skip_if_quick as well as test url loading using test_pretrained_networks. We may also want to look into hosting models on a mirror in case we loose the ones from original repo.

'memswish' has been added to unittests for activation.

The kitty test image has been updated to use a Creative Commons image from wikimedia commons.

I need a few more passes to have everything double-checked and in place. I will also run the full coverage integration tests as soon as I have access to relevant resources.

@yiheng-wang-nv would it be possible to have your review in the coming days? Thank you!

monai/networks/nets/efficientnet.py

tests/test_efficientnet.py

yiheng-wang-nv · 2021-04-07T10:04:20Z

Thanks @masadcv , I added some comments above.
Hi @wyli , does efficientnet need to support torch script?

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

wyli · 2021-04-08T07:13:49Z

could you please add test_efficientnet to the "min_tests" list of excluding cases

MONAI/tests/min_tests.py

Line 29 in d56c002

exclude_cases = [ # these cases use external dependencies

and fix/remove some of the problematic type hints, I'll make sure this PR is included in v0.5, many thanks @masadcv !

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

masadcv · 2021-04-08T11:21:18Z

could you please add test_efficientnet to the "min_tests" list of excluding cases

MONAI/tests/min_tests.py

Line 29 in d56c002

exclude_cases = [ # these cases use external dependencies

and fix/remove some of the problematic type hints, I'll make sure this PR is included in v0.5, many thanks @masadcv !

@wyli many thanks! I have done both, locally all tests are passing. It would be awesome to have this in v0.5. Once that is done, I will also add a tutorial. Do you have any suggestions for medical imaging related classification dataset that I can use for a simple tutorial?

@yiheng-wang-nv many thanks for your comments, I have addressed most of your comments. I will look into testing different shapes as you suggested.

wyli · 2021-04-08T12:02:46Z

/black
there's still an error here: https://github.com/Project-MONAI/MONAI/pull/1938/checks?check_run_id=2296192348#step:6:7760

tutorial -- we have a demo script using the IXI dataset (https://brain-development.org/ixi-dataset/) for gender classification https://github.com/Project-MONAI/tutorials/blob/master/3d_classification/torch/densenet_training_array.py perhaps you can make a new example based on it

masadcv · 2021-04-08T13:21:09Z

there's still an error here: https://github.com/Project-MONAI/MONAI/pull/1938/checks?check_run_id=2296192348#step:6:7760

tutorial -- we have a demo script using the IXI dataset (https://brain-development.org/ixi-dataset/) for gender classification https://github.com/Project-MONAI/tutorials/blob/master/3d_classification/torch/densenet_training_array.py perhaps you can make a new example based on it

I am looking into this error now. Should resolve soon.
Thanks for the pointing me in the direction of tutorials.

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

tests/test_efficientnet.py

wyli

thanks, it looks nice!

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

yiheng-wang-nv · 2021-04-08T14:34:39Z

could you please add test_efficientnet to the "min_tests" list of excluding cases

MONAI/tests/min_tests.py

Line 29 in d56c002

exclude_cases = [ # these cases use external dependencies

and fix/remove some of the problematic type hints, I'll make sure this PR is included in v0.5, many thanks @masadcv !

@wyli many thanks! I have done both, locally all tests are passing. It would be awesome to have this in v0.5. Once that is done, I will also add a tutorial. Do you have any suggestions for medical imaging related classification dataset that I can use for a simple tutorial?

@yiheng-wang-nv many thanks for your comments, I have addressed most of your comments. I will look into testing different shapes as you suggested.

Many thanks @masadcv , this PR looks nice!

masadcv · 2021-04-08T15:18:31Z

Seems to be failing on unrelated test now:
https://github.com/Project-MONAI/MONAI/pull/1938/checks?check_run_id=2297721328

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* adding init efficientnet support Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * fixing flake8 and further refactoring Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * adding unittests for efficiennet Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * making unittests backwards compatible python<3.8 Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * fixed kitty unittests file path Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * adding docstrings and minor refactoring Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * fix flake8-py3 failing test Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * generalize drop_connect for n-dim, fix/add unittests, remove assert Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * fix failing unittest, CC0-license image for test Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * refactoring code for review Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * WIP fix mypy type hint errors Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * fix cuda test error Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * WIP fix test errors Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * adding non-default shape tests Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * remove 3d case from non-default shape test Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> * refactoring and updating docs Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk> Co-authored-by: Yiheng Wang <68361391+yiheng-wang-nv@users.noreply.github.com> Co-authored-by: Wenqi Li <wenqil@nvidia.com> Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>

masadcv added 6 commits April 4, 2021 00:01

adding init efficientnet support

259541f

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

fixing flake8 and further refactoring

12bc476

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

adding unittests for efficiennet

9cda5e2

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

making unittests backwards compatible python<3.8

c086516

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

fixed kitty unittests file path

b718b3b

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

Merge branch 'master' into 1090-efficientnet-support

fe0acdb

wyli requested a review from yiheng-wang-nv April 5, 2021 23:39

masadcv added 2 commits April 6, 2021 02:45

adding docstrings and minor refactoring

cea0d90

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

fix flake8-py3 failing test

dfc0a6e

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

wyli reviewed Apr 6, 2021

View reviewed changes

masadcv added 2 commits April 6, 2021 14:03

generalize drop_connect for n-dim, fix/add unittests, remove assert

0940161

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

fix failing unittest, CC0-license image for test

d83a053

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

yiheng-wang-nv reviewed Apr 7, 2021

View reviewed changes

yiheng-wang-nv and others added 2 commits April 7, 2021 18:05

Merge branch 'master' into 1090-efficientnet-support

e96ab04

refactoring code for review

f2606e7

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

masadcv force-pushed the 1090-efficientnet-support branch from 928c91d to f2606e7 Compare April 8, 2021 01:29

Merge branch 'master' into 1090-efficientnet-support

eb0c4d8

WIP fix mypy type hint errors

89febad

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

masadcv added 2 commits April 8, 2021 14:30

fix cuda test error

3d2fa09

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

WIP fix test errors

410a406

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

wyli reviewed Apr 8, 2021

View reviewed changes

tests/test_efficientnet.py Show resolved Hide resolved

wyli approved these changes Apr 8, 2021

View reviewed changes

masadcv added 2 commits April 8, 2021 15:16

adding non-default shape tests

45c2d91

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

remove 3d case from non-default shape test

d51249a

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

refactoring and updating docs

1774a48

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

masadcv changed the title ~~[WIP] Adding EfficientNetsB0-B7 support~~ Adding EfficientNetsB0-B7 support Apr 8, 2021

wyli enabled auto-merge (squash) April 9, 2021 00:17

wyli merged commit e7a7422 into Project-MONAI:master Apr 9, 2021

masadcv deleted the 1090-efficientnet-support branch April 9, 2021 09:02

Adding EfficientNetsB0-B7 support #1938

Adding EfficientNetsB0-B7 support #1938

Uh oh!

Conversation

masadcv commented Apr 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Status

Types of changes

Uh oh!

wyli left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

masadcv commented Apr 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yiheng-wang-nv commented Apr 7, 2021

Uh oh!

wyli commented Apr 8, 2021

Uh oh!

masadcv commented Apr 8, 2021

Uh oh!

wyli commented Apr 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

masadcv commented Apr 8, 2021

Uh oh!

Uh oh!

wyli left a comment

Choose a reason for hiding this comment

Uh oh!

yiheng-wang-nv commented Apr 8, 2021

Uh oh!

masadcv commented Apr 8, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

masadcv commented Apr 3, 2021 •

edited

Loading

masadcv commented Apr 7, 2021 •

edited

Loading

wyli commented Apr 8, 2021 •

edited

Loading