A Pytorch Implementation for Incremental Multi-Target Domain Adaptation for Object Detection with Efficient Domain Transfer

Introduction

Please follow faster-rcnn and HTCN respository to setup the environment. In this project, we use Pytorch 1.6.1 and CUDA version is 10.2

Additional Information

• We use Sacred(link) to run/manage our experiments. It can be installed with:

pip install sacred

• To monitor the training, we used MongoObserver of Sacred with Omniboard. However, it is disabled by default. If you want to plug in your own monitoring. It can be done in class LoggerForSacred of experiments/exp_utils.py which will be called at every 100 iteration. By default, log are printed to stdout.
• All models are expected to be saved in all_saves folder in repo's root. This can be changed in the code.

Datasets

Datasets Preparation

• Cityscape and FoggyCityscape: Download the Cityscape dataset, see dataset preparation code in DA-Faster RCNN.
• PASCAL_VOC 07+12: Please follow the instruction to prepare VOC dataset.
• Clipart - Watercolor - Comic: Please follow the instruction to download

Datasets Format

All codes are written to fit for the format of PASCAL_VOC.
If you want to use this code on your own dataset, please arrange the dataset in the format of PASCAL, make dataset class in lib/datasets/, and add it to lib/datasets/factory.py, lib/datasets/config_dataset.py. Then, add the dataset option to lib/model/utils/parser_func.py.

For RainCityscape train and evaluation list you can find them here: train, val

Models

Pre-trained Models

In our experiments, we used two pre-trained models on ImageNet, i.e., VGG16 and ResNet101. Please download these two models from:

• VGG16: Dropbox VT Server

• ResNet50: Download the model (the one for BGR) from link

Download them and write the path in __C.VGG_PATH and __C.RESNET50_PATH at lib/model/utils/config.py.

Train Example

Don't forget to install sacred, you cannot run our code without it. To run the first step of Cityscape -> FoggyCityscape on VGG16 with our hyper-parameters :

CUDA_VISIBLE_DEVICES=$GPU_ID \
       python experiments/exp_traineval_htcn.py

If you want to do your own hyper-parameters, etc..., exp_traineval_htcn.py is fairly easy to understand to modify. We provided an example to modify this at the end. To train DTM for the next incremental step, you will have to change 'load_name' inside exp_train_dtm.py to the model obtained from previous step:

CUDA_VISIBLE_DEVICES=$GPU_ID \
       python experiments/exp_train_dtm.py

The next incremental domain adaptation, you will have to point 'dtm_load_p' to the DTM model you just trained and 'load_name' the model obtained from the first domain adaptation inside exp_traineval_htcn_inc_ida_dtm.py:

CUDA_VISIBLE_DEVICES=$GPU_ID \
       python experiments/exp_traineval_htcn_inc_ida_dtm.py

Example of how to edit hyper-parameters etc... in exp_traineval_htcn.py. Original:

if __name__ == "__main__":
    ex.run(config_updates={'cfg_file': 'cfgs/vgg16.yml',
                           'lr': 0.001,
                           'lr_decay_step': [5],
                           'max_epochs': 7,
                           'net': 'vgg16',
                           'pretrained': True,
                           'dataset_source': 'cs',
                           'dataset_target': 'cs_fg',
                           'val_datasets': ['cs_fg']},

           options={"--name": 'htcn_cs_2_cs_fg_vgg16'})

Edit for PascalVOC -> Clipart in exp_traineval_htcn.py:

if __name__ == "__main__":
    ex.run(config_updates={'cfg_file': 'cfgs/resnet50.yml',
                           'lr': 0.001,
                           'lr_decay_step': [5],
                           'max_epochs': 7,
                           'net': 'res50',
                           'pretrained': True,
                           'dataset_source': 'voc_0712',
                           'dataset_target': 'clipart',
                           'val_datasets': ['clipart']},

           options={"--name": 'NAME_OF_YOUR_EXPERIMENT'})

Test

Similar to training, modify exp_eval.py to evaluate a model. Edit 'val_datasets' for the dataset you want to test, it can be multiple datasets (it will return overall accuracy of all datasets on each class), 'model_pth' the path to the model:

CUDA_VISIBLE_DEVICES=$GPU_ID \
       python experiments/exp_eval.py 

Trained Model

• VGG16 Model of Cityscape -> FoggyCityscape -> RainCityscape(Overall mAP = 39.1): Link
• ResNet50 Model of Pascal -> Clipart -> Watercolor(Overall mAP = 39.8): Link
• ResNet50 Model of Pascal -> Clipart -> Watercolor -> Comic(Overall mAP = 38.1): Link