Table of Contents

• Introduction
• Data format
• Configuration
• Models
• Simple training pipeline
• Examples
• Citing
• License

Introduction

This project use the library of qubvel/segmentation_models most, aiming at most of CV tasks such as object detection(including the prediction of bboxes, keypoints, heatmap, ..., etc), appearance embedding learning, encoding/decoding of images and GAN.

Data format

The data is stored in .xlsx file with single sheet for non-series model and with multiple sheets for time-series model. The format is like

key1	key1	key2	key2	key2	key2
value1	value2	value1	value2	value3	value4

The keys are strings and values could be file paths or numbers. One key may correspond to several values.

Configuration

The confiuration .json files stored in folder "config" contains informations how the program interpretates the data recorded in the .xlsx file.

{
    "hello": "world",
    "this": {
        "can": {
            "be": "nested"
        }
    }
}

The values corresponding to keys of image or array are the string of file path; that of keypoints are 2n numbers(x and y of n points); that of bboxes are 4n numbers(xmin, ymin xmax, ymax of n boxes)

Models

The keras models are stored at the folder "models" which contains the .yaml for the model graph and .h5 for the model weights. The names of inputs and outputs should be consistant with the corresponding configuration.

Simple training pipeline

After setting the model(graph and pretrained weights), configuration and generation of data, just execute

Examples

Face Analysis:

• [Dataset] Facial Keypoints Detection here.
• [Data Augmentation]

• [color augmentation]
• [noise augmentation]
• [perspective transform]

• [Model]

• [backbone] VGG19 with batch-normalization, using stride 2 convolution instead of max-pooling.
• [bbox detection branch] Predict bbox of facial part.
• [hashcode prediction branch] Give the 16-digit binary hash code of facial part.
• [group mask prediction branch] Detect group of keypoints(left-eye, right-eye, mouth parts) with form of heatmap.
• [keypoint deviation] Predict the deviation of keypoint with respect to location of grid.
• [denoise variational auto encoder/decoder] Recover and filter noise of image with normally distributed coding.
• [image/recover discriminator] Discriminate the original image and the recovered or randomly generated image.

Citing

@misc{Yakubovskiy:2019,
  Author = {Pavel Yakubovskiy},
  Title = {Segmentation Models},
  Year = {2019},
  Publisher = {GitHub},
  Journal = {GitHub repository},
  Howpublished = {\url{https://github.com/qubvel/segmentation_models}}
}

License

Project is distributed under MIT Licence.