Series of projects and tutorials around using Building production grade Deep learning pipelines
-
A brief architecture for a simple/scalable Deep learning pipeline.
-
Building blocks of a real world Deep learning pipeline.
-
Explain the different building blocks.
-
HDFS vs Minio for storage layer.
-
Training on batch data, storing Data into Minio.
-
Data exploration using Spark SQL, Minio and Jupyter notebook.
-
Data preparation.
-
Hello world tensorflow + Minio.
-
Docker container to run the example.
-
Code example to run Hello World tensorflow on Minio play.
-
Push the code into Recipes.
-
Read the data from Minio using Tensorflow into memory and run a simple Logistic regression training job.
-
Create Google collab notebooks for easy running of code.
-
Live coding video with explanation on the same.
-
Push the code to repo.
- Saving the trained model in the last blog into Minio.
- Serving from the trained model that is stored in Minio.
- Check pointing the training into Minio.
- Restoring the checkpoints from Minio.
- A Neural network project.
- Save the code examples in Github.
- Create Google collab notebooks for the project.
- Live coding video with explanation on the same.
- Push the code to repo.
- Running the project in last example end to end on Kubeflow.
- Create Terraform templates to recreate the Kubeflow setup with Minio in one shot on GKE.
- Live coding video with explanation on the same.
- Push the code to repo.
- Batch reading of data stored in Minio for training and scaling the in memory training.
- Neural network training on Large dataset.
- Create Containers for the code to run, push to repo.
- Live coding video with explanation on the same.
- Distributing the tensorflow training with large datasets stored in Minio distributed setup.
- Terraform scripts for getting the one shot setup.
- Kubeflow project for the same.
- Live coding video with explanation on the same.
- Push the code to repo.
- Ingest real world data from Kafka and store it in Minio as shredded files.
- Run batch Deep learning jobs on the injested stream data into Minio.
- Live coding video with explanation on the same.
- Terraform scripts for the setup.
- Kubeflow project.
- Push the code to repo.
- Clean the data using Kafka streaming and Kafka-SQL.
- Store the cleaned data on Minio.
- Run Deep learning training jobs.
- Terraform scripts and live code video.
- Push the code to repo.
- Large scale Image processing pipeline using Convolutional Neural networks using Minio and Tensorflow.
- Live coding video.
- Terraform scripts and Kubeflow projects.
- Push the code to repo.