Skip to content

Python fundamentals with Pydantic for Databricks and Streamlit development - comprehensive tutorial with notebooks and interactive examples

License

Notifications You must be signed in to change notification settings

merca/python_basics_pydantic

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

8 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

🐍 Python Basics with Pydantic - Learning Path

Python Pydantic Learning Path License

A comprehensive, progressive learning journey through Pydantic data validation and modern Python application development. Three versions designed to take you from beginner to enterprise-level expertise.

🎯 Choose Your Learning Path

πŸ”° BASIC πŸ“Š INTERMEDIATE πŸš€ ADVANCED
Pydantic Fundamentals Database Integration Enterprise Patterns
  • βœ… Data validation basics
  • βœ… Type hints & constraints
  • βœ… Custom validators
  • βœ… JSON serialization
  • βœ… Error handling
  • βœ… Simple Streamlit UI
  • βœ… SQLAlchemy integration
  • βœ… Model inheritance
  • βœ… Database sessions
  • βœ… CRUD operations
  • βœ… Data relationships
  • βœ… Basic analytics
  • βœ… Service layer architecture
  • βœ… Repository pattern
  • βœ… Dependency injection
  • βœ… Enterprise error handling
  • βœ… Production optimization
  • βœ… Testing strategies
πŸ“Š Stats:
πŸ“ ~620 lines
πŸ“¦ 3 dependencies
⏱️ 30-60 min
πŸŽ“ Beginner friendly
πŸ“Š Stats:
πŸ“ ~1,200 lines
πŸ“¦ 6 dependencies
⏱️ 1-2 hours
πŸŽ“ Intermediate level
πŸ“Š Stats:
πŸ“ ~3,000 lines
πŸ“¦ 20+ dependencies
⏱️ 4-8 hours
πŸŽ“ Advanced patterns
Start Basic Continue Intermediate Master Advanced

πŸš€ Quick Start Guide

πŸ› οΈ Fresh Installation? New to development or setting up from scratch? Check our Developer Setup Guide for complete installation instructions from a fresh OS.

Option 1: Using the Helper Script (Recommended)

# Clone the repository
git clone https://github.com/merca/python_basics_pydantic.git
cd python_basics_pydantic

# View all available learning versions
python run_version.py --info

# Check if dependencies are available for a version
python run_version.py --check basic

# Launch any version with automatic setup
python run_version.py basic        # Start with fundamentals
python run_version.py intermediate  # Database integration  
python run_version.py advanced     # Enterprise patterns

Helper Script Features:

  • πŸ” Dependency Checking: Validates required packages are available
  • πŸš€ Auto-Launch: Creates virtual environment and installs dependencies automatically
  • πŸ“Š Version Info: Shows detailed comparison of all learning versions
  • 🎯 Smart Navigation: Handles platform differences (Windows/macOS/Linux)

Option 2: Manual Setup

⚑ With uv (Recommended - Much Faster!)

New to Pydantic? πŸ”° Start Here!

# 1. Clone the repository
git clone https://github.com/merca/python_basics_pydantic.git
cd python_basics_pydantic

# 2. Start with BASIC version
cd basic
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
uv pip install -r requirements.txt
streamlit run app.py

Already know Pydantic basics? πŸ“Š Jump to Intermediate!

# Skip to database integration
cd intermediate
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
uv pip install -r requirements.txt  
streamlit run app.py

Ready for enterprise patterns? πŸš€ Go Advanced!

# Dive into production architecture
cd advanced
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
uv pip install -r requirements.txt
streamlit run main.py

🐌 Traditional Method (Slower)

Click to expand traditional pip/venv instructions

Basic Version:

cd basic
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt
streamlit run app.py

Intermediate Version:

cd intermediate
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt
streamlit run app.py

Advanced Version:

cd advanced
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt
streamlit run main.py

πŸ“š Learning Journey Overview

πŸŽ“ Progressive Skill Building

Our three-tier approach ensures you build a solid foundation before tackling complex concepts:

πŸ”° Basic                     πŸ“Š Intermediate                πŸš€ Advanced
Single File              β†’  Modular Structure          β†’  Layered Architecture
In-Memory Data           β†’  SQLite Database           β†’  Production Patterns
Direct Integration       β†’  Session Management        β†’  Dependency Injection

πŸ† What You'll Master

By completing all three versions, you'll have expertise in:

  1. πŸ”° Fundamentals - Core Pydantic validation, type safety, and error handling
  2. πŸ“Š Data Persistence - Database integration, ORM patterns, and session management
  3. πŸ—οΈ Architecture Patterns - Service layers, repositories, and dependency injection
  4. πŸ›‘οΈ Production Readiness - Error handling, testing, and performance optimization
  5. 🎯 Best Practices - Code organization, documentation, and maintainability

🌟 Why This Learning Path?

βœ… Progressive Complexity

Each version builds naturally on the previous one. No overwhelming jumps in complexity.

βœ… Real-World Focus

Not just toy examples - each version represents how you'd actually structure applications of that scale.

βœ… Complete Applications

Every version is a fully functional employee management system you can run and explore.

βœ… Production Insights

Learn not just how to use Pydantic, but when and why to apply different patterns.

βœ… Educational First

Extensive comments, learning notes, and architectural explanations built right into the code.

πŸ“– Detailed Version Comparison

πŸ”° BASIC Version - Pydantic Fundamentals

Perfect for:

  • Python developers new to Pydantic
  • Learning core validation concepts
  • Understanding type safety benefits

Architecture:

  • Single file (app.py) - 620 lines
  • In-memory data storage (session state)
  • Direct Streamlit integration

Key Learning Points:

# Field validation and constraints
class Employee(BaseModel):
    name: str = Field(min_length=2, max_length=50)
    email: EmailStr
    salary: Decimal = Field(ge=0, decimal_places=2)

# Custom validation logic
@field_validator('birth_date')
def validate_age(cls, v):
    if calculate_age(v) < 16:
        raise ValueError('Too young')
    return v

# JSON serialization
employee = Employee(**data)
json_data = employee.model_dump(mode='json')

What You'll Build:

  • Interactive employee management form
  • Real-time validation feedback
  • JSON import/export functionality
  • Basic analytics dashboard
πŸ“Š INTERMEDIATE Version - Database Integration

Perfect for:

  • Developers ready for persistence
  • Learning SQLAlchemy + Pydantic patterns
  • Understanding model inheritance

Architecture:

  • Modular structure (5 files) - 1,200 lines
  • SQLite database with SQLAlchemy ORM
  • Proper separation of concerns

Key Learning Points:

# Model inheritance hierarchy
class BaseEmployee(BaseModel):
    # Shared fields and validation

class EmployeeCreate(BaseEmployee):
    # Fields specific to creation

class Employee(BaseEmployee):  
    # Full model with database fields
    id: Optional[int] = None
    created_at: Optional[datetime] = None

# Database session management
@contextmanager
def session_scope():
    session = SessionLocal()
    try:
        yield session
        session.commit()
    except:
        session.rollback()
        raise
    finally:
        session.close()

What You'll Build:

  • Database-backed employee system
  • Model inheritance patterns
  • CRUD operations with validation
  • Basic analytics with charts
πŸš€ ADVANCED Version - Enterprise Patterns

Perfect for:

  • Senior developers planning production apps
  • Learning enterprise architecture patterns
  • Understanding complex system design

Architecture:

  • Layered architecture (15+ files) - 3,000+ lines
  • Service layer + Repository pattern
  • Dependency injection and error handling

Key Learning Points:

# Service layer for business logic
class EmployeeService:
    def __init__(self, repository: EmployeeRepository):
        self.repository = repository
    
    def create_employee(self, data: EmployeeCreate) -> Employee:
        # Business validation and coordination

# Repository for data access
class EmployeeRepository:
    def create(self, employee: EmployeeCreate) -> Employee:
        # Database operations abstraction

# Dependency injection
def get_employee_service() -> EmployeeService:
    repository = get_employee_repository()
    return EmployeeService(repository)

What You'll Build:

  • Production-ready architecture
  • Comprehensive error handling
  • Performance-optimized database operations
  • Advanced testing patterns

πŸ› οΈ Prerequisites & Setup

System Requirements

  • Python 3.8+ (Python 3.11+ recommended)
  • Git for cloning the repository
  • Virtual environment support (venv/conda)

Development Environment

# Check Python version
python --version  # Should be 3.8+

# Clone repository
git clone https://github.com/merca/python_basics_pydantic.git
cd python_basics_pydantic

# Each version has independent setup
cd basic    # or intermediate, or advanced
python -m venv venv

IDE Recommendations

  • VS Code with Python extension
  • PyCharm (Community/Professional)
  • Cursor for AI-assisted development

πŸ“ˆ Learning Progression

Recommended Timeline

Week 1: πŸ”° Basic Version
β”œβ”€β”€ Day 1-2: Setup and basic validation
β”œβ”€β”€ Day 3-4: Custom validators and JSON operations
└── Day 5: Exercises and experimentation

Week 2: πŸ“Š Intermediate Version  
β”œβ”€β”€ Day 1-2: Database integration
β”œβ”€β”€ Day 3-4: Model inheritance and CRUD
└── Day 5: Analytics and architecture study

Week 3-4: πŸš€ Advanced Version
β”œβ”€β”€ Week 3: Service layer and repository patterns
β”œβ”€β”€ Week 4: Testing, optimization, and production concepts
└── Final project: Extend with your own features

Assessment Checkpoints

After each version, you should be able to:

πŸ”° Basic Completion:

  • Explain Pydantic's value proposition
  • Create models with field validation
  • Handle validation errors gracefully
  • Serialize/deserialize JSON data

πŸ“Š Intermediate Completion:

  • Design model inheritance hierarchies
  • Integrate Pydantic with SQLAlchemy
  • Manage database sessions properly
  • Implement CRUD operations with validation

πŸš€ Advanced Completion:

  • Design service layer architectures
  • Implement repository patterns
  • Apply dependency injection principles
  • Plan production deployment strategies

πŸ“‹ Repository Structure

python_basics_pydantic/
β”œβ”€β”€ πŸ”° basic/                     # Single-file fundamentals (620 lines)
β”‚   β”œβ”€β”€ app.py                   # Main application
β”‚   β”œβ”€β”€ requirements.txt         # 3 core dependencies
β”‚   └── README.md               # Beginner guide
β”œβ”€β”€ πŸ“Š intermediate/              # Database integration (1,200 lines)  
β”‚   β”œβ”€β”€ models/                  # Pydantic models
β”‚   β”œβ”€β”€ database/               # SQLAlchemy integration
β”‚   β”œβ”€β”€ utils/                  # Helper functions
β”‚   β”œβ”€β”€ app.py                  # Multi-page application
β”‚   β”œβ”€β”€ requirements.txt        # 6 dependencies
β”‚   └── README.md              # Intermediate guide
β”œβ”€β”€ πŸš€ advanced/                  # Enterprise patterns (3,000+ lines)
β”‚   β”œβ”€β”€ app/                    # Application layer
β”‚   β”‚   β”œβ”€β”€ components/        # Reusable UI components
β”‚   β”‚   β”œβ”€β”€ pages/            # Page modules
β”‚   β”‚   └── services/         # Business logic layer
β”‚   β”œβ”€β”€ src/                   # Core business layer
β”‚   β”‚   β”œβ”€β”€ models/           # Advanced Pydantic models
β”‚   β”‚   └── database/         # Repository pattern
β”‚   β”œβ”€β”€ data/                 # Database files
β”‚   β”œβ”€β”€ main.py              # Application entry point
β”‚   β”œβ”€β”€ requirements.txt     # 20+ production dependencies
β”‚   └── README.md           # Enterprise guide
β”œβ”€β”€ πŸƒ run_version.py           # Helper script for easy switching
β”œβ”€β”€ πŸ“š README.md               # Main learning path documentation
β”œβ”€β”€ πŸ› οΈ DEVELOPER_SETUP.md      # Complete setup guide (fresh OS)
β”œβ”€β”€ πŸ“„ LICENSE                 # MIT license
β”œβ”€β”€ βš™οΈ pyproject.toml         # Project configuration
└── 🚷 .gitignore             # Git exclusions

πŸƒ Helper Script Usage

The run_version.py script provides a convenient way to manage all learning versions:

Basic Commands

# Show version information and features
python run_version.py --info

# Check if system has required dependencies  
python run_version.py --check [basic|intermediate|advanced]

# Launch a version (creates venv, installs deps, starts app)
python run_version.py [basic|intermediate|advanced]

# Get help and see all options
python run_version.py --help

Example Workflow

# Start learning journey
python run_version.py --info          # See what's available
python run_version.py --check basic   # Verify system readiness
python run_version.py basic           # Launch basic version

# Progress to intermediate 
python run_version.py --check intermediate
python run_version.py intermediate

# Master advanced patterns
python run_version.py --check advanced  
python run_version.py advanced

πŸ“Š Repository Statistics

πŸ”° Basic Version:      πŸ“ ~620 lines,  πŸ“¦ 3 packages,  ⏱️ 30-60min
πŸ“Š Intermediate:       πŸ“ ~1,200 lines, πŸ“¦ 6 packages,  ⏱️ 1-2hrs  
πŸš€ Advanced:          πŸ“ ~3,000 lines, πŸ“¦ 20+ packages, ⏱️ 4-8hrs
─────────────────────────────────────────────────────────────────
πŸ“š Total Learning:    πŸ“ ~5,000 lines, πŸŽ“ Beginner β†’ Expert

🀝 Community & Support

Getting Help

  • πŸ“š Documentation: Each version has comprehensive README.md files
  • πŸ› οΈ Setup Issues: Check DEVELOPER_SETUP.md for installation help
  • πŸ” Helper Script: Use python run_version.py --help for quick assistance
  • πŸ“ Code Comments: Extensive educational comments explain every concept
  • πŸ§ͺ Experimentation: Break things to understand how they work
  • πŸš€ GitHub Issues: Report bugs, ask questions, or request features

Contributing

We welcome contributions! Ways to help:

  • Improve Documentation: Fix typos, add examples
  • Enhance Examples: Add new features or use cases
  • Share Feedback: Tell us about your learning experience
  • Create Extensions: Build on top of the advanced version

🌟 Success Stories

"This progressive approach helped me understand not just HOW to use Pydantic, but WHEN and WHY to apply different patterns. The advanced version gave me confidence to architect production applications."
β€” Senior Python Developer

"Starting with the basic version made Pydantic approachable. By the time I reached the advanced patterns, I understood the reasoning behind each architectural decision."
β€” Full Stack Engineer

πŸš€ Next Steps After Completion

Apply Your Knowledge

  • Personal Projects: Use these patterns in your own applications
  • Work Projects: Introduce Pydantic to your team's codebase
  • Open Source: Contribute to Pydantic or related projects

Continue Learning

  • FastAPI: Web APIs with automatic validation
  • SQLModel: Type-safe database models
  • Pydantic v3: Stay current with latest features

Share Your Success

  • Blog Posts: Write about your learning journey
  • Conference Talks: Share architectural insights
  • Mentoring: Help others learn Pydantic

πŸ“„ License

This project is licensed under the MIT License - see the LICENSE file for details.

πŸ™ Acknowledgments

  • Pydantic Team for creating an amazing validation library
  • Streamlit Team for the excellent web app framework
  • SQLAlchemy Team for powerful ORM capabilities
  • Python Community for continuous innovation

🎯 Ready to start your Pydantic journey?

πŸ”° Begin with Basic Version β†’

Transform from Pydantic newcomer to enterprise architect

Happy Learning! πŸš€

About

Python fundamentals with Pydantic for Databricks and Streamlit development - comprehensive tutorial with notebooks and interactive examples

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages