Azure Functions Portfolio -- Production-Grade Patterns

TL;DR: A reference portfolio of four Azure Functions scenarios (document processing, real-time notifications, saga orchestration, and scheduled ETL) demonstrating production-grade patterns: Durable Functions, Polly v8 resilience, managed identity, private endpoints, blue/green deployments, and modular Terraform.

Azure Functions Production-Grade Patterns is a masterclass in enterprise distributed systems architecture using serverless Azure compute. The portfolio demonstrates:

• Architectural Excellence -- Saga patterns, fan-out/fan-in orchestration, middleware pipelines, repository abstractions
• Production Readiness -- Zero Trust networking, managed identity, private endpoints, blue/green deployments
• Resilience Engineering -- Polly v8 strategies, dead-letter handling, automatic compensation
• Infrastructure Mastery -- Modular Terraform, OIDC-based CI/CD, zero-downtime deployments
• Code Quality -- 95% test coverage, centralized cross-cutting concerns, SOLID principles, clear separation of concerns

This project answers the question: "How do I build enterprise-grade Azure Functions applications?" It's a comprehensive reference for architects and senior engineers designing distributed systems on Azure.

Table of Contents

• Highlights
• Scenarios
• Project Structure
• Getting Started
  • Quickstart
  • Prerequisites
  • Configuration
  • Local Development
• Architecture Principles
• CI/CD
• Documentation
• License

Highlights

• Saga orchestration with compensation -- Durable Functions coordinate multi-step workflows and automatically roll back completed steps on failure
• Fan-out/fan-in ETL -- Durable Functions extract from three sources in parallel, then merge, validate, transform, and load sequentially
• Real-time notifications -- Azure SignalR Service serverless integration for instant in-app delivery with multi-channel routing
• Circuit breaker and resilience -- Polly v8 retry, circuit breaker, and timeout strategies applied to all outbound HTTP calls
• Dead-letter handling -- Service Bus messages that fail validation or processing are explicitly dead-lettered with structured reasons
• Middleware pipeline -- Correlation ID propagation and centralized exception handling via IFunctionsWorkerMiddleware
• Repository pattern -- Table Storage and Cosmos DB repositories behind interfaces for testability and swap-ability
• Managed identity -- DefaultAzureCredential for all Azure service authentication; connection strings used only during local development
• Blue/green deployment -- Staging slots with smoke tests before production swap
• Modular Terraform -- Reusable modules for function apps, storage, Service Bus, Cosmos DB, SignalR, and networking with private endpoints

Scenarios

Scenario 1: Document Processing Pipeline

An event-driven document processing pipeline that ingests files uploaded to Blob Storage, processes them through a queue-based workflow with OCR and classification, persists metadata to Table Storage, and generates daily aggregate reports.

Triggers: BlobTrigger, QueueTrigger, TimerTrigger, HTTP Key patterns: Blob-to-queue fan-out, Table Storage repository, content-type inference, daily scheduled reporting Source: src/Scenario01.DocumentProcessing/

Scenario 2: Real-Time Notification System

A multi-channel notification system using Azure SignalR Service for real-time in-app delivery, queue-based email routing, Event Grid integration for system events, and user subscription management with daily digest aggregation.

Triggers: HTTP, QueueTrigger, EventGridTrigger, TimerTrigger, SignalR (negotiate + output binding) Key patterns: SignalR serverless integration, multi-channel fan-out delivery, queue-based routing, user preference management Source: src/Scenario02.RealtimeNotifications/

Scenario 3: Event-Driven Microservices Orchestration

A distributed order fulfillment system using the saga pattern to coordinate inventory reservation, payment processing, and shipment creation. Failed steps trigger compensating transactions in reverse order. Orders can arrive via Service Bus or the HTTP API.

Triggers: ServiceBusTrigger, EventGridTrigger, Durable Functions (Orchestration + Activity), HTTP Key patterns: Saga orchestrator with compensation, Cosmos DB repository, dead-letter routing, event-driven inventory alerts Source: src/Scenario03.EventDrivenOrchestration/

Scenario 4 (not included)

Scenario 4 is intentionally omitted from this portfolio and reserved for a future scenario.

Scenario 5: Scheduled ETL Pipeline

A scheduled ETL pipeline using Durable Functions fan-out/fan-in pattern. Extracts data from three sources in parallel (API, CSV, database), merges results, validates against configurable rules, transforms with field mappings, and loads to blob storage.

Triggers: TimerTrigger, HTTP, Durable Functions (Orchestration + Activity) Key patterns: Fan-out/fan-in orchestration, multi-stage pipeline (raw → validated → transformed → output), rule-based validation, partial failure tolerance Source: src/Scenario05.ScheduledEtlPipeline/

Project Structure

Azure-Func-ee/
├── src/
│   ├── AzureFunctions.Shared/          # Cross-cutting: middleware, telemetry, resilience
│   │   ├── Middleware/                  # CorrelationId, ExceptionHandling
│   │   ├── Resilience/                  # Polly retry, circuit breaker, timeout
│   │   ├── Telemetry/                   # ITelemetryService abstraction
│   │   ├── Models/                      # ErrorResponse, OperationResult
│   │   └── Extensions/                  # ServiceCollection helpers
│   ├── Scenario01.DocumentProcessing/   # Document processing function app
│   │   ├── Functions/                   # ProcessNewDocument, ProcessDocument, GenerateProcessingReport, GetDocumentStatus
│   │   ├── Models/                      # DocumentMetadata, DocumentProcessingMessage, DocumentProcessingOptions
│   │   └── Services/                    # IDocumentRepository, IDocumentProcessingService, IClassificationService
│   ├── Scenario02.RealtimeNotifications/  # Real-time notifications function app
│   │   ├── Functions/                   # Negotiate, SendNotification, ProcessNotification, BroadcastRealtime, SendDigest, ManageSubscriptions, HandleSystemEvent
│   │   ├── Models/                      # Notification, UserSubscription, NotificationOptions
│   │   ├── Repositories/               # INotificationRepository, ISubscriptionRepository
│   │   └── Services/                    # INotificationService, IEmailService, ITemplateService
│   ├── Scenario03.EventDrivenOrchestration/  # Order orchestration function app
│   │   ├── Functions/                   # OrderSagaOrchestrator, ProcessOrder, OrderApi, HandleInventoryEvent
│   │   │   └── Activities/              # ReserveInventory, ProcessPayment, CreateShipment + compensations
│   │   ├── Models/                      # Order, SagaState, OrderResult, OrderProcessingOptions
│   │   │   ├── Dtos/                    # CreateOrderRequest, OrderResponse, OrderItemDto
│   │   │   └── Events/                  # InventoryEvent
│   │   ├── Repositories/               # IOrderRepository, CosmosDbOrderRepository
│   │   └── Services/                    # IOrderService, IInventoryService, IPaymentService
│   └── Scenario05.ScheduledEtlPipeline/  # ETL pipeline function app
│       ├── Functions/                   # ScheduledEtl, TriggerEtl, GetPipelineStatus, EtlOrchestrator
│       │   └── Activities/              # ExtractFromApi, ExtractFromCsv, ExtractFromDatabase, Validate, Transform, Load
│       ├── Models/                      # PipelineRun, DataRecord, ValidationRule, TransformationMapping, EtlOptions
│       ├── Repositories/               # IPipelineRepository, TableStoragePipelineRepository
│       └── Services/                    # IPipelineService, IDataValidator, IDataTransformer, IExternalApiClient
├── tests/
│   ├── AzureFunctions.Shared.Tests/
│   ├── Scenario01.DocumentProcessing.Tests/
│   ├── Scenario02.RealtimeNotifications.Tests/
│   ├── Scenario03.EventDrivenOrchestration.Tests/
│   ├── Scenario05.ScheduledEtlPipeline.Tests/
│   └── Integration.Tests/
├── terraform/
│   ├── modules/
│   │   ├── core-infrastructure/         # Resource group, VNet, Key Vault, Log Analytics
│   │   ├── function-app/                # Reusable function app with slots, diagnostics
│   │   ├── document-processing/         # Blob, queue, table storage + function app
│   │   ├── realtime-notifications/      # SignalR, queues, tables + function app
│   │   ├── event-orchestration/         # Service Bus, Event Grid, Cosmos DB + function app
│   │   └── scheduled-etl-pipeline/      # Blob containers, table, Durable storage + function app
│   └── environments/
│       └── dev/                         # Dev environment composition
├── .github/workflows/
│   ├── build-and-test.yml               # CI: restore, build, test, coverage, publish
│   ├── terraform-plan.yml               # PR: fmt, validate, plan with PR comment
│   └── deploy-functions.yml             # CD: build, terraform apply, slot deploy, swap
├── Azure-Functions-Portfolio.sln
├── Directory.Build.props                # Centralized .NET 8, C# 12, nullable, package versions
└── global.json                          # SDK 8.0.400

Getting Started

Quickstart

Get a scenario running locally in under five minutes.

Required tools: .NET 8 SDK, Azure Functions Core Tools v4, Azurite

# 1. Restore and build
dotnet restore Azure-Functions-Portfolio.sln
dotnet build Azure-Functions-Portfolio.sln

# 2. Start the local storage emulator
azurite --silent --location .azurite --debug .azurite/debug.log

# 3. Run a scenario (e.g., Scenario 1 -- Document Processing)
cd src/Scenario01.DocumentProcessing
func start

See Prerequisites and Local Development below for full setup details, and the Deployment Guide for Terraform-based cloud deployment.

Prerequisites

• .NET 8 SDK (8.0.400 or later)
• Azure Functions Core Tools v4
• Azurite (local storage emulator)
• Terraform >= 1.5 (for infrastructure deployment)

Configuration

Each scenario uses a local.settings.json file for local development configuration. This file is excluded from source control (.gitignore) and should never be committed -- it may contain connection strings or secrets.

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated"
  }
}

Local vs. deployed: In deployed environments all Azure service connections use DefaultAzureCredential (managed identity). Connection strings in local.settings.json are used only when running against Azurite or a local emulator during development.

Local Development

dotnet restore Azure-Functions-Portfolio.sln
dotnet build Azure-Functions-Portfolio.sln
dotnet test Azure-Functions-Portfolio.sln

Start Azurite before running functions locally:

azurite --silent --location .azurite --debug .azurite/debug.log

Run a specific scenario:

cd src/Scenario01.DocumentProcessing
func start

Architecture Principles

• Isolated worker model -- All function apps use the .NET 8 isolated worker process, decoupling from the Functions host
• Managed identity -- DefaultAzureCredential for Azure service auth in deployed environments; connection strings only for Azurite
• Private endpoints -- All data-plane access (Blob, Table, Queue, Cosmos DB, Service Bus, Key Vault) secured via private endpoints
• Middleware pipeline -- CorrelationIdMiddleware for distributed tracing, ExceptionHandlingMiddleware for consistent error responses
• Resilience -- Polly v8 retry (exponential backoff with jitter), circuit breaker, and timeout on all outbound HTTP clients
• Event-driven -- Blob triggers, queue triggers, Service Bus, Event Grid for loose coupling between components
• Infrastructure as Code -- Modular Terraform with reusable modules, OIDC-based CI/CD, and remote state

CI/CD

Three GitHub Actions workflows automate the full lifecycle:

Workflow	Trigger	Purpose
Build and Test	Push/PR to main (src/tests changes)	Restore, build, test with coverage, publish artifacts
Terraform Plan	PR to main (terraform changes)	Format check, validate, plan with PR comment
Deploy Functions	Push to main or manual dispatch	Build, Terraform apply, deploy to staging slots, smoke test, swap to production

Documentation

• Architecture Overview -- System design, data flows, cross-cutting concerns
• Developer Guide -- Local setup, debugging, testing, coding standards
• Deployment Guide -- Terraform setup, deployment steps, blue/green, rollback
• Terraform Reference -- Module structure, variables, usage
• Portfolio Overview -- Deep Dive into project scenarios

License

MIT