Skip to content

Dev#4

Merged
Jenish-1235 merged 48 commits intomainfrom
dev
Dec 19, 2025
Merged

Dev#4
Jenish-1235 merged 48 commits intomainfrom
dev

Conversation

@Jenish-1235
Copy link
Copy Markdown
Contributor

This pull request introduces major improvements to configuration management, application health monitoring, and infrastructure setup for the NeroSpatial Backend. The changes standardize environment variable handling, provide robust health endpoints for production readiness, and enhance developer experience with clearer documentation and workflow updates.

Configuration and Environment Management

  • Added a comprehensive .env.example template with detailed documentation for all required environment variables, including Azure, PostgreSQL, Redis, JWT, and OpenTelemetry settings.
  • Refactored config.py to support hierarchical configuration loading (Azure App Configuration, Key Vault, .env), added validation, and new computed properties for connection URLs.

Health Monitoring and Application State

  • Implemented new health endpoints /health, /ready, and /live in api/health.py for Kubernetes and production monitoring, with detailed dependency checks and metadata reporting.
  • Introduced AppState dataclass (core/app_state.py) to centralize all application resources, startup metadata, and cleanup logic.
  • Updated core/__init__.py to expose new shared utilities, exceptions, and models for easier imports and code organization. [1] [2] [3]

Infrastructure and Developer Experience

  • Updated CI workflow (.github/workflows/ci.yml) to use Python 3.13.7 and added Redis and PostgreSQL as services for integration testing. [1] [2] [3]
  • Updated Dockerfile to use Python 3.13.7 for both build and runtime stages. [1] [2]
  • Improved README.md with detailed infrastructure setup instructions, health endpoint documentation, and configuration hierarchy explanation.

Other Minor Changes

  • Added package docstring to api/__init__.py for clarity.

harii55 and others added 30 commits December 13, 2025 03:17
…g for audio/video streams, include comprehensive tests for gateway components.
…or Redis settings, and add Redis service to Docker Compose. Include tests for Redis client functionality.
- Split core/models.py into domain modules
- Enhance user/auth models with production-grade features
- Update dependencies: pydantic[email] for EmailStr validation
- Update tests to match new model structure
- All imports remain backward compatible
- All 48 tests passing
- Add secondary index (user_sessions:{user_id} SET) for O(1) lookups
- Implement grace period (10-min TTL) on disconnect for session reuse
- Throttle activity updates to 5 minutes (16,500x reduction)
- Add comprehensive integration tests for end-to-end verification
- Ensure no ghost sessions (both keys cleaned up properly)

Performance:
- Activity updates: 100 ops/sec → 0.006 ops/sec
- User sessions lookup: O(n) SCAN → O(k) SET lookup
…ling and OAuth provider requirements

- Changed IP address fields in models to use IPvAnyAddress for better validation.
- Made oauth_provider a required field in User and UserContext models.
- Removed deprecated OAuthTokens model and updated related tests.
- Enhanced tests to validate new model behaviors and requirements.
- Add docker-compose.infra.yml with PostgreSQL, Redis, Jaeger
- Add scripts/init-db.sql with complete database schema
- Add scripts/generate-keys.sh for JWT key generation
- Add scripts/setup-keyvault.sh for Azure Key Vault setup
- Add PostgreSQL, Redis, JWT, OpenTelemetry settings
- Add startup timeout and retry configuration
- Add pool size configuration for future use
- Create .env.example template
- Update .gitignore for secrets
- Create core/config_loader.py with retry logic
- Add environment validation (strict for prod/staging)
- Add exponential backoff retry for Azure connections
- Add azure-appconfiguration dependency
- Create core/app_state.py with AppState container
- Define DatabasePool and RedisClient protocols
- Create core/database.py with pool factory stub
- Create core/redis.py with client factory stub
- Add startup metadata tracking
- Add 7-phase startup sequence with logging
- Add configuration loading from Azure
- Add connection initialization and verification
- Add graceful shutdown with cleanup
- Add AppState dependency injection
- Add /health with detailed dependency checks
- Add /ready for Kubernetes readiness probe
- Add /live for Kubernetes liveness probe
- Update Docker health check to use /ready
- Add uptime and metadata to health response
- Add tests for environment validation
- Add tests for AppState lifecycle
- Add tests for health endpoints
- Update README with infrastructure setup
- Move all core module tests to tests/core/ directory
- Create tests/core/__init__.py
- Keep API tests (test_health_endpoints.py) in tests/ root
- Maintains test organization by module structure
…uirements

- Introduce azure-appconfiguration version 1.7.2 with its dependencies
- Update project dependencies to include azure-appconfiguration
- Ensure compatibility with existing azure-core and other related packages
- Implement SessionCleanupService to periodically clean up stale session IDs from Redis.
- Integrate session cleanup into the application lifespan management.
- Add distributed locking mechanism to prevent concurrent cleanup operations.
- Enhance Redis client with batch operations for efficient key existence checks.
- Introduce comprehensive unit and integration tests for the cleanup service and its interactions with Redis.

This update improves session management reliability and ensures stale sessions are efficiently removed, enhancing overall application performance.
Jenish-1235 and others added 18 commits December 19, 2025 12:45
- Add redis_max_connections setting to configure maximum Redis connections.
- Refactor main.py to initialize Redis client with new connection settings.
- Update health check endpoints to verify Redis connection status.
- Remove unused database and Redis client creation functions.
- Introduce idempotent session keys for WebSocket connections to improve session management.
- Implement batch operations in Redis client for efficient session handling.
- Update tests to reflect changes in session management and Redis interactions.

These enhancements improve the application's performance and reliability in managing Redis connections and user sessions.
…ultiple files

- Simplified string formatting in config.py, main.py, and various model files for improved readability.
- Refactored method signatures in auth.py, keyvault.py, and test files to enhance consistency and clarity.
- Cleaned up exception representation in exceptions.py for better debugging output.
…, atomic pipelines, and improved connection tracking; fix test mocks
@Jenish-1235 Jenish-1235 merged commit 5f42f62 into main Dec 19, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants