Add Structured Logging #22
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add structured logging infrastructure with JSON and console formatters. Files Created: - src/eventkit/logging/__init__.py - src/eventkit/logging/config.py Configuration: - Dual formatters: JSON for prod, colored console for dev - Context propagation via contextvars - ISO timestamps, log level, logger name in all logs Settings Added: - EVENTKIT_LOG_LEVEL (default: INFO) - EVENTKIT_JSON_LOGS (default: False) Call configure_logging() in FastAPI app creation.
Add logging to API router with context propagation and timing: Changes: - Use structlog.get_logger() instead of stdlib logging - Bind request context (request_id, stream) for all logs - Log request_received with method, path, event_type - Log request_completed with status_code, duration_ms - Log request_failed with error, error_type, status_code - Clear context after request (in finally block) - Move logging configuration to lifespan (startup) Metrics Logged: - duration_ms: Request processing time - event_type: Type of event received - status_code: HTTP response status Context Variables: - request_id: UUID for tracing - stream: Event stream name All tests passing.
Add logging to event processing pipeline: Changes: - Log event_received (DEBUG): event_type, stream - Log adaptation_failed (WARN): error, event_type, stream - Log event_adapted (DEBUG): event_type - Log event_sequenced (DEBUG): partition Never logs full event payloads (PII safety). All tests passing.
Add logging to buffer operations with performance metrics: Changes: - Log buffer_add (DEBUG): partition, buffer_size - Log buffer_flush_start (INFO): partition, event_count - Log buffer_flush_complete (INFO): partition, event_count, duration_ms Performance tracking with timing measurements for flush operations. All tests passing.
Add logging to storage operations: EventStore (FirestoreEventStore): - Log store_write_start (INFO): event_count - Log store_write_complete (INFO): event_count, duration_ms, collection - Log store_write_failed (ERROR): event_count, error, error_type, duration_ms ErrorStore (FirestoreErrorStore): - Log error_stored (WARN): stream, error Never logs full event payloads (PII safety). All tests passing.
Add comprehensive tests for structlog configuration: Configuration Tests: - Log level setting (DEBUG, INFO) - JSON mode configuration - Console mode configuration - Logger functionality after config Context Propagation Tests: - Context variable binding - Context variable clearing All tests passing.
Update LOCAL_DEV.md and CLAUDE.md with structured logging guides: LOCAL_DEV.md: - Development vs production logging - Log level configuration - JSON logs for production - Example output for both modes - Log levels and what they mean - What is/is not logged (PII safety) - Context propagation for tracing CLAUDE.md: - How to use structlog correctly - What to log at each level - What NEVER to log (PII, tokens, etc) - Context propagation patterns - Performance timing examples Complete structured logging implementation.
Rename test file and update assertions: - tests/unit/logging/test_config.py → test_logging_config.py (avoid conflict) - LOG_LEVEL → EVENTKIT_LOG_LEVEL - Add EVENTKIT_JSON_LOGS assertions All tests passing.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #8
Implement structured logging with dual formatters (JSON/console) for production observability.
Changes
Infrastructure (8 commits):
What's Logged:
Context Propagation:
Modes:
Tests:
Configuration