Create trait-based storage backend abstraction with WASM compatibility

[avrabe]

Summary

Create a flexible, trait-based storage backend abstraction that supports multiple storage implementations and is fully compatible with WASM32-WASIP2 targets, incorporating patterns from loriot-websocket-mcp.

Background

Current storage in mcp-auth is tightly coupled to file system operations and platform-specific code. We need:

• Trait-based abstraction for different storage backends
• WASM-compatible implementations using WASI interfaces
• Integration of persistence patterns from loriot-websocket-mcp
• Support for both synchronous and asynchronous operations

Implementation Tasks

Core Storage Traits

• Define StorageBackend trait with async operations
• Create PersistenceStrategy trait for different storage patterns
• Implement DataFormat trait for serialization (JSON, JSONL, binary)
• Add StorageMetrics trait for monitoring and health checks

Storage Backend Implementations

• FileStorage - Current file-based implementation (native only)
• WasiStorage - WASM-compatible file operations using WASI
• MemoryStorage - In-memory storage for testing and WASM
• JsonLinesStorage - Based on loriot-websocket-mcp patterns
• CompositeStorage - Multi-tier memory + persistence

Persistence Patterns (from loriot-websocket-mcp)

• JSON Lines format support with configurable storage
• Two-tier memory management (in-memory + file backup)
• Configurable memory limits and eviction policies
• Automatic deduplication strategies
• Backup rotation and versioning

WASM Compatibility Layer

• WASI filesystem interface integration
• Feature-flagged compilation for different targets
• Component Model storage interfaces
• Browser-compatible storage using IndexedDB/localStorage

Configuration System

• Storage configuration traits and implementations
• Environment-specific storage selection
• Runtime storage backend switching
• Migration support between storage backends

Security & Encryption

• Refactor encryption to work with trait-based storage
• WASM-compatible crypto implementations
• Secure key management for different backends
• Data integrity verification

Integration Points

MCP Auth Integration

• Migrate existing mcp-auth storage to use new traits
• Maintain backward compatibility with existing storage
• Add migration tools for existing installations

Caching Integration

• Interface with generalized caching framework (Extract and generalize caching framework from studio-mcp #28)
• Cache-aware storage operations
• Storage-level cache invalidation hooks

Configuration Examples

// Native file storage
StorageConfig::File {
    path: "auth.db".into(),
    encryption: EncryptionConfig::AES256,
    backup_count: 3,
}

// WASM memory storage  
StorageConfig::Memory {
    max_entries: 1000,
    persistence: None,
}

// JSON Lines with two-tier management
StorageConfig::JsonLines {
    file_path: "data.jsonl".into(),
    memory_limit_entries: 2000,
    sync_interval: Duration::from_secs(60),
    deduplication: DeduplicationStrategy::TimestampBased,
}

Acceptance Criteria

• All storage backends compile for both native and WASM targets
• Performance comparable to current file-based implementation
• Full test coverage including WASM integration tests
• Migration path from existing storage implementations
• Comprehensive documentation and examples
• JSON Lines patterns from loriot-websocket-mcp integrated

Related Issues

• WASM32-WASIP2 Target Support (Add WASM32-WASIP2 target support for WebAssembly deployment #26)
• Generalized Caching Framework (Extract and generalize caching framework from studio-mcp #28)
• Data Persistence Patterns (#TBD)

References

• loriot-websocket-mcp storage patterns
• WASI filesystem specification