Skip to content

Comments

feat(wiki-gen): implement P0 storage connectors (Local + Git)#28

Merged
TelivANT merged 4 commits intomainfrom
feat/storage-connectors-p0
Feb 21, 2026
Merged

feat(wiki-gen): implement P0 storage connectors (Local + Git)#28
TelivANT merged 4 commits intomainfrom
feat/storage-connectors-p0

Conversation

@TelivANT
Copy link
Owner

📦 Storage Connectors - P0 Phase 1

✅ Implemented (2/4)

1. StorageConnector Trait

  • 7 core methods: connect, list_files, read_file, exists, metadata, clone_to_temp, name
  • FileEntry and FileMetadata structs
  • Full async/await support
  • Send + Sync bounds

2. LocalConnector

  • Tokio async file operations
  • Path resolution
  • Metadata support
  • ✅ Tests passing

3. GitConnector

  • git2 integration
  • Token authentication
  • SSH key authentication
  • Clone to temp directory
  • Support for GitHub/GitLab/Gitee/Bitbucket
  • ✅ Tests written (requires network)

📝 Code Stats

  • storage/mod.rs: 52 lines (trait definition)
  • storage/local.rs: 72 lines (local filesystem)
  • storage/git.rs: 175 lines (git clone)
  • storage/tests.rs: 48 lines (unit tests)
  • Total: 347 lines

🔧 Technical Highlights

  1. Unified Abstraction - StorageConnector trait provides consistent interface
  2. Async Design - Full async/await with Tokio runtime
  3. Authentication - Git Token + SSH Key support
  4. Error Handling - Unified WikiGenError::Storage variant

📋 Remaining (P0)

  • S3Connector (AWS S3 + compatible storage)
  • WebDavConnector (WebDAV protocol)

🎯 Progress

  • P0: 50% (2/4 complete)
  • Overall: 12% (2/17 complete)

🧪 Testing

cargo test -p memoryos-wiki-gen --lib storage

📚 Related

  • Design doc: docs/STORAGE_CONNECTORS.md
  • Issue: Storage connector system for wiki-gen

Storage Connectors Design:
- 17 connectors planned (P0: 4, P1: 6, P2: 7)
- Excluded: Container storage, special protocols
- Focus: Enterprise practical scenarios
- Timeline: 6 weeks for full implementation

P0 (Week 1):
- LocalConnector (existing)
- GitConnector (GitHub/GitLab/Gitee)
- S3Connector (AWS/MinIO/Wasabi)
- WebDavConnector (坚果云/Nextcloud)

P1 (Week 2-3):
- OSS/COS/OBS (阿里云/腾讯云/华为云)
- SMB/NFS/SFTP (企业文件共享)

P2 (Week 4-6):
- Cloud drives (OneDrive/Google Drive/Dropbox)
- Cloud storage (GCS/Azure Blob)
- Chinese drives (百度网盘/阿里云盘)

Updated:
- docs/state.json: phase = Storage Connectors
- WORK_LOG.md: added current task
Storage Connector System:
- Define StorageConnector trait with 7 methods
- FileEntry and FileMetadata structs
- Async/await support

Implemented Connectors:
1. LocalConnector - Local filesystem access
   - Tokio async file operations
   - Path resolution
   - Metadata support

2. GitConnector - Git repository cloning
   - git2 integration
   - Token authentication
   - SSH key authentication
   - Clone to temp directory
   - Support for GitHub/GitLab/Gitee/etc

Error Handling:
- Added Storage error variant
- Proper error propagation
- JoinError handling for async tasks

Tests:
- LocalConnector unit test (passing)
- GitConnector integration test (ignored, requires network)

Dependencies:
- git2 = "0.19"
- uuid = { version = "1", features = ["v4"] }

Next: S3Connector and WebDavConnector (P0 remaining)
Progress Update:
- P0 Phase 1 complete: LocalConnector + GitConnector
- 347 lines of code implemented
- Tests passing
- PR #28 created

Updated:
- docs/state.json: progress 12%
- WORK_LOG.md: added completed items
- docs/STORAGE_CONNECTORS.md: marked Local + Git as implemented

Next: S3Connector + WebDavConnector
@TelivANT TelivANT merged commit 31b82a5 into main Feb 21, 2026
3 checks passed
@TelivANT TelivANT deleted the feat/storage-connectors-p0 branch February 21, 2026 02:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant