feat: refactor codebase to microservices monorepo#56
Merged
Conversation
Contributor
|
Caution Review failedFailed to post review comments 📝 WalkthroughWalkthroughThis pull request restructures the project from a monolithic application into a distributed monorepo architecture with three packages: a shared utilities package (fastfetchbot-shared) containing models and common functions, an API server (apps/api) for web-based content scraping, and a Telegram bot service (apps/telegram-bot) that communicates with the API. The CI/CD workflow is updated to build and publish container images for both services to GitHub Container Registry. Changes
Sequence Diagram(s)sequenceDiagram
actor User as User/<br/>Telegram
participant TBot as Telegram Bot<br/>Service
participant API as API Server<br/>Service
participant Scraper as Scraper<br/>Module
participant Ext as External<br/>Service
participant DB as MongoDB
User->>TBot: /start or message<br/>with URL
activate TBot
TBot->>TBot: Parse message,<br/>extract URLs
alt URL Detected
TBot->>API: POST /scraper/getItem<br/>(url, api_key)
activate API
API->>Scraper: Dispatch to<br/>appropriate scraper
activate Scraper
Scraper->>Ext: Fetch page data<br/>(HTTP, API calls)
activate Ext
Ext-->>Scraper: Raw data
deactivate Ext
Scraper->>Scraper: Parse & transform<br/>into MetadataItem
Scraper-->>API: Structured item
deactivate Scraper
API->>DB: Save item<br/>(if enabled)
activate DB
DB-->>API: Confirmation
deactivate DB
API-->>TBot: MetadataItem JSON
deactivate API
TBot->>TBot: Format message<br/>via template
TBot->>TBot: Package media<br/>(download, resize)
TBot->>User: Send message<br/>+ media
else Error/Unknown
TBot->>User: Error or<br/>help message
end
deactivate TBot
sequenceDiagram
participant Client as External Client
participant API as API Server
participant InfoSvc as InfoExtract<br/>Service
participant MGR as ScraperManager
participant Scraper as Specific<br/>Scraper
participant Cache as Cache/DB
Client->>API: POST /scraper/getItem<br/>(url, params)
activate API
API->>InfoSvc: new InfoExtractService<br/>(UrlMetadata)
activate InfoSvc
InfoSvc->>InfoSvc: Detect source<br/>category
alt Known Source
InfoSvc->>Scraper: Dispatch via<br/>service_classes registry
activate Scraper
else Legacy Source
InfoSvc->>MGR: init_scraper(category)
activate MGR
MGR->>Scraper: Create processor
MGR-->>InfoSvc: Processor
deactivate MGR
activate Scraper
end
Scraper->>Scraper: Fetch & parse<br/>content
Scraper-->>InfoSvc: Item dict
deactivate Scraper
InfoSvc->>InfoSvc: Post-process item<br/>(telegraph, PDF, DB)
InfoSvc->>Cache: Optionally persist
activate Cache
Cache-->>InfoSvc: OK
deactivate Cache
InfoSvc-->>API: Final item
deactivate InfoSvc
API-->>Client: JSON response
deactivate API
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120+ minutes This is a substantial monorepo restructuring involving 150+ new files, significant module consolidation (extracting shared utilities into a new package), two new microservices (API and Telegram bot), comprehensive scraper implementations for 10+ social platforms, database model definitions, template systems, and CI/CD pipeline updates. The heterogeneity of changes (infrastructure, services, scrapers, templates, tests) combined with complex multi-component interactions and high logic density across modules demands careful verification of API contracts, service communication flows, database persistence logic, and error handling pathways. Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Feb 20, 2026
This was referenced Mar 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
New Features
Infrastructure
Documentation