diff --git a/README.md b/README.md index f987de4..246660e 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,514 @@ -# DevPulse Phase 2/7 Completed ✅ +
+# 🔮 DevPulse + +### The AI-powered developer intelligence platform + +**Track your GitHub activity, understand your coding patterns, and discover the perfect open-source project to contribute to — all in one place.** + +[![TypeScript](https://img.shields.io/badge/TypeScript-3178C6?style=flat-square&logo=typescript&logoColor=white)](https://www.typescriptlang.org/) +[![NestJS](https://img.shields.io/badge/NestJS-E0234E?style=flat-square&logo=nestjs&logoColor=white)](https://nestjs.com/) +[![Next.js](https://img.shields.io/badge/Next.js-000000?style=flat-square&logo=nextdotjs&logoColor=white)](https://nextjs.org/) +[![Python](https://img.shields.io/badge/Python-3776AB?style=flat-square&logo=python&logoColor=white)](https://python.org/) +[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-4169E1?style=flat-square&logo=postgresql&logoColor=white)](https://postgresql.org/) +[![Redis](https://img.shields.io/badge/Redis-DC382D?style=flat-square&logo=redis&logoColor=white)](https://redis.io/) +[![Docker](https://img.shields.io/badge/Docker-2496ED?style=flat-square&logo=docker&logoColor=white)](https://docker.com/) + +
+ +--- + +## 📖 Table of Contents + +- [What is DevPulse?](#-what-is-devpulse) +- [Features](#-features) +- [Architecture](#-architecture) +- [Tech Stack](#-tech-stack) +- [Project Structure](#-project-structure) +- [Local Setup](#-local-setup) +- [Environment Variables](#-environment-variables) +- [API Overview](#-api-overview) +- [OS Finder — Deep Dive](#-os-finder--deep-dive) +- [AI Features](#-ai-features) +- [Database Schema](#-database-schema) + +--- + +## 🤔 What is DevPulse? + +DevPulse is a full-stack developer intelligence platform that connects to your GitHub account and gives you deep insights about your own engineering activity. Beyond analytics, it features **OS Finder** — an AI-powered open-source contribution discovery engine that understands your actual skill level, language preferences, and experience to surface perfectly-matched repositories for you to contribute to. + +Think of it as your personal developer co-pilot: it knows what you write, how often you write it, and what open-source projects would be a realistic, rewarding next step for your growth. + +--- + +## ✨ Features + +### 📊 GitHub Activity Dashboard +- Syncs your entire GitHub history — repositories, commits, and pull requests +- Visual commit heatmap and streak tracker +- Language distribution charts and coding pattern analysis +- Commit frequency by day-of-week and hour-of-day breakdowns +- Live sync status via **WebSocket** (Socket.IO) + +### 🧠 AI Weekly Digest +- Every week, an AI generates a personalised markdown digest of your coding activity +- Highlights what you built, patterns in your workflow, and what to focus on next +- Backed by OpenRouter (GPT-4o-mini) with a friendly, senior-dev tone + +### ⭐ PR Quality Scoring +- Every pull request you make is automatically scored **1–10** by AI +- The score is based on title clarity, body completeness, and professionalism +- Score and reason are stored and surfaced in your dashboard + +### 🔍 OS Finder — Open Source Discovery Engine +- Describe what you want to contribute to in plain English, or use advanced filters +- AI parses your query into structured GitHub search filters +- Repos are ranked by a custom **NCF (New Contributor Friendliness) Score** +- Filters include: language, difficulty, domain, repo size, license, activity recency +- Watchlist system to save, track, and annotate repos you're planning to contribute to + +### 📝 Community Posts +- Create, edit, and delete developer posts with comments +- Markdown-style content support + +### 🔐 Authentication +- GitHub OAuth 2.0 via Passport.js +- JWT access tokens (15 min) + refresh tokens (7 days) +- Silent refresh on token expiry — no repeated logins + +### ⚡ Real-time Updates +- WebSocket gateway (Socket.IO) for live sync progress events + +--- + +## 🏗 Architecture + +DevPulse is a **monorepo** with three independent services orchestrated via Docker Compose: + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Docker Compose │ +│ │ +│ ┌─────────────┐ ┌──────────────┐ ┌───────────────────┐ │ +│ │ Frontend │───▶│ Backend │───▶│ Analytics │ │ +│ │ (Next.js) │ │ (NestJS) │ │ (Flask/Python) │ │ +│ │ Port 3001 │ │ Port 3000 │ │ Port 5001 │ │ +│ └─────────────┘ └──────┬───────┘ └───────────────────┘ │ +│ │ │ +│ ┌───────┴────────┐ │ +│ │ │ │ +│ ┌──────▼──────┐ ┌──────▼──────┐ │ +│ │ PostgreSQL │ │ Redis │ │ +│ │ Port 5433 │ │ Port 6379 │ │ +│ └─────────────┘ └─────────────┘ │ +└─────────────────────────────────────────────────────────────────┘ +``` + +**Communication:** +- **Frontend → Backend**: HTTP rewrites via Next.js proxy + custom API routes for long-running AI calls +- **Backend → Analytics**: Internal HTTP (service URL via env) +- **Backend → GitHub**: REST API v3 with encrypted token storage per user +- **Backend → OpenRouter**: AI completions for digests, PR scoring, and OS Finder queries +- **Backend ↔ Frontend**: WebSocket (Socket.IO) for real-time sync events + +--- + +## 🛠 Tech Stack + +### Frontend +| Tool | Purpose | +|------|---------| +| **Next.js 14** | React framework, SSR pages, API routes | +| **TypeScript** | Type safety | +| **Tailwind CSS** | Utility-first styling | +| **Framer Motion** | Page and component animations | +| **Socket.IO Client** | Real-time sync progress | + +### Backend +| Tool | Purpose | +|------|---------| +| **NestJS 11** | Modular Node.js framework | +| **TypeScript** | Type safety | +| **TypeORM** | ORM + migrations | +| **PostgreSQL** | Primary database | +| **Redis (ioredis)** | Caching layer (search results, saved repos, issues) | +| **Passport.js** | GitHub OAuth + JWT strategy | +| **Socket.IO** | Real-time WebSocket gateway | +| **OpenRouter API** | AI inference (GPT-4o-mini) | +| **node-cron** | Scheduled jobs (GitHub sync every 6h, weekly digests) | +| **class-validator** | DTO validation | +| **crypto-js** | Encrypted GitHub token storage | + +### Analytics Microservice +| Tool | Purpose | +|------|---------| +| **Python 3 / Flask** | Lightweight HTTP microservice | +| **Custom analysers** | Commit pattern & language distribution analysis | + +### Infrastructure +| Tool | Purpose | +|------|---------| +| **Docker + Docker Compose** | Containerised dev and production environment | +| **PostgreSQL 15** | Relational data store | +| **Redis 7** | Caching and rate-limit protection | + +--- + +## 📂 Project Structure + +``` +DevPulse/ +├── docker-compose.yml # Orchestrates all services +├── apps/ +│ ├── backend/ # NestJS API server +│ │ ├── src/ +│ │ │ ├── auth/ # GitHub OAuth + JWT (Passport) +│ │ │ ├── users/ # User entity and profile +│ │ │ ├── github/ # GitHub API client +│ │ │ ├── github-sync/ # Sync engine (repos, commits, PRs) + cron +│ │ │ │ └── pr-score.service.ts # AI PR quality scoring +│ │ │ ├── os-finder/ # 🔍 Open Source Discovery Engine +│ │ │ │ ├── os-finder.service.ts # Core search logic +│ │ │ │ ├── ai-query-builder.service.ts # NL → filters via AI +│ │ │ │ ├── github-query-builder.ts # Filter → GitHub query string +│ │ │ │ ├── ncf-scorer.service.ts # NCF Score computation +│ │ │ │ ├── repo-health.service.ts # Repository health flags +│ │ │ │ ├── os-finder-cache.service.ts # Redis cache layer +│ │ │ │ └── entities/ # SavedRepo, OsFinderSearch +│ │ │ ├── digests/ # AI weekly digest generation + cron +│ │ │ ├── posts/ # Community posts +│ │ │ ├── comments/ # Post comments +│ │ │ ├── analytics/ # Analytics microservice bridge +│ │ │ ├── realtime/ # Socket.IO WebSocket gateway +│ │ │ ├── shared/ +│ │ │ │ └── ai.service.ts # OpenRouter AI client (shared) +│ │ │ ├── common/ # Guards, decorators, cache module, utils +│ │ │ └── database/ +│ │ │ └── migrations/ # 10 TypeORM migration files +│ │ └── packages/ +│ │ └── shared-types/ # Shared TypeScript interfaces (OS Finder) +│ │ +│ ├── frontend/ # Next.js 14 app +│ │ ├── pages/ +│ │ │ ├── index.tsx # Landing page +│ │ │ ├── api/ +│ │ │ │ └── os-finder-ai-search.ts # Long-running AI proxy route +│ │ │ ├── auth/callback.tsx # OAuth callback handler +│ │ │ └── dashboard/ +│ │ │ ├── index.tsx # Main dashboard +│ │ │ ├── commits.tsx # Commit analytics +│ │ │ ├── prs.tsx # Pull request history + scores +│ │ │ ├── digest.tsx # Weekly AI digest viewer +│ │ │ ├── settings.tsx # User settings +│ │ │ └── os-finder/ +│ │ │ ├── index.tsx # OS Finder search UI +│ │ │ ├── saved.tsx # Watchlist manager +│ │ │ └── [owner]/[repo].tsx # Repo detail + issues +│ │ ├── components/ # Reusable UI components +│ │ ├── context/AuthContext.tsx # Auth state + fetchWithAuth +│ │ ├── hooks/ # Custom React hooks +│ │ └── next.config.js # Proxy rewrites to backend +│ │ +│ └── analytics/ # Python Flask microservice +│ ├── app.py # Commit + language analysis endpoints +│ └── services/ # CommitAnalyser, LanguageAnalyser +│ +└── load-tests/ # Load testing scripts +``` + +--- + +## 🚀 Local Setup + +### Prerequisites + +- [Docker Desktop](https://www.docker.com/products/docker-desktop/) (includes Docker Compose) +- [Node.js 20+](https://nodejs.org/) *(only needed for local non-Docker dev)* +- A GitHub OAuth App (for auth) +- An [OpenRouter](https://openrouter.ai/) API key (for AI features) + +### 1. Clone the Repository + +```bash +git clone https://github.com/psychic-coder/DevPulse.git +cd DevPulse +``` + +### 2. Configure Environment Variables + +Copy and fill in the backend environment file: + +```bash +cp apps/backend/.env.example apps/backend/.env +``` + +Edit `apps/backend/.env` — see the [Environment Variables](#-environment-variables) section below for all required values. + +### 3. Create a GitHub OAuth App + +1. Go to [GitHub Developer Settings → OAuth Apps](https://github.com/settings/developers) +2. Click **New OAuth App** +3. Set: + - **Homepage URL**: `http://localhost:3001` + - **Authorization callback URL**: `http://localhost:3000/auth/github/callback` +4. Copy the **Client ID** and **Client Secret** into your `.env` + +### 4. Start with Docker Compose + +```bash +# Build and start all services (first time takes a few minutes) +docker compose up --build + +# Or run in background +docker compose up --build -d +``` + +This starts: +| Service | URL | +|---------|-----| +| **Frontend** | http://localhost:3001 | +| **Backend API** | http://localhost:3000 | +| **Analytics** | http://localhost:5001 | +| **PostgreSQL** | localhost:5433 | +| **Redis** | localhost:6379 | + +> **Note:** Database migrations run automatically on backend startup via TypeORM `migrationsRun: true`. + +### 5. Trigger a GitHub Sync + +After logging in via GitHub OAuth, your data won't appear until you trigger a sync: + +```bash +# Via the dashboard Settings page → "Sync GitHub Data" button +# Or via API: +curl -X POST http://localhost:3000/sync/github \ + -H "Authorization: Bearer YOUR_JWT_TOKEN" +``` + +--- + +## 🔑 Environment Variables + +All variables go in `apps/backend/.env`: + +| Variable | Required | Description | +|----------|----------|-------------| +| `NODE_ENV` | ✅ | `development` \| `production` \| `test` | +| `PORT` | ✅ | Backend port (default: `3000`) | +| `APP_URL` | ✅ | Frontend URL (e.g. `http://localhost:3001`) | +| `BACKEND_URL` | ✅ | Backend URL (e.g. `http://localhost:3000`) | +| `GITHUB_CLIENT_ID` | ✅ | From your GitHub OAuth App | +| `GITHUB_CLIENT_SECRET` | ✅ | From your GitHub OAuth App | +| `GITHUB_CALLBACK_URL` | ✅ | e.g. `http://localhost:3000/auth/github/callback` | +| `JWT_SECRET` | ✅ | Min 32-char random string | +| `JWT_EXPIRY` | ✅ | Access token lifetime (e.g. `15m`) | +| `JWT_REFRESH_SECRET` | ✅ | Min 32-char random string | +| `JWT_REFRESH_EXPIRY` | ✅ | Refresh token lifetime (e.g. `7d`) | +| `DATABASE_URL` | ✅ | PostgreSQL connection string | +| `ENCRYPTION_SECRET` | ✅ | Min 32-char string — encrypts stored GitHub tokens | +| `REDIS_URL` | ⬜ | Redis connection URL (defaults to no cache) | +| `OPENROUTER_API_KEY` | ⬜ | For AI features (digest, PR scoring, OS Finder AI search) | +| `OPENROUTER_MODEL` | ⬜ | Model name (default: `gpt-4o-mini`) | +| `ANALYTICS_SERVICE_URL` | ⬜ | Python analytics service (default: `http://localhost:5001`) | +| `DATABASE_SYNCHRONIZE` | ⬜ | `true` for auto-sync schema in dev (use migrations in prod) | + +> **Tip:** Generate secrets with: `openssl rand -hex 32` + +--- + +## 📡 API Overview + +All API routes are prefixed by the backend base URL and require a JWT `Authorization: Bearer ` header unless noted. + +### Auth +| Method | Endpoint | Description | +|--------|----------|-------------| +| `GET` | `/auth/github` | Initiate GitHub OAuth flow | +| `GET` | `/auth/github/callback` | OAuth callback (redirects with JWT) | +| `POST` | `/auth/refresh` | Exchange refresh token for new access token | +| `GET` | `/auth/session` | Validate current session | + +### GitHub Sync +| Method | Endpoint | Description | +|--------|----------|-------------| +| `POST` | `/sync/github` | Trigger full GitHub data sync for current user | +| `GET` | `/sync/github` | Get current sync status | +| `GET` | `/sync/github/streaks` | Get commit streak data | + +### Analytics +| Method | Endpoint | Description | +|--------|----------|-------------| +| `GET` | `/analytics/me` | Get full analytics for current user | + +### Digests +| Method | Endpoint | Description | +|--------|----------|-------------| +| `GET` | `/digests/me` | Get weekly AI digest for current user | + +### OS Finder +| Method | Endpoint | Description | +|--------|----------|-------------| +| `GET` | `/os-finder/search` | Standard filtered repository search | +| `POST` | `/os-finder/search/ai` | AI-powered natural language search | +| `GET` | `/os-finder/repo/:owner/:repo` | Detailed repo info + NCF score + health | +| `GET` | `/os-finder/repo/:owner/:repo/issues` | Beginner-friendly open issues | +| `POST` | `/os-finder/saved` | Save a repo to your watchlist | +| `GET` | `/os-finder/saved` | Get your watchlist | +| `PATCH` | `/os-finder/saved/:id` | Update notes/status on a saved repo | +| `DELETE` | `/os-finder/saved/:id` | Remove from watchlist | +| `GET` | `/os-finder/history` | View past searches | + +### Posts & Comments +| Method | Endpoint | Description | +|--------|----------|-------------| +| `GET` | `/posts` | List all posts | +| `POST` | `/posts` | Create a post | +| `GET/PUT/DELETE` | `/posts/:id` | Read, update, or delete a post | +| `GET/POST` | `/posts/:id/comments` | List or add comments | + +--- + +## 🔍 OS Finder — Deep Dive + +OS Finder is the flagship feature of DevPulse. It is a personalised open-source repository discovery engine built on top of the GitHub Search API, enhanced with AI and user profile data. + +### How It Works + +``` +User Query / Filters + │ + ▼ +┌───────────────────┐ +│ AI Query Builder │ ← Converts NL query to structured OsFinderFilters +│ (GPT-4o-mini) │ using user's language profile + experience level +└────────┬──────────┘ + │ + ▼ +┌───────────────────┐ +│ GitHub Query │ ← Builds valid GitHub Search API query string +│ Builder │ (handles multi-language splitting, text OR-terms) +└────────┬──────────┘ + │ + ▼ +┌───────────────────┐ +│ GitHub Search API │ ← Parallel queries per language (max 3), merged +└────────┬──────────┘ + │ + ▼ +┌───────────────────┐ +│ Health + NCF │ ← Per-repo staggered parallel analysis (50ms stagger) +│ Processing │ Repo health flags + contributor-friendliness score +└────────┬──────────┘ + │ + ▼ +┌───────────────────┐ +│ Ranking + Cache │ ← Sort by NCF × langMatchScore, cache in Redis 30min +└────────┬──────────┘ + │ + ▼ + Results +``` + +### NCF Score (New Contributor Friendliness) + +Each repo is scored out of 10 based on: + +| Signal | Weight | What it measures | +|--------|--------|-----------------| +| Good First Issues | 2.5 | Volume of beginner-labelled issues | +| Help Wanted Issues | 1.0 | Active maintainer requests for contribution | +| CONTRIBUTING.md | 1.5 | Has a contributing guide | +| Issue Response Time | 2.0 | How fast maintainers respond to new issues | +| New Contributor PRs | 1.5 | Historical rate of first-time contributor merges | +| README Quality | 0.5 | Presence and length of README | +| Code of Conduct | 0.5 | Has a CoC file | +| PR Merge Rate | 0.5 | Overall PR acceptance rate | + +### Repository Health Flags + +Each result includes boolean health flags: +- `isArchived` — repo is read-only +- `isStale` — no commits in 90+ days +- `noContributing` — missing CONTRIBUTING.md +- `noReadme` — missing README +- `lowPRMergeRate` — < 30% PRs merged +- `forkHeavy` — more forks than stars (sign of dead project) +- `noExternalContribs` — only the owner has committed +- `lowIssueEngagement` — very few issues +- `slowMaintainerResp` — maintainer takes 14+ days to respond + +### Filter Options + +| Filter | Type | Description | +|--------|------|-------------| +| `languages` | `string[]` | Programming languages (auto-detected from your DevPulse profile) | +| `languageMode` | `strict \| any_of` | Match all languages or any | +| `difficulty` | `beginner \| intermediate \| advanced` | Auto-inferred from your commit history | +| `contributionTypes` | `string[]` | bug_fix, feature, docs, tests, i18n, etc. | +| `domains` | `string[]` | web, devtools, ai_ml, mobile, data, infra, etc. | +| `repoSize` | `small \| medium \| large \| any` | By star count ranges | +| `lastCommitDays` | `number` | Max days since last commit | +| `minOpenIssues` | `number` | Minimum open issue count | +| `hasContributing` | `boolean` | Must have CONTRIBUTING.md | +| `hasCodeOfConduct` | `boolean` | Must have CODE_OF_CONDUCT.md | +| `licenseTypes` | `string[]` | e.g. `['MIT', 'Apache-2.0']` | +| `prMergeRate` | `number` | Minimum % of PRs merged | + +--- + +## 🤖 AI Features + +DevPulse uses **OpenRouter** (compatible with OpenAI API) to power three distinct AI use cases: + +### 1. Weekly Digest Generation +- Runs weekly via `node-cron` +- Analyses commits, PRs, streaks, and language usage from the past 7 days +- Writes a friendly, structured markdown digest with sections: Week in Review, What You Crushed, Patterns Noticed, Focus Suggestion + +### 2. PR Quality Scoring +- Runs automatically after each GitHub sync +- Scores each PR 1–10 based on title, body, and professionalism +- Stored alongside the PR record for display in the dashboard + +### 3. OS Finder AI Query Builder +- Natural language → structured `OsFinderFilters` JSON +- Uses your DevPulse profile (top languages, experience level, average PR score) as context +- Falls back gracefully to keyword-based regex matching if the AI call fails or times out (15s timeout) +- The resulting query is split per language to work within GitHub Search API limitations + +--- + +## 🗄 Database Schema + +DevPulse uses **PostgreSQL** with TypeORM migrations. Key tables: + +| Table | Description | +|-------|-------------| +| `users` | GitHub user profile + encrypted token | +| `repositories` | Synced GitHub repos | +| `commits` | Commit history with additions/deletions | +| `pull_requests` | PRs with AI quality scores | +| `posts` | Community posts | +| `comments` | Comments on posts | +| `digests` | Weekly AI digest records | +| `saved_repos` | OS Finder watchlist (stores full NCF score as `jsonb`) | +| `os_finder_searches` | Search history with filters applied | + +--- + +## 🤝 Contributing + +1. Fork the repo and create a feature branch from `main` +2. Follow the existing module structure for backend features +3. All new backend routes should use the `@UseGuards(JwtAuthGuard)` and `@CurrentUser()` decorator pattern +4. Run `npm run lint` and `npm run test` in `apps/backend` before submitting a PR +5. Open a PR with a clear title and description — DevPulse will score it 😄 + +--- + +
+ +Built with ❤️ by [psychic-coder](https://github.com/psychic-coder) + +
diff --git a/apps/backend/Dockerfile b/apps/backend/Dockerfile index bdf7558..0ea1a0d 100644 --- a/apps/backend/Dockerfile +++ b/apps/backend/Dockerfile @@ -2,10 +2,10 @@ FROM node:20-alpine AS builder WORKDIR /app -COPY package*.json ./ +COPY apps/backend/package*.json ./ RUN npm ci -COPY . . +COPY apps/backend/ . RUN npm run build FROM node:20-alpine AS runner @@ -13,7 +13,7 @@ FROM node:20-alpine AS runner WORKDIR /app ENV NODE_ENV=production -COPY package*.json ./ +COPY apps/backend/package*.json ./ RUN npm ci --only=production COPY --from=builder /app/dist ./dist diff --git a/apps/backend/packages/shared-types/os-finder.types.ts b/apps/backend/packages/shared-types/os-finder.types.ts new file mode 100644 index 0000000..1745e4b --- /dev/null +++ b/apps/backend/packages/shared-types/os-finder.types.ts @@ -0,0 +1,91 @@ +export type Difficulty = 'beginner' | 'intermediate' | 'advanced'; +export type ContributionType = 'bug_fix' | 'feature' | 'documentation' | 'tests' | 'i18n' | 'performance' | 'ui_design' | 'security'; +export type Domain = 'web' | 'devtools' | 'ai_ml' | 'mobile' | 'data' | 'infrastructure' | 'education' | 'games' | 'finance'; +export type RepoSize = 'small' | 'medium' | 'large' | 'any'; +export type LanguageMode = 'strict' | 'any_of'; +export type SavedRepoStatus = 'saved' | 'contributed' | 'skipped'; + +export interface NCFScoreBreakdown { + total: number; + goodFirstIssue: number; + helpWanted: number; + contributingFile: number; + issueResponseTime: number; + newContribPR: number; + readmeQuality: number; + codeOfConduct: number; + prMergeRate: number; +} + +export interface RepoHealthFlags { + isArchived: boolean; + isStale: boolean; + noContributing: boolean; + noReadme: boolean; + lowPRMergeRate: boolean; + forkHeavy: boolean; + noExternalContribs: boolean; + lowIssueEngagement: boolean; + slowMaintainerResp: boolean; +} + +export interface OsFinderRepoResult { + githubRepoId: number; + owner: string; + name: string; + fullName: string; + description: string | null; + language: string | null; + stars: number; + forks: number; + openIssues: number; + lastCommitAt: string; + licenseType: string | null; + htmlUrl: string; + ncfScore: NCFScoreBreakdown; + langMatchScore: number; // 0.0 to 1.0 + openLabels: string[]; + healthFlags: RepoHealthFlags; + isSaved: boolean; + alreadyContrib: boolean; +} + +export interface OsFinderFilters { + // Language + languages: string[]; // default: user's top 3 from DevPulse commit data + languageMode: 'strict' | 'any_of'; // default: 'any_of' + + // Difficulty (if not set, auto-detect from user's DevPulse data) + difficulty?: 'beginner' | 'intermediate' | 'advanced'; + + // Contribution type (multi-select) + contributionTypes: ContributionType[]; + + // Domain (multi-select — maps to GitHub topic tags) + domains: Domain[]; + + // Repo size by stars + repoSize: RepoSize; + + // Activity filters + lastCommitDays: number; // default 90 — exclude repos with no commits in N days + minOpenIssues: number; // default 3 — repo must have at least N open issues + issueFreshDays: number; // default 60 — issues must be created/updated within N days + + // Health filters + hasContributing: boolean; // default true — must have CONTRIBUTING.md + hasCodeOfConduct: boolean; // default false — optional filter + licenseTypes: string[]; // default [] (any) — e.g. ['MIT', 'Apache-2.0'] + prMergeRate: number; // default 30 — minimum % of PRs merged + includeAlreadyContributed?: boolean; +} + +export interface OsFinderSearchResponse { + results: OsFinderRepoResult[]; + total: number; + page: number; + filtersApplied: OsFinderFilters; + filtersRelaxed: Partial | null; + relaxationNote: string | null; + aiModeUsed: boolean; +} diff --git a/apps/backend/src/app.module.ts b/apps/backend/src/app.module.ts index 8c2c781..88ee2a2 100644 --- a/apps/backend/src/app.module.ts +++ b/apps/backend/src/app.module.ts @@ -17,6 +17,7 @@ import { DigestsModule } from './digests/digests.module'; import { CacheModule } from './common/cache/cache.module'; import { AnalyticsModule } from './analytics/analytics.module'; import { RealtimeModule } from './realtime/realtime.module'; +import { OsFinderModule } from './os-finder/os-finder.module'; @Module({ imports: [ @@ -35,6 +36,7 @@ import { RealtimeModule } from './realtime/realtime.module'; DigestsModule, AnalyticsModule, RealtimeModule, + OsFinderModule, ], controllers: [AppController], providers: [ diff --git a/apps/backend/src/database/migrations/202605210001-create-os-finder-tables.ts b/apps/backend/src/database/migrations/202605210001-create-os-finder-tables.ts new file mode 100644 index 0000000..aa16f56 --- /dev/null +++ b/apps/backend/src/database/migrations/202605210001-create-os-finder-tables.ts @@ -0,0 +1,65 @@ +import { MigrationInterface, QueryRunner } from 'typeorm'; + +export class CreateOsFinderTables1779200000000 implements MigrationInterface { + name = 'CreateOsFinderTables1779200000000'; + + public async up(queryRunner: QueryRunner): Promise { + await queryRunner.query(` + CREATE TABLE "saved_repos" ( + "id" uuid NOT NULL DEFAULT uuid_generate_v4(), + "user_id" uuid NOT NULL, + "github_repo_id" bigint NOT NULL, + "owner" character varying NOT NULL, + "name" character varying NOT NULL, + "full_name" character varying NOT NULL, + "description" text, + "language" character varying, + "stars" integer NOT NULL DEFAULT 0, + "forks" integer NOT NULL DEFAULT 0, + "open_issues" integer NOT NULL DEFAULT 0, + "ncf_score" double precision, + "lang_match_score" double precision, + "last_commit_at" TIMESTAMP, + "has_contributing" boolean DEFAULT false, + "license_type" character varying, + "html_url" character varying NOT NULL, + "saved_at" TIMESTAMP NOT NULL DEFAULT now(), + "notes" text, + "status" character varying NOT NULL DEFAULT 'saved', + CONSTRAINT "PK_saved_repos_id" PRIMARY KEY ("id"), + CONSTRAINT "UQ_saved_repos_user_github" UNIQUE ("user_id", "github_repo_id"), + CONSTRAINT "FK_saved_repos_user" FOREIGN KEY ("user_id") REFERENCES "users"("id") ON DELETE CASCADE + ) + `); + + await queryRunner.query(` + CREATE INDEX "IDX_saved_repos_user" ON "saved_repos" ("user_id") + `); + + await queryRunner.query(` + CREATE TABLE "os_finder_searches" ( + "id" uuid NOT NULL DEFAULT uuid_generate_v4(), + "user_id" uuid NOT NULL, + "query_text" text, + "filters_applied" jsonb NOT NULL, + "result_count" integer, + "ai_query_used" boolean NOT NULL DEFAULT false, + "github_query" text, + "created_at" TIMESTAMP NOT NULL DEFAULT now(), + CONSTRAINT "PK_os_finder_searches_id" PRIMARY KEY ("id"), + CONSTRAINT "FK_os_finder_searches_user" FOREIGN KEY ("user_id") REFERENCES "users"("id") ON DELETE CASCADE + ) + `); + + await queryRunner.query(` + CREATE INDEX "idx_os_finder_searches_user" ON "os_finder_searches" ("user_id", "created_at" DESC) + `); + } + + public async down(queryRunner: QueryRunner): Promise { + await queryRunner.query(`DROP INDEX IF EXISTS "idx_os_finder_searches_user"`); + await queryRunner.query(`DROP TABLE IF EXISTS "os_finder_searches"`); + await queryRunner.query(`DROP INDEX IF EXISTS "IDX_saved_repos_user"`); + await queryRunner.query(`DROP TABLE IF EXISTS "saved_repos"`); + } +} diff --git a/apps/backend/src/database/migrations/202605210002-alter-saved-repos-ncf-score.ts b/apps/backend/src/database/migrations/202605210002-alter-saved-repos-ncf-score.ts new file mode 100644 index 0000000..d54512a --- /dev/null +++ b/apps/backend/src/database/migrations/202605210002-alter-saved-repos-ncf-score.ts @@ -0,0 +1,24 @@ +import { MigrationInterface, QueryRunner } from 'typeorm'; + +export class AlterSavedReposNcfScore1779200000001 implements MigrationInterface { + name = 'AlterSavedReposNcfScore1779200000001'; + + public async up(queryRunner: QueryRunner): Promise { + await queryRunner.query(` + ALTER TABLE "saved_repos" + ALTER COLUMN "ncf_score" TYPE jsonb + USING CASE + WHEN ncf_score IS NULL THEN NULL + ELSE jsonb_build_object('total', ncf_score) + END + `); + } + + public async down(queryRunner: QueryRunner): Promise { + await queryRunner.query(` + ALTER TABLE "saved_repos" + ALTER COLUMN "ncf_score" TYPE double precision + USING (ncf_score->>'total')::double precision + `); + } +} diff --git a/apps/backend/src/database/typeorm.config.ts b/apps/backend/src/database/typeorm.config.ts index b90863c..44d5b68 100644 --- a/apps/backend/src/database/typeorm.config.ts +++ b/apps/backend/src/database/typeorm.config.ts @@ -7,13 +7,26 @@ import { PullRequest } from '../github-sync/entities/pull-request.entity'; import { Post } from '../posts/entities/post.entity'; import { Comment } from '../comments/entities/comment.entity'; import { Digest } from '../digests/entities/digest.entity'; +import { SavedRepo } from '../os-finder/entities/saved-repo.entity'; +import { OsFinderSearch } from '../os-finder/entities/os-finder-search.entity'; export const createTypeOrmOptions = (): TypeOrmModuleOptions => ({ type: 'postgres', url: process.env.DATABASE_URL, - entities: [User, Repository, Commit, PullRequest, Post, Comment, Digest], + entities: [ + User, + Repository, + Commit, + PullRequest, + Post, + Comment, + Digest, + SavedRepo, + OsFinderSearch, + ], migrations: [join(__dirname, './migrations/*{.ts,.js}')], migrationsRun: true, synchronize: process.env.DATABASE_SYNCHRONIZE === 'true', logging: process.env.DATABASE_LOGGING === 'true', }); + diff --git a/apps/backend/src/os-finder/ai-query-builder.service.ts b/apps/backend/src/os-finder/ai-query-builder.service.ts new file mode 100644 index 0000000..a3732f0 --- /dev/null +++ b/apps/backend/src/os-finder/ai-query-builder.service.ts @@ -0,0 +1,219 @@ +import { Injectable, Logger } from '@nestjs/common'; +import { AiService } from '../shared/ai.service'; +import { OsFinderFilters, Difficulty, ContributionType, Domain, RepoSize } from '../../packages/shared-types/os-finder.types'; + +@Injectable() +export class AiQueryBuilderService { + private readonly logger = new Logger(AiQueryBuilderService.name); + + constructor(private readonly aiService: AiService) {} + + async buildFilters( + userQuery: string, + userCtx: { topLanguages: string[]; inferredLevel: string; avgPRScore: number } + ): Promise<{ filters: OsFinderFilters; keywords: string[]; aiModeUsed: boolean; fallbackUsed: boolean }> { + const prompt = `You are a GitHub repository search assistant inside DevPulse. +Given a developer's natural language request and their profile, +extract a structured search filter object. Respond ONLY with valid JSON. + +Developer profile: +- Top languages: ${userCtx.topLanguages.join(', ')} +- Experience level (inferred): ${userCtx.inferredLevel} +- Avg PR quality score: ${userCtx.avgPRScore.toFixed(1)}/10 + +User request: "${userQuery}" + +Return this exact JSON structure: +{ + "languages": string[], + "languageMode": "strict" | "any_of", + "difficulty": "beginner" | "intermediate" | "advanced", + "contributionTypes": string[], + "domains": string[], + "repoSize": "small" | "medium" | "large" | "any", + "lastCommitDays": number, + "hasContributing": boolean, + "hasCodeOfConduct": boolean, + "licenseTypes": string[], + "keywords": string[] +} + +Rules: +- If user doesn't mention language, use their top language from profile +- If user says 'nothing too complex' or 'beginner-friendly', set difficulty: beginner +- If user says 'active community', set lastCommitDays: 30 +- For licenses mentioned (e.g. 'mit', 'apache'), populate the "licenseTypes" array (e.g. ["MIT"]) rather than putting it in keywords. +- Keep the keywords array extremely minimal (0 to 2 items max). Only extract high-value specific search terms that are NOT already captured by languages, domains, or license types. Never include common stop words, or words like "application", "tool", "license", "library", "framework", "repo", "project". +- Never invent filters not in the schema above +- Only respond with JSON — no preamble, no markdown backticks`; + + const timeoutMs = 15000; + let aiResponse: string | null = null; + let fallbackUsed = false; + + try { + // 5-second hard timeout + aiResponse = await Promise.race([ + this.aiService.generateText(prompt), + new Promise((_, reject) => + setTimeout(() => reject(new Error('AI Call Timeout')), timeoutMs) + ), + ]); + } catch (error) { + this.logger.warn(`AI Query Builder failed or timed out: ${error instanceof Error ? error.message : String(error)}. Using keyword fallback.`); + fallbackUsed = true; + } + + if (!aiResponse || fallbackUsed) { + return { + ...this.parseFallback(userQuery, userCtx), + aiModeUsed: true, + fallbackUsed: true, + }; + } + + try { + // Clean possible JSON formatting issues (like markdown ticks) + let cleaned = aiResponse.trim(); + if (cleaned.startsWith('```json')) { + cleaned = cleaned.substring(7); + } + if (cleaned.startsWith('```')) { + cleaned = cleaned.substring(3); + } + if (cleaned.endsWith('```')) { + cleaned = cleaned.substring(0, cleaned.length - 3); + } + cleaned = cleaned.trim(); + + const parsed = JSON.parse(cleaned); + + const filters: OsFinderFilters = { + languages: Array.isArray(parsed.languages) ? parsed.languages : [userCtx.topLanguages[0] || 'typescript'], + languageMode: parsed.languageMode || 'any_of', + difficulty: parsed.difficulty || (userCtx.inferredLevel as Difficulty), + contributionTypes: Array.isArray(parsed.contributionTypes) ? parsed.contributionTypes as ContributionType[] : [], + domains: Array.isArray(parsed.domains) ? parsed.domains as Domain[] : [], + repoSize: parsed.repoSize || 'any', + lastCommitDays: typeof parsed.lastCommitDays === 'number' ? parsed.lastCommitDays : 90, + minOpenIssues: 3, + issueFreshDays: 60, + hasContributing: typeof parsed.hasContributing === 'boolean' ? parsed.hasContributing : true, + hasCodeOfConduct: typeof parsed.hasCodeOfConduct === 'boolean' ? parsed.hasCodeOfConduct : false, + licenseTypes: Array.isArray(parsed.licenseTypes) ? parsed.licenseTypes : [], + prMergeRate: 30, + }; + + const keywords = Array.isArray(parsed.keywords) ? parsed.keywords : []; + + return { + filters, + keywords, + aiModeUsed: true, + fallbackUsed: false, + }; + } catch (parseError) { + this.logger.error(`Failed to parse AI JSON response: ${parseError instanceof Error ? parseError.message : String(parseError)}. Content was: ${aiResponse}`); + return { + ...this.parseFallback(userQuery, userCtx), + aiModeUsed: true, + fallbackUsed: true, + }; + } + } + + private parseFallback( + query: string, + userCtx: { topLanguages: string[]; inferredLevel: string } + ): { filters: OsFinderFilters; keywords: string[] } { + const text = query.toLowerCase(); + + // 1. Detect language keywords + const COMMON_LANGUAGES = ['typescript', 'javascript', 'python', 'rust', 'go', 'cpp', 'c++', 'java', 'ruby', 'php', 'html', 'css', 'swift', 'kotlin']; + const detectedLanguages: string[] = []; + COMMON_LANGUAGES.forEach(lang => { + // match full word + const regex = new RegExp(`\\b${lang.replace('+', '\\+')}\\b`, 'i'); + if (regex.test(text)) { + detectedLanguages.push(lang); + } + }); + + // Default to user's top language if none found + if (detectedLanguages.length === 0 && userCtx.topLanguages.length > 0) { + detectedLanguages.push(userCtx.topLanguages[0]); + } else if (detectedLanguages.length === 0) { + detectedLanguages.push('typescript'); // final fallback + } + + // 2. Detect difficulty + let difficulty: Difficulty = userCtx.inferredLevel as Difficulty; + if (/\b(beginner|simple|easy|first issue|starter|newbie)\b/i.test(text)) { + difficulty = 'beginner'; + } else if (/\b(intermediate|medium|average)\b/i.test(text)) { + difficulty = 'intermediate'; + } else if (/\b(advanced|complex|hard|expert|niche|guru)\b/i.test(text)) { + difficulty = 'advanced'; + } + + // 3. Detect domains + const domainKeywords: { domain: Domain; keywords: string[] }[] = [ + { domain: 'web', keywords: ['web', 'frontend', 'backend', 'api', 'react', 'nextjs', 'nodejs', 'website'] }, + { domain: 'devtools', keywords: ['cli', 'tool', 'terminal', 'devtools', 'npm', 'scripts'] }, + { domain: 'ai_ml', keywords: ['ml', 'ai', 'learning', 'gpt', 'llm', 'neural', 'weights', 'inference'] }, + { domain: 'mobile', keywords: ['mobile', 'android', 'ios', 'flutter', 'react native', 'phone', 'app'] }, + { domain: 'data', keywords: ['data', 'visualization', 'science', 'database', 'sql', 'analysis'] }, + { domain: 'infrastructure', keywords: ['devops', 'infrastructure', 'docker', 'kubernetes', 'cloud', 'ci', 'cd'] }, + { domain: 'education', keywords: ['learn', 'education', 'tutorial', 'course', 'school', 'academy'] }, + { domain: 'games', keywords: ['game', 'gameplay', 'gamedev', 'unity', 'unreal', 'physics', 'engine'] }, + { domain: 'finance', keywords: ['finance', 'blockchain', 'crypto', 'trading', 'ledger', 'wallet'] }, + ]; + + const detectedDomains: Domain[] = []; + domainKeywords.forEach(dk => { + const match = dk.keywords.some(keyword => { + const regex = new RegExp(`\\b${keyword}\\b`, 'i'); + return regex.test(text); + }); + if (match) { + detectedDomains.push(dk.domain); + } + }); + + // 4. Remaining keywords extraction + // Split into words, exclude common stop words, languages and domain terms + const stopWords = new Set(['i', 'want', 'to', 'contribute', 'something', 'related', 'in', 'the', 'a', 'an', 'and', 'for', 'with', 'on', 'in', 'of', 'at', 'by', 'project', 'projects', 'repo', 'repos', 'repository', 'repositories']); + const allWords = text.split(/[^a-zA-Z0-9\+\#]/).filter(Boolean); + + const keywords = allWords.filter(word => { + if (word.length < 3) return false; + if (stopWords.has(word)) return false; + if (COMMON_LANGUAGES.includes(word)) return false; + // also filter out domain match terms + const isDomainTerm = domainKeywords.some(dk => dk.keywords.includes(word)); + if (isDomainTerm) return false; + return true; + }); + + const filters: OsFinderFilters = { + languages: detectedLanguages, + languageMode: 'any_of', + difficulty, + contributionTypes: [], + domains: detectedDomains, + repoSize: 'any', + lastCommitDays: text.includes('active community') ? 30 : 90, + minOpenIssues: 3, + issueFreshDays: 60, + hasContributing: true, + hasCodeOfConduct: false, + licenseTypes: [], + prMergeRate: 30, + }; + + return { + filters, + keywords: keywords.slice(0, 5), // take first 5 keywords + }; + } +} diff --git a/apps/backend/src/os-finder/dto/save-repo.dto.ts b/apps/backend/src/os-finder/dto/save-repo.dto.ts new file mode 100644 index 0000000..9289102 --- /dev/null +++ b/apps/backend/src/os-finder/dto/save-repo.dto.ts @@ -0,0 +1,71 @@ +import { IsNotEmpty, IsNumber, IsString, IsOptional, IsBoolean, IsEnum } from 'class-validator'; +import type { SavedRepoStatus } from '../../../packages/shared-types/os-finder.types'; + +export class SaveRepoDto { + @IsNotEmpty() + @IsNumber() + githubRepoId: number; + + @IsNotEmpty() + @IsString() + owner: string; + + @IsNotEmpty() + @IsString() + name: string; + + @IsNotEmpty() + @IsString() + fullName: string; + + @IsOptional() + @IsString() + description?: string | null; + + @IsOptional() + @IsString() + language?: string | null; + + @IsOptional() + @IsNumber() + stars?: number; + + @IsOptional() + @IsNumber() + forks?: number; + + @IsOptional() + @IsNumber() + openIssues?: number; + + @IsOptional() + ncfScore?: any; + + @IsOptional() + @IsNumber() + langMatchScore?: number | null; + + @IsOptional() + @IsString() + lastCommitAt?: string | null; + + @IsOptional() + @IsBoolean() + hasContributing?: boolean; + + @IsOptional() + @IsString() + licenseType?: string | null; + + @IsNotEmpty() + @IsString() + htmlUrl: string; + + @IsOptional() + @IsString() + notes?: string | null; + + @IsOptional() + @IsEnum(['saved', 'contributed', 'skipped']) + status?: SavedRepoStatus = 'saved'; +} diff --git a/apps/backend/src/os-finder/dto/search-query.dto.ts b/apps/backend/src/os-finder/dto/search-query.dto.ts new file mode 100644 index 0000000..f6395ec --- /dev/null +++ b/apps/backend/src/os-finder/dto/search-query.dto.ts @@ -0,0 +1,115 @@ +import { IsOptional, IsString, IsEnum, IsNumber, IsBoolean, IsArray, IsInt, Min, Max } from 'class-validator'; +import { Transform } from 'class-transformer'; +import type { Difficulty, ContributionType, Domain, RepoSize, LanguageMode } from '../../../packages/shared-types/os-finder.types'; + +export class SearchQueryDto { + @IsOptional() + @IsArray() + @IsString({ each: true }) + @Transform(({ value }) => { + if (typeof value === 'string') return value.split(',').map(s => s.trim()).filter(Boolean); + return value; + }) + languages?: string[]; + + @IsOptional() + @IsEnum(['strict', 'any_of']) + languageMode?: LanguageMode = 'any_of'; + + @IsOptional() + @IsEnum(['beginner', 'intermediate', 'advanced']) + difficulty?: Difficulty; + + @IsOptional() + @IsArray() + @Transform(({ value }) => { + if (typeof value === 'string') return value.split(',').map(s => s.trim()).filter(Boolean); + return value; + }) + contributionTypes?: ContributionType[]; + + @IsOptional() + @IsArray() + @Transform(({ value }) => { + if (typeof value === 'string') return value.split(',').map(s => s.trim()).filter(Boolean); + return value; + }) + domains?: Domain[]; + + @IsOptional() + @IsEnum(['small', 'medium', 'large', 'any']) + repoSize?: RepoSize = 'any'; + + @IsOptional() + @IsInt() + @Min(1) + @Transform(({ value }) => value !== undefined ? parseInt(value, 10) : undefined) + lastCommitDays?: number = 90; + + @IsOptional() + @IsInt() + @Min(0) + @Transform(({ value }) => value !== undefined ? parseInt(value, 10) : undefined) + minOpenIssues?: number = 3; + + @IsOptional() + @IsInt() + @Min(1) + @Transform(({ value }) => value !== undefined ? parseInt(value, 10) : undefined) + issueFreshDays?: number = 60; + + @IsOptional() + @IsBoolean() + @Transform(({ value }) => { + if (value === 'true' || value === true) return true; + if (value === 'false' || value === false) return false; + return value; + }) + hasContributing?: boolean = true; + + @IsOptional() + @IsBoolean() + @Transform(({ value }) => { + if (value === 'true' || value === true) return true; + if (value === 'false' || value === false) return false; + return value; + }) + hasCodeOfConduct?: boolean = false; + + @IsOptional() + @IsArray() + @IsString({ each: true }) + @Transform(({ value }) => { + if (typeof value === 'string') return value.split(',').map(s => s.trim()).filter(Boolean); + return value; + }) + licenseTypes?: string[]; + + @IsOptional() + @IsNumber() + @Min(0) + @Max(100) + @Transform(({ value }) => value !== undefined ? parseFloat(value) : undefined) + prMergeRate?: number = 30; + + @IsOptional() + @IsBoolean() + @Transform(({ value }) => { + if (value === 'true' || value === true) return true; + if (value === 'false' || value === false) return false; + return value; + }) + includeAlreadyContributed?: boolean = false; + + @IsOptional() + @IsInt() + @Min(1) + @Transform(({ value }) => value !== undefined ? parseInt(value, 10) : undefined) + page?: number = 1; + + @IsOptional() + @IsInt() + @Min(1) + @Transform(({ value }) => value !== undefined ? parseInt(value, 10) : undefined) + limit?: number = 30; +} diff --git a/apps/backend/src/os-finder/dto/update-saved-repo.dto.ts b/apps/backend/src/os-finder/dto/update-saved-repo.dto.ts new file mode 100644 index 0000000..de87369 --- /dev/null +++ b/apps/backend/src/os-finder/dto/update-saved-repo.dto.ts @@ -0,0 +1,12 @@ +import { IsOptional, IsString, IsEnum } from 'class-validator'; +import type { SavedRepoStatus } from '../../../packages/shared-types/os-finder.types'; + +export class UpdateSavedRepoDto { + @IsOptional() + @IsString() + notes?: string | null; + + @IsOptional() + @IsEnum(['saved', 'contributed', 'skipped']) + status?: SavedRepoStatus; +} diff --git a/apps/backend/src/os-finder/entities/os-finder-search.entity.ts b/apps/backend/src/os-finder/entities/os-finder-search.entity.ts new file mode 100644 index 0000000..09a314a --- /dev/null +++ b/apps/backend/src/os-finder/entities/os-finder-search.entity.ts @@ -0,0 +1,42 @@ +import { + Entity, + PrimaryGeneratedColumn, + Column, + ManyToOne, + CreateDateColumn, + Index, + JoinColumn, +} from 'typeorm'; +import { User } from '../../users/entities/user.entity'; + +@Entity('os_finder_searches') +@Index(['userId', 'createdAt']) +export class OsFinderSearch { + @PrimaryGeneratedColumn('uuid') + id: string; + + @Column({ name: 'user_id', type: 'uuid' }) + userId: string; + + @ManyToOne(() => User, { onDelete: 'CASCADE' }) + @JoinColumn({ name: 'user_id' }) + user: User; + + @Column({ name: 'query_text', type: 'text', nullable: true }) + queryText: string | null; + + @Column({ name: 'filters_applied', type: 'jsonb' }) + filtersApplied: any; + + @Column({ name: 'result_count', type: 'int', nullable: true }) + resultCount: number | null; + + @Column({ name: 'ai_query_used', type: 'boolean', default: false }) + aiQueryUsed: boolean; + + @Column({ name: 'github_query', type: 'text', nullable: true }) + githubQuery: string | null; + + @CreateDateColumn({ name: 'created_at' }) + createdAt: Date; +} diff --git a/apps/backend/src/os-finder/entities/saved-repo.entity.ts b/apps/backend/src/os-finder/entities/saved-repo.entity.ts new file mode 100644 index 0000000..30d0c65 --- /dev/null +++ b/apps/backend/src/os-finder/entities/saved-repo.entity.ts @@ -0,0 +1,80 @@ +import { + Entity, + PrimaryGeneratedColumn, + Column, + ManyToOne, + CreateDateColumn, + Unique, + Index, + JoinColumn, +} from 'typeorm'; +import { User } from '../../users/entities/user.entity'; + +@Entity('saved_repos') +@Unique(['userId', 'githubRepoId']) +@Index(['userId']) +export class SavedRepo { + @PrimaryGeneratedColumn('uuid') + id: string; + + @Column({ name: 'user_id', type: 'uuid' }) + userId: string; + + @ManyToOne(() => User, { onDelete: 'CASCADE' }) + @JoinColumn({ name: 'user_id' }) + user: User; + + @Column({ name: 'github_repo_id', type: 'bigint' }) + githubRepoId: number; + + @Column({ type: 'varchar' }) + owner: string; + + @Column({ type: 'varchar' }) + name: string; + + @Column({ name: 'full_name', type: 'varchar' }) + fullName: string; + + @Column({ type: 'text', nullable: true }) + description: string | null; + + @Column({ type: 'varchar', nullable: true }) + language: string | null; + + @Column({ type: 'int', default: 0 }) + stars: number; + + @Column({ type: 'int', default: 0 }) + forks: number; + + @Column({ name: 'open_issues', type: 'int', default: 0 }) + openIssues: number; + + @Column({ name: 'ncf_score', type: 'jsonb', nullable: true }) + ncfScore: any | null; + + @Column({ name: 'lang_match_score', type: 'double precision', nullable: true }) + langMatchScore: number | null; + + @Column({ name: 'last_commit_at', type: 'timestamp', nullable: true }) + lastCommitAt: Date | null; + + @Column({ name: 'has_contributing', type: 'boolean', default: false }) + hasContributing: boolean; + + @Column({ name: 'license_type', type: 'varchar', nullable: true }) + licenseType: string | null; + + @Column({ name: 'html_url', type: 'varchar' }) + htmlUrl: string; + + @CreateDateColumn({ name: 'saved_at' }) + savedAt: Date; + + @Column({ type: 'text', nullable: true }) + notes: string | null; + + @Column({ type: 'varchar', default: 'saved' }) + status: string; // 'saved' | 'contributed' | 'skipped' +} diff --git a/apps/backend/src/os-finder/github-query-builder.spec.ts b/apps/backend/src/os-finder/github-query-builder.spec.ts new file mode 100644 index 0000000..62ee792 --- /dev/null +++ b/apps/backend/src/os-finder/github-query-builder.spec.ts @@ -0,0 +1,137 @@ +import { GitHubQueryBuilder } from './github-query-builder'; +import { OsFinderFilters } from '../../packages/shared-types/os-finder.types'; + +describe('GitHubQueryBuilder', () => { + const baseFilters: OsFinderFilters = { + languages: [], + languageMode: 'any_of', + contributionTypes: [], + domains: [], + repoSize: 'any', + lastCommitDays: 90, + minOpenIssues: 3, + issueFreshDays: 60, + hasContributing: true, + hasCodeOfConduct: false, + licenseTypes: [], + prMergeRate: 30, + }; + + const getPushedDateStr = (days: number): string => { + const d = new Date(); + d.setDate(d.getDate() - days); + return d.toISOString().slice(0, 10); + }; + + it('1. should generate base query with defaults', () => { + const query = GitHubQueryBuilder.build(baseFilters); + const expectedPushed = getPushedDateStr(90); + expect(query).toContain('archived:false'); + expect(query).toContain(`pushed:>=${expectedPushed}`); + }); + + it('2. should append single language filter', () => { + const filters = { ...baseFilters, languages: ['typescript'] }; + const query = GitHubQueryBuilder.build(filters); + expect(query).toContain('language:typescript'); + }); + + it('3. should group multiple languages with OR', () => { + const filters = { ...baseFilters, languages: ['typescript', 'python'] }; + const query = GitHubQueryBuilder.build(filters); + expect(query).toContain('(language:typescript OR language:python)'); + }); + + it('4. should use user context languages as fallback', () => { + const query = GitHubQueryBuilder.build(baseFilters, { topLanguages: ['javascript', 'go'], inferredLevel: 'beginner' }); + expect(query).toContain('(language:javascript OR language:go)'); + }); + + it('5. should handle small repo size stars filter', () => { + const filters = { ...baseFilters, repoSize: 'small' as const }; + const query = GitHubQueryBuilder.build(filters); + expect(query).toContain('stars:1..999'); + }); + + it('6. should handle medium repo size stars filter', () => { + const filters = { ...baseFilters, repoSize: 'medium' as const }; + const query = GitHubQueryBuilder.build(filters); + expect(query).toContain('stars:1000..10000'); + }); + + it('7. should handle large repo size stars filter', () => { + const filters = { ...baseFilters, repoSize: 'large' as const }; + const query = GitHubQueryBuilder.build(filters); + expect(query).toContain('stars:10001..100000'); + }); + + it('8. should append good-first-issues filter for beginner difficulty', () => { + const filters = { ...baseFilters, difficulty: 'beginner' as const }; + const query = GitHubQueryBuilder.build(filters); + expect(query).toContain('good-first-issues:>0'); + }); + + it('9. should append help-wanted-issues filter for intermediate difficulty', () => { + const filters = { ...baseFilters, difficulty: 'intermediate' as const }; + const query = GitHubQueryBuilder.build(filters); + expect(query).toContain('help-wanted-issues:>0'); + }); + + it('10. should infer beginner level from user context', () => { + const query = GitHubQueryBuilder.build(baseFilters, { topLanguages: [], inferredLevel: 'beginner' }); + expect(query).toContain('good-first-issues:>0'); + }); + + it('11. should infer intermediate level from user context', () => { + const query = GitHubQueryBuilder.build(baseFilters, { topLanguages: [], inferredLevel: 'intermediate' }); + expect(query).toContain('help-wanted-issues:>0'); + }); + + it('12. should handle single domain mapped to topics', () => { + const filters = { ...baseFilters, domains: ['devtools' as const] }; + const query = GitHubQueryBuilder.build(filters); + expect(query).toContain('(cli OR developer-tools OR devtools OR terminal)'); + }); + + it('13. should handle multiple domains mapped to topics', () => { + const filters = { ...baseFilters, domains: ['devtools' as const, 'ai_ml' as const] }; + const query = GitHubQueryBuilder.build(filters); + expect(query).toContain('cli'); + expect(query).toContain('machine-learning'); + }); + + it('14. should append single license filter', () => { + const filters = { ...baseFilters, licenseTypes: ['MIT'] }; + const query = GitHubQueryBuilder.build(filters); + expect(query).toContain('license:mit'); + }); + + it('15. should append multiple license filters grouped with OR', () => { + const filters = { ...baseFilters, licenseTypes: ['MIT', 'Apache-2.0'] }; + const query = GitHubQueryBuilder.build(filters); + expect(query).toContain('(mit OR apache-2.0)'); + }); + + it('16. should prepend keywords to query', () => { + const query = GitHubQueryBuilder.build(baseFilters, undefined, ['cli-tool', 'rust helper']); + expect(query.startsWith('cli-tool "rust helper"')).toBe(true); + }); + + it('17. should combine all filters correctly', () => { + const filters = { + ...baseFilters, + languages: ['typescript'], + difficulty: 'beginner' as const, + domains: ['web' as const], + repoSize: 'small' as const, + licenseTypes: ['MIT'], + }; + const query = GitHubQueryBuilder.build(filters, undefined, ['editor']); + expect(query).toContain('editor'); + expect(query).toContain('language:typescript'); + expect(query).toContain('good-first-issues:>0'); + expect(query).toContain('stars:1..999'); + expect(query).toContain('web'); + expect(query).toContain('license:mit'); + }); +}); diff --git a/apps/backend/src/os-finder/github-query-builder.ts b/apps/backend/src/os-finder/github-query-builder.ts new file mode 100644 index 0000000..787411a --- /dev/null +++ b/apps/backend/src/os-finder/github-query-builder.ts @@ -0,0 +1,109 @@ +import { OsFinderFilters } from '../../packages/shared-types/os-finder.types'; + +export class GitHubQueryBuilder { + static build( + filters: OsFinderFilters, + userCtx?: { topLanguages: string[]; inferredLevel: string }, + keywords?: string[] + ): string { + const parts: string[] = []; + + // 1. Language filter + const langs = filters.languages && filters.languages.length > 0 + ? filters.languages + : (userCtx?.topLanguages || []); + + if (langs.length > 0) { + const langQueries = langs.map(lang => `language:${lang}`); + if (langQueries.length === 1) { + parts.push(langQueries[0]); + } else { + parts.push(`(${langQueries.join(' OR ')})`); + } + } + + // 2. Exclude archived + parts.push('archived:false'); + + // 3. Last commit days (activity) + if (filters.lastCommitDays) { + const date = new Date(); + date.setDate(date.getDate() - filters.lastCommitDays); + const YYYYMMDD = date.toISOString().slice(0, 10); + parts.push(`pushed:>=${YYYYMMDD}`); + } + + // 4. Repo size (stars) + if (filters.repoSize && filters.repoSize !== 'any') { + const STAR_RANGE_MAP = { + small: '1..999', + medium: '1000..10000', + large: '10001..100000', + }; + const range = STAR_RANGE_MAP[filters.repoSize as keyof typeof STAR_RANGE_MAP]; + if (range) { + parts.push(`stars:${range}`); + } + } + + // 5. Difficulty issue filters + const diff = filters.difficulty || userCtx?.inferredLevel; + if (diff === 'beginner') { + parts.push('good-first-issues:>0'); + } else if (diff === 'intermediate') { + parts.push('help-wanted-issues:>0'); + } + + // 6. Domains / Topics + if (filters.domains && filters.domains.length > 0) { + const DOMAIN_TOPIC_MAP: Record = { + web: ['web', 'frontend', 'backend', 'react', 'nextjs', 'nodejs'], + devtools: ['cli', 'developer-tools', 'devtools', 'terminal'], + ai_ml: ['machine-learning', 'artificial-intelligence', 'llm'], + mobile: ['mobile', 'react-native', 'flutter', 'ios', 'android'], + data: ['data-science', 'analytics', 'pandas', 'visualization'], + infrastructure: ['devops', 'docker', 'kubernetes', 'ci-cd', 'cloud'], + education: ['education', 'learning', 'tutorial', 'course'], + games: ['game', 'game-development', 'gamedev'], + finance: ['finance', 'fintech', 'trading', 'blockchain'], + }; + + const topics: string[] = []; + filters.domains.forEach(domain => { + const mapped = DOMAIN_TOPIC_MAP[domain]; + if (mapped) { + topics.push(...mapped); + } + }); + + if (topics.length > 0) { + if (topics.length === 1) { + parts.push(`topic:${topics[0]}`); + } else { + parts.push(`(${topics.join(' OR ')})`); + } + } + } + + // 7. License types + if (filters.licenseTypes && filters.licenseTypes.length > 0) { + const licenses = filters.licenseTypes.map(lic => lic.toLowerCase()); + if (licenses.length === 1) { + parts.push(`license:${licenses[0]}`); + } else { + parts.push(`(${licenses.join(' OR ')})`); + } + } + + // 8. Keywords + if (keywords && keywords.length > 0) { + // Escape spaces or quote them if needed, or join simply + const kw = keywords.filter(Boolean).map(k => k.includes(' ') ? `"${k}"` : k); + if (kw.length > 0) { + parts.unshift(...kw); + } + } + + return parts.join(' '); + } +} diff --git a/apps/backend/src/os-finder/ncf-scorer.service.spec.ts b/apps/backend/src/os-finder/ncf-scorer.service.spec.ts new file mode 100644 index 0000000..e8ef569 --- /dev/null +++ b/apps/backend/src/os-finder/ncf-scorer.service.spec.ts @@ -0,0 +1,163 @@ +import { Test, TestingModule } from '@nestjs/testing'; +import { NcfScorerService } from './ncf-scorer.service'; +import { OsFinderCacheService } from './os-finder-cache.service'; + +describe('NcfScorerService', () => { + let service: NcfScorerService; + let cacheService: OsFinderCacheService; + + const mockCacheService = { + getRepoIssues: jest.fn(), + setRepoIssues: jest.fn(), + getPRStats: jest.fn(), + setPRStats: jest.fn(), + }; + + beforeEach(async () => { + const module: TestingModule = await Test.createTestingModule({ + providers: [ + NcfScorerService, + { provide: OsFinderCacheService, useValue: mockCacheService }, + ], + }).compile(); + + service = module.get(NcfScorerService); + cacheService = module.get(OsFinderCacheService); + }); + + afterEach(() => { + jest.clearAllMocks(); + if ((global.fetch as any).mockRestore) { + (global.fetch as any).mockRestore(); + } + }); + + it('should be defined', () => { + expect(service).toBeDefined(); + }); + + it('should compute full NCF score when all indicators are optimal', async () => { + mockCacheService.getRepoIssues.mockResolvedValue(null); + mockCacheService.getPRStats.mockResolvedValue({ mergeRate: 80 }); + + const freshDate = new Date(); + freshDate.setDate(freshDate.getDate() - 5); + + // Mock global fetch + const fetchSpy = jest.spyOn(global, 'fetch').mockImplementation((url: any) => { + let data: any = {}; + if (url.includes('/issues?labels=good first issue')) { + data = [{ id: 1, updated_at: freshDate.toISOString() }]; + } else if (url.includes('/issues?labels=help wanted')) { + data = [{ id: 2 }]; + } else if (url.includes('/community/profile')) { + data = { + files: { + contributing: { html_url: 'contrib' }, + code_of_conduct: { html_url: 'coc' }, + }, + }; + } else if (url.includes('/readme')) { + data = { size: 5000 }; + } else if (url.includes('/pulls')) { + data = [ + { + merged_at: freshDate.toISOString(), + author_association: 'FIRST_TIME_CONTRIBUTOR', + }, + ]; + } else if (url.includes('/issues?state=closed')) { + data = [ + { + created_at: freshDate.toISOString(), + closed_at: freshDate.toISOString(), + }, + ]; + } + + return Promise.resolve({ + ok: true, + status: 200, + headers: new Map([ + ['X-RateLimit-Remaining', '1000'], + ['X-RateLimit-Reset', '1234567'], + ]), + json: () => Promise.resolve(data), + } as any); + }); + + const breakdown = await service.computeNCFScore('owner', 'repo', 'token', {}); + + expect(breakdown.goodFirstIssue).toBe(2.5); + expect(breakdown.helpWanted).toBe(1.0); + expect(breakdown.contributingFile).toBe(1.5); + expect(breakdown.codeOfConduct).toBe(0.5); + expect(breakdown.readmeQuality).toBe(0.5); + expect(breakdown.prMergeRate).toBe(0.5); + expect(breakdown.issueResponseTime).toBe(2.0); + expect(breakdown.newContribPR).toBe(1.5); + expect(breakdown.total).toBe(10.0); + }); + + it('should compute low NCF score when all indicators are absent or poor', async () => { + mockCacheService.getRepoIssues.mockResolvedValue(null); + mockCacheService.getPRStats.mockResolvedValue({ mergeRate: 5 }); + + const oldDate = new Date(); + oldDate.setDate(oldDate.getDate() - 100); + + const fetchSpy = jest.spyOn(global, 'fetch').mockImplementation((url: any) => { + let data: any = {}; + if (url.includes('/issues?labels=good first issue')) { + data = []; + } else if (url.includes('/issues?labels=help wanted')) { + data = []; + } else if (url.includes('/community/profile')) { + data = { + files: { + contributing: null, + code_of_conduct: null, + }, + }; + } else if (url.includes('/readme')) { + data = { size: 100 }; + } else if (url.includes('/pulls')) { + data = []; + } else if (url.includes('/issues?state=closed')) { + // slow close time: 40 days + const start = new Date(); + start.setDate(start.getDate() - 50); + const end = new Date(); + end.setDate(end.getDate() - 10); + data = [ + { + created_at: start.toISOString(), + closed_at: end.toISOString(), + }, + ]; + } + + return Promise.resolve({ + ok: true, + status: 200, + headers: new Map([ + ['X-RateLimit-Remaining', '1000'], + ['X-RateLimit-Reset', '1234567'], + ]), + json: () => Promise.resolve(data), + } as any); + }); + + const breakdown = await service.computeNCFScore('owner', 'repo', 'token', {}); + + expect(breakdown.goodFirstIssue).toBe(0); + expect(breakdown.helpWanted).toBe(0); + expect(breakdown.contributingFile).toBe(0); + expect(breakdown.codeOfConduct).toBe(0); + expect(breakdown.readmeQuality).toBe(0); + expect(breakdown.prMergeRate).toBe(0); + expect(breakdown.issueResponseTime).toBe(0); // slow avgDaysToClose (40 days) => 0 points + expect(breakdown.newContribPR).toBe(0); + expect(breakdown.total).toBe(1.0); // base score of 1.0 minimum + }); +}); diff --git a/apps/backend/src/os-finder/ncf-scorer.service.ts b/apps/backend/src/os-finder/ncf-scorer.service.ts new file mode 100644 index 0000000..977af7c --- /dev/null +++ b/apps/backend/src/os-finder/ncf-scorer.service.ts @@ -0,0 +1,229 @@ +import { Injectable, Logger } from '@nestjs/common'; +import { OsFinderCacheService } from './os-finder-cache.service'; +import { NCFScoreBreakdown } from '../../packages/shared-types/os-finder.types'; +import { GithubRateLimitError } from './repo-health.service'; + +@Injectable() +export class NcfScorerService { + private readonly logger = new Logger(NcfScorerService.name); + private readonly GITHUB_API_BASE = 'https://api.github.com'; + + constructor(private readonly cacheService: OsFinderCacheService) {} + + private async fetchGithub(url: string, token: string): Promise { + const response = await fetch(url, { + headers: { + Authorization: `Bearer ${token}`, + Accept: 'application/vnd.github+json', + 'User-Agent': 'DevPulse Backend', + }, + }); + + const remaining = response.headers.get('X-RateLimit-Remaining'); + const reset = response.headers.get('X-RateLimit-Reset'); + + if (remaining) { + const remainingVal = parseInt(remaining, 10); + if (remainingVal < 50) { + const resetTime = reset ? parseInt(reset, 10) * 1000 : Date.now() + 60000; + this.logger.warn(`GitHub API Rate limit low during NCF score compute: ${remainingVal} remaining. Reset in ${Math.round((resetTime - Date.now()) / 1000)}s`); + throw new GithubRateLimitError(resetTime); + } + } + + if (!response.ok) { + if (response.status === 404) { + return null; + } + throw new Error(`GitHub API NCF fetch failed: ${response.status} ${response.statusText} on ${url}`); + } + + return response.json(); + } + + async computeNCFScore( + owner: string, + repo: string, + token: string, + repoDetails: any + ): Promise { + const breakdown: NCFScoreBreakdown = { + total: 1.0, // base score + goodFirstIssue: 0, + helpWanted: 0, + contributingFile: 0, + issueResponseTime: 0, + newContribPR: 0, + readmeQuality: 0, + codeOfConduct: 0, + prMergeRate: 0, + }; + + try { + let score = 0; + + // 1. Open and Fresh 'good first issue' labels (weight 25% = 2.5 pts) + let goodFirstIssues = await this.cacheService.getRepoIssues(owner, repo, ['good first issue']); + if (!goodFirstIssues) { + goodFirstIssues = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/issues?labels=good first issue&state=open&per_page=10`, + token + ); + if (goodFirstIssues) { + await this.cacheService.setRepoIssues(owner, repo, ['good first issue'], goodFirstIssues); + } + } + + if (Array.isArray(goodFirstIssues) && goodFirstIssues.length > 0) { + // Check for freshness: updated within 60 days + const sixtyDaysAgo = Date.now() - 60 * 24 * 60 * 60 * 1000; + const hasFresh = goodFirstIssues.some( + issue => !issue.pull_request && new Date(issue.updated_at).getTime() > sixtyDaysAgo + ); + if (hasFresh) { + breakdown.goodFirstIssue = 2.5; + score += 2.5; + } + } + + // 2. Has 'help wanted' labels (weight 10% = 1.0 pt) + let helpWanted = await this.cacheService.getRepoIssues(owner, repo, ['help wanted']); + if (!helpWanted) { + helpWanted = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/issues?labels=help wanted&state=open&per_page=5`, + token + ); + if (helpWanted) { + await this.cacheService.setRepoIssues(owner, repo, ['help wanted'], helpWanted); + } + } + + if (Array.isArray(helpWanted) && helpWanted.length > 0) { + breakdown.helpWanted = 1.0; + score += 1.0; + } + + // Fetch Community Profile & README metadata + const communityProfile = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/community/profile`, + token + ); + + if (communityProfile) { + // 3. CONTRIBUTING file (weight 15% = 1.5 pts) + if (communityProfile.files?.contributing) { + breakdown.contributingFile = 1.5; + score += 1.5; + } + + // 7. Code of Conduct present (weight 5% = 0.5 pts) + if (communityProfile.files?.code_of_conduct) { + breakdown.codeOfConduct = 0.5; + score += 0.5; + } + } + + // 6. README quality: check word count proxy via README file size (weight 5% = 0.5 pts) + const readmeMeta = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/readme`, + token + ); + if (readmeMeta && readmeMeta.size > 2000) { + breakdown.readmeQuality = 0.5; + score += 0.5; + } + + // 8. PR Merge Rate (weight 5% = 0.5 pts) + const prStatsCache = await this.cacheService.getPRStats(owner, repo); + let prStats = prStatsCache; + if (!prStats) { + const closedPRs = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/pulls?state=closed&per_page=30`, + token + ); + if (Array.isArray(closedPRs) && closedPRs.length > 0) { + const mergedCount = closedPRs.filter(pr => pr.merged_at).length; + const mergeRate = (mergedCount / closedPRs.length) * 100; + prStats = { mergeRate }; + await this.cacheService.setPRStats(owner, repo, prStats); + } else { + prStats = { mergeRate: 100 }; + } + } + + if (prStats.mergeRate > 50) { + breakdown.prMergeRate = 0.5; + score += 0.5; + } else if (prStats.mergeRate > 30) { + breakdown.prMergeRate = 0.25; + score += 0.25; + } + + // 4. Avg issue response time proxy: close time of closed issues (weight 20% = 2.0 pts) + const closedIssues = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/issues?state=closed&per_page=20`, + token + ); + + let avgCloseDays = 999; + if (Array.isArray(closedIssues) && closedIssues.length > 0) { + const cleanIssues = closedIssues.filter(item => !item.pull_request); + if (cleanIssues.length > 0) { + let totalCloseDays = 0; + cleanIssues.forEach(issue => { + const created = new Date(issue.created_at).getTime(); + const closed = new Date(issue.closed_at || Date.now()).getTime(); + totalCloseDays += (closed - created) / (1000 * 60 * 60 * 24); + }); + avgCloseDays = totalCloseDays / cleanIssues.length; + } + } + + if (avgCloseDays < 3) { + breakdown.issueResponseTime = 2.0; + score += 2.0; + } else if (avgCloseDays < 7) { + breakdown.issueResponseTime = 1.5; + score += 1.5; + } else if (avgCloseDays < 14) { + breakdown.issueResponseTime = 1.0; + score += 1.0; + } else if (avgCloseDays < 30) { + breakdown.issueResponseTime = 0.5; + score += 0.5; + } + + // 5. Recent merged PR from first time contributor (weight 15% = 1.5 pts) + const closedPRs = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/pulls?state=closed&per_page=20`, + token + ); + + if (Array.isArray(closedPRs) && closedPRs.length > 0) { + const sixtyDaysAgo = Date.now() - 60 * 24 * 60 * 60 * 1000; + const newContribPR = closedPRs.find(pr => { + if (!pr.merged_at) return false; + const mergedTime = new Date(pr.merged_at).getTime(); + if (mergedTime < sixtyDaysAgo) return false; + // check association + return pr.author_association === 'FIRST_TIME_CONTRIBUTOR' || pr.author_association === 'NONE'; + }); + + if (newContribPR) { + breakdown.newContribPR = 1.5; + score += 1.5; + } + } + + // Scale final score: base score of 1.0 up to a maximum of 10.0 + breakdown.total = Math.max(1.0, Math.min(score, 10.0)); + } catch (error) { + if (error instanceof GithubRateLimitError) { + throw error; + } + this.logger.error(`Error computing NCF score for ${owner}/${repo}: ${error instanceof Error ? error.message : String(error)}`); + } + + return breakdown; + } +} diff --git a/apps/backend/src/os-finder/os-finder-cache.service.ts b/apps/backend/src/os-finder/os-finder-cache.service.ts new file mode 100644 index 0000000..f58a8f6 --- /dev/null +++ b/apps/backend/src/os-finder/os-finder-cache.service.ts @@ -0,0 +1,96 @@ +import { Injectable, Logger } from '@nestjs/common'; +import { CacheService } from '../common/cache/cache.service'; +import * as crypto from 'crypto'; + +@Injectable() +export class OsFinderCacheService { + private readonly logger = new Logger(OsFinderCacheService.name); + + constructor(private readonly cacheService: CacheService) {} + + private md5(val: string): string { + return crypto.createHash('md5').update(val).digest('hex'); + } + + // 1. Search Results Cache + async getSearchResults(userId: string, filters: any): Promise { + const sortedFilters = this.sortObjectKeys(filters); + const filterHash = this.md5(JSON.stringify(sortedFilters)); + const key = `os_finder:${userId}:${filterHash}`; + return this.cacheService.get(key); + } + + async setSearchResults(userId: string, filters: any, results: any): Promise { + const sortedFilters = this.sortObjectKeys(filters); + const filterHash = this.md5(JSON.stringify(sortedFilters)); + const key = `os_finder:${userId}:${filterHash}`; + await this.cacheService.set(key, results, 1800); // 30 minutes TTL + } + + // 2. Repo Health Flags Cache + async getRepoHealth(owner: string, repo: string): Promise { + const key = `gh_repo_health:${owner.toLowerCase()}/${repo.toLowerCase()}`; + return this.cacheService.get(key); + } + + async setRepoHealth(owner: string, repo: string, health: any): Promise { + const key = `gh_repo_health:${owner.toLowerCase()}/${repo.toLowerCase()}`; + await this.cacheService.set(key, health, 14400); // 4 hours TTL + } + + // 3. Issues List Cache + async getRepoIssues(owner: string, repo: string, labels: string[]): Promise { + const labelHash = this.md5(labels.sort().join(',')); + const key = `gh_repo_issues:${owner.toLowerCase()}/${repo.toLowerCase()}:${labelHash}`; + return this.cacheService.get(key); + } + + async setRepoIssues(owner: string, repo: string, labels: string[], issues: any[]): Promise { + const labelHash = this.md5(labels.sort().join(',')); + const key = `gh_repo_issues:${owner.toLowerCase()}/${repo.toLowerCase()}:${labelHash}`; + await this.cacheService.set(key, issues, 3600); // 1 hour TTL + } + + // 4. PR Stats Cache + async getPRStats(owner: string, repo: string): Promise { + const key = `gh_repo_prstats:${owner.toLowerCase()}/${repo.toLowerCase()}`; + return this.cacheService.get(key); + } + + async setPRStats(owner: string, repo: string, stats: any): Promise { + const key = `gh_repo_prstats:${owner.toLowerCase()}/${repo.toLowerCase()}`; + await this.cacheService.set(key, stats, 21600); // 6 hours TTL + } + + // 5. Watchlist / Saved Repos Cache + async getSavedRepos(userId: string): Promise { + const key = `os_finder_saved:${userId}`; + return this.cacheService.get(key); + } + + async setSavedRepos(userId: string, saved: any[]): Promise { + const key = `os_finder_saved:${userId}`; + await this.cacheService.set(key, saved, 300); // 5 minutes TTL + } + + async invalidateSavedRepos(userId: string): Promise { + const key = `os_finder_saved:${userId}`; + await this.cacheService.delete(key); + } + + // Utility to recursively sort object keys to ensure consistent hashes + private sortObjectKeys(obj: any): any { + if (obj === null || typeof obj !== 'object') { + return obj; + } + if (Array.isArray(obj)) { + return obj.map(item => this.sortObjectKeys(item)); + } + const sortedKeys = Object.keys(obj).sort(); + const result: any = {}; + sortedKeys.forEach(key => { + result[key] = this.sortObjectKeys(obj[key]); + }); + return result; + } +} diff --git a/apps/backend/src/os-finder/os-finder.controller.ts b/apps/backend/src/os-finder/os-finder.controller.ts new file mode 100644 index 0000000..a372edf --- /dev/null +++ b/apps/backend/src/os-finder/os-finder.controller.ts @@ -0,0 +1,109 @@ +import { + Controller, + Get, + Post, + Patch, + Delete, + Body, + Query, + Param, + UseGuards, + HttpCode, + HttpStatus, + Logger, +} from '@nestjs/common'; +import { JwtAuthGuard } from '../common/guards/jwt.guard'; +import { CurrentUser } from '../common/decorators/current-user.decorator'; +import { User } from '../users/entities/user.entity'; +import { OsFinderService } from './os-finder.service'; +import { SearchQueryDto } from './dto/search-query.dto'; +import { SaveRepoDto } from './dto/save-repo.dto'; +import { UpdateSavedRepoDto } from './dto/update-saved-repo.dto'; + +@Controller('os-finder') +@UseGuards(JwtAuthGuard) +export class OsFinderController { + private readonly logger = new Logger(OsFinderController.name); + + constructor(private readonly osFinderService: OsFinderService) {} + + @Get('search') + async search( + @CurrentUser('sub') userId: string, + @Query() queryDto: SearchQueryDto, + ) { + this.logger.log(`User ${userId} initiated a standard OS Finder search`); + return this.osFinderService.search(userId, queryDto); + } + + @Post('search/ai') + @HttpCode(HttpStatus.OK) + async searchAi( + @CurrentUser('sub') userId: string, + @Body() body: { query: string }, + ) { + this.logger.log(`User ${userId} initiated an AI-powered OS Finder search with query: "${body.query}"`); + return this.osFinderService.searchAi(userId, body); + } + + @Get('repo/:owner/:repo') + async getRepoDetail( + @CurrentUser('sub') userId: string, + @Param('owner') owner: string, + @Param('repo') repo: string, + ) { + this.logger.log(`User ${userId} requested repo details for ${owner}/${repo}`); + return this.osFinderService.getRepoDetail(userId, owner, repo); + } + + @Get('repo/:owner/:repo/issues') + async getRepoIssues( + @CurrentUser('sub') userId: string, + @Param('owner') owner: string, + @Param('repo') repo: string, + ) { + this.logger.log(`User ${userId} requested open issues for ${owner}/${repo}`); + return this.osFinderService.getRepoIssues(userId, owner, repo); + } + + @Post('saved') + async saveRepo( + @CurrentUser('sub') userId: string, + @Body() saveRepoDto: SaveRepoDto, + ) { + this.logger.log(`User ${userId} is saving/updating repo ${saveRepoDto.fullName} in watchlist`); + return this.osFinderService.saveRepo(userId, saveRepoDto); + } + + @Get('saved') + async getSavedRepos(@CurrentUser('sub') userId: string) { + this.logger.log(`User ${userId} requested saved repos watchlist`); + return this.osFinderService.getSavedRepos(userId); + } + + @Patch('saved/:id') + async updateSavedRepo( + @CurrentUser('sub') userId: string, + @Param('id') id: string, + @Body() updateSavedRepoDto: UpdateSavedRepoDto, + ) { + this.logger.log(`User ${userId} is updating saved repo status/notes for item ${id}`); + return this.osFinderService.updateSavedRepo(userId, id, updateSavedRepoDto); + } + + @Delete('saved/:id') + @HttpCode(HttpStatus.NO_CONTENT) + async deleteSavedRepo( + @CurrentUser('sub') userId: string, + @Param('id') id: string, + ) { + this.logger.log(`User ${userId} is removing repo item ${id} from watchlist`); + await this.osFinderService.deleteSavedRepo(userId, id); + } + + @Get('history') + async getSearchHistory(@CurrentUser('sub') userId: string) { + this.logger.log(`User ${userId} requested search history`); + return this.osFinderService.getSearchHistory(userId); + } +} diff --git a/apps/backend/src/os-finder/os-finder.module.ts b/apps/backend/src/os-finder/os-finder.module.ts new file mode 100644 index 0000000..12f5fb9 --- /dev/null +++ b/apps/backend/src/os-finder/os-finder.module.ts @@ -0,0 +1,39 @@ +import { Module } from '@nestjs/common'; +import { TypeOrmModule } from '@nestjs/typeorm'; +import { CacheModule } from '../common/cache/cache.module'; +import { User } from '../users/entities/user.entity'; +import { Commit } from '../github-sync/entities/commit.entity'; +import { PullRequest } from '../github-sync/entities/pull-request.entity'; +import { AiService } from '../shared/ai.service'; +import { SavedRepo } from './entities/saved-repo.entity'; +import { OsFinderSearch } from './entities/os-finder-search.entity'; +import { OsFinderController } from './os-finder.controller'; +import { OsFinderService } from './os-finder.service'; +import { OsFinderCacheService } from './os-finder-cache.service'; +import { RepoHealthService } from './repo-health.service'; +import { NcfScorerService } from './ncf-scorer.service'; +import { AiQueryBuilderService } from './ai-query-builder.service'; + +@Module({ + imports: [ + CacheModule, + TypeOrmModule.forFeature([ + User, + SavedRepo, + OsFinderSearch, + Commit, + PullRequest, + ]), + ], + controllers: [OsFinderController], + providers: [ + OsFinderService, + OsFinderCacheService, + RepoHealthService, + NcfScorerService, + AiQueryBuilderService, + AiService, + ], + exports: [OsFinderService], +}) +export class OsFinderModule {} diff --git a/apps/backend/src/os-finder/os-finder.service.ts b/apps/backend/src/os-finder/os-finder.service.ts new file mode 100644 index 0000000..2275f61 --- /dev/null +++ b/apps/backend/src/os-finder/os-finder.service.ts @@ -0,0 +1,770 @@ +import { Injectable, Logger, BadRequestException, NotFoundException } from '@nestjs/common'; +import { InjectRepository } from '@nestjs/typeorm'; +import { Repository as TypeOrmRepository, Not, IsNull } from 'typeorm'; +import { User } from '../users/entities/user.entity'; +import { SavedRepo } from './entities/saved-repo.entity'; +import { OsFinderSearch } from './entities/os-finder-search.entity'; +import { Commit } from '../github-sync/entities/commit.entity'; +import { PullRequest } from '../github-sync/entities/pull-request.entity'; +import { OsFinderCacheService } from './os-finder-cache.service'; +import { RepoHealthService, GithubRateLimitError } from './repo-health.service'; +import { NcfScorerService } from './ncf-scorer.service'; +import { AiQueryBuilderService } from './ai-query-builder.service'; +import { GitHubQueryBuilder } from './github-query-builder'; +import { decryptToken } from '../common/utils/crypto.util'; +import { + OsFinderFilters, + OsFinderRepoResult, + OsFinderSearchResponse, + NCFScoreBreakdown, + RepoHealthFlags, + SavedRepoStatus +} from '../../packages/shared-types/os-finder.types'; +import { SearchQueryDto } from './dto/search-query.dto'; +import { SaveRepoDto } from './dto/save-repo.dto'; +import { UpdateSavedRepoDto } from './dto/update-saved-repo.dto'; + +@Injectable() +export class OsFinderService { + private readonly logger = new Logger(OsFinderService.name); + private readonly GITHUB_API_BASE = 'https://api.github.com'; + + constructor( + @InjectRepository(User) + private readonly userRepository: TypeOrmRepository, + @InjectRepository(SavedRepo) + private readonly savedRepoRepository: TypeOrmRepository, + @InjectRepository(OsFinderSearch) + private readonly searchRepository: TypeOrmRepository, + @InjectRepository(Commit) + private readonly commitRepository: TypeOrmRepository, + @InjectRepository(PullRequest) + private readonly pullRequestRepository: TypeOrmRepository, + private readonly cacheService: OsFinderCacheService, + private readonly repoHealthService: RepoHealthService, + private readonly ncfScorerService: NcfScorerService, + private readonly aiQueryBuilderService: AiQueryBuilderService, + ) {} + + private async fetchGithub(url: string, token: string): Promise { + const response = await fetch(url, { + headers: { + Authorization: `Bearer ${token}`, + Accept: 'application/vnd.github+json', + 'User-Agent': 'DevPulse Backend', + }, + }); + + if (response.status === 403 || response.status === 429) { + const remaining = response.headers.get('X-RateLimit-Remaining'); + const reset = response.headers.get('X-RateLimit-Reset'); + if (remaining && parseInt(remaining, 10) < 50) { + const resetTime = reset ? parseInt(reset, 10) * 1000 : Date.now() + 60000; + throw new GithubRateLimitError(resetTime); + } + } + + if (!response.ok) { + throw new Error(`GitHub search failure: ${response.status} ${response.statusText} on ${url}`); + } + + return response.json(); + } + + private async fetchGithubWithLanguages( + filters: OsFinderFilters, + userCtx: any, + keywords?: string[] + ): Promise<{ items: any[]; total_count: number }> { + const langs = filters.languages && filters.languages.length > 0 + ? filters.languages + : []; + + if (langs.length === 0) { + const query = GitHubQueryBuilder.build({ ...filters, languages: [] }, userCtx, keywords); + const res = await this.fetchGithub( + `${this.GITHUB_API_BASE}/search/repositories?q=${encodeURIComponent(query)}&sort=updated&per_page=30`, + userCtx.githubToken + ); + return { + items: res?.items || [], + total_count: res?.total_count || 0 + }; + } + + const targetLangs = langs.slice(0, 3); + const promises = targetLangs.map(async (lang) => { + const query = GitHubQueryBuilder.build({ ...filters, languages: [lang] }, userCtx, keywords); + try { + const res = await this.fetchGithub( + `${this.GITHUB_API_BASE}/search/repositories?q=${encodeURIComponent(query)}&sort=updated&per_page=30`, + userCtx.githubToken + ); + return { + items: res?.items || [], + total_count: res?.total_count || 0 + }; + } catch (err) { + if (err instanceof GithubRateLimitError) { + throw err; + } + this.logger.error(`Failed github search API query for language ${lang}: ${err instanceof Error ? err.message : String(err)}`); + return { items: [], total_count: 0 }; + } + }); + + const results = await Promise.all(promises); + + const seenIds = new Set(); + const mergedItems: any[] = []; + let totalCount = 0; + + for (const res of results) { + if (res && res.items) { + totalCount += res.total_count || 0; + for (const item of res.items) { + if (!seenIds.has(item.id)) { + seenIds.add(item.id); + mergedItems.push(item); + } + } + } + } + + return { + items: mergedItems.slice(0, 30), + total_count: totalCount, + }; + } + + // Step 2: Load user context (languages + avg PR score + inferred difficulty) + async loadUserContext(userId: string): Promise<{ + topLanguages: string[]; + inferredLevel: string; + avgPRScore: number; + githubToken: string; + githubUsername: string; + }> { + const user = await this.userRepository.findOne({ where: { id: userId } }); + if (!user) { + throw new BadRequestException('User not found'); + } + + const token = decryptToken(user.githubToken); + + // Get languages from commits + const commitLangs = await this.commitRepository + .createQueryBuilder('commit') + .innerJoinAndSelect('commit.repository', 'repository') + .where('commit.userId = :userId', { userId }) + .andWhere('repository.language IS NOT NULL') + .select('repository.language', 'language') + .addSelect('COUNT(commit.id)', 'count') + .groupBy('repository.language') + .orderBy('count', 'DESC') + .limit(3) + .getRawMany(); + + const topLanguages = commitLangs.map(cl => cl.language); + + // Inferred experience level and PR scores + const prs = await this.pullRequestRepository.find({ + where: { userId, prScore: Not(IsNull()) }, + }); + const avgPRScore = prs.length > 0 ? prs.reduce((sum, pr) => sum + pr.prScore, 0) / prs.length : 0; + + // commits history in days + const earliestCommit = await this.commitRepository.findOne({ + where: { userId }, + order: { committedAt: 'ASC' }, + }); + const commitHistoryDays = earliestCommit + ? (Date.now() - new Date(earliestCommit.committedAt).getTime()) / (1000 * 60 * 60 * 24) + : 0; + + let inferredLevel = 'beginner'; + if (commitHistoryDays >= 365 && avgPRScore > 7.5) { + inferredLevel = 'advanced'; + } else if (commitHistoryDays >= 90 && avgPRScore >= 5.0) { + inferredLevel = 'intermediate'; + } + + return { + topLanguages, + inferredLevel, + avgPRScore, + githubToken: token, + githubUsername: user.githubUsername, + }; + } + + // Main 14-step search orchestration + async search(userId: string, queryDto: SearchQueryDto): Promise { + const userCtx = await this.loadUserContext(userId); + + // Setup filter options with DTO values or defaults + const filters: OsFinderFilters = { + languages: queryDto.languages || userCtx.topLanguages, + languageMode: queryDto.languageMode || 'any_of', + difficulty: queryDto.difficulty || (userCtx.inferredLevel as any), + contributionTypes: queryDto.contributionTypes || [], + domains: queryDto.domains || [], + repoSize: queryDto.repoSize || 'any', + lastCommitDays: queryDto.lastCommitDays !== undefined ? queryDto.lastCommitDays : 90, + minOpenIssues: queryDto.minOpenIssues !== undefined ? queryDto.minOpenIssues : 3, + issueFreshDays: queryDto.issueFreshDays !== undefined ? queryDto.issueFreshDays : 60, + hasContributing: queryDto.hasContributing !== undefined ? queryDto.hasContributing : true, + hasCodeOfConduct: queryDto.hasCodeOfConduct !== undefined ? queryDto.hasCodeOfConduct : false, + licenseTypes: queryDto.licenseTypes || [], + prMergeRate: queryDto.prMergeRate !== undefined ? queryDto.prMergeRate : 30, + includeAlreadyContributed: queryDto.includeAlreadyContributed || false, + }; + + // Step 3: Check cache + const cacheHit = await this.cacheService.getSearchResults(userId, filters); + if (cacheHit) { + this.logger.log(`Serving search results from cache for user ${userId}`); + return cacheHit; + } + + // Step 4-5-12: Execute search and handle relaxation loop + let activeFilters = { ...filters }; + let githubQuery = GitHubQueryBuilder.build(activeFilters, userCtx); + let rawResults: any = null; + let filtersRelaxed: Partial | null = null; + let relaxationNote: string | null = null; + + const relaxationOrder = [ + { field: 'lastCommitDays', from: 90, to: 180, message: "last commit window relaxed to 180 days" }, + { field: 'minOpenIssues', from: 3, to: 1, message: "minimum open issues relaxed to 1" }, + { field: 'issueFreshDays', from: 60, to: 180, message: "issue freshness relaxed to 180 days" }, + { field: 'prMergeRate', from: 30, to: 10, message: "PR merge rate threshold relaxed to 10%" }, + { field: 'hasContributing', from: true, to: false, message: "CONTRIBUTING.md requirement removed" }, + ]; + + try { + rawResults = await this.fetchGithubWithLanguages(activeFilters, userCtx); + + // Loop relaxation + if ((!rawResults || rawResults.total_count < 3) && rawResults.items) { + filtersRelaxed = {}; + const notes: string[] = []; + + for (const step of relaxationOrder) { + const field = step.field as keyof OsFinderFilters; + if (activeFilters[field] === step.from) { + (activeFilters as any)[field] = step.to; + (filtersRelaxed as any)[field] = step.to; + notes.push(step.message); + + const retryRes = await this.fetchGithubWithLanguages(activeFilters, userCtx); + + if (retryRes && retryRes.total_count >= 3) { + rawResults = retryRes; + break; + } else if (retryRes) { + rawResults = retryRes; + } + } + } + + if (notes.length > 0) { + relaxationNote = `No repos matched all filters. We relaxed filters: ${notes.join('; ')}.`; + } + } + } catch (err) { + if (err instanceof GithubRateLimitError) { + throw err; + } + this.logger.error(`Failed github search API query: ${err instanceof Error ? err.message : String(err)}`); + rawResults = { items: [], total_count: 0 }; + } + + const items = rawResults?.items || []; + + // Load already-contributed repos + const userPrs = await this.pullRequestRepository + .createQueryBuilder('pr') + .innerJoinAndSelect('pr.repository', 'repository') + .where('pr.userId = :userId', { userId }) + .getMany(); + const userContributed = new Set(userPrs.map(pr => pr.repository?.fullName).filter(Boolean)); + + // Load saved repos watchlist to flag isSaved + const saved = await this.savedRepoRepository.find({ where: { userId } }); + const savedRepoIds = new Set(saved.map(s => Number(s.githubRepoId))); + + // Step 6: Parallel health check processing (Staggered Promise Pool) + const processedResults = await this.processHealthAndNcfInParallel( + items, + userCtx.githubToken, + userCtx.topLanguages, + userContributed, + savedRepoIds + ); + + // Apply filters and exclusions + let results = processedResults; + + // Step 9: Already contributed exclusion + if (!filters.includeAlreadyContributed) { + results = results.filter(r => !r.alreadyContrib); + } + + // Step 10: Overwhelming repos guard (Star > 100k) + // If not advanced+large, push to bottom, or just split + const majorRepos = results.filter(r => r.stars > 100000); + const regularRepos = results.filter(r => r.stars <= 100000); + + // Sort regular repos: primary: (ncfScore * langMatchScore), secondary: lastCommitAt DESC + regularRepos.sort((a, b) => { + const scoreA = (a.ncfScore?.total || 0) * a.langMatchScore; + const scoreB = (b.ncfScore?.total || 0) * b.langMatchScore; + if (scoreA !== scoreB) { + return scoreB - scoreA; + } + return new Date(b.lastCommitAt).getTime() - new Date(a.lastCommitAt).getTime(); + }); + + // Concat regular then major repos + const finalResults = [...regularRepos, ...majorRepos]; + + const response: OsFinderSearchResponse = { + results: finalResults, + total: rawResults.total_count || 0, + page: queryDto.page || 1, + filtersApplied: filters, + filtersRelaxed, + relaxationNote, + aiModeUsed: false, + }; + + // Step 13: Cache results + await this.cacheService.setSearchResults(userId, filters, response); + + // Step 14: Log search asynchronously + this.searchRepository.save({ + userId, + queryText: null, + filtersApplied: filters, + resultCount: finalResults.length, + aiQueryUsed: false, + githubQuery, + }).catch(err => this.logger.error(`Failed to log search query: ${err.message}`)); + + return response; + } + + // AI Search orchestrator + async searchAi(userId: string, body: { query: string }): Promise { + const userCtx = await this.loadUserContext(userId); + + const { filters, keywords, fallbackUsed } = await this.aiQueryBuilderService.buildFilters( + body.query, + userCtx + ); + + const cacheHit = await this.cacheService.getSearchResults(userId, filters); + if (cacheHit) { + cacheHit.aiModeUsed = true; + if (fallbackUsed) { + cacheHit.relaxationNote = "AI query builder unavailable. Using keyword matching instead."; + } + return cacheHit; + } + + let activeFilters = { ...filters }; + let githubQuery = GitHubQueryBuilder.build(activeFilters, userCtx, keywords); + let rawResults: any = null; + let filtersRelaxed: Partial | null = null; + let relaxationNote: string | null = fallbackUsed ? "AI query builder unavailable. Using keyword matching instead." : null; + + try { + rawResults = await this.fetchGithubWithLanguages(activeFilters, userCtx, keywords); + } catch (err) { + if (err instanceof GithubRateLimitError) { + throw err; + } + this.logger.error(`Failed github search AI query: ${err instanceof Error ? err.message : String(err)}`); + rawResults = { items: [], total_count: 0 }; + } + + const items = rawResults?.items || []; + + // Load already-contributed repos + const userPrs = await this.pullRequestRepository + .createQueryBuilder('pr') + .innerJoinAndSelect('pr.repository', 'repository') + .where('pr.userId = :userId', { userId }) + .getMany(); + const userContributed = new Set(userPrs.map(pr => pr.repository?.fullName).filter(Boolean)); + + // Load saved repos + const saved = await this.savedRepoRepository.find({ where: { userId } }); + const savedRepoIds = new Set(saved.map(s => Number(s.githubRepoId))); + + // Staggered promise health + NCF + const processedResults = await this.processHealthAndNcfInParallel( + items, + userCtx.githubToken, + userCtx.topLanguages, + userContributed, + savedRepoIds + ); + + let results = processedResults; + if (!filters.includeAlreadyContributed) { + results = results.filter(r => !r.alreadyContrib); + } + + const majorRepos = results.filter(r => r.stars > 100000); + const regularRepos = results.filter(r => r.stars <= 100000); + + regularRepos.sort((a, b) => { + const scoreA = (a.ncfScore?.total || 0) * a.langMatchScore; + const scoreB = (b.ncfScore?.total || 0) * b.langMatchScore; + if (scoreA !== scoreB) { + return scoreB - scoreA; + } + return new Date(b.lastCommitAt).getTime() - new Date(a.lastCommitAt).getTime(); + }); + + const finalResults = [...regularRepos, ...majorRepos]; + + const response: OsFinderSearchResponse = { + results: finalResults, + total: rawResults.total_count || 0, + page: 1, + filtersApplied: filters, + filtersRelaxed: null, + relaxationNote, + aiModeUsed: true, + }; + + await this.cacheService.setSearchResults(userId, filters, response); + + this.searchRepository.save({ + userId, + queryText: body.query, + filtersApplied: filters, + resultCount: finalResults.length, + aiQueryUsed: true, + githubQuery, + }).catch(err => this.logger.error(`Failed to log search query: ${err.message}`)); + + return response; + } + + // Staggered parallel health + NCF processor (staggered queue) + private async processHealthAndNcfInParallel( + items: any[], + token: string, + topLanguages: string[], + userContributed: Set, + savedRepoIds: Set + ): Promise { + const results: OsFinderRepoResult[] = []; + const activePromises: Promise[] = []; + + for (let i = 0; i < items.length; i++) { + const item = items[i]; + + // Concurrency limit = 10 + while (activePromises.length >= 10) { + await Promise.race(activePromises); + } + + // Stagger start: 50ms wait (prevents burst rate limiting while keeping total time short) + if (i > 0) { + await new Promise(resolve => setTimeout(resolve, 50)); + } + + const promise = (async () => { + const owner = item.owner.login; + const repo = item.name; + + try { + const healthFlags = await this.repoHealthService.computeRepoHealth(owner, repo, token, item); + const ncfScore = await this.ncfScorerService.computeNCFScore(owner, repo, token, item); + + // Language match calculation (weights: Rank 1: 1.0, Rank 2: 0.6, Rank 3: 0.3) + let langMatchScore = 0.0; + if (item.language) { + const index = topLanguages.indexOf(item.language); + if (index === 0) langMatchScore = 1.0; + else if (index === 1) langMatchScore = 0.6; + else if (index === 2) langMatchScore = 0.3; + } + + const alreadyContrib = userContributed.has(item.full_name); + const isSaved = savedRepoIds.has(Number(item.id)); + + return { + githubRepoId: Number(item.id), + owner, + name: repo, + fullName: item.full_name, + description: item.description || null, + language: item.language || null, + stars: item.stargazers_count || 0, + forks: item.forks_count || 0, + openIssues: item.open_issues_count || 0, + lastCommitAt: item.pushed_at || item.updated_at, + licenseType: item.license?.name || item.license?.spdx_id || null, + htmlUrl: item.html_url, + ncfScore, + langMatchScore, + openLabels: ['good first issue', 'help wanted'], // labels we target + healthFlags, + isSaved, + alreadyContrib, + }; + } catch (err) { + this.logger.warn(`Staggered health check failed for ${item.full_name}: ${err instanceof Error ? err.message : String(err)}`); + // Default fallback results if rate limited + return { + githubRepoId: Number(item.id), + owner, + name: repo, + fullName: item.full_name, + description: item.description || null, + language: item.language || null, + stars: item.stargazers_count || 0, + forks: item.forks_count || 0, + openIssues: item.open_issues_count || 0, + lastCommitAt: item.pushed_at || item.updated_at, + licenseType: item.license?.name || item.license?.spdx_id || null, + htmlUrl: item.html_url, + ncfScore: { + total: 0.0, // Health check pending + goodFirstIssue: 0, + helpWanted: 0, + contributingFile: 0, + issueResponseTime: 0, + newContribPR: 0, + readmeQuality: 0, + codeOfConduct: 0, + prMergeRate: 0, + }, + langMatchScore: 0.0, + openLabels: [], + healthFlags: { + isArchived: false, + isStale: false, + noContributing: false, + noReadme: false, + lowPRMergeRate: false, + forkHeavy: false, + noExternalContribs: false, + lowIssueEngagement: false, + slowMaintainerResp: false, + }, + isSaved: savedRepoIds.has(Number(item.id)), + alreadyContrib: userContributed.has(item.full_name), + }; + } + })(); + + activePromises.push(promise); + promise.then(res => { + results.push(res); + const idx = activePromises.indexOf(promise); + if (idx > -1) { + activePromises.splice(idx, 1); + } + }); + } + + await Promise.all(activePromises); + return results; + } + + // GET /os-finder/repo/:owner/:repo + async getRepoDetail(userId: string, owner: string, repo: string): Promise { + const userCtx = await this.loadUserContext(userId); + const token = userCtx.githubToken; + + const detailsUrl = `${this.GITHUB_API_BASE}/repos/${owner}/${repo}`; + const repoDetails = await this.fetchGithub(detailsUrl, token); + if (!repoDetails) { + throw new NotFoundException(`GitHub Repository ${owner}/${repo} not found`); + } + + const healthFlags = await this.repoHealthService.computeRepoHealth(owner, repo, token, repoDetails); + const ncfScore = await this.ncfScorerService.computeNCFScore(owner, repo, token, repoDetails); + + // Fetch top 5 contributors + const contributors = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/contributors?per_page=5`, + token + ).catch(() => []); + + // Fetch last 5 merged PRs + const recentPRs = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/pulls?state=closed&per_page=15`, + token + ).catch(() => []); + const mergedPRs = Array.isArray(recentPRs) ? recentPRs.filter(pr => pr.merged_at).slice(0, 5) : []; + + // Check if saved + const saved = await this.savedRepoRepository.findOne({ + where: { userId, githubRepoId: repoDetails.id }, + }); + + return { + githubRepoId: repoDetails.id, + owner: repoDetails.owner?.login, + name: repoDetails.name, + fullName: repoDetails.full_name, + description: repoDetails.description, + language: repoDetails.language, + stars: repoDetails.stargazers_count, + forks: repoDetails.forks_count, + openIssues: repoDetails.open_issues_count, + lastCommitAt: repoDetails.pushed_at || repoDetails.updated_at, + licenseType: repoDetails.license?.name || repoDetails.license?.spdx_id || null, + htmlUrl: repoDetails.html_url, + ncfScore, + healthFlags, + contributors: Array.isArray(contributors) ? contributors.map(c => ({ login: c.login, avatarUrl: c.avatar_url, contributions: c.contributions })) : [], + recentPRs: mergedPRs.map(pr => ({ id: pr.id, title: pr.title, url: pr.html_url, author: pr.user?.login, authorAvatar: pr.user?.avatar_url, mergedAt: pr.merged_at })), + savedId: saved?.id || null, + notes: saved?.notes || null, + status: saved?.status || null, + }; + } + + // GET /os-finder/repo/:owner/:repo/issues (beginner open issues list) + async getRepoIssues(userId: string, owner: string, repo: string): Promise { + const userCtx = await this.loadUserContext(userId); + // Fetch labels combined + const labels = ['good first issue', 'help wanted']; + + // Check cache + const cacheHit = await this.cacheService.getRepoIssues(owner, repo, labels); + if (cacheHit) { + return cacheHit; + } + + const url = `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/issues?state=open&per_page=30`; + const response = await this.fetchGithub(url, userCtx.githubToken); + + if (Array.isArray(response)) { + // Filter out PRs, and retain ones matching labels + const issues = response.filter(item => { + if (item.pull_request) return false; + const itemLabels = item.labels?.map((l: any) => l.name.toLowerCase()) || []; + return itemLabels.some((l: string) => labels.includes(l)); + }).map(issue => ({ + id: issue.id, + number: issue.number, + title: issue.title, + url: issue.html_url, + createdAt: issue.created_at, + updatedAt: issue.updated_at, + commentsCount: issue.comments, + labels: issue.labels?.map((l: any) => l.name) || [], + })); + + await this.cacheService.setRepoIssues(owner, repo, labels, issues); + return issues; + } + + return []; + } + + // POST /os-finder/saved (watchlist) + async saveRepo(userId: string, dto: SaveRepoDto): Promise { + let saved = await this.savedRepoRepository.findOne({ + where: { userId, githubRepoId: dto.githubRepoId }, + }); + + if (!saved) { + saved = this.savedRepoRepository.create({ + userId, + githubRepoId: dto.githubRepoId, + owner: dto.owner, + name: dto.name, + fullName: dto.fullName, + description: dto.description || null, + language: dto.language || null, + stars: dto.stars || 0, + forks: dto.forks || 0, + openIssues: dto.openIssues || 0, + ncfScore: dto.ncfScore && typeof dto.ncfScore === 'object' ? dto.ncfScore : null, + langMatchScore: dto.langMatchScore || null, + lastCommitAt: dto.lastCommitAt ? new Date(dto.lastCommitAt) : null, + hasContributing: dto.hasContributing || false, + licenseType: dto.licenseType || null, + htmlUrl: dto.htmlUrl, + notes: dto.notes || null, + status: dto.status || 'saved', + }); + } else { + Object.assign(saved, { + notes: dto.notes !== undefined ? dto.notes : saved.notes, + status: dto.status || saved.status, + }); + } + + const res = await this.savedRepoRepository.save(saved); + await this.cacheService.invalidateSavedRepos(userId); + return res; + } + + // GET /os-finder/saved (watchlist) + async getSavedRepos(userId: string): Promise { + // Check cache + const cacheHit = await this.cacheService.getSavedRepos(userId); + if (cacheHit) { + return cacheHit; + } + + const list = await this.savedRepoRepository.find({ + where: { userId }, + order: { savedAt: 'DESC' }, + }); + + await this.cacheService.setSavedRepos(userId, list); + return list; + } + + // PATCH /os-finder/saved/:id + async updateSavedRepo(userId: string, id: string, dto: UpdateSavedRepoDto): Promise { + const saved = await this.savedRepoRepository.findOne({ where: { id, userId } }); + if (!saved) { + throw new NotFoundException(`Saved repo item ${id} not found`); + } + + if (dto.notes !== undefined) { + saved.notes = dto.notes; + } + if (dto.status !== undefined) { + saved.status = dto.status; + } + + const res = await this.savedRepoRepository.save(saved); + await this.cacheService.invalidateSavedRepos(userId); + return res; + } + + // DELETE /os-finder/saved/:id + async deleteSavedRepo(userId: string, id: string): Promise { + const saved = await this.savedRepoRepository.findOne({ where: { id, userId } }); + if (!saved) { + throw new NotFoundException(`Saved repo item ${id} not found`); + } + + await this.savedRepoRepository.remove(saved); + await this.cacheService.invalidateSavedRepos(userId); + } + + // GET /os-finder/history + async getSearchHistory(userId: string): Promise { + return this.searchRepository.find({ + where: { userId }, + order: { createdAt: 'DESC' }, + take: 10, + }); + } +} diff --git a/apps/backend/src/os-finder/repo-health.service.ts b/apps/backend/src/os-finder/repo-health.service.ts new file mode 100644 index 0000000..dfdf9e1 --- /dev/null +++ b/apps/backend/src/os-finder/repo-health.service.ts @@ -0,0 +1,181 @@ +import { Injectable, Logger } from '@nestjs/common'; +import { OsFinderCacheService } from './os-finder-cache.service'; +import { RepoHealthFlags } from '../../packages/shared-types/os-finder.types'; + +export class GithubRateLimitError extends Error { + constructor(public readonly resetTime: number) { + super('GitHub API Rate Limit Threshold reached'); + this.name = 'GithubRateLimitError'; + } +} + +@Injectable() +export class RepoHealthService { + private readonly logger = new Logger(RepoHealthService.name); + private readonly GITHUB_API_BASE = 'https://api.github.com'; + + constructor(private readonly cacheService: OsFinderCacheService) {} + + private async fetchGithub(url: string, token: string): Promise { + const response = await fetch(url, { + headers: { + Authorization: `Bearer ${token}`, + Accept: 'application/vnd.github+json', + 'User-Agent': 'DevPulse Backend', + }, + }); + + const remaining = response.headers.get('X-RateLimit-Remaining'); + const reset = response.headers.get('X-RateLimit-Reset'); + + if (remaining) { + const remainingVal = parseInt(remaining, 10); + if (remainingVal < 50) { + const resetTime = reset ? parseInt(reset, 10) * 1000 : Date.now() + 60000; + this.logger.warn(`GitHub API Rate limit low: ${remainingVal} remaining. Reset in ${Math.round((resetTime - Date.now()) / 1000)}s`); + throw new GithubRateLimitError(resetTime); + } + } + + if (!response.ok) { + if (response.status === 404) { + return null; + } + if (response.status === 202) { + // Contributor stats are compiling, return 202 + return { status: 202 }; + } + throw new Error(`GitHub API call failed: ${response.status} ${response.statusText} on ${url}`); + } + + return response.json(); + } + + async computeRepoHealth( + owner: string, + repo: string, + token: string, + repoDetails: any // passed in to save API calls since we get it from Search API + ): Promise { + const cacheHit = await this.cacheService.getRepoHealth(owner, repo); + if (cacheHit) { + return cacheHit; + } + + const healthFlags: RepoHealthFlags = { + isArchived: false, + isStale: false, + noContributing: false, + noReadme: false, + lowPRMergeRate: false, + forkHeavy: false, + noExternalContribs: false, + lowIssueEngagement: false, + slowMaintainerResp: false, + }; + + try { + // 1. isArchived (from search details) + healthFlags.isArchived = !!repoDetails?.archived; + + // 2. isStale: pushed_at > 180 days ago + const pushedTime = new Date(repoDetails?.pushed_at || repoDetails?.updated_at || Date.now()).getTime(); + const staleTime = 180 * 24 * 60 * 60 * 1000; + healthFlags.isStale = (Date.now() - pushedTime) > staleTime; + + // 3. forkHeavy: forks > stars * 1.5 AND stars < 500 + const stars = repoDetails?.stargazers_count || repoDetails?.stars || 0; + const forks = repoDetails?.forks_count || repoDetails?.forks || 0; + healthFlags.forkHeavy = forks > (stars * 1.5) && stars < 500; + + // Fetch Community Profile (CONTRIBUTING, README, CoC) + const communityProfile = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/community/profile`, + token + ); + + if (communityProfile) { + healthFlags.noContributing = !communityProfile.files?.contributing; + healthFlags.noReadme = !communityProfile.files?.readme; + } else { + // Fallback if not found + healthFlags.noContributing = true; + healthFlags.noReadme = true; + } + + // Fetch recent closed PRs to compute lowPRMergeRate + const closedPRStatsCache = await this.cacheService.getPRStats(owner, repo); + let prStats = closedPRStatsCache; + + if (!prStats) { + const closedPRs = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/pulls?state=closed&per_page=30`, + token + ); + if (Array.isArray(closedPRs) && closedPRs.length > 0) { + const mergedCount = closedPRs.filter(pr => pr.merged_at).length; + const mergeRate = (mergedCount / closedPRs.length) * 100; + prStats = { mergeRate }; + await this.cacheService.setPRStats(owner, repo, prStats); + } else { + prStats = { mergeRate: 100 }; // assume fine if no PRs + } + } + + healthFlags.lowPRMergeRate = prStats.mergeRate < 20; + + // Fetch contributor stats to compute noExternalContribs + const contribStats = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/stats/contributors`, + token + ); + + if (Array.isArray(contribStats) && contribStats.length > 0) { + const sortedContribs = contribStats + .map((c: any) => c.total || 0) + .sort((a: number, b: number) => b - a); + + const totalCommits = sortedContribs.reduce((sum: number, commits: number) => sum + commits, 0); + if (totalCommits > 0) { + const top3Commits = sortedContribs.slice(0, 3).reduce((sum: number, commits: number) => sum + commits, 0); + healthFlags.noExternalContribs = (top3Commits / totalCommits) > 0.95; + } + } + + // Fetch closed issues to compute lowIssueEngagement and slowMaintainerResp + const closedIssues = await this.fetchGithub( + `${this.GITHUB_API_BASE}/repos/${owner}/${repo}/issues?state=closed&per_page=20`, + token + ); + + if (Array.isArray(closedIssues) && closedIssues.length > 0) { + // filter out PRs + const cleanIssues = closedIssues.filter(item => !item.pull_request); + if (cleanIssues.length > 0) { + const zeroCommentClosed = cleanIssues.filter(issue => (issue.comments || 0) === 0).length; + healthFlags.lowIssueEngagement = (zeroCommentClosed / cleanIssues.length) > 0.60; + + // Compute average close time + let totalCloseDays = 0; + cleanIssues.forEach(issue => { + const created = new Date(issue.created_at).getTime(); + const closed = new Date(issue.closed_at || Date.now()).getTime(); + totalCloseDays += (closed - created) / (1000 * 60 * 60 * 24); + }); + const avgCloseDays = totalCloseDays / cleanIssues.length; + healthFlags.slowMaintainerResp = avgCloseDays > 30; + } + } + + // Save to cache + await this.cacheService.setRepoHealth(owner, repo, healthFlags); + } catch (error) { + if (error instanceof GithubRateLimitError) { + throw error; + } + this.logger.error(`Error computing health for ${owner}/${repo}: ${error instanceof Error ? error.message : String(error)}`); + } + + return healthFlags; + } +} diff --git a/apps/backend/src/shared/ai.service.ts b/apps/backend/src/shared/ai.service.ts index e9a621f..064040b 100644 --- a/apps/backend/src/shared/ai.service.ts +++ b/apps/backend/src/shared/ai.service.ts @@ -64,7 +64,6 @@ export class AiService { body: JSON.stringify({ model, messages, - reasoning: { enabled: process.env.OPENROUTER_REASONING !== 'false' }, max_tokens: 800, }), }); diff --git a/apps/frontend/Dockerfile b/apps/frontend/Dockerfile index 51bfe6d..2ab8dfe 100644 --- a/apps/frontend/Dockerfile +++ b/apps/frontend/Dockerfile @@ -2,10 +2,15 @@ FROM node:18-alpine AS builder WORKDIR /app -COPY package*.json ./ +COPY apps/frontend/package*.json ./ RUN npm ci --legacy-peer-deps -COPY . . +COPY apps/frontend/ . +COPY apps/backend/packages/ /backend/packages/ +ARG BACKEND_URL +ARG NEXT_PUBLIC_BACKEND_URL +ENV BACKEND_URL=${BACKEND_URL} +ENV NEXT_PUBLIC_BACKEND_URL=${NEXT_PUBLIC_BACKEND_URL} ENV NEXT_TELEMETRY_DISABLED=1 RUN npm run build diff --git a/apps/frontend/components/dashboard/DashboardShell.tsx b/apps/frontend/components/dashboard/DashboardShell.tsx index d78d57b..2068405 100644 --- a/apps/frontend/components/dashboard/DashboardShell.tsx +++ b/apps/frontend/components/dashboard/DashboardShell.tsx @@ -3,13 +3,14 @@ import { motion } from "framer-motion"; import { ReactNode } from "react"; import { Navbar } from "../Navbar"; -type RouteKey = "dashboard" | "commits" | "prs" | "digest" | "settings"; +type RouteKey = "dashboard" | "commits" | "prs" | "digest" | "settings" | "os-finder"; const tabs: Array<{ key: RouteKey; label: string; href: string }> = [ { key: "dashboard", label: "Overview", href: "/dashboard" }, { key: "commits", label: "Commits", href: "/dashboard/commits" }, { key: "prs", label: "PRs", href: "/dashboard/prs" }, { key: "digest", label: "Digest", href: "/dashboard/digest" }, + { key: "os-finder", label: "OS Finder", href: "/dashboard/os-finder" }, { key: "settings", label: "Settings", href: "/dashboard/settings" }, ]; diff --git a/apps/frontend/next.config.js b/apps/frontend/next.config.js index ffb3c59..f99b753 100644 --- a/apps/frontend/next.config.js +++ b/apps/frontend/next.config.js @@ -36,6 +36,11 @@ const nextConfig = { source: '/digests/:path*', destination: `${backendUrl}/digests/:path*`, }, + { + // All os-finder routes EXCEPT /search/ai (that goes to the custom API route) + source: '/os-finder/:path((?!search/ai$).*)', + destination: `${backendUrl}/os-finder/:path*`, + }, ]; }, }; diff --git a/apps/frontend/pages/api/os-finder-ai-search.ts b/apps/frontend/pages/api/os-finder-ai-search.ts new file mode 100644 index 0000000..6ffc1a8 --- /dev/null +++ b/apps/frontend/pages/api/os-finder-ai-search.ts @@ -0,0 +1,60 @@ +import type { NextApiRequest, NextApiResponse } from 'next'; +import http from 'http'; +import https from 'https'; + +// Disable Next.js body parsing — we forward raw body +export const config = { + api: { + bodyParser: false, + // No response size limit, no timeout constraint from Next.js side + responseLimit: false, + }, +}; + +export default function handler(req: NextApiRequest, res: NextApiResponse) { + // Only allow POST + if (req.method !== 'POST') { + res.status(405).json({ message: 'Method not allowed' }); + return; + } + + const backendUrl = process.env.BACKEND_URL || 'http://localhost:3000'; + const parsed = new URL(`${backendUrl}/os-finder/search/ai`); + + const isHttps = parsed.protocol === 'https:'; + const transport = isHttps ? https : http; + + const options: http.RequestOptions = { + hostname: parsed.hostname, + port: parsed.port || (isHttps ? 443 : 80), + path: parsed.pathname, + method: 'POST', + headers: { + ...req.headers, + host: parsed.host, + }, + // 60-second socket timeout on the backend-facing connection + timeout: 60000, + }; + + const proxyReq = transport.request(options, (proxyRes) => { + res.writeHead(proxyRes.statusCode || 200, proxyRes.headers); + proxyRes.pipe(res, { end: true }); + }); + + proxyReq.on('timeout', () => { + proxyReq.destroy(); + if (!res.headersSent) { + res.status(504).json({ message: 'AI search timed out. Please try again.' }); + } + }); + + proxyReq.on('error', (err) => { + if (!res.headersSent) { + res.status(502).json({ message: 'Backend connection error', detail: err.message }); + } + }); + + // Pipe incoming request body to proxy request + req.pipe(proxyReq, { end: true }); +} diff --git a/apps/frontend/pages/dashboard/os-finder/[owner]/[repo].tsx b/apps/frontend/pages/dashboard/os-finder/[owner]/[repo].tsx new file mode 100644 index 0000000..0e9fff1 --- /dev/null +++ b/apps/frontend/pages/dashboard/os-finder/[owner]/[repo].tsx @@ -0,0 +1,644 @@ +import { useState, useEffect } from "react"; +import Link from "next/link"; +import { useRouter } from "next/router"; +import { motion } from "framer-motion"; +import { DashboardShell } from "../../../../components/dashboard/DashboardShell"; +import { useAuth } from "../../../../context/AuthContext"; +import { useDashboardData } from "../../../../hooks/useDashboardData"; + +type RepoDetailData = { + githubRepoId: number; + owner: string; + name: string; + fullName: string; + description: string | null; + language: string | null; + stars: number; + forks: number; + openIssues: number; + lastCommitAt: string; + licenseType: string | null; + htmlUrl: string; + ncfScore: { + total: number; + goodFirstIssue: number; + helpWanted: number; + contributingFile: number; + issueResponseTime: number; + newContribPR: number; + readmeQuality: number; + codeOfConduct: number; + prMergeRate: number; + }; + healthFlags: { + isArchived: boolean; + isStale: boolean; + noContributing: boolean; + noReadme: boolean; + lowPRMergeRate: boolean; + forkHeavy: boolean; + noExternalContribs: boolean; + lowIssueEngagement: boolean; + slowMaintainerResp: boolean; + }; + contributors: Array<{ login: string; avatarUrl: string; contributions: number }>; + recentPRs: Array<{ id: number; title: string; url: string; author: string; authorAvatar: string; mergedAt: string }>; + savedId: string | null; + notes: string | null; + status: "saved" | "contributed" | "skipped" | null; +}; + +export default function OsFinderRepoDetailPage() { + const router = useRouter(); + const { owner, repo } = router.query as { owner?: string; repo?: string }; + const { fetchWithAuth } = useAuth(); + const { user, loading: dashboardLoading } = useDashboardData(); + + // Detail & Issues state + const [detail, setDetail] = useState(null); + const [issues, setIssues] = useState([]); + const [loadingDetail, setLoadingDetail] = useState(true); + const [loadingIssues, setLoadingIssues] = useState(true); + + // Watchlist Local settings State + const [isSaved, setIsSaved] = useState(false); + const [notes, setNotes] = useState(""); + const [status, setStatus] = useState<"saved" | "contributed" | "skipped">("saved"); + const [updatingWatchlist, setUpdatingWatchlist] = useState(false); + + useEffect(() => { + if (!dashboardLoading && !user) { + router.replace("/"); + } + }, [dashboardLoading, router, user]); + + useEffect(() => { + if (user && owner && repo) { + loadDetails(); + loadIssues(); + } + }, [user, owner, repo]); + + const loadDetails = async () => { + setLoadingDetail(true); + try { + const res = await fetchWithAuth(`/os-finder/repo/${owner}/${repo}`); + if (res.ok) { + const data = await res.json(); + setDetail(data); + setIsSaved(!!data.savedId); + setNotes(data.notes || ""); + setStatus(data.status || "saved"); + } else { + console.error("Failed to load repo details"); + } + } catch (err) { + console.error(err); + } finally { + setLoadingDetail(false); + } + }; + + const loadIssues = async () => { + setLoadingIssues(true); + try { + const res = await fetchWithAuth(`/os-finder/repo/${owner}/${repo}/issues`); + if (res.ok) { + const data = await res.json(); + setIssues(data || []); + } + } catch (err) { + console.error("Failed to load repo issues:", err); + } finally { + setLoadingIssues(false); + } + }; + + const handleWatchlistSave = async () => { + if (!detail) return; + setUpdatingWatchlist(true); + + try { + if (isSaved) { + // Update details (status + notes) + const savedListRes = await fetchWithAuth("/os-finder/saved"); + if (savedListRes.ok) { + const savedList = await savedListRes.json(); + const match = savedList.find((s: any) => Number(s.githubRepoId) === detail.githubRepoId); + if (match) { + const updRes = await fetchWithAuth(`/os-finder/saved/${match.id}`, { + method: "PATCH", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ notes, status }), + }); + if (updRes.ok) { + alert("Watchlist updated successfully."); + } + } + } + } else { + // Save new item + const saveDto = { + githubRepoId: detail.githubRepoId, + owner: detail.owner, + name: detail.name, + fullName: detail.fullName, + description: detail.description, + language: detail.language, + stars: detail.stars, + forks: detail.forks, + openIssues: detail.openIssues, + ncfScore: detail.ncfScore, + langMatchScore: 1.0, + lastCommitAt: detail.lastCommitAt, + hasContributing: !detail.healthFlags.noContributing, + licenseType: detail.licenseType, + htmlUrl: detail.htmlUrl, + notes, + status, + }; + + const saveRes = await fetchWithAuth("/os-finder/saved", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify(saveDto), + }); + + if (saveRes.ok) { + setIsSaved(true); + alert("Repository added to Watchlist."); + } + } + } catch (err) { + console.error(err); + } finally { + setUpdatingWatchlist(false); + } + }; + + const handleWatchlistRemove = async () => { + if (!detail) return; + if (!confirm("Are you sure you want to remove this repo from your watchlist?")) return; + + setUpdatingWatchlist(true); + try { + const savedListRes = await fetchWithAuth("/os-finder/saved"); + if (savedListRes.ok) { + const savedList = await savedListRes.json(); + const match = savedList.find((s: any) => Number(s.githubRepoId) === detail.githubRepoId); + if (match) { + const delRes = await fetchWithAuth(`/os-finder/saved/${match.id}`, { method: "DELETE" }); + if (delRes.ok) { + setIsSaved(false); + setNotes(""); + alert("Removed from Watchlist."); + } + } + } + } catch (err) { + console.error(err); + } finally { + setUpdatingWatchlist(false); + } + }; + + const getNcfColorClass = (score: number) => { + if (score >= 7.5) return "text-emerald-400 bg-emerald-950/40 border-emerald-900/50"; + if (score >= 5.0) return "text-amber-400 bg-amber-950/40 border-amber-900/50"; + return "text-sky-400 bg-sky-950/40 border-sky-900/50"; + }; + + const getNcfBarColor = (score: number) => { + if (score >= 7.5) return "bg-emerald-500 shadow-[0_0_8px_rgba(16,185,129,0.3)]"; + if (score >= 5.0) return "bg-amber-500 shadow-[0_0_8px_rgba(245,158,11,0.3)]"; + return "bg-sky-500 shadow-[0_0_8px_rgba(59,130,246,0.3)]"; + }; + + if (dashboardLoading || !user) return null; + + return ( + +
+ + + + + Back to Discovery Engine + + + {detail && ( + + View on GitHub + + + + + )} +
+ + {loadingDetail || !detail ? ( + /* Loading skeleton */ +
+
+
+
+
+
+
+
+
+ ) : ( +
+ {/* Main info, NCF bars and health flags */} +
+ {/* Header info card */} +
+
+

{detail.name}

+ {detail.language && ( + {detail.language} + )} + {detail.licenseType && ( + + {detail.licenseType} + + )} +
+ +

+ {detail.description || "No description provided."} +

+ + {/* Counter badges */} +
+
+ {detail.stars.toLocaleString()} + Stars +
+
+ {detail.forks.toLocaleString()} + Forks +
+
+ {detail.openIssues.toLocaleString()} + Open Issues +
+
+
+ + {/* NCF Score Breakdown Card */} +
+
+
+

New Contributor Friendliness (NCF)

+

8 metrics representing maintainer responsiveness and code accessibility.

+
+ + {Number(detail.ncfScore.total).toFixed(1)} / 10 + +
+ + {/* NCF Bars grid */} +
+ {/* Metric Item: Good First Issues */} +
+
+ Good First Issues Label (25%) + {detail.ncfScore.goodFirstIssue} / 2.5 +
+
+
+
+
+ + {/* Metric Item: Help Wanted */} +
+
+ Help Wanted Label (10%) + {detail.ncfScore.helpWanted} / 1.0 +
+
+
+
+
+ + {/* Metric Item: Contributing Guide */} +
+
+ CONTRIBUTING.md file present (15%) + {detail.ncfScore.contributingFile} / 1.5 +
+
+
+
+
+ + {/* Metric Item: Issue Response Time */} +
+
+ Issue Closure Responsiveness (20%) + {detail.ncfScore.issueResponseTime} / 2.0 +
+
+
+
+
+ + {/* Metric Item: New Contributor PR */} +
+
+ Recent Merged External PRs (15%) + {detail.ncfScore.newContribPR} / 1.5 +
+
+
+
+
+ + {/* Metric Item: Code of Conduct */} +
+
+ Code of Conduct present (5%) + {detail.ncfScore.codeOfConduct} / 0.5 +
+
+
+
+
+ + {/* Metric Item: Readme Quality */} +
+
+ README quality & detail (5%) + {detail.ncfScore.readmeQuality} / 0.5 +
+
+
+
+
+ + {/* Metric Item: PR Merge Rate */} +
+
+ Pull Request Merge Rate (5%) + {detail.ncfScore.prMergeRate} / 0.5 +
+
+
+
+
+
+
+ + {/* Beginner Issues list section */} +
+

Open Onboarding Issues

+

Active tickets marked with good-first-issue or help-wanted labels.

+ + {loadingIssues ? ( +
+
+
+
+ ) : issues.length === 0 ? ( +
+ No issues matching new contributor labels were found. +
+ ) : ( + + )} +
+
+ + {/* Right sidebar: Watchlist panel, Community profile details, PR lists */} +
+ {/* Watchlist Manager Panel */} +
+

Watchlist Settings

+ +
+
+ Status + +
+ +
+ Contribution Notes +