Feat/gateway spec and platform integration#228
Open
glassBead-tc wants to merge 35 commits intomainfrom
Open
Conversation
…s, dgm-specs, infra, research-workflows, self-improvement
Fetch the public Dedalus marketplace catalog at startup, filter to open (no-auth) servers, and present them as virtual upstreams through GatewayRuntime. Tool execution routes through the Dedalus chat completions API with mcp_servers=["slug"]. - Add DedalusMarketplaceRuntime and CompositeGatewayRuntime - Wire into both HTTP and MCP server bootstrap paths - Activate when DEDALUS_API_KEY env var is set - Gateway manifest gracefully falls back to empty when file missing - Pass DEDALUS_API_KEY through docker-compose.yml - Includes prior repo cleanup (spec/doc/observability removals) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…e-integration Feat/dedalus marketplace integration
Dedalus's build system expects a manifest.json at the repo root defining the server type, entry point, and user-configurable env vars. Without it the bun bundler has no entry point to target. Entry point: dist/index.js (built from src/index.ts via tsc) Tools declared explicitly: thoughtbox_search, thoughtbox_execute
Confirmed from dedalus-refs and build logs:
- StreamableHTTP transport is correct
- Auth header contract matches our implementation
- Dedalus build system uses bun
- DXT manifest.json is required at repo root (was the build failure cause)
Remaining unknown: ESM ("type": "module") vs CJS mismatch with
reference servers. Will be confirmed/falsified by next deploy attempt.
SCAFFOLD_CONTRACT: mostly resolved
SERVER_LIST_DISCOVERY: still unresolved, proceeding with public catalog approach
Dedalus's build platform uses bun. Aligning our toolchain to match: - All pnpm script references replaced with bun run - pnpm-lock.yaml removed, bun.lock added - packageManager field removed (corepack doesn't support bun) - fix: suppress false-positive unicorn/require-post-message-target-origin on Node.js worker_threads postMessage (not browser window.postMessage) - build, test, embed scripts all verified passing
chore: switch package manager from pnpm to bun
Replace complex build chain (tsc + shx + madge + embed scripts) with a single bun build invocation that we know works. Entry point moves from dist/index.js to index.js at repo root. This removes the dependency on npx, tsc, shx, and shell scripts that may not be available or behave differently on Dedalus's build platform.
These are local reference repos used for investigation only. They contain large .git pack files (~98MB) that don't belong in this repo and aren't needed for deployment. Added dedalus-refs/ to .gitignore — files remain locally.
Tests hub-channel.js which is dormant — not relevant to the Marketplace gateway deployment.
…r-marketplace chore: simplify build to single bun build command for Marketplace
Meeting transcript with participant names — should never have been committed. Removed from tracking and added to .gitignore.
src/marketplace.ts — thoughtbox_search + thoughtbox_execute only. No Hub, Observatory, knowledge, SQLite, or evaluation wiring. Build drops from 1014 to 552 modules, removes better-sqlite3 native addon entirely.
…odules chore: lean marketplace entry point, gateway only
Replace FileSystemKnowledgeStorage SQLite implementation with a stub. Not needed for the Marketplace gateway deployment. Removes better-sqlite3 and @types/better-sqlite3 from package.json. Build now has zero native addons.
…odules chore: remove better-sqlite3 native dependency
TypeScript build blocked after 7 attempts with zero error output. Spec for minimal Python implementation using Dedalus Python SDK. Two tools, no native addons, no build step, ~100 lines.
Working TS server on Dedalus Marketplace (dedalus-marketplace-crawler-ts) uses prepare: tsc so bun install produces dist/ automatically. Our prepare was running husky (sets up git hooks, produces nothing). Dedalus never got a dist/index.js to run. Also fix FileSystemKnowledgeStorage stub to satisfy TypeScript.
…odules docs: Python gateway spec + remove better-sqlite3
manifest.json pointed to index.js at root. prepare:tsc produces dist/index.js. Dedalus looks for index.js, finds nothing, fails. All three now aligned: manifest, package.json main, tsc outDir.
fix: align manifest entry_point with tsc output
Match the working pattern from dedalus-labs/brave-search-mcp: - build script uses tsc instead of bun build - Dockerfile uses npm install + single-stage alpine image - start script points to dist/marketplace.js - PORT defaults to 8000 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix: align build and Dockerfile with Dedalus deploy pattern
Dedalus builder runs its own bun build step, ignoring the build script. It resolves the entry point from the main field. Previously this pointed at dist/index.js which doesn't exist in the repo. Now points at src/marketplace.ts so the bundler can find it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix: point main at source entry point for Dedalus bun bundler
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix: strip deps to gateway-only for Dedalus deploy
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix: match tsconfig to brave-search-mcp
Matches brave-search-mcp and marketplace-crawler-ts exactly: - No main field - build: tsc + chmod - prepare: npm run build - No engines, no extra scripts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix: match package.json to working Dedalus servers
- Add comprehensive YAML spec for Thoughtbox Gateway - Add platform integration points doc for Dedalus discussion - Read mcp_tool_results instead of model summary text - Add Worker resourceLimits to execute sandbox Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Return native content blocks (images/audio/resources) instead of stringifying structured MCP tool payloads - Use .at(-1) instead of .find() to get the final tool result when the model retries or calls the same tool multiple times Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Detail composite credential model (API key + OAuth token) for Option B in the platform integration doc. References MCP spec auth model and existing Python SDK OAuth/DPoP implementation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Working Dedalus servers (brave-search-mcp, marketplace-crawler-ts) deploy without manifest.json. Ours pointed entry_point at dist/index.js which doesn't exist after clone — likely cause of silent bun build failure. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
|
Too many files changed for review. ( |
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.