release 20251022 #21
Merged
release 20251022 #21
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add Groq provider with OpenAI-compatible API support (requires API key) - Add DeepSeek provider using web scraping from documentation - Fix Tokenflux provider JSON parsing by simplifying struct definitions - Update README with comprehensive provider documentation - Add configuration examples for all new providers - Support 8 providers total: PPInfra, OpenRouter, Gemini, Vercel, GitHub AI, Tokenflux, Groq, DeepSeek - Generate updated JSON files for all providers
- Add complete OpenAI template with 35 models including GPT-5, GPT-4.1, o1/o3/o4 series, DALL-E, Whisper, TTS, and embeddings - Add comprehensive Anthropic template with 8 Claude models (Opus 4.1, Opus 4, Sonnet 4, 3.7 Sonnet, 3.5 Sonnet variants, Haiku) - Implement intelligent multi-pattern matching system via 'match' arrays for handling versioned model IDs - Add auto-configuration for unmatched API models with smart capability detection - Support template-based fallback when no API key is provided - Update configuration examples with OpenAI and Anthropic provider settings - Enhance model fetching to handle 65+ OpenAI models and 8 Anthropic models - Add comprehensive logging for matched vs auto-configured models The new matching system resolves API model ID variations (e.g., gpt-5-nano-2025-08-07 → gpt-5-nano template) while providing intelligent defaults for unknown models, significantly improving coverage and reliability.
…mplementations Update all documentation files to reflect the successful implementation of OpenAI and Anthropic providers with advanced template matching system: - Update README.md with OpenAI (65+ models) and Anthropic (8 models) support status - Add template-based vs direct conversion implementation patterns - Update CLAUDE.md Next Steps section marking OpenAI/Anthropic as completed - Enhanced .claude/provider_implementer.md with template matching system documentation - Updated .claude/data_converter.md with template-based provider examples - Corrected .claude/format_validator.md aggregated JSON format and validation commands - Added comprehensive API key configuration documentation - Updated example commands to include anthropic provider Key documentation improvements: - Template matching system with multi-pattern support via 'match' arrays - Auto-configuration system for unmatched models with intelligent capability detection - Clear distinction between template-based providers (OpenAI, Anthropic) and direct conversion providers (PPInfra, OpenRouter) - Updated validation tools and format specifications - Comprehensive implementation guides for both provider patterns
…h or tags - Remove automatic triggers on push to main branch to prevent continuous execution - Fix YAML syntax error in PR body by removing problematic heredoc syntax - Update API key environment variables to include OpenAI, Anthropic, Groq, and Gemini - Simplify workflow to only create PRs on manual dispatch, not on push events - Remove redundant direct commit step for push events
The YAML syntax error was caused by escaped backticks in the PR body text. Removed the backslash escapes to fix the workflow validation.
- Replace inline multi-line string with heredoc + --body-file approach for PR creation - Simplify complex conditional expression in release body to avoid syntax issues - Use standard shell heredoc (cat > file << EOF) instead of problematic YAML multi-line strings - Verified YAML syntax is now valid with PyYAML parser This should completely resolve the line 133 syntax error and any other YAML formatting issues.
- Add PartialEq derive to ModelType enum for test comparisons - Fix test assertion for ModelType::Other -> ModelType::Chat in groq provider - Add #[allow(dead_code)] to unused struct fields that are needed for JSON deserialization - Fix groq provider display name formatting logic: - Add 'oss' -> 'OSS' special case - Handle numeric suffixes like '120b' -> '120B' - Update test expectations to match actual function behavior - All tests now pass and compilation is clean Affected files: - src/models/model_info.rs: Added PartialEq trait - src/providers/groq.rs: Fixed test and formatting logic - src/providers/*.rs: Added dead_code allowance for JSON deserialization structs
…el support - Add SiliconFlow provider implementation similar to Ollama template-based approach - Create src/providers/siliconflow.rs with Provider trait implementation - Add siliconflow module reference in src/providers/mod.rs - Register provider in src/main.rs fetch functions - Create templates/siliconflow.json with 15 model definitions - Generate dist/siliconflow.json output file - Add agent documentation files for future provider development
* feat: complete migration from Rust to TypeScript - Migrate all 12 provider implementations to TypeScript - Replace CLI framework from clap to commander.js - Update HTTP client from reqwest to axios - Convert web scraping from scraper crate to cheerio - Update GitHub Actions workflow for Node.js environment - Fix API response structure handling for all providers - Resolve Gemini embedding model validation errors - Clean up all Rust source code and configuration files - Generate 855 models from 7 active providers * refactor: migrate package manager to pnpm and separate build output - Switch from npm to pnpm for better performance and disk efficiency - Remove npm lock files and node_modules dependencies - Update TypeScript config to output compiled files to build/ directory - Separate data files (dist/) from compiled artifacts (build/) - Update package.json scripts and GitHub Actions for pnpm workflow - Update all documentation to use pnpm commands - Add build/ directory to .gitignore - Clean up mixed file structure for better organization * build(vite): migrate from tsc/Jest to Vite/Vitest; update scripts and docs - Replace ad-hoc ts-node tests with Vitest specs\n- Add vitest.config.ts; add tests for normalizer/validator/aggregator/json-writer/config\n- Switch to Vite-only builds (library + CLI) with vite.config.ts and vite.cli.config.ts\n- Update scripts: dev/start run 'fetch-all' directly\n- Remove Jest and legacy test scripts\n- Update README and README-TS to reflect Vite build and new usage * build(vite): refactor to vitest and vite * build(vite): migration to rolldown vite * build(dev): use vite-node for dev; add dev:watch; docs updated - Switch dev to vite-node to run CLI with Vite transpilation\n- Add dev:watch for automatic reload on changes\n- Update README docs to reflect vite-node usage * chore: update deps
- Enable corepack and set up pnpm@9 using pnpm/action-setup\n- Add pnpm cache dependency path\n- Use node build/cli.js for fetch commands (start now defaults to fetch-all)
…); remove corepack step - Use pnpm/action-setup@v2 with explicit version to install pnpm binary\n- Keep Node cache for pnpm and lockfile path\n- Drop corepack to avoid conflicts on runners without pre-enabled corepack
…stall pnpm afterwards) - Remove cache: pnpm from actions/setup-node to avoid calling pnpm before installation\n- Rely on actions/cache for pnpm store and lockfile caching
* feat: simplify models.dev integration * fix(build): pass build * chore: add ignored file * feat: finish models.dev support * feat: renmame manual-templates
* feat: add better log adn refresh data * ci: update cdn uploader * feat: add compress for json
* fix(fetcher): retry models.dev requests * fix: merge ppio overrides into ppinfra * chore: update new config
* fix: align capabilities format with models.dev * chore: update models and providers
* feat(schema): migrate reasoning to object and add search config * refactor(schema): rename 'enabled' to 'supported' in reasoning/search
* fix: qwen-plus models * fix: qwen-flash models * fix: qwen-turbo models * fix: qwq-plus models * feat: qwen-long * feat: qvq-max * fix: qwen3-vl-plus * feat: qwen-vl-ocr * feat: qwen3-coder-plus * feat: qwen-mt * fix: qwen3-next-80b-a3b * fix: qwen3-235b-a22b * fix: qwen3-30b-a3b * fix: qwen3-32b * fix: qwen3-14b * fix: qwen3-8b * fix: qwen3-4b * fix: qwen3-1.7b * fix: qwen3-0.6b * fix: qwen-max * chore: update dashscope provider data
* fix: align DeepSeek manual template format * chore: update deepseek provider
* fix(manual-templates): enrich siliconflow template * fix: rebuild siliconflow template models * chore: update dist file
* fix(openrouter): flatten reasoning toggle * fix(openrouter): align reasoning tuning with api * fix: remove models-dev openrouter
* fix: force live fetch for jiekou provider * fix(commands): always run jiekou provider override * feat: update data
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+180
to
+186
| const contextLength = toNumber(model.top_provider?.context_length); | ||
| const maxOutputTokens = toNumber(model.top_provider?.max_completion_tokens); | ||
|
|
||
| const modelType = determineModelType(model, normalizedId); | ||
| const context = contextLength ?? 0; | ||
| const maxTokens = maxOutputTokens ?? context; | ||
| const vision = modelHasVisionCapability(modalities); |
There was a problem hiding this comment.
When a model from the OpenRouter API lacks context_length but specifies max_completion_tokens, the mapper sets context to 0 while leaving maxTokens as the provided value. This produces ModelInfo objects with contextLength=0 and a positive maxTokens, which violates JsonValidator.validateModelInfo (context length must be greater than 0 and max tokens cannot exceed context length). Providers containing such models will always fail validation and either be skipped or emit invalid data. Consider defaulting context to maxTokens (or removing the model) whenever context_length is absent so the output remains consistent and passes validation.
Useful? React with 👍 / 👎.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.