Skip to content

Add feature: Support for multiple models on Azure and Azure model mapper#3344

Closed
shendongming wants to merge 8 commits intoChatGPTNextWeb:mainfrom
sdm2345:azure-model-mapper
Closed

Add feature: Support for multiple models on Azure and Azure model mapper#3344
shendongming wants to merge 8 commits intoChatGPTNextWeb:mainfrom
sdm2345:azure-model-mapper

Conversation

@shendongming
Copy link
Copy Markdown

Added support for multiple models on Azure; in practice, we will deploy several models on Azure, such as gpt-4, gpt-3.5, etc.

Two changes:

  1. Added a new configuration item: AZURE_OPENAI_MODEL_MAPPER
  2. Extended the configuration item: AZURE_URL

The changes are as follows:

AZURE_URL (optional)

Example: https://{azure-resource-url}/openai/deployments/{deploy-name}

Example: https://xxx.openai.azure.com/openai/deployments/{deploy-name}

Azure deploy url.

If {deploy-name} is using the template mode, then it will automatically replace the path based on the model selected by the client.
If your model name is different from the deployment name, then you need to set the AZURE_OPENAI_MODEL_MAPPER parameter.

AZURE_OPENAI_MODEL_MAPPER (optional)

Default: Empty
Example: gpt-3.5-turbo=gpt-35-turbo means map gpt-3.5-turbo to gpt-35-turbo

If you are deploying ChatGPT using Azure OpenAI, it is recommended to set the AZURE_OPENAI_MODEL_MAPPER.
The session summarization feature relies on the gpt-3.5-turbo model, unless the name of your Azure deployment is the same as it.

Copy link
Copy Markdown
Contributor

@H0llyW00dzZ H0llyW00dzZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this default: "2023-04", must made separate for azure as example, because it will affected for openai and become bad hallucination. if force default: "2023-04", for all service provider

Comment thread app/constant.ts Outdated

export const KnowledgeCutOffDate: Record<string, string> = {
default: "2021-09",
default: "2023-04",
Copy link
Copy Markdown
Contributor

@H0llyW00dzZ H0llyW00dzZ Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this default: "2023-04", must made separate for azure as example, because it will affected for openai and become bad hallucination. if force default: "2023-04", for all service provider

@H0llyW00dzZ
Copy link
Copy Markdown
Contributor

Warning

The KnowledgeCutOffDate is intended for system prompts. Exercise caution when changing it, as it must be correctly aligned with the models.

@shendongming
Copy link
Copy Markdown
Author

Thank you
I have set KnowledgeCutOffDate to 2021-09
@H0llyW00dzZ

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 29, 2024

Someone is attempting to deploy a commit to the NextChat Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions
Copy link
Copy Markdown
Contributor

Your build has completed!

Preview deployment

@lloydzhou lloydzhou requested a review from Dogtiti July 2, 2024 16:54
Comment thread README_CN.md

### `AZURE_OPENAI_MODEL_MAPPER` (optional)
> Default: Empty
> Example: `gpt-3.5-turbo=gpt-35-turbo` 意思是 `gpt-3.5-turbo` 映射到 `gpt-35-turbo`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

看起来似乎是一个容易在配置文件中实现的方案 @Dogtiti

@lloydzhou lloydzhou added good first issue Good for newcomers planned planned feature, will support in the future labels Jul 3, 2024
Comment thread README_CN.md
> Example: `gpt-3.5-turbo=gpt-35-turbo` 意思是 `gpt-3.5-turbo` 映射到 `gpt-35-turbo`

如果你使用azure openai 来部署的chatgpt,建议设定AZURE_OPENAI_MODEL_MAPPER .
会话摘要功能依赖gpt-3.5-turbo 这个model,除非你的azure deployment的名字和它相同
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

可以直接使用CUSTOM_MODELS中配置gpt-4-1106-preview=gpt-4-turbo的方式,等于后面既作为展示名称,也作为deploy_name使用

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

你好,这个最新 readme 文档有了吗?使用 azure 部署的遇到一些麻烦。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

good first issue Good for newcomers planned planned feature, will support in the future

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants