Skip to content

Update partitionKey path definitions for cosmosDatabase containers.#240

Merged
crickman merged 8 commits intomainfrom
fix-cosmosdb-partitioning
Aug 28, 2023
Merged

Update partitionKey path definitions for cosmosDatabase containers.#240
crickman merged 8 commits intomainfrom
fix-cosmosdb-partitioning

Conversation

@crickman
Copy link
Copy Markdown
Contributor

@crickman crickman commented Aug 23, 2023

Motivation and Context

Customer issue reported in #238 related to partition scheme in cosmosdb containers.

Description

Update partition-key path definition for ChatMessage, ChatPartitipant, and MemorySource.

Existing deployments will continue to work in their current mode.

CosmosDB containers must be removed and redeployed to realize the peformance update related to the partition scheme.

Contribution Checklist

@github-actions github-actions Bot added webapi Pull requests that update .net code PR: ready for review labels Aug 23, 2023
@crickman crickman requested a review from TaoChenOSU August 23, 2023 03:01
@dluc
Copy link
Copy Markdown
Collaborator

dluc commented Aug 23, 2023

is there any table that should be partitioned by user?

Comment thread webapi/Storage/Repository.cs
Comment thread webapi/Controllers/ChatHistoryController.cs
Comment thread webapi/Models/Storage/ChatParticipant.cs Outdated
Comment thread scripts/deploy/main.bicep
@crickman
Copy link
Copy Markdown
Contributor Author

TryFindByIdAsync

is there any table that should be partitioned by user?

No sir...this model is "chat-centric"

Comment thread scripts/deploy/main.bicep
@dluc
Copy link
Copy Markdown
Collaborator

dluc commented Aug 23, 2023

Another perf detail that came up is the consistency level. The tables are setup for "strong consistency" which could be relaxed to reduce latency.

@crickman crickman marked this pull request as ready for review August 23, 2023 17:45
@crickman crickman self-assigned this Aug 23, 2023
@crickman crickman removed PR: in progress preview Not even close to completed labels Aug 23, 2023
dluc
dluc previously approved these changes Aug 24, 2023
@crickman crickman changed the title Update patitionKey path definitions for cosmosDatabase containers. Update partitionKey path definitions for cosmosDatabase containers. Aug 24, 2023
alliscode
alliscode previously approved these changes Aug 28, 2023
Comment thread scripts/deploy/main.bicep Outdated
@crickman crickman dismissed stale reviews from alliscode and dluc via 8034b9e August 28, 2023 20:44
@crickman crickman requested review from alliscode and dluc August 28, 2023 20:45
@crickman crickman added this pull request to the merge queue Aug 28, 2023
Merged via the queue into main with commit 679bd07 Aug 28, 2023
@crickman crickman deleted the fix-cosmosdb-partitioning branch August 28, 2023 20:51
teamleader-dev pushed a commit to vlink-group/chat-copilot that referenced this pull request Oct 7, 2024
…icrosoft#240)

### Motivation and Context
Customer issue reported in
microsoft#238 related to
partition scheme in cosmosdb containers.

### Description
Update partition-key path definition for `ChatMessage`,
`ChatPartitipant`, and `MemorySource`.

Existing deployments will continue to work in their current mode.

CosmosDB containers must be removed and redeployed to realize the
peformance update related to the partition scheme.

### Contribution Checklist
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [Contribution
Guidelines](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
kb0039 pushed a commit to aaronba/chat-copilot that referenced this pull request Jan 8, 2025
…icrosoft#240)

### Motivation and Context
Customer issue reported in
microsoft#238 related to
partition scheme in cosmosdb containers.

### Description
Update partition-key path definition for `ChatMessage`,
`ChatPartitipant`, and `MemorySource`.

Existing deployments will continue to work in their current mode.

CosmosDB containers must be removed and redeployed to realize the
peformance update related to the partition scheme.

### Contribution Checklist
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [Contribution
Guidelines](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
jdtoombs pushed a commit to jdtoombs/chat-copilot that referenced this pull request Apr 10, 2025
* updated dev

* tweaks to deployments to share capacity in canada
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deployment Issues related to deploying Chat-Copilot external dependency issue Issues related to external dependencies (e.g. Azure) performance webapi Pull requests that update .net code

Projects

No open projects

Development

Successfully merging this pull request may close these issues.

CosmosDb partitioning strategy causing high cost/low perf, max quota/too many requests

4 participants