docs(local-ai): document local chat routing#2383
Merged
graycyrus merged 3 commits intoMay 22, 2026
Merged
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe Local AI docs are revised to state local AI is opt-in, add Chat and Reasoning as routable workloads, clarify cloud-default behavior, refine lightweight/medium vs heavy routing heuristics, and document unified ChangesLocal AI routing opt-in and provider configuration
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
gitbooks/features/model-routing/local-ai.md (2)
39-39: 💤 Low valueConsider clarifying "when local AI is enabled".
The phrase "when local AI is enabled" could be more precise. Does it mean
local_ai.runtime_enabled = true, or does it require specificlocal_ai.usage.*flags to be set? Given the dual configuration mechanism (usage flags and provider fields), explicit criteria would reduce ambiguity.That said, the linked router reference may already provide this detail.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@gitbooks/features/model-routing/local-ai.md` at line 39, Clarify the ambiguous phrase "when local AI is enabled" by specifying the exact configuration checks the router uses: state that the router picks a local provider only if local_ai.runtime_enabled is true and the relevant local_ai.usage.* flag for the hint category (or the corresponding provider field) allows that workload, and note that Ollama (or the configured local provider) must be reachable; update the sentence referencing the router to explicitly mention these config keys (local_ai.runtime_enabled, local_ai.usage.* and provider fields) so readers know which flags must be set for the local preference to apply.
23-24: 💤 Low valueConsider aligning the Implementation column with other rows.
The Chat and Reasoning rows show configuration field syntax in the Implementation column, while other rows reference actual source file paths. For consistency, consider either:
- Referencing the router/provider code that handles these workloads (e.g.,
src/openhuman/providers/router.rsor similar)- Or adding a note that routing is config-driven and pointing to the Opting in section
The configuration syntax is already well-documented in lines 71-78.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@gitbooks/features/model-routing/local-ai.md` around lines 23 - 24, The Implementation column for the Chat and Reasoning rows should be made consistent with other rows: instead of just showing the config syntax (Config::chat_provider and Config::reasoning_provider), update the cell to either reference the module/function that performs routing for those workloads (the router/provider implementation that dispatches to local models) or add a short note pointing readers to the "Opting in" section that explains config-driven routing; locate the routing code by searching for the router/provider implementation that handles chat/reasoning and update the markdown to mention that symbol or the "Opting in" section accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@gitbooks/features/model-routing/local-ai.md`:
- Around line 71-78: Explain in the doc that the boolean flags
local_ai.usage.embeddings, local_ai.usage.heartbeat,
local_ai.usage.learning_reflection, and local_ai.usage.subconscious are feature
toggles that enable/disable use of local AI for those capability groups, while
the provider fields (embeddings_provider, heartbeat_provider, learning_provider,
subconscious_provider and the other *_provider fields such as agentic_provider,
coding_provider, memory_provider) select the specific runtime when the
corresponding feature is enabled; state the precedence (if a usage flag is false
the local provider is ignored even if a *_provider is set; if the usage flag is
true and the *_provider is unset/blank or set to "cloud" the cloud/default route
is used; setting a provider (e.g., embeddings_provider="ollama:...") with the
usage flag true routes that workload on-device), give a short recommendation to
either set both (enable flag + provider) or just set the flag to false to force
cloud, and add a brief sentence listing/pointing out agentic_provider,
coding_provider, memory_provider as additional capability-specific providers
with a link or reference to the detailed provider docs.
---
Nitpick comments:
In `@gitbooks/features/model-routing/local-ai.md`:
- Line 39: Clarify the ambiguous phrase "when local AI is enabled" by specifying
the exact configuration checks the router uses: state that the router picks a
local provider only if local_ai.runtime_enabled is true and the relevant
local_ai.usage.* flag for the hint category (or the corresponding provider
field) allows that workload, and note that Ollama (or the configured local
provider) must be reachable; update the sentence referencing the router to
explicitly mention these config keys (local_ai.runtime_enabled, local_ai.usage.*
and provider fields) so readers know which flags must be set for the local
preference to apply.
- Around line 23-24: The Implementation column for the Chat and Reasoning rows
should be made consistent with other rows: instead of just showing the config
syntax (Config::chat_provider and Config::reasoning_provider), update the cell
to either reference the module/function that performs routing for those
workloads (the router/provider implementation that dispatches to local models)
or add a short note pointing readers to the "Opting in" section that explains
config-driven routing; locate the routing code by searching for the
router/provider implementation that handles chat/reasoning and update the
markdown to mention that symbol or the "Opting in" section accordingly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 1a03e392-b534-4ae7-b0e7-6302aac5d4ab
📒 Files selected for processing (1)
gitbooks/features/model-routing/local-ai.md
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (3)
gitbooks/features/model-routing/local-ai.md (3)
78-80: ⚡ Quick winComplete the precedence documentation for legacy flags vs provider fields.
Lines 78-80 state that
*_providerfields are the source of truth and legacy flags don't override them after migration, which partially addresses the past review comment. However, the detailed precedence rules are still not fully explained.Consider adding a brief clarification of the interaction:
- When is a legacy
usage.*flag consulted?- What happens if
local_ai.usage.embeddings = falsebutembeddings_provider = "ollama:all-minilm"?- Should users set both or just one?
📋 Suggested addition after line 80
Add a subsection that clarifies the precedence:
### Legacy flag behavior The `local_ai.usage.*` booleans (e.g., `local_ai.usage.embeddings`) are only used during preset application and initial migration. After migration, the corresponding `*_provider` field (e.g., `embeddings_provider`) becomes the sole routing control: - If `embeddings_provider = "ollama:all-minilm"` → routes to Ollama regardless of the legacy flag - If `embeddings_provider` is unset, blank, or `"cloud"` → routes to cloud regardless of the legacy flag **Recommendation**: Use the `*_provider` fields directly in your configuration. The legacy `usage.*` flags are maintained for backward compatibility with existing presets.Based on learnings from past review: The relationship between
local_ai.usage.*flags and*_providerfields needs clarification including precedence rules and user guidance.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@gitbooks/features/model-routing/local-ai.md` around lines 78 - 80, Add a concise "Legacy flag behavior" subsection after the existing paragraph clarifying that Config::workload_local_model (and related routing logic) treats the `*_provider` fields as the definitive source of truth once migration/presets complete; explain that `local_ai.usage.*` booleans are only consulted during preset application/initial migration and do not override provider fields afterwards, give two explicit examples (e.g., `embeddings_provider = "ollama:all-minilm"` routes on-device regardless of `local_ai.usage.embeddings = false`, and an unset/blank/"cloud" `*_provider` forces cloud routing regardless of legacy flags), and add a short user recommendation to prefer setting `*_provider` fields directly.
39-39: ⚡ Quick winConsider breaking the routing logic into separate sentences for clarity.
This sentence describes two distinct routing behaviors (lightweight/medium vs heavy hints) with multiple conditions in a single statement. Breaking it into two or three sentences would improve readability.
📝 Suggested restructure
-For **lightweight or medium chat hints** (`hint:reaction`, `hint:classify`, `hint:format`, `hint:sentiment`, `hint:summarize`, `hint:medium`, `hint:tool_lite`), the [router](README.md) can prefer the local provider only when `local_ai.runtime_enabled = true` and the configured local provider is reachable. Heavy hints (`hint:reasoning`, `hint:agentic`, `hint:coding`) stay cloud unless the matching workload provider field is explicitly configured locally. +For **lightweight or medium chat hints** (`hint:reaction`, `hint:classify`, `hint:format`, `hint:sentiment`, `hint:summarize`, `hint:medium`, `hint:tool_lite`), the [router](README.md) can prefer the local provider when `local_ai.runtime_enabled = true` and the configured local provider is reachable. + +**Heavy hints** (`hint:reasoning`, `hint:agentic`, `hint:coding`) always stay cloud by default. They only route locally when the matching workload provider field (e.g., `reasoning_provider`, `agentic_provider`, `coding_provider`) is explicitly configured with a local provider string.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@gitbooks/features/model-routing/local-ai.md` at line 39, Split the long sentence into two clearer sentences: one that states routing for lightweight/medium chat hints (list the hint tokens: hint:reaction, hint:classify, hint:format, hint:sentiment, hint:summarize, hint:medium, hint:tool_lite) is allowed to prefer the local provider only when local_ai.runtime_enabled = true and the configured local provider is reachable, and a second sentence that states heavy hints (hint:reasoning, hint:agentic, hint:coding) remain cloud by default unless the matching workload provider field is explicitly configured locally; keep the same terminology (“router”, “local provider reachable”, and “matching workload provider field”) so the revised lines in local-ai.md are easier to scan and understand.
88-90: 💤 Low valueOptional: Vary sentence structure to avoid repetition.
Three consecutive bullet points begin with "You," which creates a slightly repetitive rhythm. Consider varying the structure for one or two of them.
✏️ Possible variations
- You ingest large volumes of email / chat and want **embeddings to never leave the machine**. - You want **summary-tree building** to work offline. -- You're privacy-sensitive about background reflection ("subconscious") loops. +- **Background reflection** ("subconscious") loops require privacy-sensitive handling.or
-- You ingest large volumes of email / chat and want **embeddings to never leave the machine**. +- **Large volumes of email / chat** ingestion where embeddings should never leave the machine. - You want **summary-tree building** to work offline. - You're privacy-sensitive about background reflection ("subconscious") loops.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@gitbooks/features/model-routing/local-ai.md` around lines 88 - 90, The three consecutive bullets all start with "You," creating repetition; change one or two bullet leads to vary sentence structure - e.g., reword "You ingest large volumes of email / chat and want **embeddings to never leave the machine**" to "Keep embeddings local when ingesting large volumes of email/chat," or change "You want **summary-tree building** to work offline" to "Enable summary‑tree building to run offline," and/or convert "You're privacy‑sensitive about background reflection ("subconscious") loops" to "Privacy‑sensitive users may want to disable background reflection ("subconscious") loops" so the list reads less repetitive while preserving meaning.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@gitbooks/features/model-routing/local-ai.md`:
- Around line 78-80: Add a concise "Legacy flag behavior" subsection after the
existing paragraph clarifying that Config::workload_local_model (and related
routing logic) treats the `*_provider` fields as the definitive source of truth
once migration/presets complete; explain that `local_ai.usage.*` booleans are
only consulted during preset application/initial migration and do not override
provider fields afterwards, give two explicit examples (e.g.,
`embeddings_provider = "ollama:all-minilm"` routes on-device regardless of
`local_ai.usage.embeddings = false`, and an unset/blank/"cloud" `*_provider`
forces cloud routing regardless of legacy flags), and add a short user
recommendation to prefer setting `*_provider` fields directly.
- Line 39: Split the long sentence into two clearer sentences: one that states
routing for lightweight/medium chat hints (list the hint tokens: hint:reaction,
hint:classify, hint:format, hint:sentiment, hint:summarize, hint:medium,
hint:tool_lite) is allowed to prefer the local provider only when
local_ai.runtime_enabled = true and the configured local provider is reachable,
and a second sentence that states heavy hints (hint:reasoning, hint:agentic,
hint:coding) remain cloud by default unless the matching workload provider field
is explicitly configured locally; keep the same terminology (“router”, “local
provider reachable”, and “matching workload provider field”) so the revised
lines in local-ai.md are easier to scan and understand.
- Around line 88-90: The three consecutive bullets all start with "You,"
creating repetition; change one or two bullet leads to vary sentence structure -
e.g., reword "You ingest large volumes of email / chat and want **embeddings to
never leave the machine**" to "Keep embeddings local when ingesting large
volumes of email/chat," or change "You want **summary-tree building** to work
offline" to "Enable summary‑tree building to run offline," and/or convert
"You're privacy‑sensitive about background reflection ("subconscious") loops" to
"Privacy‑sensitive users may want to disable background reflection
("subconscious") loops" so the list reads less repetitive while preserving
meaning.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 6e10e340-4ff9-4fc9-b989-d76cba5911da
📒 Files selected for processing (1)
gitbooks/features/model-routing/local-ai.md
graycyrus
approved these changes
May 22, 2026
Contributor
graycyrus
left a comment
graycyrus
left a comment
There was a problem hiding this comment.
Looks good, nice work!
senamakel
pushed a commit
to aqilaziz/openhuman
that referenced
this pull request
May 23, 2026
Co-authored-by: Aqil Aziz <aqilaziz@users.noreply.github.com>
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refs #2375
Testing
Summary by CodeRabbit