Skip to content

feat: add set_local_voice and list_cloud_capabilities tools#1

Draft
Copilot wants to merge 3 commits into
mainfrom
copilot/vscode-mnybr24d-kgnk
Draft

feat: add set_local_voice and list_cloud_capabilities tools#1
Copilot wants to merge 3 commits into
mainfrom
copilot/vscode-mnybr24d-kgnk

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 14, 2026

Users had no programmatic way for the AI agent to change the local TTS voice by name or report available cloud provider status. Adds two new tools with full three-way registration (TOOLS schema + _exec_tool_impl + _EL_TOOL_NAMES).

set_local_voice

Sets tts_voice_idx by matching a partial voice name against all installed SAPI + Windows OneCore neural voices, then broadcasts voice_settings to all connected UI clients.

Match priority: exact → whole-word boundary → substring — prevents false positives like "Ava" matching "Lava Voice".

# User: "set local voice to Microsoft Ava"
# → tool call: set_local_voice(voice_name="Ava")
# → returns: "Local voice set to 'Microsoft Ava Online (Natural) - English (United States)'."
# → broadcasts voice_settings to all WS clients (UI selector updates live)

On no match, returns the full list of available voice names.

list_cloud_capabilities

Returns a live snapshot of all 8 configured cloud LLM providers (OpenAI, GitHub, Anthropic, Groq, Gemini, DeepSeek, Mistral, xAI) with ✅/🔑 key status and model lists, plus ElevenLabs TTS/STT status and the currently active provider/model. Gives the agent a factual answer when a user asks what cloud services are available.

Alignment

All 72 tools remain in sync across all three surfaces — gaps silently break ElevenLabs ConvAI tool dispatch.

Original prompt

do you more capability via the cloud?

1. **Chronological Review**: The conversation began with the user requesting to set a local voice to "Microsoft Ava." The user then inquired about the feasibility of this request. Following this, the user provided an executive summary detailing the integration of the ULTRON and dqikfox/Vision codebases into a unified AI-automation system, including a flowchart and a comprehensive file inventory. The user also discussed overlaps and conflicts between the two systems, integration points, and proposed a unified project structure. The conversation then shifted to the user asking if more capabilities could be accessed via the cloud.
  1. Intent Mapping:

    • "set local voice to microsoft ava" - User's request to configure a specific voice setting.
    • "is this possible?" - User's inquiry about the feasibility of the previous request.
    • "do you more capability via the cloud?" - User's question regarding additional capabilities available through cloud services.
  2. Technical Inventory:

    • Technologies mentioned include FastAPI, WebSocket, OpenAI, PyAutoGUI, PyTesseract, scikit-learn, and cv2.
    • The integration of two codebases (ULTRON and Vision) was discussed, focusing on their functionalities and how they can work together.
  3. Code Archaeology:

    • The user provided an executive summary that included a detailed inventory of files and their purposes within the ULTRON and Vision components.
    • Specific classes and functions were highlighted, such as AIBrain, FileSorter, and SystemAutomation.
  4. Progress Assessment:

    • The integration of the two systems was outlined, with proposed adapters and API routes for communication between components.
    • The user has made significant progress in detailing the architecture and integration points but has not yet implemented the discussed changes.
  5. Context Validation:

    • All critical information regarding the integration of the two systems, including file inventories and proposed architecture, has been captured for continuation.
  6. Recent Commands Analysis:

    • Last Agent Commands: The user requested to set a local voice and inquired about the possibility of this action. The user also provided an extensive executive summary of the integration project.
    • Tool Results Summary: The conversation included a detailed summary of the integration project, outlining the architecture, file inventories, and proposed changes. The user has not received specific tool results as the conversation has been primarily focused on requests and summaries.
    • Pre-Summary State: The agent was actively discussing the integration of the ULTRON and Vision systems and addressing user inquiries about capabilities.
    • Operation Context: The recent commands were executed to clarify the user's goals regarding voice settings and to explore the potential for cloud capabilities.
1. Conversation Overview: - Primary Objectives: The user requested to set a local voice to "Microsoft Ava" and inquired about its feasibility. They also sought to understand if more capabilities could be accessed via the cloud. - Session Context: The conversation transitioned from voice configuration requests to a detailed executive summary of the integration of the ULTRON and dqikfox/Vision codebases, including architecture and file inventories. - User Intent Evolution: The user initially focused on voice settings and then shifted to discussing the integration of two systems, indicating a broader interest in enhancing system capabilities.
  1. Technical Foundation:
  • FastAPI: Used for the web server framework in the Vision component.
  • OpenAI: Integrated for AI functionalities in the ULTRON component.
  • PyAutoGUI, PyTesseract, scikit-learn, cv2: Libraries mentioned for automation, OCR, machine learning, and image processing.
  1. Codebase Status:
  • live_chat_app.py:
    • Purpose: Main back-end server for handling audio and chat input.
    • Current State: Summarized as part of the integration project, with proposed API routes for communication.
    • Key Code Segments: Includes handling voice commands and routing to ULTRON components.
  • Additional files from both ULTRON and Vision were detailed in the executive summary.
  1. Problem Resolution:
  • Issues Encountered: Overlaps in command handling between ULTRON and Vision systems.
  • Solutions Implemented: Proposed adapters and API routes to streamline communication between the two systems.
  1. Progress Tracking:
  • Completed Tasks: The user has outlined the integration architecture and provided a comprehensive summary.
  • Partially Complete Work: Implementation of the proposed changes and adapters is pending.
  • Validated Outcomes: No specific outcomes have been validated yet as the focus has been on planning and outlining.
  1. Active Work State:
  • Current Focus: The user was discussing the integration of the ULTRON and Vision systems and ex...

Created from VS Code.

dqikfox and others added 2 commits April 11, 2026 23:01
- testpaths: 'tests/' -> '.' (tests live in repo root as test_*.py)
- Remove --cov=elite_ (no elite_ package target) and --cov-fail-under=70
  (fails on a fresh clone since backend requires the server running)
- pylint max-line-length: 100 -> 120 to match ruff config

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI changed the title [WIP] Explore additional capabilities via the cloud feat: add set_local_voice and list_cloud_capabilities tools Apr 14, 2026
Copilot AI requested a review from dqikfox April 14, 2026 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants