Feat: Skills System — composable, hot-swappable capability bundles for Agents

[Nisarg38]

Feature Type

Would make my life easier

Feature Description

Summary

Agents today support tools and instructions, but there's no reusable abstraction for packaging domain-specific capabilities into composable, self-contained units. This proposal introduces Skills — bundles of instructions + tools that can be added to an agent at init time or activated/deactivated mid-conversation by the LLM itself.

---

Motivation

As agents grow in complexity, developers end up with monolithic instruction blobs and flat tool lists. Skills solve this by letting you:

• Compose agents from reusable capability modules (weather, calendar, CRM, etc.)
• Dynamically load/unload capabilities mid-conversation based on user needs
• Define skills as directories (skill.md + tools.py) for easy sharing and version control

---

Design

┌─────────────────────────────────────────────────┐
│                    Agent                        │
│                                                 │
│  base instructions + composed skill instructions│
│                                                 │
│  ┌─────────────┐  ┌─────────────┐              │
│  │ Skill:      │  │ Skill:      │  ← active    │
│  │ weather     │  │ calendar    │    skills     │
│  │ ┌─────────┐ │  │ ┌─────────┐ │              │
│  │ │ instrs  │ │  │ │ instrs  │ │              │
│  │ │ + tools │ │  │ │ + tools │ │              │
│  │ └─────────┘ │  │ └─────────┘ │              │
│  └─────────────┘  └─────────────┘              │
│                                                 │
│  ┌────────────────────────────────────────┐     │
│  │         SkillRegistry                  │     │
│  │  ┌─────────┐ ┌─────────┐ ┌─────────┐  │     │
│  │  │ crm     │ │ search  │ │ email   │  │     │
│  │  └─────────┘ └─────────┘ └─────────┘  │     │
│  │         ↑ activate_skill()             │     │
│  │         ↓ deactivate_skill()           │     │
│  └────────────────────────────────────────┘     │
└─────────────────────────────────────────────────┘

---

API

1. Define a skill as a directory

skills/weather/skill.md

---
name: weather
description: Get current weather for a city
---
Use the get_weather tool to look up conditions for any city.

skills/weather/tools.py

@function_tool
async def get_weather(city: str) -> str:
    ...

2. Load and use skills

from livekit.agents import Agent, Skill, SkillRegistry
from livekit.agents.skills import load_skill_from_directory

# Pre-activate skills at init
agent = Agent(
    instructions="You are a helpful assistant.",
    skills=[load_skill_from_directory("./skills/weather")],
)

# Or let the LLM self-activate from a registry
registry = SkillRegistry.from_directory("./skills/")
agent = Agent(
    instructions="You are a helpful assistant.",
    skill_registry=registry,  # injects activate/deactivate meta-tools
)

# Programmatic mid-conversation activation
await agent.add_skill("weather")
await agent.remove_skill("weather")

---

Key Design Decisions

• Skill extends Toolset — inherits full tool pipeline integration (ToolContext, AgentActivity, provider serialization) for free
• Markdown frontmatter for skill.md — zero external dependencies, parsed with stdlib re
• Meta-tools (activate_skill / deactivate_skill) — when a registry is present, the LLM can self-manage its capabilities
• Instruction composition — base instructions are never mutated; _compose_instructions() rebuilds from base + active skills on every change

PR

#5152

Workarounds / Alternatives

No response

Additional Context

No response

[longcw]

this is a good, we are adding more features around the Toolset, this may relate to #5140

[Nisarg38]

This would def integrate better with the new Toolset feature, I can try work off that

[Nisarg38]

Experimenting with this #5167