User Story
I want to have a dynamic tool installation & execution system for AI agents via whitelisted container images (Podman/Docker compatible)
so that I can safely extend my agent’s capabilities without requiring elevated system privileges.
Context
As a platform developer, I want an AI agent running in the Pengine agentic engine to dynamically discover, install, and execute tools via whitelisted OCI-compatible container images, using a rootless container runtime (Podman preferred), so that the agent can safely extend its capabilities (e.g., file management, CLI utilities) without compromising system security or requiring root access.
The system should provide a native tool management capability that allows the AI agent to dynamically discover, install, and use additional tools through simple, natural language commands.
Users should be able to interact with the agent in an intuitive way, without needing to understand underlying container or installation mechanics.
Expected Behavior
- Expose a built-in tool management interface
- Be aware of:
- All available tools (from a registry)
- All installed tools (active capabilities)
- Guide the user on how to interact with tools via natural language
Example User Interactions
- Discover available tools “Show me all available tools” → Agent responds with a list of installable tools, including descriptions and capabilities.
- Install a tool “Install tool weather”or “Add the weather tool” → Agent installs the tool and confirms availability.
- View installed tools / capabilities “Show me all your capabilities" or “What tools do you have installed?” → Agent lists all currently installed tools and what they can do.
Acceptance Criteria
[ ] - Discover available tools from a central registry
[ ] - Request tool installation via structured message
[ ] - Execute commands inside an isolated Podman container (rootless)
[ ] - Capture and return command output to the user
Security Requirements
- Only whitelisted container images (OCI-compliant) may be installed or executed
- All execution happens inside rootless, sandboxed containers
- No host access unless explicitly granted (e.g., scoped volume mounts)
- Image integrity must be verified (digest pinning)
- No requirement for
sudo or elevated privileges
System Features
- Tool metadata discovery (name, version, capabilities, commands)
- Rootless container execution via Podman
- Optional fallback to Docker if available
- Secure resource limits (CPU, memory)
- Structured output capture via message interface
- Graceful error handling
Proof of Concept (PoC)
- Installing the
pengine/file-manager tool from the whitelist
- Running it via Podman
- Reading a file from a mounted path
- Returning file contents through the agent interface
⚙️ Functional Specification (Podman-first)
1. Architecture Overview
┌─────────────────────────────────────────────────────────┐
│ Pengine AI Engine │
│ │
│ ┌──────────────┐ ┌──────────────────────────────┐ │
│ │ Agent Core │────▶│ Tool Registry │ │
│ │ (LLM + Plan) │ │ (Whitelisted OCI Images) │ │
│ └──────┬───────┘ └──────────────────────────────┘ │
│ │ │
│ ┌──────▼───────┐ ┌──────────────────────────────┐ │
│ │ Message │────▶│ Execution Engine │ │
│ │ Interface │ │ (Podman / Docker Runtime) │ │
│ └──────────────┘ └──────────────────────────────┘ │
└─────────────────────────────────────────────────────────┘
Components
| Component |
Responsibility |
| Agent Core |
LLM planning and tool selection |
| Tool Registry |
Whitelisted OCI images |
| Execution Engine |
Executes via Podman (primary) or Docker (fallback) |
| Runtime Adapter |
Abstracts Podman vs Docker CLI |
| Message Interface |
Handles tool request/response |
Runtime Strategy
Engine Selection Logic
- If Podman is available → use Podman
- Else if Docker is available → use Docker
- Else → trigger Setup Wizard
Command Abstraction
All execution uses a unified interface:
Where:
engine = podman (preferred)
engine = docker (fallback)
🔄 Tool Invocation Protocol
Unchanged — runtime-agnostic.
🔐 Security Model
Rootless Execution
- Containers run without root privileges
- No daemon socket dependency
- No special user group required
Benefits
- No root daemon
- Reduced attack surface
- Safer for desktop environments
User Story
I want to have a dynamic tool installation & execution system for AI agents via whitelisted container images (Podman/Docker compatible)
so that I can safely extend my agent’s capabilities without requiring elevated system privileges.
Context
As a platform developer, I want an AI agent running in the Pengine agentic engine to dynamically discover, install, and execute tools via whitelisted OCI-compatible container images, using a rootless container runtime (Podman preferred), so that the agent can safely extend its capabilities (e.g., file management, CLI utilities) without compromising system security or requiring root access.
The system should provide a native tool management capability that allows the AI agent to dynamically discover, install, and use additional tools through simple, natural language commands.
Users should be able to interact with the agent in an intuitive way, without needing to understand underlying container or installation mechanics.
Expected Behavior
Example User Interactions
Acceptance Criteria
[ ] - Discover available tools from a central registry
[ ] - Request tool installation via structured message
[ ] - Execute commands inside an isolated Podman container (rootless)
[ ] - Capture and return command output to the user
Security Requirements
sudoor elevated privilegesSystem Features
Proof of Concept (PoC)
pengine/file-managertool from the whitelist⚙️ Functional Specification (Podman-first)
1. Architecture Overview
┌─────────────────────────────────────────────────────────┐
│ Pengine AI Engine │
│ │
│ ┌──────────────┐ ┌──────────────────────────────┐ │
│ │ Agent Core │────▶│ Tool Registry │ │
│ │ (LLM + Plan) │ │ (Whitelisted OCI Images) │ │
│ └──────┬───────┘ └──────────────────────────────┘ │
│ │ │
│ ┌──────▼───────┐ ┌──────────────────────────────┐ │
│ │ Message │────▶│ Execution Engine │ │
│ │ Interface │ │ (Podman / Docker Runtime) │ │
│ └──────────────┘ └──────────────────────────────┘ │
└─────────────────────────────────────────────────────────┘
Components
Runtime Strategy
Engine Selection Logic
Command Abstraction
All execution uses a unified interface:
Where:
engine = podman(preferred)engine = docker(fallback)🔄 Tool Invocation Protocol
Unchanged — runtime-agnostic.
🔐 Security Model
Rootless Execution
Benefits