[Kernel Feature] Model Lifecycle Manager - Systemd-Based LLM Service Management

[mikejmorgan-ai]

Model Lifecycle Manager - Systemd-Based LLM Service Management

Part of: Kernel-Level AI Enhancements (Tier 1 - User-Space)

Description

Manage LLM models as first-class system services using systemd. This brings "systemctl for AI models" to Cortex Linux.

Effort: 2-3 weeks | Bounty: $150

The Solution

cortex model register llama-70b --path meta-llama/Llama-2-70b-hf --backend vllm --gpus 0,1
cortex model start llama-70b
cortex model status
cortex model enable llama-70b  # auto-start on boot

Features

• Systemd service generation for any LLM backend
• Multi-backend support (vLLM, llama.cpp, Ollama, TGI)
• Automatic GPU memory configuration
• Health check monitoring with auto-restart
• Resource limits via systemd (CPU, memory, PIDs)
• Security hardening (NoNewPrivileges, ProtectSystem)
• SQLite database for configuration persistence

Acceptance Criteria

• cortex model register creates valid systemd service
• cortex model start/stop works reliably
• Health checks trigger auto-restart on failure
• Unit tests pass with >80% coverage

Files

Complete implementation available:

• model_lifecycle.py (~800 lines)
• test_model_lifecycle.py (~600 lines)
• README_MODEL_LIFECYCLE.md

Priority

High - Core infrastructure for AI-native OS vision

[yaya1738]

Interested in this bounty.

Relevant experience:

• systemd service management and unit file generation
• Python CLI tools (argparse, click)
• SQLite for configuration persistence
• Built production autonomous systems with health monitoring

Approach:

• Dynamic systemd unit file generation per model/backend
• Support vLLM, llama.cpp, Ollama, TGI backends
• Health check monitoring with configurable endpoints
• Resource limits via systemd (CPUQuota, MemoryMax, etc.)
• Security hardening (NoNewPrivileges, ProtectSystem, etc.)

Key components:

1. ModelRegistry - SQLite-backed model configuration
2. ServiceGenerator - systemd unit file templating
3. HealthMonitor - HTTP/gRPC health checks
4. ResourceManager - GPU/CPU/memory configuration
5. CLI - cortex model register/start/stop/status/enable

Can start immediately. Already have working prototypes for #222 and #223.

[yaya1738]

Update: Prototype Complete

Working implementation with all features:

Test Results: 36/36 passing

TestBackends (6 tests) ✓
TestModelConfig (6 tests) ✓
TestModelRegistry (7 tests) ✓
TestServiceGenerator (7 tests) ✓
TestServiceController (5 tests) ✓
TestHealthMonitor (2 tests) ✓
TestCLI (1 test) ✓
TestEndToEnd (2 tests) ✓

Implementation:

• model_lifecycle.py (~650 lines)
• 4 backends: vLLM, llama.cpp, Ollama, TGI
• SQLite-backed model registry
• Dynamic systemd unit file generation
• Security hardening (NoNewPrivileges, ProtectSystem, etc.)
• Health monitoring via HTTP endpoints

Demo:

# Register model
cortex model register llama-70b \
  --path meta-llama/Llama-2-70b-hf \
  --backend vllm \
  --gpus 0,1,2,3

# Start service
cortex model start llama-70b

# Check status
cortex model status llama-70b
# Shows: active, memory usage, health status

# Enable auto-start
cortex model enable llama-70b

# View logs
cortex model logs llama-70b -n 100

Generated systemd service includes:

• Resource limits (CPUQuota, MemoryMax)
• Security hardening
• Health check on startup
• Auto-restart on failure
• Journal logging

Ready to submit PR. This is my third prototype for cortexlinux bounties - also have working code for #222 (32/32 tests) and #223 (20/20 tests).