Skip to content

feat: add macOS Metal GPU detection for local embeddings#1

Open
lexgenius wants to merge 1 commit intomainfrom
feat/metal-gpu-support
Open

feat: add macOS Metal GPU detection for local embeddings#1
lexgenius wants to merge 1 commit intomainfrom
feat/metal-gpu-support

Conversation

@lexgenius
Copy link
Copy Markdown
Owner

Summary

Currently the plugin only probes for CUDA (NVIDIA) and Vulkan (AMD/Intel) on Linux. On macOS it falls back to CPU-only, even though node-llama-cpp supports Metal natively.

This PR adds Apple Metal GPU detection to _detect_gpu_backend() in both execute.py and helpers/qmd_client.py.

Changes

  • Added import sys to helpers/qmd_client.py
  • Added Metal detection block to _detect_gpu_backend() in both files:
    • Checks sys.platform == darwin
    • Verifies Metal framework exists at /System/Library/Frameworks/Metal.framework
    • Returns metal to enable GPU-accelerated embeddings on Apple Silicon and Intel Macs

Detection order (after this PR)

  1. CUDA — Linux/NVIDIA
  2. Vulkan — Linux AMD/Intel
  3. Metal — macOS Apple Silicon + Intel Macs (new)
  4. CPU fallback

Testing

  • Verified single Metal block insertion in each file (no duplicates)
  • Detection runs only on darwin platforms, no impact on Linux behavior

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant