Skip to content

Latest commit

 

History

History
269 lines (190 loc) · 13.7 KB

README.md

File metadata and controls

269 lines (190 loc) · 13.7 KB

Zixir

Zixir

Repository: github.com/Zixir-lang/Zixir · Created and maintained by Leo Louvar.

Zixir is an immutable AI automation language that bakes workflow orchestration, resource limits, and observability into one runtime—no Airflow + Redis + Prometheus glue. Small, expression-oriented, on a three-tier runtime: Elixir (orchestrator), Zig (engine), Python (specialist).

Zixir is its own language (own grammar and semantics), implemented with Elixir, compiling to Zig, and calling into Python. You write .zixir source; it is not Elixir or Zig syntax. Zixir is immutable by design: variables cannot be reassigned, so code is easier to reason about and less prone to bugs than in mutable-by-default languages.

Who it's for: Developers and teams building AI automation, agentic workflows, and ML pipelines who prefer a single, expression-oriented language and runtime over managing Airflow, K8s, Redis, and custom YAML. Best fit for engineers who like Elixir/FP, want pattern matching and type inference, and need built-in fault tolerance and observability without extra infra.

Why Zixir?

Problem: Production AI pipelines usually need Airflow, K8s, Redis, Prometheus, custom fault-tolerance code, and YAML.

Solution: One language. Orchestration, checkpointing, resource limits, circuit breakers, and built-in caching (ETS + disk, no Redis) are in the runtime.

Strength Trade-off
Immutable by default (single-assignment; no in-place mutation) —
Built-in caching (ETS + disk) —
Pattern matching (native; unique among workflow tools) —
Interactive REPL —
Fault tolerance (supervision, circuit breakers, retries) —
Less infra, faster dev, type safety Learning Elixir/Zixir; smaller ecosystem; newer project

Why a three-tier runtime?

Each tier does one job well; together: orchestration, speed, and ecosystem.

Tier Role Strength
Elixir Orchestrator Concurrency, fault tolerance, supervision (“let it crash”), OTP. Coordinates tasks and keeps the system up.
Zig Engine Predictable performance, no GC, small binaries. Hot paths: parsing, math, core ops (NIFs).
Python Specialist ML (PyTorch, TensorFlow), data (pandas, numpy), APIs. Use existing libraries without rewriting.

Elixir orchestrates and restarts failed workers; Zig runs hot-path code; Python handles ML and data.

How it works

Zixir has its own grammar (let, expressions, array/map indexing arr[i] and map["key"], engine.op(args), python "module" "function" (args), literals, binary ops, 25+ built-in functions). Source is parsed in Elixir into a Zixir AST, then either:

  • Interpreted — Zixir.eval(source) evaluates the AST in Elixir; engine.* calls run in Zig NIFs, python calls go to Python via a port.
  • Compiled — Zixir.Compiler.compile(source) type-checks, optimizes, and emits Zig; the Zig is compiled to a native binary or run JIT.

Your code stays Zixir-only; the runtime is Elixir + Zig (engine/codegen) + Python (libraries).

Zixir vs. alternatives (honest assessment)

Feature Zixir Airflow Kubeflow Prefect
External infrastructure Elixir runtime only* Redis + DB Kubernetes Minimal
Setup time ~20 min ~2 hours ~2 days ~1 hour
Lines of code (typical ML pipeline) 50–150 200–500 500–1000 150–400
Workflow orchestration ✅ Built-in ✅ ✅ ✅
Checkpointing ✅ Any type ⚠️ JSON only (XCom) ✅ ✅
Resource limits ✅ Code-level primitives ✅ Config (e.g. execution_timeout) ✅ YAML ✅ Decorators
Fault tolerance ✅ Supervision + circuit breakers ⚠️ Basic ⚠️ K8s ⚠️ Basic
Observability ✅ Built-in ⚠️ Manual ⚠️ Complex ⚠️ UI
Pattern matching ✅ Native ❌ ❌ ❌
Interactive REPL ✅ ❌ ❌ ❌
Type inference ✅ ❌ ❌ ❌
Immutable (no reassignment; safer, easier to reason about) ✅ ❌ ❌ ❌
Native performance ✅ Zig NIFs ❌ ❌ ❌
LSP Support ✅ mix zixir.lsp ❌ ❌ ❌

* Elixir + Zig (build-time) required; no Redis, K8s, or separate DB for workflows.

Bottom line: One language replaces the usual Airflow + Redis + Prometheus stack. Requires Elixir and Zig; once set up, you get orchestration, limits, and observability without 3–5 external services.

Layout (three-tier flow)

flowchart TB
  Source[Zixir source / eval]

  subgraph T1["Tier 1: Elixir - Orchestrator"]
    Intent[Intent / routing]
    Memory[Memory / state]
    Supervisor[Supervision]
  end

  subgraph T2["Tier 2: Zig - Engine"]
    NIF[NIFs]
    Math[Math, parsing, core ops]
  end

  subgraph T3["Tier 3: Python - Specialist"]
    Port[Port bridge]
    Libs[ML, data, scripts]
  end

  Source --> Intent
  Intent --> Memory
  Intent --> Supervisor
  Intent -->|hot path| NIF
  Intent -->|library calls| Port
  NIF --> Math
  Port --> Libs
Loading

📚 Zixir Language complete guide

Zixir Language complete guide — Learn the language from zero to real projects. This is what the complete guide shows:

What's inside

Part Content
Part 1: Getting Started (1–5) Installation (all platforms), your first program, REPL, Hello world with explanations
Part 2: Language Fundamentals (6–12) Variables and all data types (Int, Float, Bool, String, Arrays, Maps), map indexing, type conversions, built-in functions, exercises
Part 3: Control Flow (13–17) If/else, while and for loops, pattern matching, decision-making patterns
Part 4: Functions (18–22) Definition and calling, parameters vs arguments, lambdas, recursion, scope and best practices
Part 5: Power Features (23–28) 25+ built-in functions, pipe operator (|>), engine operations (22 Zig NIFs), Python integration, performance
Part 6: Real-World Projects (29–35) Data pipeline, AI text analysis, LLM integration, workflow automation
Part 7: Mastery (36–40) Best practices, performance tips, debugging guide, patterns and anti-patterns
Appendices Grammar reference, quick reference card, common patterns, engine operations table

Key features: Progressive learning, 20+ complete examples, 4 real projects, exercises with solutions, visual aids, error-handling focus, AI/automation emphasis.

Guide stats: 40+ pages · 20+ code examples · 4 projects · 6 exercises per chapter · 25+ built-in functions · 22 engine ops · pipe operator, modulo, map indexing, type conversions · full grammar reference.

PDF: A PDF copy is available at docs/Zixir Language complete guide.pdf (same path as the guide, .pdf for download). Website: zixir-lang.github.io/Zixir/Zixir%20Language%20complete%20guide.pdf. To rebuild locally: Node — npx md-to-pdf "docs/Zixir Language complete guide.md" (output is already named with spaces); or pandoc — ./scripts/build-guide-pdf.sh (Unix) or .\scripts\build-guide-pdf.ps1 (Windows; requires pandoc and LaTeX). To enable automatic PDF build on push: create .github/workflows/build-guide-pdf.yml from scripts/build-guide-pdf-workflow.yml (skip the first 3 comment lines).

Requirements

Requirement Notes
Elixir 1.14+ / OTP 25+ Runtime
Zig 0.15+ Build-time only; run mix zig.get after mix deps.get (Zigler)
~100 MB disk, file system For persistence, checkpoints, cache
Python 3.8+ (optional) For ML/specialist calls; set config :zixir, :python_path if not on PATH

Platforms: Windows, macOS, Linux. Zigler compiles NIFs at mix compile; Python script: priv/python/port_bridge.py.

Optional: MLIR (Phase 4) — For extra optimizations (vectorization, CSE, constant folding) you can add the Beaver dependency on Unix: {:beaver, "~> 0.4"} in mix.exs. Without it, the compiler still runs AST-level optimizations. See docs/MLIR_AND_PYTHON.md. Windows: Beaver/Kinda not supported; use default (no Beaver).

New to Zixir? For step-by-step install of Elixir, Zig, and Python per OS, see SETUP_GUIDE.md.

Quick start

First-time install

git clone https://github.com/Zixir-lang/Zixir.git
cd Zixir
git checkout v7.1.0
mix deps.get
mix zig.get   # after deps.get, for Zigler
mix compile
mix phx.server   # start web UI

Then open http://localhost:4000 in your browser.

Clean build (troubleshoot or reinstall)

Use this when you need a clean build, are troubleshooting, or reinstalling:

# 1. Clone the repository
git clone https://github.com/Zixir-lang/Zixir.git

# 2. Navigate into the project directory (CRITICAL!)
cd Zixir

# 3. Reset any local changes and fetch tags
git checkout -- .
git fetch origin --tags
git checkout v7.1.0

# 4. Full clean
mix clean
mix deps.clean --all

# 5. Get dependencies FIRST
mix deps.get

# 6. Install Zig
mix zig.get

# 7. Compile
mix compile

# 8. Start server
mix phx.server   # start web UI

Then open http://localhost:4000 in your browser.

For Python specialist: ensure Python is on PATH or set in config; recommend a virtualenv. For VectorDB (nine backends: memory, chroma, pinecone, weaviate, qdrant, milvus, pgvector, redis, azure), see docs/VECTORDB_BACKENDS.md for pip install and setup.

Usage

Zixir language (source)

Run Zixir source with eval/1 or run/1:

Zixir.eval("engine.list_sum([1.0, 2.0, 3.0])")
# => {:ok, 6.0}

Zixir.run("let x = 5\nlet y = 5\nx + y")
# => 10

Run a .zixir file (from repo root):

mix zixir.run examples/hello.zixir

Run from any directory: Build a release (mix release), add _build/prod/rel/zixir/bin to PATH, then use zixir_run.sh / zixir_run.bat with a path to a .zixir file. See SETUP_GUIDE.md.

Grammar, types, and standard library: see docs/LANGUAGE.md.

Elixir API: Zixir.run_engine/2 (hot path → Zig NIFs), Zixir.call_python/3 (library calls → Python). Intent/routing: Zixir.Intent. See project_Analysis_for_fork.md for architecture.

Test

mix test

Verification

After Setup, run mix zixir.run examples/hello.zixir. Expected: 11.0. For JIT run: mix zixir run examples/enterprise_test.zr (expected: 28.75). Windows: scripts\verify.ps1. If "mix is not recognized", install Elixir and add to PATH.

Implementation Status

✅ Fully Implemented

Feature Status Notes
Parser Complete Recursive descent; tokenization, expressions, control flow, modulo (%), array and map indexing, pipe operator (|>), try/catch parsing; unclosed-block error (no hang)
Interpreter Complete 25+ built-in functions, pipe operator (|>), array and map indexing, try/catch evaluation, closures, graceful engine error handling
Engine NIFs Complete 20+ Zig operations (sum, product, dot, etc.)
Zig Backend Complete Codegen, functions, optimization passes
Type System Complete Inference, lambda/map/struct types
MLIR Optional Phase 4: with Beaver (Unix) → full MLIR; without → AST optimizations (CSE, constant folding, LICM). See docs/MLIR_AND_PYTHON.md.
Quality/Drift Complete Validation, detection, auto-fix
Experiment Complete A/B testing framework, statistics
Python Port Working Zixir.call_python/3 via ports
Workflow Complete Steps, retries, checkpoints, sandboxing
Observability Complete Logging, metrics, tracing, alerts
Cache Complete ETS + disk caching
CLI/REPL Complete Variable persistence, function persistence, all commands functional
Portable CLI Working zixir_run.sh / zixir_run.bat from release; run from any path
LSP Server ✅ Ready mix zixir.lsp + VS Code integration
Package Manager Complete Zixir.Package: resolve, install (Git/path), list, cache; zixir.toml manifest
VectorDB Complete Nine backends (memory, chroma, pinecone, weaviate, qdrant, milvus, pgvector, redis, azure); cloud resilience (pooling, backoff, circuit breaker, caching, health, metrics). See docs/VECTORDB_BACKENDS.md.

⚠️ Partially Implemented

Feature Status Notes
Python FFI Implemented Port-based (PythonFFI) default; NIF path (PythonNIF + priv/python_nif.zig) when NIF built; Zixir.Python auto-selects
GPU Implemented Detection (CUDA/ROCm/Metal); codegen, compile, and launcher execution; requires nvcc/hipcc/Metal SDK

❌ Aspirational / Not Implemented

None at this time.

Note: See PROJECT_ANALYSIS.md for detailed implementation status. For how MLIR (Phase 4) fits with the three-tier runtime and Python, see docs/MLIR_AND_PYTHON.md. To contribute, see CONTRIBUTING.md. For roadmap and use cases, see ROADMAP.md and docs/USE_CASES.md.

Author & License

Zixir is created and maintained by Leo Louvar. Apache-2.0 — see LICENSE. Contributors: CONTRIBUTORS.md.