Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
430 changes: 405 additions & 25 deletions Cargo.lock

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ members = [
"crates/aingle_minimal", # IoT-optimized minimal node
"crates/aingle_contracts", # Smart Contracts (DSL + WASM Runtime)
"crates/aingle_viz", # DAG Visualization Server
"crates/aingle_wal", # Write-Ahead Log (clustering)
"crates/aingle_raft", # Raft consensus (clustering)

# ── Examples ────────────────────────────────────────────────────
"examples/iot_sensor_network",
Expand Down
88 changes: 86 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@
<p align="center">
<a href="https://github.com/ApiliumCode/aingle/actions/workflows/ci.yml"><img src="https://github.com/ApiliumCode/aingle/actions/workflows/ci.yml/badge.svg" alt="Build Status"></a>
<a href="./LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License"></a>
<a href="https://www.rust-lang.org/"><img src="https://img.shields.io/badge/rust-1.70%2B-orange.svg" alt="Rust"></a>
<a href="https://www.rust-lang.org/"><img src="https://img.shields.io/badge/rust-1.83%2B-orange.svg" alt="Rust"></a>
<a href="https://github.com/ApiliumCode/mayros"><img src="https://img.shields.io/badge/Powers-Mayros%20AI-blueviolet?logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0id2hpdGUiPjxwYXRoIGQ9Ik0xMiAyQzYuNDggMiAyIDYuNDggMiAxMnM0LjQ4IDEwIDEwIDEwIDEwLTQuNDggMTAtMTBTMTcuNTIgMiAxMiAyem0wIDNjMS42NiAwIDMgMS4zNCAzIDNzLTEuMzQgMy0zIDMtMy0xLjM0LTMtMyAxLjM0LTMgMy0zem0wIDE0LjJjLTIuNSAwLTQuNzEtMS4yOC02LTMuMjIuMDMtMS45OSA0LTMuMDggNi0zLjA4IDEuOTkgMCA1Ljk3IDEuMDkgNiAzLjA4LTEuMjkgMS45NC0zLjUgMy4yMi02IDMuMjJ6Ii8+PC9zdmc+" alt="Powers Mayros AI"></a>
</p>

<p align="center">
Expand Down Expand Up @@ -156,6 +157,73 @@ Interactive D3.js dashboard. Watch your DAG evolve in real-time. Filter, search,

---

## Clustering

AIngle supports multi-node clustering via Raft consensus for high availability and horizontal scalability. Writes are replicated to all nodes; reads can be served from any node with optional quorum consistency.

### Quick Start (3-node cluster)

```bash
# Node 1 — bootstrap leader
aingle-cortex --port 8081 \
--cluster --cluster-node-id 1 \
--cluster-secret "your-secret-at-least-16-chars" \
--cluster-wal-dir ./data/node1/wal \
--db-path ./data/node1/graph.sled

# Node 2 — joins via node 1
aingle-cortex --port 8082 \
--cluster --cluster-node-id 2 \
--cluster-peers 127.0.0.1:8081 \
--cluster-secret "your-secret-at-least-16-chars" \
--cluster-wal-dir ./data/node2/wal \
--db-path ./data/node2/graph.sled

# Node 3 — joins via node 1
aingle-cortex --port 8083 \
--cluster --cluster-node-id 3 \
--cluster-peers 127.0.0.1:8081 \
--cluster-secret "your-secret-at-least-16-chars" \
--cluster-wal-dir ./data/node3/wal \
--db-path ./data/node3/graph.sled
```

### With TLS encryption

```bash
# Auto-generated self-signed certs (development)
aingle-cortex --port 8081 --cluster --cluster-node-id 1 \
--cluster-secret "your-secret" --cluster-tls

# Custom certificates (production)
aingle-cortex --port 8081 --cluster --cluster-node-id 1 \
--cluster-secret "your-secret" --cluster-tls \
--cluster-tls-cert /path/to/cert.pem \
--cluster-tls-key /path/to/key.pem
```

### Cluster endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/v1/cluster/status` | GET | Node role, leader ID, current term |
| `/api/v1/cluster/members` | GET | All cluster members and their state |
| `/api/v1/cluster/join` | POST | Add a new node to the cluster |
| `/api/v1/cluster/leave` | POST | Gracefully remove a node |
| `/api/v1/cluster/wal/stats` | GET | WAL segment count and disk usage |
| `/api/v1/cluster/wal/verify` | POST | Verify WAL integrity (checksums) |

### Features

- **Raft consensus** — automatic leader election, log replication, and membership changes
- **Streaming snapshots** — 512KB chunked transfer with per-chunk ACK for large datasets
- **Write-Ahead Log** — crash-safe durability with segment rotation and integrity verification
- **TLS encryption** — optional TLS for inter-node communication (self-signed or custom certs)
- **Constant-time auth** — cluster secret verified with timing-safe comparison
- **Quorum reads** — optional strong consistency for read operations

---

## Architecture

```
Expand All @@ -177,6 +245,12 @@ Interactive D3.js dashboard. Watch your DAG evolve in real-time. Filter, search,
│ │ Graph │ │ Engine │ │ (Privacy) │ │ Runtime │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ └───────────┘ │
├────────────────────────────────────────────────────────────────────────┤
│ CONSENSUS LAYER │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌───────────┐ │
│ │ Raft │ │ WAL │ │ Streaming │ │ TLS │ │
│ │ (openraft) │ │ (Durability) │ │ Snapshots │ │ (mTLS) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ └───────────┘ │
├────────────────────────────────────────────────────────────────────────┤
│ NETWORK LAYER │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌───────────┐ │
│ │ Kitsune P2P │ │ CoAP │ │ Gossip │ │ mDNS │ │
Expand All @@ -199,6 +273,9 @@ cd aingle
# Build
cargo build --workspace --release

# Build with clustering support
cargo build -p aingle_cortex --features cluster --release

# Test
cargo test --workspace

Expand All @@ -208,7 +285,7 @@ cargo doc --workspace --no-deps --open

### Prerequisites

- **Rust** 1.70 or later
- **Rust** 1.83 or later
- **libsodium-dev** (cryptography)
- **libssl-dev** (TLS)
- **pkg-config**
Expand Down Expand Up @@ -264,6 +341,13 @@ cargo doc --workspace --no-deps --open
| `aingle_logic` | Prolog-style reasoning engine |
| `aingle_graph` | Semantic graph database |

### Clustering & Consensus

| Component | Purpose |
|-----------|---------|
| `aingle_raft` | Raft consensus (leader election, log replication, streaming snapshots) |
| `aingle_wal` | Write-Ahead Log for crash-safe durability |

### Security & Privacy

| Component | Purpose |
Expand Down
2 changes: 1 addition & 1 deletion crates/aingle_ai/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "aingle_ai"
version = "0.4.2"
version = "0.5.0"
description = "AI integration layer for AIngle - Ineru, Nested Learning, Kaneru"
license = "Apache-2.0 OR LicenseRef-Commercial"
repository = "https://github.com/ApiliumCode/aingle"
Expand Down
2 changes: 1 addition & 1 deletion crates/aingle_contracts/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "aingle_contracts"
version = "0.4.2"
version = "0.5.0"
description = "Smart Contracts DSL and WASM Runtime for AIngle"
license = "Apache-2.0 OR LicenseRef-Commercial"
repository = "https://github.com/ApiliumCode/aingle"
Expand Down
19 changes: 14 additions & 5 deletions crates/aingle_cortex/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "aingle_cortex"
version = "0.4.2"
version = "0.5.0"
description = "Córtex API - REST/GraphQL/SPARQL interface for AIngle semantic graphs"
license = "Apache-2.0 OR LicenseRef-Commercial"
repository = "https://github.com/ApiliumCode/aingle"
Expand All @@ -20,6 +20,7 @@ sparql = ["dep:spargebra"]
auth = ["dep:jsonwebtoken", "dep:argon2"]
p2p = ["dep:quinn", "dep:rustls", "dep:rcgen", "dep:ed25519-dalek", "dep:hex"]
p2p-mdns = ["p2p", "dep:mdns-sd", "dep:if-addrs"]
cluster = ["p2p", "dep:aingle_wal", "dep:aingle_raft", "dep:openraft", "dep:tokio-rustls", "dep:rustls-pemfile"]
full = ["rest", "graphql", "sparql", "auth"]

[[bin]]
Expand All @@ -28,10 +29,10 @@ path = "src/main.rs"

[dependencies]
# Core AIngle crates
aingle_graph = { version = "0.4", path = "../aingle_graph", features = ["sled-backend"] }
aingle_logic = { version = "0.4", path = "../aingle_logic" }
aingle_zk = { version = "0.4", path = "../aingle_zk" }
ineru = { version = "0.4", path = "../ineru" }
aingle_graph = { version = "0.5", path = "../aingle_graph", features = ["sled-backend"] }
aingle_logic = { version = "0.5", path = "../aingle_logic" }
aingle_zk = { version = "0.5", path = "../aingle_zk" }
ineru = { version = "0.5", path = "../ineru" }

# Web framework
axum = { version = "0.8", features = ["ws", "macros"] }
Expand Down Expand Up @@ -68,6 +69,7 @@ rand = "0.9"

# Hashing
blake3 = "1.8"
subtle = "2.6"

# Streaming
tokio-stream = { version = "0.1", features = ["sync"] }
Expand All @@ -92,6 +94,13 @@ rustls = { version = "0.23", default-features = false, features = ["ring", "std"
rcgen = { version = "0.13", optional = true }
ed25519-dalek = { version = "2", features = ["rand_core"], optional = true }
hex = { version = "0.4", optional = true }
# Clustering (optional)
aingle_wal = { version = "0.5", path = "../aingle_wal", optional = true }
aingle_raft = { version = "0.5", path = "../aingle_raft", optional = true }
openraft = { version = "0.10.0-alpha.17", features = ["serde", "type-alias"], optional = true }
tokio-rustls = { version = "0.26", default-features = false, features = ["ring"], optional = true }
rustls-pemfile = { version = "2", optional = true }

dirs = "6"
mdns-sd = { version = "0.18", optional = true }
if-addrs = { version = "0.13", optional = true }
Expand Down
Loading
Loading