Conversation
…nd add unit tests Replace the heuristic-based InformationIntegrationTheory.calculate_phi() with a tractable bipartition mutual-information approximation (Tononi 2004): - Convert subsystem dicts to numeric vectors via recursive flattening - Enumerate all non-trivial bipartitions at subsystem level (63 cuts) - φ = min MI across all cuts, with noise-floor suppression for idle states - Preserve contradiction penalty from self-model validator - Add 'phi' field to WebSocket broadcast payload for acceptance criteria - 27 unit tests: idle→φ=0, active→φ>0, penalty, helpers, performance <50ms Co-authored-by: Steake <530040+Steake@users.noreply.github.com>
…nd test coverage Co-authored-by: Steake <530040+Steake@users.noreply.github.com>
|
@copilot PRs #114 (schema contracts) and #117 (Global Workspace) have merged to main, causing conflicts with this branch. Please rebase onto current main, resolve conflicts, and mark ready for review. The φ calculator must integrate cleanly with the schema layer and the already-merged Global Workspace broadcaster. |
There was a problem hiding this comment.
Pull request overview
Implements a tractable Integrated Information Theory (IIT) φ calculation in the unified consciousness engine using a bipartition mutual-information approximation, and adds unit tests to validate behavior and latency.
Changes:
- Replaced the previous heuristic/stub φ calculation with a subsystem-level bipartition MI approximation (histogram-binned entropy) and contradiction penalty integration.
- Added a new unit test suite covering idle/active behavior, helper flattening, contradiction penalty effects, and a latency target.
- Added a
.gitignoreentry to exclude query-related test JSON artifacts.
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
backend/core/unified_consciousness_engine.py |
Adds numpy-based bipartition MI φ calculator and includes phi in WebSocket update payload. |
tests/backend/test_iit_phi_calculator.py |
Adds comprehensive unit tests for φ computation, helpers, and a runtime threshold check. |
.gitignore |
Ignores query_*test*.json files produced during local testing. |
| def _binned_entropy(values: np.ndarray, num_bins: int = _NUM_BINS) -> float: | ||
| """Shannon entropy (bits) estimated via histogram binning.""" | ||
| if values.size < 2: | ||
| return 0.0 | ||
|
|
||
| # Count non-empty/non-zero values as information | ||
| info_count = 0 | ||
| for key, value in subsystem.items(): | ||
| if value: # Non-empty, non-zero, non-None | ||
| if isinstance(value, (list, dict)): | ||
| info_count += len(value) if value else 0 | ||
| elif isinstance(value, (int, float)): | ||
| info_count += 1 if value != 0 else 0 | ||
| elif isinstance(value, str): | ||
| info_count += len(value.split()) if value.strip() else 0 | ||
| else: | ||
| info_count += 1 | ||
|
|
||
| return float(info_count) | ||
|
|
||
| def _calculate_integration(self, subsystem1: Dict[str, Any], subsystem2: Dict[str, Any]) -> float: | ||
| """Calculate integration between two subsystems""" | ||
| # Look for shared concepts, cross-references, or causal relationships | ||
| shared_concepts = 0 | ||
|
|
||
| # Simple heuristic: look for overlapping keys or values | ||
| keys1 = set(subsystem1.keys()) | ||
| keys2 = set(subsystem2.keys()) | ||
| shared_keys = keys1.intersection(keys2) | ||
| shared_concepts += len(shared_keys) | ||
|
|
||
| # Look for cross-references in values (simplified) | ||
| values1 = str(subsystem1).lower() | ||
| values2 = str(subsystem2).lower() | ||
|
|
||
| # Count word overlaps as integration | ||
| words1 = set(values1.split()) | ||
| words2 = set(values2.split()) | ||
| shared_words = words1.intersection(words2) | ||
| shared_concepts += len(shared_words) | ||
|
|
||
| return float(shared_concepts) | ||
| # ptp == 0 means all values are identical → zero entropy. | ||
| if np.ptp(values) == 0: | ||
| return 0.0 | ||
| counts, _ = np.histogram(values, bins=num_bins) | ||
| total = counts.sum() | ||
| if total == 0: | ||
| return 0.0 | ||
| probs = counts / total | ||
| probs = probs[probs > 0] | ||
| return float(-np.sum(probs * np.log2(probs))) | ||
|
|
||
| @classmethod | ||
| def _bipartition_mi( | ||
| cls, | ||
| vectors: List[np.ndarray], | ||
| partition_a: tuple, | ||
| partition_b: tuple, | ||
| ) -> float: | ||
| """Mutual information across a subsystem-level bipartition. | ||
|
|
||
| MI(A; B) = H(A) + H(B) − H(A ∪ B), clamped to ≥ 0. | ||
| """ | ||
| vec_a = np.concatenate([vectors[i] for i in partition_a]) | ||
| vec_b = np.concatenate([vectors[i] for i in partition_b]) | ||
| vec_all = np.concatenate([vec_a, vec_b]) | ||
|
|
||
| h_a = cls._binned_entropy(vec_a) | ||
| h_b = cls._binned_entropy(vec_b) | ||
| h_all = cls._binned_entropy(vec_all) | ||
|
|
||
| return max(0.0, h_a + h_b - h_all) |
There was a problem hiding this comment.
The mutual-information estimate uses histogram entropy, but _binned_entropy() is called separately on vec_a, vec_b, and vec_all with np.histogram(..., bins=num_bins) and no fixed range/bin-edges. This means each entropy is computed with potentially different bin edges (because min/max differs per vector), which makes H(A)+H(B)-H(A∪B) inconsistent and can produce arbitrary MI values (then masked by the max(0.0, ...) clamp). Consider computing shared bin edges (or at least a shared (min,max) range derived from vec_all) and using that same binning for A, B, and A∪B so the MI estimate is mathematically coherent and stable across partitions.
| def test_compute_under_50ms(self, iit, active_state): | ||
| # Warm up | ||
| iit.calculate_phi(active_state) | ||
|
|
||
| iterations = 50 | ||
| timings = [] | ||
| for _ in range(iterations): | ||
| start = time.perf_counter() | ||
| iit.calculate_phi(active_state) | ||
| timings.append((time.perf_counter() - start) * 1000) | ||
|
|
||
| avg_ms = sum(timings) / len(timings) | ||
| max_ms = max(timings) | ||
|
|
||
| assert avg_ms < 50, f"φ avg {avg_ms:.1f} ms exceeds 50 ms target" | ||
| assert max_ms < 50, f"φ worst-case {max_ms:.1f} ms exceeds 50 ms target" |
There was a problem hiding this comment.
This test hard-fails on a strict wall-clock threshold (avg_ms < 50 and max_ms < 50) over 50 iterations. On shared/variable CI runners this is likely to be flaky even when the code is fine. Consider making it an opt-in performance test (marker/env var), or relaxing the assertion (e.g., higher worst-case bound, percentile-based check, or skip under constrained CI) while still keeping the ≤50ms target documented.
…orkspace), #118 (EmergenceDetector), #119 (transparency), #120 (ChromaDB) Resolve .gitignore conflict (keep both gitignore entries). unified_consciousness_engine.py auto-merged cleanly: IIT φ calculator coexists with GlobalWorkspace broadcaster. All 51 tests pass (30 IIT + 21 GlobalWorkspace). Co-authored-by: Steake <530040+Steake@users.noreply.github.com>
Merged main into the branch at |
…owledge-endpoints # Conflicts: # backend/unified_server.py # godelOS/core_kr/knowledge_store/__init__.py
Description
InformationIntegrationTheory.calculate_phi()used a naive heuristic (shared word/key counting) that didn't implement actual IIT mathematics. Replaced with a tractable Tononi 2004 bipartition MI approximation using numpy only.Algorithm:
Key properties:
phifield added to WebSocket broadcast payload (alongside existingphi_measure)Integration with merged PRs:
unified_consciousness_engine.py.gitignoreconflict resolved (kept both gitignore patterns)Related Issues
Test Evidence
30 unit tests in
tests/backend/test_iit_phi_calculator.py:Coverage: idle→φ=0, active→φ>0, monotonicity with activity, contradiction penalty, recursive flattening helpers, MI non-negativity, worst-case latency <50ms.
Post-merge verification across all affected test suites:
94 tests pass across all affected files, 0 regressions. CodeQL: 0 alerts.
Checklist
pytest tests/)black .andisort .)Original prompt
🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.