Skip to content

Integrate Antithesis using MOOG into Cardano node test suite #3316

@saratomaz

Description

@saratomaz

We want to integrate Antithesis to run the existing Cardano node tests in an autonomous simulation environment with fault injection (network delays, crashes, timing reordering, etc.), in order to find concurrency and distributed-system bugs that are hard to catch in CI alone.

MOOG will be used to invoke Antithesis runs, the full system under test, including Cardano nodes, supporting services, and the pytest test runner, will be provided via prebuilt Docker images.

The goal is to reuse the current pytest-based test suite unchanged where possible, while enabling deterministic execution and replay of failures within Antithesis.

Motivation

  • The current test suite runs pytest-based integration tests against a running Cardano cluster.
  • CI validates expected behavior, but does not systematically explore alternative schedules, timing variations, or failure scenarios.
  • Antithesis allows us to:
    • explore many possible executions of the same workload,
    • replay failures deterministically,
    • find bugs that do not reproduce reliably in CI.

High-level approach

  • Build Docker images using the existing Nix-based build logic:
    • Cardano node and related binaries
    • Python test environment
  • Publish Docker images on a public registry (like DockerHub or GHCR).
  • Use MOOG only to invoke Antithesis
    • Image references will be immutable (e.g. git SHA or digest)

Required changes

Containerization

  • Create Docker images for:
    • Cardano node services
    • Pytest test runner
  • Move Nix-based builds to image build time.

Test adjustments

  • Run the existing pytest test suite in Antithesis unchanged.
  • Identify environment and execution assumptions (e.g. networking, paths, timing).
  • Fix issues that prevent reliable execution under fault injection (e.g. configuration, retries for eventual consistency).
  • Optionally introduce Antithesis Python SDK assertions where they add value.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions