CARD: Towards Conditional Design of Multi-agent Topological Structures

CARD (Conditional Design of Multi-agent Topological Structures) is a novel framework that leverages Large Language Models (LLMs) to build collaborative agent systems using dynamic graph structures. It integrates Graph Neural Networks (GNNs) with multi-agent systems to enable intelligent agent collaboration and reasoning.

Authors: Tongtong Wu, Yanming Li, Ziye Tang, Chen Jiang, Linhao Luo, Guilin Qi, Shirui Pan, Gholamreza Haffari

Features

• Dynamic Graph-Based Agent Collaboration: Build agent networks with learnable spatial and temporal connections
• GNN-Enhanced Reasoning: Use Graph Neural Networks to optimize agent collaboration patterns
• Flexible Agent System: Support for various agent types including code writing, mathematical reasoning, analysis, and more
• External Tool Integration: Built-in tools for web search, code execution, RAG, and more
• Multiple LLM Support: Compatible with GPT-4, Claude, DeepSeek, Llama, and other models
• Benchmark Evaluation: Ready-to-use experiments on MMLU, HumanEval, and GSM8K datasets

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         CARD Framework                          │
├─────────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
│  │   Agents    │  │   Tools     │  │   Dynamic Information   │  │
│  │ - CodeWriter│  │ - Search    │  │ - LLM Profiles          │  │
│  │ - MathSolver│  │ - Executor  │  │ - Tool Capabilities     │  │
│  │ - Analyze   │  │ - RAG       │  │ - Knowledge Sources     │  │
│  └──────┬──────┘  └──────┬──────┘  └────────────┬────────────┘  │
│         │                │                      │               │
│         └────────────────┼──────────────────────┘               │
│                          ▼                                      │
│              ┌───────────────────────────┐                      │
│              │   Graph Neural Network    │                      │
│              │   (GCN + Feature Fusion)  │                      │
│              └───────────┬───────────────┘                      │
│                          ▼                                      │
│              ┌───────────────────────┐                          │
│              │   Dynamic Graph       │                          │
│              │   - Spatial Edges     │                          │
│              │   - Temporal Edges    │                          │
│              └───────────────────────┘                          │
└─────────────────────────────────────────────────────────────────┘

Installation

# Clone the repository
git clone https://github.com/Warma10032/CARD.git
cd CARD

# Install dependencies
pip install -r requirements.txt

# Set up environment variables (copy from template.env)
cp template.env .env
# Edit .env with your API keys

Experiments

Run benchmarks on standard datasets:

MMLU

Train

python experiments/run_mmlu.py \
    --phase train \
    --eval_group "cycle" \
    --mode FullConnected \
    --num_iterations 10 \
    --agent_nums 5 \
    --batch_size 8 \
    --optimized_spatial

Eval

python experiments/run_mmlu.py \
    --phase eval \
    --eval_group "model_group_1" \
    --mode FullConnected \
    --num_iterations 10 \
    --agent_nums 5 \
    --batch_size 8 \
    --optimized_spatial

Configuration

Configure agents and node layouts in JSON files:

{
  "model_group_1": [
    {"role": "Math Expert", "llm_name": "gpt-4o"},
    {"role": "Code Expert", "llm_name": "gpt-4o"}
  ]
}

Requirements

• Python 3.10+
• PyTorch 2.3+
• Transformers
• PyTorch Geometric
• OpenAI API key (or other LLM providers)

Citation

If you use CARD in your research, please cite:

@inproceedings{card2026,
  title = {CARD: Towards Conditional Design of Multi-agent Topological Structures},
  author = {Tongtong Wu and Yanming Li and Ziye Tang and Chen Jiang and Linhao Luo and Guilin Qi and Shirui Pan and Gholamreza Haffari},
  booktitle = {ICLR},
  year = {2026}
}

Or cite the paper directly:

• Paper PDF

Acknowledgments

• Various open-source search and tool providers
• This code refers to GDesigner