This repository provides infrastructure-as-code examples and tooling to help node operators deploy and manage xmtpd nodes. xmtpd (XMTP daemon) is the node software that powers the testnet and will power the mainnet of the decentralized XMTP network.
Each node should be configured for high availability (HA) across all required components, including the database, xmtpd, and the MLS validation service.
Database:
- 2vCPU
- 8GB RAM
- Postgres 16.0 or newer
- 20ms commit latency
- 250MB/s throughput
xmtpd:
- 2vCPU
- 2GiB memory
- 1GB/s network link
MLS validation service:
- 2vCPU
- 512MiB memory
Choose your infrastructure approach:
-
Use Terraform if you need to provision the underlying cloud infrastructure.
-
Use Helm charts if you have an existing Kubernetes cluster or want to deploy on managed Kubernetes services.
You can use this Terraform tooling if you need to provision underlying cloud infrastructure on AWS/ECS.
XMTPD Terraform Modules describes how to use Terraform modules to provision AWS/ECS infrastructure for xmtpd nodes.
You can use these Helm charts to deploy xmtpd into an existing Kubernetes cluster or on managed Kubernetes services.
Install xmtpd on Kubernetes using Helm charts describes how to install xmtpd on Kubernetes using Helm charts.
Optionally, if you are using Google Kubernetes Engine, you can run xmtpd on GKE with Nginx ingress and Let's Encrypt.
Deploy xmtpd on Google Kubernetes Engine secured by SSL/TLS describes how to secure your deployment with HTTPS and ingress.
Node operators must implement robust security protocols across all layers of their deployment. The following practices are recommended for production environments:
-
Transport Layer Security (TLS): Enforce TLS for all external communications with automated certificate management. See TLS configuration for Kubernetes or AWS load balancer TLS configuration.
-
Secrets management: Use platform-native secret stores with appropriate access controls. See examples: Kubernetes Secrets for database credentials and signing keys, or AWS Secrets Manager integration for ECS deployments.
-
Network segmentation: Deploy databases in private subnets with security group rules restricting access to VPC CIDR blocks only. See AWS RDS security group configuration and API security groups.
-
Access control: Implement least-privilege IAM roles and Kubernetes service accounts with granular permissions. See ECS task execution role and Kubernetes service accounts.
-
Private key protection: Securely generate and store node signing keys in secrets management systems and register keys with blockchain before deployment. Keys stored as
XMTPD_SIGNER_PRIVATE_KEYin Kubernetes Secrets or AWS Secrets Manager. -
Database security: Use managed PostgreSQL with SSL connections, automated backups, and deletion protection. See Aurora PostgreSQL configuration.
-
Pod security standards: Configure security contexts with read-only root filesystems, drop unnecessary Linux capabilities, and run as non-root users. See security context examples in xmtpd values.
-
Health monitoring and alerting: Deploy Prometheus-based monitoring with service discovery and health checks. See Prometheus setup guide to learn how to automatically scrape metrics from xmtpd pods using PodMonitor, visualize metrics in Grafana, and set alerts.
- Data retention and pruning: Implement automated database pruning to prevent data bloat and maintain node performance. See database pruning guide and prune CronJob configuration for details.
To learn more about the networking architecture xmtpd uses, see XMTP Node Communication APIs in the xmtpd repo.
Currently, xmtpd APIs are implemented using the Connect-RPC library, which allows gRPC and gRPC-Web clients out of the box.
Because this library uses HTTP2 instead of gRPC, it also relies on requests headers to properly function, including CORS headers.
When using a load balancer in front of xmtpd, make sure it correctly forwards all headers, including CORS.
Contributions are welcome! See the contributing guidelines for details on how to get involved.