Skip to content

Deploying to Production

This guide covers deploying Meridian from development to production.

Development Environment

./launch_dev.sh

This starts the full stack in Docker Compose: Bus API, sidecar fleet, reference plugins, Management UI. All sibling repos are mounted at /workspace/.

Choosing a Bus Backend

Backend Best For Latency Durability
NATS Development, general purpose Low Optional
Kafka Production, audit compliance Medium Strong (infinite retention)
Aeron Market making, HFT Ultra-low (~44ns/op) None (in-memory)
Cloud (SQS, Pub/Sub, Service Bus) Cloud-native deployments Variable Managed

Set via environment variable:

BUS_BACKEND=kafka    # or: nats, aeron, cloud

For regulatory compliance (SEC 17a-4, MiFID II), use Kafka with the durable profile — it provides append-only, infinite-retention storage suitable for audit logs.

Choosing a Storage Backend

Backend Best For
TimescaleDB Time-series data, position history, trade records
ClickHouse Analytics, large-scale aggregation

Set via environment variable:

STORAGE_BACKEND=timescaledb    # or: clickhouse

Plugin Deployment

Plugins are containerized and managed by the instance manager:

  1. Package your plugin as a Docker image
  2. Push to the deployment's container registry
  3. Register via Management UI or API
  4. Instance manager starts the plugin with a paired sidecar

Each plugin gets: - Its own sidecar (ACL-scoped to its pool type) - Health monitoring (heartbeat, liveness, readiness) - Automatic state recovery on restart - Telemetry collection (Prometheus metrics)

ACL Configuration

Each deployment has an isolated ACL namespace: deployment.{id}.**

Plugins can only access topics their pool type authorizes: - A DGM plugin can publish to platform.data.* but not platform.oms.* - An OMS plugin can publish to platform.oms.* but not platform.data.* - ACL grants are requested at registration and approved by the deployment admin

Monitoring

Sidecars collect and report: - Message throughput (publish/subscribe rates per topic) - Latency distributions (p50, p95, p99) - Plugin health status (heartbeat age, uptime) - Error rates and types

Metrics are available via Prometheus endpoints for Grafana dashboards.

Production Checklist

  • Bus backend set to Kafka (durable profile) for audit compliance
  • Storage backend configured (TimescaleDB or ClickHouse)
  • ACL grants reviewed per plugin
  • Health check endpoints verified
  • Monitoring dashboards configured
  • Backup schedule configured
  • Container images tagged with semver (not latest)

Coming Soon

The following production features are on the roadmap:

  • Multi-cloud Helm charts (AWS, GCP, Azure, on-prem)
  • OIDC/SSO authentication
  • Multi-region deployments
  • Disaster recovery (cross-region failover)
  • Co-location node affinity (NY4, LD4, TY3)

Next Steps