Sidecar Pattern¶

Every plugin in Meridian runs alongside a sidecar process. The plugin talks only to the sidecar on localhost:9191 via gRPC. The sidecar handles everything else: routing messages to and from the bus, enforcing access control, validating schemas, logging for audit, monitoring health, and recovering state on restart.

The plugin never touches the message bus directly. It never communicates with other plugins. It never reaches out to external services without going through the sidecar. This is the foundation of Meridian's security and isolation model.

Why a Sidecar¶

The alternative is embedding bus logic, ACL enforcement, audit logging, and state recovery directly into each SDK. The sidecar approach provides four advantages:

Advantage	Description
Language independence	The SDK is a thin gRPC client. All complex logic (ACL, audit, state recovery, schema translation) lives in the sidecar, written once in Go. Adding a new language SDK means writing a gRPC stub, not reimplementing infrastructure.
Consistent security	ACL enforcement and audit logging happen at the infrastructure level. A plugin cannot bypass access control or skip audit logging because it never has direct access to the bus.
Upgradeability	The sidecar can be upgraded independently of plugins. Security patches, protocol improvements, and new bus backends ship without requiring plugin vendors to rebuild or redeploy.
Zero trust	The plugin operates in a zero-trust environment. It cannot forge message metadata, skip schema validation, or access topics it has not been granted. The sidecar is the enforcement boundary.

Network Model¶

The plugin and its sidecar share a network namespace (typically a Kubernetes pod). The plugin connects to 127.0.0.1:9191 (loopback only). The sidecar connects outbound to the Bus API on port 8443 using mTLS when TLS is enabled.

+---------------------------------------+
|  Pod / Container Group                |
|                                       |
|  +-----------+     +------------+     |
|  |  Plugin   |---->|  Sidecar   |-----------> Bus API (:8443)
|  | (any lang)|gRPC |   (Go)     |     |
|  |           |:9191|            |     |
|  +-----------+     +------------+     |
|                                       |
+---------------------------------------+

The plugin is unaware of the bus topology, cloud provider, other plugin instances, or network configuration. Its only contract is localhost:9191.

Three Operations¶

The sidecar exposes three operations to the plugin via gRPC:

Publish¶

Send a message to a topic on the bus.

Publish(topic, payload) -> (accepted, message_id, error)

The plugin provides the topic and the raw payload. The sidecar wraps the payload in an envelope, stamps metadata, checks ACL, validates the schema, and forwards to the Bus API. The plugin receives back whether the message was accepted and the assigned message_id.

Open a long-lived stream of messages matching a topic pattern.

Subscribe(topic_pattern) -> stream of Envelope

The plugin provides a glob pattern (e.g., platform.data.market.*). The sidecar checks ACL, opens the upstream subscription, and streams matching messages back to the plugin. The stream stays open until the plugin disconnects or the sidecar shuts down.

Request (Request/Reply)¶

A synchronous request-reply pattern built on top of Publish and Subscribe. Used for kernel API calls (e.g., CreateOrder, GetPositions).

What the Sidecar Handles¶

Responsibility	Description
ACL enforcement	Every publish and subscribe operation is checked against the local ACL cache before forwarding. Unauthorized operations are rejected immediately.
Schema validation	Messages must conform to declared protobuf schemas. The sidecar may reject malformed payloads.
Audit logging	All messages are written to the immutable audit log. Every message is observable at three correlated points: the publishing sidecar, the Bus API, and the subscribing sidecar.
Health monitoring	The sidecar sends heartbeats to the Bus API, exposes liveness and readiness probes, and tracks six health dimensions.
State recovery	On restart, the sidecar rebuilds plugin state from durable stores before accepting traffic.
Entitlements	The sidecar checks the plugin's license and feature flags at startup.
Envelope stamping	The sidecar assigns `message_id`, `trace_id`, `source_id`, and `timestamp_ns` to every outbound message.
Schema translation	When enabled, the sidecar translates between schema versions so that plugins at different versions can interoperate.

Publish Flow¶

When a plugin publishes a message, the sidecar executes the following steps in order:

1. Payload size check
   Reject if payload exceeds max_payload_bytes.

2. ACL readiness gate
   Reject all traffic until initial ACL recovery has completed.

3. ACL permission check
   Check the local ACL cache for a PUBLISH or PUBLISH_SUBSCRIBE grant
   matching the target topic. Reject if no grant exists.

4. Envelope stamping
   Assign message_id (UUID v4), trace_id (UUID v4), source_id (instance ID),
   and timestamp_ns (Unix nanoseconds). The plugin must not set these fields.

5. Schema translation
   If schema translation is enabled, translate the payload to the target
   schema version.

6. State sync routing
   If the topic contains ".state.", route through the state sync outbox
   for guaranteed delivery.

7. Forward to Bus API
   Send the stamped envelope to the upstream Bus API.

When a plugin subscribes to a topic pattern, the sidecar executes:

1. Subscription count check
   Reject if the plugin has reached max_subscriptions.

2. ACL readiness gate
   Reject until initial ACL recovery has completed.

3. ACL permission check
   Check the local ACL cache for a SUBSCRIBE or PUBLISH_SUBSCRIBE grant
   matching the topic pattern. Reject if no grant exists.

4. Open upstream subscription
   Register the subscription with the Bus API.

5. Inbound buffering
   When a cache manager is configured, buffer inbound messages using
   high-water/low-water marks to apply backpressure.

6. Schema translation
   If enabled, translate inbound messages to the plugin's schema version.

7. Cache population
   If caching is enabled, populate the local reference data cache.

8. Stream to plugin
   Deliver matching messages to the plugin over the gRPC stream.

Envelope Metadata¶

The sidecar stamps every outbound message with metadata that the plugin must not set:

Field	Description
`message_id`	UUID v4, unique per message across the platform
`trace_id`	UUID v4, end-to-end trace across all hops
`source_id`	Instance ID of the originating sidecar
`timestamp_ns`	Unix nanoseconds when the sidecar received the message
`ts_event`	When the event occurred at the source (plugin may provide; sidecar defaults to `ts_init`)
`ts_init`	When the sidecar first received and timestamped the message

If the plugin sets message_id, trace_id, source_id, or timestamp_ns, the sidecar overwrites them.

Registration¶

Before the sidecar accepts any plugin traffic, it registers with the Bus API:

RegisterInstanceRequest {
  instance_id       -- Unique identifier for this plugin instance
  pool_type         -- Functional classification (e.g., "SignalEnginePool", "OMSPool")
  image             -- Container image reference
  requested_grants  -- ACL grants the plugin requests (topic + action pairs)
  tags              -- Capability tags (e.g., "oms", "ems", "dgm")
}

The Bus API responds with:

RegisterInstanceResponse {
  accepted       -- Whether registration succeeded
  acl_version    -- Current ACL version for this instance
  pending_count  -- Number of requested grants awaiting operator approval
  error          -- Non-empty on failure
}

Registration failure is fatal. The sidecar process exits.

Requested grants are not immediately active. The platform operator must approve each grant before it takes effect. Until approved, the sidecar rejects publish/subscribe operations on those topics.

Startup Sequence (8 Steps)¶

The sidecar executes these steps in strict order on startup. Steps 1 through 3 must succeed before any plugin traffic is accepted.

Step	Name	Blocks Traffic	Fatal on Failure
1	Load identity	Yes	Yes
2	Register with Bus API	Yes	Yes
3	ACL recovery	Yes	Yes
4	State recovery	Yes	No (best-effort)
5	Database hydration	Yes	No (best-effort)
6	Fast-forward	Yes	No (best-effort)
7	Signal readiness	No	Yes
8	Initial subscriptions	No	No (best-effort)

ACL recovery (step 3) must precede state recovery (step 4). This ordering ensures the sidecar has permissions to retrieve state data.

Step 7 starts an HTTP health probe server on :8080: - GET /healthz -- liveness probe (200 for healthy/degraded, 503 for unhealthy) - GET /readyz -- readiness probe (200 only after step 7 completes)

Step 8 publishes an instance.online event to notify the platform that this instance is available.

Health Monitoring¶

Heartbeats¶

The sidecar sends heartbeats to the Bus API every 10 seconds (configurable). Each heartbeat reports six health dimensions:

Dimension	Description
`bus_connectivity`	Whether the sidecar can reach the Bus API
`acl_status`	Whether the local ACL cache has been populated
`identity_status`	Whether the instance identity was loaded
`upstream_latency`	Round-trip time of the most recent heartbeat RPC
`message_rate`	Messages per second (publish + subscribe) over the last interval
`error_rate`	Errors per second over the last interval

Health Status¶

The overall health_status is computed from these dimensions:

Status	Condition
`unhealthy`	`bus_connectivity` is false or `identity_status` is false
`degraded`	`acl_status` is false or `error_rate` exceeds threshold (default: 10/sec)
`healthy`	All other cases

ACL Drift Detection¶

When the acl_version in the heartbeat response exceeds the local ACL version, the sidecar triggers an ACL refresh by calling GetACL on the Bus API and updating the local cache.

Liveness and Readiness¶

Probe	Endpoint	Returns 200 When
Liveness	`GET /healthz`	Status is `healthy` or `degraded`
Readiness	`GET /readyz`	Startup step 7 has completed

Graceful Shutdown¶

On receiving SIGINT or SIGTERM, the sidecar:

Transitions to draining (readiness probes return 503)
Publishes instance.offline event (best-effort, 5-second timeout)
Drains in-flight gRPC RPCs
Stops the HTTP health probe server
Cancels the heartbeat loop and all active subscriptions
Transitions to shutdown

Plugins should register a shutdown hook (on_stop()) to complete any in-progress work before the sidecar terminates the gRPC connection.

Error Handling¶

The sidecar returns standard gRPC status codes:

Code	Meaning	Plugin Action
`UNAVAILABLE`	Sidecar not ready (ACL recovery pending)	Retry after backoff
`PERMISSION_DENIED`	No ACL grant for the requested topic	Do not retry without operator intervention
`INVALID_ARGUMENT`	Payload exceeds size limit	Reduce payload size
`RESOURCE_EXHAUSTED`	Subscription limit reached	Close unused subscriptions before retrying
`INTERNAL`	Upstream or sidecar error	Retry with exponential backoff

Plugins should implement automatic reconnection to the sidecar gRPC endpoint with exponential backoff (initial: 100ms, max: 30s, jitter: +/- 20%).

Next Steps¶

Kernel Model -- the unified execution kernel and order lifecycle
Plugin Types -- detailed guide for each plugin role
SDK Core Concepts -- building on the platform