Sync

State chain blocks are synchronised between peers using a dedicated libp2p stream protocol. This ensures all nodes converge on the same chain state, even if they were offline when blocks were published.

Sync protocol

Property

Value

Protocol ID

/xe/statechain-sync/1.0.0

Gossip topic

xe/statechain

Transport

libp2p stream (JSON over TCP)

Page size

64 blocks per response

Max blocks per sync

10,000

Stream deadline

60 seconds

Rate limit

30-second cooldown per peer

Message types

Request

type syncRequest struct {
    TipIndex int64 `json:"tip_index"` // client's latest known block index
}

A value of -1 means the client has no blocks (not even genesis) and wants everything from index 1 onward. Genesis (index 0) is never sent over sync -- it is configured locally.

TipIndex is range-checked before any arithmetic: values below -1, or above the server's own tip, get an empty syncResponse rather than a page of blocks.

Response

type syncResponse struct {
    Blocks  []*Block `json:"blocks"`
    HasMore bool     `json:"has_more"`
}

Responses are paginated. If HasMore is true, the client should expect additional response pages on the same stream.

Sync flow

Client Node                          Server Node
    │                                    │
    │  open stream                       │
    │───────────────────────────────────►│
    │                                    │
    │  syncRequest{TipIndex: 5}          │
    │───────────────────────────────────►│
    │  close write                       │
    │                                    │
    │  syncResponse{Blocks:[6..69],      │
    │               HasMore: true}       │
    │◄───────────────────────────────────│
    │                                    │
    │  syncResponse{Blocks:[70..100],    │
    │               HasMore: false}      │
    │◄───────────────────────────────────│
    │                                    │
    │  stream closed                     │

1. The client opens a stream to the server using the sync protocol ID.
2. The client sends a syncRequest with its current tip index, then closes the write side.
3. The server reads the request and sends back all blocks after the client's tip, paginated in chunks of 64.
4. Each page is a JSON-encoded syncResponse. The last page has HasMore: false.
5. The client applies each received block via chain.AddBlock().

Trigger mechanisms

Sync is triggered in two ways:

On peer connection

When a new peer connects, the node automatically initiates an outbound sync:

New peer connected
    │
    ▼
Rate limit check (30s cooldown)
    │
    ▼
Open sync stream → send tip index → receive blocks

This ensures nodes catch up immediately when they join the network or reconnect after downtime.

Via gossip

New state chain blocks are broadcast to all peers via the xe/statechain gossip topic. When a node receives a gossip block, it calls chain.AddBlock() directly.

Gap-triggered resync

If a gossiped block would leave a hole behind the tip (the node missed earlier blocks), AddBlock rejects it with ErrGap. The gossip ingest treats that rejection as a demand signal -- the missing range exists on some peer -- and pulls the gap closed instead of waiting.

Gossip block arrives
    │
    ▼
AddBlock → ErrGap
    │
    ▼
Backoff check (30s window)
    │
    ▼
Sync from up to 3 connected peers, stopping at the first success

[!NOTE] Why this exists Connect-time sync is one-shot. Once connections became long-lived, a node whose single sync attempt failed (for example because the peer was mid-replay during a rolling deploy) would sit behind forever, gap-rejecting every later block. The gap-triggered resync is the retry path, throttled to one attempt per backoff window.

Rate limiting

Both inbound and outbound sync are rate-limited independently:

Direction

Cooldown

Purpose

Inbound

30 seconds per peer

Prevents a peer from flooding the server with sync requests

Outbound

30 seconds per peer

Prevents redundant sync requests to the same peer

The rate limiter automatically cleans up stale entries when checking for allowance. Gap-triggered resync has its own separate 30-second backoff, independent of the per-peer cooldowns.

Startup replay

On startup, the chain replays all stored blocks to rebuild the in-memory KV state:

1. NewChain() applies genesis ops.
2. replay() reads all stored blocks (by index) and re-verifies each one -- index contiguity, hash, prev_hash linkage, and signatures against the state as of the previous block -- before applying its ops.
3. After replay, the chain verifies that a valid sys.dao_keyset exists.
4. Sync handlers are registered so the node can catch up from peers.

[!WARNING] Replay refuses to boot on a mismatch A gap, a bad hash, a broken prev_hash link, or a signature that no longer validates aborts startup. The node fails loudly rather than booting into silent divergence from the rest of the network.

[!NOTE] Deterministic rebuild Because operations are pure key-value mutations applied in index order, every node that has the same blocks will arrive at the same KV state. There is no non-deterministic input.

Size limits

Limit

Value

Description

syncPageSize

64

Blocks per response page

syncMaxBlocks

10,000

Maximum total blocks per sync session

syncMaxRequestSize

1,024 bytes

Maximum sync request payload

syncMaxResponseSize

10 MiB

Read limit applied to the sync response stream

syncStreamDeadline

60 seconds

Stream timeout

syncCooldown

30 seconds

Per-peer rate limit interval

Error handling

• Block validation failures during sync are logged but do not abort the sync. The node continues processing remaining blocks -- a single invalid block does not poison the session.
• Stream errors (timeout, disconnect) terminate the sync. The node will retry on the next peer connection, or sooner if a gossiped block trips the gap-triggered resync.
• Rate-limited requests are silently dropped by the server.