Most Solana "real-time" APIs advertise latency numbers that measure just one hop. "Sub-second gRPC" usually means the time from their node seeing the slot to their server parsing the transaction — before anything actually reaches you. If you care about total wall-clock time from block confirmation to the moment your bot's webhook handler fires, you have to measure every hop in between.
This post breaks down the architecture behind MadeOnSol's copy-trade signal pipeline, which delivers HMAC-signed webhooks to subscribers in 1–1.5 seconds from the moment a tracked KOL wallet's trade confirms on Solana. Every number below is measured end-to-end on the production path, not marketing latency.
The end-to-end budget
Total wall-clock time is the sum of every stage. Here's what that looks like on our production path, measured at p50:
| Stage | Typical latency |
|---|
| On-chain confirmation | ~400ms (processed commitment) |
| Kaldera gRPC → kol-tracker | ~50ms (Frankfurt/NY closest) |
| Transaction parse + DB insert | ~80ms |
Postgres NOTIFY → signal-evaluator LISTEN | under 10 ms |
| Rule match + signal insert | ~30ms |
HMAC sign + POST to subscriber | ~200–500ms (network-bound) |
| Total (p50) | ~1.0s |
| Total (p95) | ~1.5s |
The two biggest line items are Solana's own confirmation time and the subscriber's network RTT — both outside our control. Everything else we own, and that's where the engineering decisions live.
Stage 1: Ingest, with two concurrent gRPC streams
Any pipeline that wants sub-second latency needs two things at the network edge: low physical distance to a validator, and redundancy.
We run a single kol-tracker service that subscribes to two Kaldera gRPC nodes concurrently — one in Frankfurt, one in New York. For each incoming trade, it deduplicates on transaction signature using a short-lived in-memory cache, so whichever node delivers first wins. The other is thrown away.
const streams = [
createGrpcStream(FRANKFURT_URL),
createGrpcStream(NEW_YORK_URL),
];
const seenSignatures = new LRU({ max: 10_000, ttl: 30_000 });
for (const stream of streams) {
stream.on("data", (msg) => {
const sig = msg.transaction.signature;
if (seenSignatures.has(sig)) return;
seenSignatures.set(sig, true);
handleTrade(msg);
});
}
Dual-region saves roughly 30ms at the median and significantly more at p99, where single-region tail latencies spike when a node is backed up. It also means one node going down is invisible to our downstream consumers — as long as one stream is working, signals flow.
Stage 2: Parse, enrich, and insert
kol-tracker parses the raw transaction, extracts the token mint, SOL amount, direction (buy/sell), and KOL wallet, then inserts a row into kol_trades. Enrichment (token metadata, KOL name, PnL context) is done inline where cheap and deferred to background jobs where expensive — we never want enrichment to sit in the critical path of a signal firing.
The insert triggers a Postgres NOTIFY on a single fan-out channel, madeonsol_events:
CREATE TRIGGER kol_trade_notify
AFTER INSERT ON kol_trades
FOR EACH ROW EXECUTE FUNCTION notify_kol_trade();
CREATE FUNCTION notify_kol_trade() RETURNS trigger AS $$
BEGIN
PERFORM pg_notify(
'madeonsol_events',
json_build_object(
'type', 'kol:trade',
'data', row_to_json(NEW)
)::text
);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
pg_notify is one of those Postgres features that doesn't get enough attention. It's in-process, requires no extra infrastructure, and delivery is measured in milliseconds on a healthy database. For an event bus where all producers and consumers already have a DB connection, it's essentially free and always fast.
Importantly, payload is capped at 8KB. The trade row fits comfortably. For anything larger, you'd notify with just an ID and have consumers re-fetch — we've never needed to.
Stage 3: Fan-out, rule evaluation, signal emission
A separate PM2 service, signal-evaluator, does nothing but LISTEN madeonsol_events and evaluate every incoming kol:trade against active copy-trade rules:
listenClient.on("notification", async (msg) => {
const event = JSON.parse(msg.payload);
if (event.type !== "kol:trade") return;
const rules = await getActiveRulesForKol(event.data.kol_wallet);
for (const rule of rules) {
if (!ruleMatches(rule, event.data)) continue;
await emitSignal(rule, event.data);
}
});
Keeping this service single-purpose matters for two reasons:
- Isolation of failure modes. If the webhook delivery logic has a bug or a memory leak, it doesn't take kol-tracker down with it. The ingest pipeline keeps writing to
kol_trades regardless of whether signals are firing.
- Independent scaling. If webhook fan-out becomes CPU-bound, we can run multiple signal-evaluator replicas without touching ingestion. Postgres
LISTEN delivers to every connected listener, so horizontal fan-out is a deploy config change, not code.
Active rules are cached in memory and refreshed on a pg_notify from the copytrade_subscriptions table — so subscription changes propagate in milliseconds, not on a polling interval.
Stage 4: Deliver, with two lanes
Each subscription picks one or both of two delivery modes:
- Webhook:
signAndPost generates an HMAC-SHA256 signature over ${timestamp}.${body} and POSTs the signal to the subscriber's URL. Retries are aggressive and short — 3 attempts with 0ms / 5s / 30s backoff — because a stale signal is worse than a failed delivery.
- WebSocket: a separate service (
ws-streaming) holds open WS connections from browsers and bots. Signals destined for WS get pg_notify'd on a copytrade:signal channel, where ws-streaming picks them up and fans them to user-scoped connections.
Both lanes run in parallel. A subscription with delivery_mode: ["webhook", "websocket"] sees the signal fire simultaneously on both — not sequentially.
Why we don't use Kafka, Redis Streams, or a queue
Every "real-time events" post online seems to default to Kafka or Redis Streams. For this pipeline we deliberately didn't, and the reasoning is worth spelling out:
- Kafka: over-engineered for our volume (a few thousand trades per hour at peak). We'd add a cluster, ZooKeeper or KRaft, a new on-call surface, and a new set of failure modes — to replace
pg_notify that already works and is already monitored.
- Redis Streams: closer to the right size, but adds a second datastore we have to keep in sync with Postgres. Right now, the trade is in exactly one place. Adding Redis means writing transactional code to keep them consistent, and consistency bugs in event pipelines are the worst kind of bugs.
- Job queues (BullMQ, RabbitMQ): signals are time-sensitive. Queues optimize for throughput and durability; we need low tail latency.
pg_notify delivers in under 10 ms; a queue round-trip is typically 20–100ms with poll intervals, retries, and broker hops.
The rule of thumb we've arrived at: if your event bus only needs to fan out to a few consumers, they already connect to Postgres, and your throughput fits comfortably within NOTIFY's 8KB/message limit, pg_notify is strictly better than any external broker. When those assumptions break, revisit. Not before.
Failure modes we actually see
In production, the latency distribution is not symmetric. The long tail comes from a few specific failure modes, each with a specific mitigation:
- Slow subscriber webhooks. Our
POST has an 8-second timeout. A subscriber whose server takes 7 seconds to reply still gets the signal, but the delivery thread was blocked that entire time. We use Promise.all over rule matches so one slow subscriber doesn't block others — classic mistake to avoid.
- Kaldera node flap. A node going dark triggers a reconnect loop with exponential backoff. During the reconnect, we're single-streamed, and latency at the ingest stage jumps. The mitigation is dual-region: if Frankfurt flaps, NY is already there.
