A practical telemetry model for measuring first-seen latency, decode and decision time, transaction landing, commitment, retries, and tail failures across a Solana trading pipeline.
Rrpc edge Team·Jul 22, 2026·12 min read
Frequently asked questions
What should I monitor in a Solana trading bot?
Monitor every stage from source receipt through decode, decision, signing, submission, first network observation, processed, confirmed, and finalized. Track latency histograms, landing rate, expiry rate, retry count, stream gaps, fork outcomes, and expected versus realized execution value.
How do I measure Solana transaction landing latency?
Start the clock when the client submits signed bytes and stop it on independent network evidence matched by signature. Record first-seen, processed, confirmed, and finalized as separate timestamps. A gateway acknowledgement only measures acceptance by the gateway.
Should Solana latency dashboards show average or p99?
Show p50, p95, and p99 plus timeout and missing-observation rates. Average latency hides the congested tail where competitive transactions are most likely to miss. Keep distributions segmented by transaction class, route, region, and leader distance.
Can I put transaction signatures in metric labels?
No. Signatures, wallet addresses, blockhashes, and trace IDs create unbounded metric cardinality. Keep them in traces or structured logs. Metrics should use bounded attributes such as route, region, transaction class, outcome, and commitment.
What is the difference between first-seen, processed, confirmed, and finalized telemetry?
First-seen is an observation at the propagation layer and may belong to a losing fork. Processed means one validator executed the transaction. Confirmed means a supermajority voted for the block. Finalized means the block is rooted. Store every transition rather than overwriting one status field.
A green RPC health check can coexist with a losing trading system. The endpoint may answer while your
stream lags, decoder queues, strategy misses its deadline, sender targets a stale leader, or transaction
expires unseen.
The unit of observability should be the full action path:
Joins retries or rebuilt transactions representing one business action
Signature
A concrete Solana transaction is signed
Joins submission with on-chain observations
One intent may produce several signatures if a blockhash expires and the transaction is rebuilt. One
signature should never represent two intents.
That distinction prevents a common reporting error. Counting signatures as independent trades inflates
attempt volume, while collapsing every retry into one row can hide duplicate execution risk.
Use a stable internal intent_id across rebuilds. Attach signature only after signing.
Store append-only events rather than updating one mutable status row. A minimal schema looks like this:
telemetry-event.ts
type TradingTelemetryEvent = { observedAt: string; // wall-clock UTC for cross-system joins monotonicNs?: bigint; // local duration math event: string; // e.g. "tx.first_seen" opportunityId?: string; intentId?: string; signature?: string; traceId?: string; strategy: string; transactionClass?: string; source?: "shred" | "grpc" | "rpc"; route?: "tpu" | "jito" | "rpc-forward"; clientRegion
The type is illustrative, not a protocol standard. Keep names stable in your own system. OpenTelemetry
semantic conventions provide common names
for standard operations; Solana-specific attributes will remain custom until a shared convention
exists.
Wall-clock timestamps are necessary when joining events across machines. They can jump when a clock is
corrected. Local duration math should use a monotonic clock that only moves forward.
Cross-machine subtraction still depends on clock synchronization. Monitor clock offset and uncertainty as
first-class infrastructure metrics. If uncertainty is larger than the latency difference you are trying to
claim, the benchmark cannot support the claim.
Propagate trace context through internal HTTP, gRPC, and queue boundaries. The OpenTelemetry
context propagation guide explains how trace
and parent identifiers correlate work across services.
Avoid sending internal baggage to public RPC endpoints unless required and reviewed. Never place keys,
wallet secrets, customer identifiers, or strategy parameters in trace baggage.
An asynchronous on-chain observation is not always a child of the submit request. Link the observation span
to the submission trace using the signature or intent record. Forcing every asynchronous relationship into
a parent-child tree makes traces misleading.
Avoid using signature, blockhash, wallet address, account address, slot, trace ID, or error message as a
metric label. Those values have high or unbounded cardinality and create a new time series for nearly every
event.
OpenTelemetry's metrics guide explains that each
unique attribute combination needs aggregation state. It also documents the SDK cardinality limit and
overflow behavior. Put high-cardinality identifiers in traces or structured logs instead.
Logs should be machine-queryable and redacted at the source. Never log private keys, seed phrases, raw
authorization headers, signed strategy configuration, or customer payloads by default.
Attach trace_id and span_id when available. The OpenTelemetry
signals model separates traces, metrics, and logs while
allowing them to describe the same underlying activity.
First-seen is not a Solana RPC commitment. It is a propagation-layer observation. Processed, confirmed,
and finalized represent progressively stronger states.
The official signatureSubscribe method can
notify a client when a signature reaches the requested commitment. The
getSignatureStatuses method supports explicit
status checks and reconciliation. Use an independent observation path when measuring a sender so the
system under test is not grading its own result.
"98% landing rate" is meaningless without eligibility rules. Define the denominator before the test.
For example:
landing-rate-contract.txt
eligible attempts:- signed with a valid recent blockhash- passed local simulation or was intentionally sent without it- submitted before the opportunity deadline- had enough balance for fee and execution- not canceled by strategy policylanding rate = landed eligible intents / all eligible intents
Report excluded attempts by reason. Otherwise a strategy can improve its displayed landing rate by silently
dropping difficult transactions before they enter the denominator.
Track both intent and attempt views:
Intent landing rate: did the business action land at least once?
Attempt landing rate: what share of submitted signatures landed?
Duplicate execution rate: did one intent land more than once?
Expiry rate: did every signature for an intent age out?
The last two expose retry policies that look successful while creating economic risk.
Group landing rate and time-to-land by fee bucket and route. A rising fee can correlate with congestion,
so an unadjusted chart may misleadingly show high fees performing worse. Segment by congestion regime,
transaction class, and writable-account set when the data permits.
The economic view stops infrastructure optimization from becoming a latency contest with no business
outcome. A route that saves time but costs more than the edge is not an improvement.
99.9% of expected stream sequence received over a rolling window.
99% of decoded events complete within the workload-specific budget.
Eligible-intent landing rate stays above the route-specific floor.
Duplicate execution remains zero.
Clock offset remains below the benchmark uncertainty budget.
Choose numbers from measured baselines and business tolerance. Do not copy a vendor SLA into a strategy
SLO without testing whether it predicts fills.
Measure the path from first-seen to landed.
rpc edge provides co-located decoded shreds, Yellowstone gRPC, RPC, and transaction delivery with published benchmark methodology. Bring your workload and compare the full path.
Solana trading observability is a correlation problem. The system must join an early signal to one business
intent, every signed attempt, independent network observations, commitment transitions, and the final
economic outcome.
Instrument the boundaries once. Keep exact evidence in logs and traces, distributions in metrics, and
high-cardinality identifiers away from metric labels. The result is a dashboard that explains missed fills
instead of merely proving that an endpoint answered.