Building an Idempotent Crypto Order Pipeline in Node.js

Crypto trading code often looks simple in examples: calculate a signal, call an exchange API, wait for a fill, repeat. Production systems are less forgiving. A single retry at the wrong layer can create a duplicate order. A websocket reconnect can replay the same event twice. A partial fill can arrive before the REST response that created the order. If the bot is running with leverage, those small software mistakes can become expensive very quickly.

The fix is not only "better error handling." A trading bot needs an order pipeline that is idempotent by design. In practical terms, every command should be safe to retry, every external event should be safe to process more than once, and every state transition should be explicit enough that the engine can recover after a crash.

This article walks through a Node.js architecture for building that kind of pipeline. The examples use TypeScript-style JavaScript, but the concepts apply to any runtime.

What idempotency means for trading bots

An idempotent operation can be repeated without changing the outcome beyond the first successful execution. For a crypto order pipeline, that means:

• Submitting the same trade intent twice should not open two positions.
• Receiving the same fill event twice should not double-count inventory.
• Retrying after a timeout should reconcile with the exchange before creating anything new.
• Restarting the process should continue from durable state, not memory.

The key is to separate a trade intent from an exchange order.

A trade intent is your internal decision: "buy 0.02 BTCUSDT because strategy X crossed above threshold Y at candle Z." An exchange order is the external object created on Binance, Bybit, OKX, Coinbase, or another venue. Your system controls the intent. The exchange controls the order lifecycle. The pipeline joins both with stable identifiers.

Core data model

Start with a durable table or collection for intents. SQLite, Postgres, Redis Streams with persistence, or any reliable database can work. For a serious bot, Postgres is usually a good default.

type TradeIntent = {
  id: string;                 // internal UUID
  strategyId: string;
  symbol: string;
  side: "buy" | "sell";
  quantity: string;
  orderType: "market" | "limit";
  limitPrice?: string;
  clientOrderId: string;      // sent to exchange
  status:
    | "created"
    | "submitting"
    | "open"
    | "partially_filled"
    | "filled"
    | "cancelled"
    | "rejected"
    | "unknown";
  exchangeOrderId?: string;
  filledQuantity: string;
  createdAt: string;
  updatedAt: string;
};

The most important field is clientOrderId. Many exchanges let you provide a client-generated ID when placing an order. Use it. Make it deterministic for the trade intent, for example:

const clientOrderId = `lucro_${strategyId}_${signalId}_${side}`;

If your strategy emits the same signal again after a restart, it should produce the same clientOrderId. That gives your bot a natural deduplication key both internally and at the exchange.

Step 1: write intent before touching the exchange

Do not call the exchange first and write your database later. If your process crashes between those two steps, you may have a live order that your system does not know exists.

Instead, write the intent first:

async function createTradeIntent(db, signal) {
  const clientOrderId = buildClientOrderId(signal);

  return db.tradeIntent.upsert({
    where: { clientOrderId },
    update: {},
    create: {
      id: crypto.randomUUID(),
      strategyId: signal.strategyId,
      symbol: signal.symbol,
      side: signal.side,
      quantity: signal.quantity,
      orderType: signal.orderType,
      limitPrice: signal.limitPrice,
      clientOrderId,
      status: "created",
      filledQuantity: "0"
    }
  });
}

The upsert is the first idempotency guard. If the same signal is processed twice, the pipeline returns the existing intent instead of creating a second one.

Step 2: use a single worker to submit pending intents

The submitter should claim work atomically. In Postgres, this is commonly done with a transaction and FOR UPDATE SKIP LOCKED, or with a queue system that supports acknowledgements.

async function claimNextIntent(db) {
  return db.transaction(async (tx) => {
    const intent = await tx.queryOne(`
      select * from trade_intents
      where status in ('created', 'unknown')
      order by created_at asc
      for update skip locked
      limit 1
    `);

    if (!intent) return null;

    await tx.query(`
      update trade_intents
      set status = 'submitting', updated_at = now()
      where id = $1
    `, [intent.id]);

    return intent;
  });
}

This prevents two bot processes from submitting the same intent at the same time. That matters when you run multiple workers for reliability or deploy a new version while the old one is still shutting down.

Step 3: reconcile before retrying after ambiguous failures

Exchange APIs fail in different ways. A clear validation error is easy: mark the intent as rejected. A network timeout is harder. The exchange may have received the order, created it, and failed to return the response.

For ambiguous failures, do not blindly retry a fresh order. First query by clientOrderId.

async function submitIntent(exchange, db, intent) {
  try {
    const order = await exchange.placeOrder({
      symbol: intent.symbol,
      side: intent.side,
      type: intent.orderType,
      quantity: intent.quantity,
      price: intent.limitPrice,
      clientOrderId: intent.clientOrderId
    });

    await markOpen(db, intent.id, order);
  } catch (err) {
    if (isAmbiguousNetworkError(err)) {
      const existing = await exchange.findOrderByClientId(
        intent.symbol,
        intent.clientOrderId
      );

      if (existing) {
        await markOpen(db, intent.id, existing);
      } else {
        await markUnknown(db, intent.id);
      }
      return;
    }

    if (isExchangeValidationError(err)) {
      await markRejected(db, intent.id, err.message);
      return;
    }

    throw err;
  }
}

The unknown state is intentional. It tells the pipeline, "we are not sure what happened, so the next attempt must reconcile before doing anything else." That is much safer than pretending every timeout means failure.

Step 4: process fills with event IDs

Most exchanges provide order updates over websocket streams. Those streams can reconnect, replay, skip, or deliver messages in surprising order. Treat them as an optimization, not the only source of truth.

Store processed exchange event IDs when available. If the venue does not provide stable event IDs, create a fingerprint from fields such as order ID, trade ID, execution timestamp, fill quantity, and price.

async function processFillEvent(db, event) {
  const eventKey = buildExecutionEventKey(event);

  await db.transaction(async (tx) => {
    const inserted = await tx.processedEvent.insertIfAbsent({
      key: eventKey,
      receivedAt: new Date().toISOString()
    });

    if (!inserted) return;

    const intent = await tx.tradeIntent.findByClientOrderId(event.clientOrderId);
    if (!intent) {
      await tx.unmatchedExchangeEvent.create({ event });
      return;
    }

    const nextFilled = decimal(intent.filledQuantity).plus(event.fillQuantity);
    const nextStatus = nextFilled.gte(intent.quantity)
      ? "filled"
      : "partially_filled";

    await tx.tradeIntent.update(intent.id, {
      filledQuantity: nextFilled.toString(),
      status: nextStatus,
      updatedAt: new Date().toISOString()
    });
  });
}

The transaction is doing two things: deduplicating the event and updating the order state. If the same fill is replayed later, insertIfAbsent fails and the inventory is not counted twice.

Step 5: run a reconciliation loop

Even with websockets, every trading bot should have a slower reconciliation loop. Every few seconds or minutes, scan non-terminal intents and compare them with the exchange.

async function reconcileOpenOrders(exchange, db) {
  const intents = await db.tradeIntent.findMany({
    status: ["submitting", "open", "partially_filled", "unknown"]
  });

  for (const intent of intents) {
    const order = await exchange.findOrderByClientId(
      intent.symbol,
      intent.clientOrderId
    );

    if (!order && intent.status === "unknown") {
      await markCreated(db, intent.id);
      continue;
    }

    if (!order) continue;

    await updateFromExchangeSnapshot(db, intent, order);
  }
}

This loop is your recovery mechanism. If a websocket disconnects, the database still converges. If a worker dies after placing an order but before storing the response, the next reconciliation pass can attach the exchange order ID. If the exchange reports a cancellation or rejection, your internal state eventually catches up.

Practical details that reduce risk

Use exact decimal arithmetic for quantities and prices. JavaScript floating point math is not suitable for balances, tick sizes, or PnL calculations. Libraries like decimal.js are cheap compared with a rounding bug.

Keep position accounting separate from order state. Orders are events. Positions are derived exposure. Your position table should be updated from fills, deposits, withdrawals, funding, and fees, not from the fact that an order was submitted.

Rate-limit every exchange adapter. A reconciliation loop that becomes too aggressive during an outage can hit API limits, which makes recovery slower. Use a shared limiter per exchange account.

Record raw exchange payloads. Normalized fields are useful for logic, but raw payloads are essential when debugging disputes, adapter bugs, and edge cases.

Add a kill switch. If the pipeline sees repeated unknown states, unexpected position drift, or too many rejected orders, it should stop submitting new intents until a human or a supervisory process clears the condition.

A simple mental model

A resilient trading bot is not a script that places orders. It is a state machine that happens to talk to an exchange.

The safest version of the pipeline is:

• Create a durable intent.
• Attach a deterministic client order ID.
• Claim work atomically.
• Submit to the exchange.
• Reconcile before retrying ambiguous failures.
• Deduplicate fill events.
• Continuously compare local state with exchange state.

This design will not make a strategy profitable by itself, but it removes a large class of operational mistakes. In automated crypto trading, that is a real edge. The market is already hard enough; the bot should not be creating accidental duplicate risk.

I am building Lucromatic to make this kind of automation easier to test, run, and monitor. Demo: try.lucromatic.com