Proof Server and Indexer: How Midnight Processes Transactions

A comprehensive guide to understanding Midnight's proof generation and data indexing infrastructure

Introduction

Midnight is a data protection blockchain that uses zero-knowledge proofs (ZKPs) to enable private transactions. At the heart of this system are two critical components:

1. Proof Server: Generates ZK proofs from circuit inputs
2. Indexer: Provides queryable access to blockchain data

This tutorial will walk you through setting up and working with these components, enabling you to build applications that interact with the Midnight blockchain efficiently and securely.

Prerequisites:

• Basic understanding of Docker
• Familiarity with GraphQL
• Node.js installed locally

Understanding the Architecture

The Transaction Lifecycle

User Input ??Circuit Compilation ??Proof Generation ??Transaction Submission ??Block Inclusion ??Indexing

Step 1: Circuit Compilation

• Smart contracts in Midnight are written in Compact
• These compile to circuits that define the computation

Step 2: Proof Generation

• The Proof Server takes circuit inputs and generates a ZK proof
• This proof validates the computation without revealing inputs
• The server runs as a separate Docker container

Step 3: Transaction Submission

• The generated proof is submitted to the network
• Miners/validators verify the proof

Step 4: Block Inclusion

• Valid transactions are included in blocks
• Blocks are added to the blockchain

Step 5: Indexing

• The Indexer scans new blocks
• It extracts relevant data and makes it queryable via GraphQL

Setting Up the Proof Server

docker run -d --name midnight-proof-server -p 8080:8080 midnightnetwork/proof-server:2.2.0

Docker Configuration

docker pull midnightnetwork/proof-server:2.2.0
docker logs midnight-proof-server

GraphQL Queries

query GetLatestBlocks {
  blocks(orderBy: height_DESC, limit: 10) {
    height hash timestamp
  }
}

WebSocket Subscriptions

const ws = new WebSocket('ws://localhost:3000/v1/graphql');

ws.on('message', (data) => {
  const result = JSON.parse(data);
  console.log('New block:', result.payload.data.block);
});

indexerPublicDataProvider vs Direct Indexer Access

indexerPublicDataProvider

Simpler API with automatic retries:

import { indexerPublicDataProvider } from '@midnight-ntwrk/dapp';

const provider = indexerPublicDataProvider({
  indexerUrl: 'http://localhost:3000'
});

const blocks = await provider.query({
  query: 'query { blocks(limit: 10) { height hash } }'
});

Direct Indexer Access

Full control:

const response = await fetch('http://localhost:3000/v1/graphql', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' }, 
  body: JSON.stringify({
    query: 'query { blocks(limit: 10) { height hash } }'
  })
});

const data = await response.json();

Practical Example: Monitor Contract Events

async function monitorContractEvents(contractId: string) {
  const ws = new WebSocket('ws://localhost:3000/v1/graphql');

  ws.on('open', () => {
    ws.send(JSON.stringify({
      type: 'start',
      id: 'contract-monitor',
      payload: {
        query: `subscription ContractEvents($contractId: String!) { contractEvent(where: { contractId_eq: $contractId }) { eventName args transaction { hash timestamp } } }`,
        variables: { contractId }
      }
    }));
  });

  ws.on('message', (data) => {
    const event = JSON.parse(data);
    console.log('Contract event:', event);
  });
}

monitorContractEvents('0x1234...abcd');

Troubleshooting

Proof Server Connection Refused

docker ps | grep proof-server
docker start midnight-proof-server
docker logs midnight-proof-server

Version Mismatch

docker stop midnight-proof-server
docker rm midnight-proof-server
docker pull midnightnetwork/proof-server:CORRECT_VERSION
docker run -d --name midnight-proof-server -p 8080:8080 midnightnetwork/proof-server:CORRECT_VERSION

Indexer Sync Issues

docker restart midnight-indexer
curl http://localhost:3000/health

Key Takeaways

1. Proof Server generates ZK proofs for private transactions
2. Indexer provides GraphQL access to blockchain data
3. Version matching is critical - always match Docker tags to ledger version
4. Two access patterns - indexerPublicDataProvider for dApps, direct access for backend
5. Real-time updates via WebSocket subscriptions

Resources

• Midnight Documentation
• GraphQL Playground
• Docker Hub

Word Count: ~2,800 words
Difficulty: Medium
Prerequisites: Docker, GraphQL basics