Configuration

Overview

Configuring the Batcher involves creating a unified configuration object that defines global settings, blockchain adapters, and batching behavior. This guide walks through the complete configuration process—from instantiating adapters to launching the batcher service.

Understanding the configuration system helps you:

Wire multiple blockchain adapters to a single batcher
Configure different batching strategies per blockchain
Set appropriate confirmation levels for your use case
Customize HTTP server and event system behavior

Target Audience

Developers setting up the Batcher who need to:

Connect to one or more blockchain networks
Configure batching behavior for different chains
Set up the HTTP server for receiving inputs
Customize confirmation levels and retry behavior

Configuration Architecture

The batcher supports two configuration approaches:

1. Dynamic Configuration (Recommended)

Use the builder pattern to wire adapters and criteria progressively after instantiation (before initialization):

// 1. Create batcher with base config (no adapters yet)
const batcher = createNewBatcher({
  pollingIntervalMs: 1000,
  port: 3334,
  enableHttpServer: true,
  adapters: {} // Start with no adapters
}, storage);

// 2. Dynamically add adapters with their criteria
batcher
  .addBlockchainAdapter("evm", evmAdapter, { 
    criteriaType: "time", 
    timeWindowMs: 5000 
  })
  .addBlockchainAdapter("midnight", midnightAdapter, { 
    criteriaType: "size", 
    maxBatchSize: 10 
  })
  .setDefaultTarget("evm");

// 3. Initialize and start
await batcher.init();

This approach offers maximum flexibility—adapters can be added progressively before initialization.

2. Unified Configuration (Alternative)

Define everything in a single configuration object:

const config: PaimaBatcherConfig = {
  pollingIntervalMs: 1000,
  port: 3334,
  enableHttpServer: true,
  
  // Define adapters upfront
  adapters: {
    evm: evmAdapter,
    midnight: midnightAdapter
  },
  
  defaultTarget: "evm",
  
  // Per-adapter batching criteria
  batchingCriteria: {
    evm: { criteriaType: "time", timeWindowMs: 5000 },
    midnight: { criteriaType: "size", maxBatchSize: 10 }
  },
  
  confirmationLevel: "wait-receipt"
};

const batcher = createNewBatcher(config, storage);
await batcher.init();

This approach is more concise for static configurations but less flexible.

When to Use Each Approach

Dynamic configuration: When adapters are conditionally enabled or loaded from plugins
Unified configuration: When all adapters are known at startup and configuration is static

Configuration Must Be Done Before Initialization

All configuration methods (addBlockchainAdapter, setDefaultTarget, setBatchingCriteria) must be called before calling init() or runBatcher(). Once the batcher is initialized, the configuration is locked and cannot be modified.

const batcher = createNewBatcher({ /* ... */ }, storage);

// ✅ OK: Adding adapters before init()
batcher.addBlockchainAdapter("evm", evmAdapter, { /* ... */ });

await batcher.init();

// ❌ ERROR: Cannot add adapters after init()
batcher.addBlockchainAdapter("midnight", midnightAdapter, { /* ... */ });
// Throws: "Cannot add adapters after batcher has been initialized"

Configuration Steps (Dynamic Approach)

The dynamic configuration approach follows these steps:

Step 1: Create Base Configuration

Create the base configuration object with global settings. Adapters are not included here—they'll be added dynamically in Step 3:

import { type PaimaBatcherConfig } from "@effectstream/batcher";

const baseConfig: PaimaBatcherConfig = {
  // Polling frequency (how often to check if batching criteria are met)
  pollingIntervalMs: 1000,  // Check every 1 second
  
  // HTTP server configuration
  port: 3334,
  enableHttpServer: true,  // Enable REST API endpoints
  
  // Event system for monitoring
  enableEventSystem: true,  // Enable state transition events
  
  // Signature verification namespace
  namespace: "paima_batcher",  // Used for signature message construction
  
  // Start with empty adapters - we'll add them dynamically
  adapters: {}
}

Global Settings Reference

Setting	Type	Default	Description
`pollingIntervalMs`	`number`	`1000`	How often to check if batching criteria are met (milliseconds)
`port`	`number`	`3000`	HTTP server port
`enableHttpServer`	`boolean`	`true`	Whether to start the HTTP REST API
`enableEventSystem`	`boolean`	`false`	Whether to enable state transition events
`namespace`	`string`	`"paima_batcher"`	Namespace for signature verification
`maxRetries`	`number`	`3`	Maximum retry attempts for failed transactions
`retryDelayMs`	`number`	`1000`	Delay between retry attempts (milliseconds)

tip

Set pollingIntervalMs to match your shortest batching time window. For example, if using timeWindowMs: 5000, a pollingIntervalMs of 1000 ensures responsive batch submission.

Step 2: Instantiate Blockchain Adapters

Instantiate each blockchain adapter you want to use:

import { PaimaL2DefaultAdapter } from "@effectstream/batcher";

const evmAdapter = new PaimaL2DefaultAdapter(
  "0x742d35Cc6634C0532925a3b844Bc9e7595f0bEb",
  "0x59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d",
  0n,
  "mainEvmRPC"
);

Adapter Details

For comprehensive information about adapters, their responsibilities, and examples for different blockchains, see Core Concepts - BlockchainAdapter.

Step 3: Instantiate Batcher and Add Adapters Dynamically

Now create the batcher instance and use the builder pattern to wire adapters:

import { createNewBatcher, FileStorage } from "@effectstream/batcher";

// Create storage
const storage = new FileStorage("./batcher-data");

// Create batcher using the factory function
const batcher = createNewBatcher(baseConfig, storage);

// Wire adapters using addBlockchainAdapter()
batcher
  .addBlockchainAdapter(
    "evm",              // Target name
    evmAdapter,         // Adapter instance
    {                   // Batching criteria for this adapter
      criteriaType: "time",
      timeWindowMs: 5000
    }
  )
  .addBlockchainAdapter(
    "midnight",
    midnightAdapter,
    {
      criteriaType: "size",
      maxBatchSize: 10
    }
  )
  .addBlockchainAdapter(
    "nft",
    nftAdapter,
    {
      criteriaType: "hybrid",
      timeWindowMs: 10000,
      maxBatchSize: 20
    }
  );

The `createNewBatcher()` Factory Function

createNewBatcher<T>(
  config: PaimaBatcherConfig<T>,
  storage?: BatcherStorage<T>
): PaimaBatcher<T>

This factory function is the recommended way to create a batcher instance. It provides a cleaner API than using the constructor directly.

The `addBlockchainAdapter()` Method

addBlockchainAdapter(
  name: string,                          // Target name for routing
  adapter: BlockchainAdapter<any>,       // Adapter instance
  criteria: BatchingCriteriaConfig       // When to submit batches
): PaimaBatcher

This method:

Wires a target name to its adapter logic
Assigns batching criteria for this specific adapter
Returns the batcher instance for method chaining
Must be called before init() - throws error if batcher is already initialized

Method Chaining

addBlockchainAdapter() returns this, enabling fluent method chaining to add multiple adapters.

Step 4: Set Default Target

Use setDefaultTarget() to specify which adapter handles inputs without an explicit target field:

batcher.setDefaultTarget("evm");

Now inputs without a target field will automatically route to the evmAdapter.

The `setDefaultTarget()` Method

setDefaultTarget(target: string): PaimaBatcher

This method:

Sets the default adapter for inputs without target field
Validates that the target exists in the adapters map
Returns the batcher instance for method chaining
Must be called before init() - throws error if batcher is already initialized

Required Adapters First

You must call addBlockchainAdapter() for a target before calling setDefaultTarget() with that target name. Otherwise, validation will fail.

How Target Routing Works

The target name (e.g., "evm", "midnight") is how inputs route to the correct adapter:

// Example input specifying target
{
  address: "0x...",
  addressType: 0,
  input: "myCommand|arg1|arg2",
  signature: "0x...",
  timestamp: "1234567890",
  target: "midnight"  // Routes to midnightAdapter
}

// Example input using default target
{
  address: "0x...",
  addressType: 0,
  input: "myCommand|arg1|arg2",
  signature: "0x...",
  timestamp: "1234567890"
  // No target specified -> routes to evmAdapter (the defaultTarget)
}

Target Names Must Match

The target field in inputs must exactly match a name passed to addBlockchainAdapter(). Mismatched targets will result in a 404 error.

Batching Criteria in Dynamic Configuration

Notice that batching criteria are specified per-adapter when calling addBlockchainAdapter():

batcher.addBlockchainAdapter(
  "evm",
  evmAdapter,
  { criteriaType: "time", timeWindowMs: 5000 }  // ← Criteria here
);

Each adapter gets its own independent batching rules. If no criteria is provided, it defaults to:

{ criteriaType: "size", maxBatchSize: 1 }  // Process immediately

Batching Criteria Types

For detailed information about all batching criteria types (time, size, hybrid, value, custom) and their use cases, see The Batching Pipeline - Batching Criteria Types.

Step 5: Initialize and Start the Batcher

With adapters wired and default target set, initialize and start the batcher:

await batcher.init();

console.log("✅ Batcher started successfully");
console.log(`🌐 HTTP Server: http://localhost:${baseConfig.port}`);

The batcher is now running and ready to accept inputs!

Complete Example (Dynamic Configuration)

Here's a complete example using the dynamic approach:

import {
  createNewBatcher,
  FileStorage,
  PaimaL2DefaultAdapter,
  MidnightAdapter
} from "@effectstream/batcher";

// 1. Instantiate adapters
const evmAdapter = new PaimaL2DefaultAdapter(
  "0x742d35Cc6634C0532925a3b844Bc9e7595f0bEb",
  Deno.env.get("EVM_PRIVATE_KEY")!,
  0n,
  "mainEvmRPC"
);

const midnightAdapter = new MidnightAdapter(
  "0xabc...",
  Deno.env.get("MIDNIGHT_WALLET_SEED")!,
  {
    indexer: "http://localhost:8088/api/v1/graphql",
    indexerWS: "ws://localhost:8088/api/v1/graphql/ws",
    node: "http://localhost:9944",
    proofServer: "http://localhost:6300",
    zkConfigPath: "./zkproof.json",
    privateStateStoreName: "batcher-state",
    privateStateId: "batcherPrivateState"
  },
  contractInstance,
  witnesses,
  contractInfo,
  0,
  "midnightRPC"
);

// 2. Create storage
const storage = new FileStorage("./batcher-data");

// 3. Create batcher with base config
const batcher = createNewBatcher({
  pollingIntervalMs: 1000,
  port: 3334,
  enableHttpServer: true,
  enableEventSystem: true,
  adapters: {} // Start empty
}, storage);

// 4. Wire adapters with their criteria
batcher
  .addBlockchainAdapter("evm", evmAdapter, {
    criteriaType: "time",
    timeWindowMs: 5000
  })
  .addBlockchainAdapter("midnight", midnightAdapter, {
    criteriaType: "hybrid",
    timeWindowMs: 10000,
    maxBatchSize: 20
  })
  .setDefaultTarget("evm");

// 5. Initialize and start
await batcher.init();

console.log("🎯 Batcher ready");
console.log("📋 Adapters: evm (default), midnight");

Alternative: Unified Configuration

If you prefer defining everything upfront, you can use the unified configuration approach:

Step 1: Define Complete Configuration

import {
  createNewBatcher,
  FileStorage,
  type PaimaBatcherConfig,
  PaimaL2DefaultAdapter
} from "@effectstream/batcher";

// Instantiate adapters first
const evmAdapter = new PaimaL2DefaultAdapter(/* ... */);
const midnightAdapter = new MidnightAdapter(/* ... */);

// Create unified configuration
const config: PaimaBatcherConfig = {
  pollingIntervalMs: 1000,
  port: 3334,
  enableHttpServer: true,
  enableEventSystem: true,
  
  // Define all adapters upfront
  adapters: {
    evm: evmAdapter,
    midnight: midnightAdapter
  },
  
  // Set default target
  defaultTarget: "evm",
  
  // Per-adapter batching criteria
  batchingCriteria: {
    evm: {
      criteriaType: "time",
      timeWindowMs: 5000
    },
    midnight: {
      criteriaType: "hybrid",
      timeWindowMs: 10000,
      maxBatchSize: 20
    }
  }
};

Step 2: Instantiate and Launch

const storage = new FileStorage("./batcher-data");
const batcher = createNewBatcher(config, storage);
await batcher.init();

This approach is more concise but offers less pre-initialization flexibility.

Confirmation Level Configuration

Confirmation levels can be configured globally or per-adapter:

// Global: applies to all adapters
confirmationLevel: "wait-receipt"

// Per-adapter: different levels for different chains
confirmationLevel: {
  evm: "no-wait",
  midnight: "wait-effectstream-processed"
}

Understanding Confirmation Levels

For detailed information about confirmation levels, their behavior in the pipeline, and when to use each, see:

Storage Configuration

Choose a storage backend for persisting pending inputs:

import { FileStorage } from "@effectstream/batcher";

const storage = new FileStorage("./batcher-data");

Available storage options:

FileStorage: Simple JSON file storage (good for development)
PostgreSQL: Production-grade relational database (coming soon)
Redis: In-memory storage with persistence (coming soon)

Storage is Critical

Storage is the single source of truth for all pending inputs. Choose a reliable storage backend for production use. The batcher survives crashes by reloading inputs from storage on restart.

Graceful Shutdown Configuration

The batcher supports graceful shutdown with customizable hooks that execute at specific phases of the shutdown process.

Shutdown Configuration

import { createNewBatcher } from "@effectstream/batcher";

const batcher = createNewBatcher({
  pollingIntervalMs: 1000,
  adapters: {},
  shutdown: {
    // Timeout for the entire shutdown process
    timeoutMs: 30000,  // 30 seconds
    
    // Signal handling configuration
    signalHandling: {
      signals: ["SIGINT", "SIGTERM"],  // Which signals to listen for
      exitCode: 0                       // Exit code after shutdown
    },
    
    // Custom hooks for each shutdown phase
    hooks: {
      preShutdown: async (batcher) => {
        console.log("🛑 Beginning graceful shutdown...");
      },
      stopAcceptingInputs: async (batcher) => {
        console.log("🚫 Stopped accepting new inputs");
      },
      waitForProcessing: async (batcher) => {
        console.log("⏳ Waiting for batch processing to complete...");
      },
      cleanup: async (batcher) => {
        console.log("🧹 Cleaning up resources...");
      },
      postShutdown: async (batcher) => {
        console.log("✅ Shutdown complete");
      }
    }
  }
}, storage);

batcher
  .addBlockchainAdapter("evm", evmAdapter, { /* criteria */ })
  .setDefaultTarget("evm");

await batcher.init();

Shutdown Hooks

The shutdown process executes in 5 distinct phases, each with an optional hook you can customize:

1. `preShutdown` Hook

Executes: At the very beginning of the shutdown process, before any shutdown logic runs.

Signature:

preShutdown?: (batcher: PaimaBatcher<T>) => Promise<void> | void

Use Cases:

Log shutdown initiation
Send notifications to monitoring systems
Set application state to "shutting down"
Begin draining external queues

Example:

preShutdown: async (batcher) => {
  console.log("🛑 Shutdown initiated");
  await notifyMonitoring({ event: "batcher_shutdown_start" });
  await externalQueue.pauseProducers();
}

2. `stopAcceptingInputs` Hook

Executes: After HTTP server is stopped and polling has ended, but before waiting for ongoing processing.

Signature:

stopAcceptingInputs?: (batcher: PaimaBatcher<T>) => Promise<void> | void

What Happens Before This Hook:

HTTP server is stopped (no new API requests accepted)
Polling interval is cleared (no new batch processing triggered)

Use Cases:

Close external connections
Stop consuming from external queues
Update load balancer health checks
Finalize any input-related state

Example:

stopAcceptingInputs: async (batcher) => {
  console.log("🚫 No longer accepting inputs");
  await messageQueue.disconnect();
  await redis.set("batcher:status", "draining");
}

3. `waitForProcessing` Hook

Executes: While waiting for any in-progress batch processing to complete.

Signature:

waitForProcessing?: (batcher: PaimaBatcher<T>) => Promise<void> | void

What Happens Before This Hook:

The batcher waits for isProcessingBatch to become false
This respects the timeoutMs setting

Use Cases:

Monitor progress of ongoing operations
Log status updates during the wait
Perform parallel cleanup that can happen during processing
Update progress bars or status dashboards

Example:

waitForProcessing: async (batcher) => {
  console.log("⏳ Waiting for ongoing batches to complete...");
  const status = await batcher.getBatchingStatus();
  console.log(`  Pending inputs: ${status.totalPendingInputs}`);
  await metrics.gauge("batcher.shutdown.pending", status.totalPendingInputs);
}

4. `cleanup` Hook

Executes: During the resource cleanup phase, after all processing has finished.

Signature:

cleanup?: (batcher: PaimaBatcher<T>) => Promise<void> | void

What Happens Before This Hook:

All batch processing has completed
Built-in cleanupResources() is called

Use Cases:

Close database connections
Flush logs and metrics
Clean up temporary files
Release locks and semaphores
Disconnect from external services

Example:

cleanup: async (batcher) => {
  console.log("🧹 Cleaning up resources...");
  await database.disconnect();
  await logger.flush();
  await redis.del("batcher:locks:*");
  await fs.remove("./temp-batch-data");
}

5. `postShutdown` Hook

Executes: At the very end, after all shutdown logic has completed successfully.

Signature:

postShutdown?: (batcher: PaimaBatcher<T>) => Promise<void> | void

Use Cases:

Final logging
Send completion notifications
Record shutdown metrics
Update external state to "stopped"

Example:

postShutdown: async (batcher) => {
  console.log("✅ Batcher shutdown complete");
  await notifyMonitoring({ event: "batcher_shutdown_complete" });
  await redis.set("batcher:status", "stopped");
  await metrics.increment("batcher.shutdowns.successful");
}

Shutdown Process Flow

The complete shutdown sequence:

1. preShutdown hook
   ↓
2. Stop HTTP server
   Stop polling interval
   ↓
3. stopAcceptingInputs hook
   ↓
4. Wait for isProcessingBatch === false (respects timeoutMs)
   ↓
5. waitForProcessing hook
   ↓
6. Call built-in cleanupResources()
   ↓
7. cleanup hook
   ↓
8. postShutdown hook
   ↓
9. Shutdown complete

Error Handling

If any hook throws an error:

Default: Shutdown process stops and error is thrown
With force: true: Error is logged but shutdown continues

// Trigger shutdown with force option
await batcher.gracefulShutdown(hooks, { 
  timeoutMs: 30000,
  force: true  // Continue shutdown even if hooks fail
});

Timeout Behavior

The timeoutMs setting applies to the entire shutdown process:

shutdown: {
  timeoutMs: 30000  // 30 second timeout for complete shutdown
}

If the timeout is reached:

A warning is logged
Shutdown completes immediately
Any pending operations are interrupted

Manual Shutdown

You can trigger shutdown manually (without signals):

// Using the Effection operation (recommended)
import { main } from "effection";

main(function* () {
  yield* batcher.runBatcher();
  
  // Later...
  yield* batcher.gracefulShutdownOp(hooks, { timeoutMs: 30000 });
});

// Using the async version
await batcher.gracefulShutdown(hooks, { timeoutMs: 30000 });

Configuration Validation

The batcher validates configuration and provides helpful error messages:

const batcher = createNewBatcher({ adapters: {} }, storage);

// ❌ Invalid default target (adapter not added yet)
batcher.setDefaultTarget("evm");
// Error: "evm" not in adapters - must call addBlockchainAdapter first

// ❌ Missing required criteria fields
batcher.addBlockchainAdapter("evm", evmAdapter, {
  criteriaType: "time"  // Error: timeWindowMs required for time criteria
});

// ✅ Correct usage
batcher
  .addBlockchainAdapter("evm", evmAdapter, {
    criteriaType: "time",
    timeWindowMs: 5000  // All required fields present
  })
  .setDefaultTarget("evm");  // Valid: "evm" exists now

// ❌ Cannot modify after initialization
await batcher.init();
batcher.addBlockchainAdapter("midnight", midnightAdapter, { /* ... */ });
// Error: "Cannot add adapters after batcher has been initialized"

Next Steps

Learn about Custom Adapters to support new blockchains
Explore Batching Criteria for advanced batch timing strategies
Review The Batching Pipeline to understand the input lifecycle
Check HTTP API Reference for endpoint documentation

Overview​

Target Audience​

Configuration Architecture​

1. Dynamic Configuration (Recommended)​

2. Unified Configuration (Alternative)​

Configuration Steps (Dynamic Approach)​

Step 1: Create Base Configuration​

Global Settings Reference​

Step 2: Instantiate Blockchain Adapters​

Step 3: Instantiate Batcher and Add Adapters Dynamically​

The createNewBatcher() Factory Function​

The addBlockchainAdapter() Method​

Step 4: Set Default Target​

The setDefaultTarget() Method​

How Target Routing Works​

Batching Criteria in Dynamic Configuration​

Step 5: Initialize and Start the Batcher​

Complete Example (Dynamic Configuration)​

Alternative: Unified Configuration​

Step 1: Define Complete Configuration​

Step 2: Instantiate and Launch​

Confirmation Level Configuration​

Storage Configuration​

Graceful Shutdown Configuration​

Shutdown Configuration​

Shutdown Hooks​

1. preShutdown Hook​

2. stopAcceptingInputs Hook​

3. waitForProcessing Hook​

4. cleanup Hook​

5. postShutdown Hook​

Shutdown Process Flow​

Error Handling​

Timeout Behavior​

Manual Shutdown​

Configuration Validation​

Next Steps​

Overview

Target Audience

Configuration Architecture

1. Dynamic Configuration (Recommended)

2. Unified Configuration (Alternative)

Configuration Steps (Dynamic Approach)

Step 1: Create Base Configuration

Global Settings Reference

Step 2: Instantiate Blockchain Adapters

Step 3: Instantiate Batcher and Add Adapters Dynamically

The `createNewBatcher()` Factory Function

The `addBlockchainAdapter()` Method

Step 4: Set Default Target

The `setDefaultTarget()` Method

How Target Routing Works

Batching Criteria in Dynamic Configuration

Step 5: Initialize and Start the Batcher

Complete Example (Dynamic Configuration)

Alternative: Unified Configuration

Step 1: Define Complete Configuration

Step 2: Instantiate and Launch

Confirmation Level Configuration

Storage Configuration

Graceful Shutdown Configuration

Shutdown Configuration

Shutdown Hooks

1. `preShutdown` Hook

2. `stopAcceptingInputs` Hook

3. `waitForProcessing` Hook

4. `cleanup` Hook

5. `postShutdown` Hook

Shutdown Process Flow

Error Handling

Timeout Behavior

Manual Shutdown

Configuration Validation

Next Steps