Skip to main content

Indexing Events

Everything the shielded pool records in a block is a Substrate event of the ShieldedPool pallet. An indexer, an explorer or a wallet backend reads them from each block's System.Events, with @orbinum/protocol or polkadot-api. This page lists the events, shows how to read and follow them, and how to rebuild the commitment tree's leaves from them.

No EVM logs

The precompiles emit no EVM logs. A shield sent through the ShieldedPool precompile has a receipt with an empty logs array, and eth_getLogs returns nothing for it. Its Shielded event is a Substrate event; see Matching an EVM shield to its event.

Events​

EventFieldsEmitted by
Shieldeddepositor, amount, commitment, encrypted_memo, leaf_indexshield, and once per entry of shield_batch
NullifiersSpentnullifiers (2)private_transfer
CommitmentsInsertedcommitments (≤ 2), encrypted_memos (≤ 2), leaf_indices (≤ 2)private_transfer
Unshieldednullifier, amount, recipient, change_commitment, change_encrypted_memo, change_leaf_indexunshield. The three change_* fields are empty for a total unshield
MerkleRootUpdatedold_root, new_root, tree_sizeEvery leaf inserted, by any call
TreeSealedtree_id, final_root, first_leaf_index, leaf_countThe insert that fills a tree
FeeCreditedvalidator, asset_id, amountEvery spend: its fee is credited to the block author's validator account
FeesClaimedvalidator, asset_id, amountclaim_fees
AssetRegistered, AssetVerified, AssetUnverifiedasset_idThe asset registry's governance calls
FieldTypeDecoded by polkadot-api as
commitment, commitments, change_commitment, nullifier, nullifiers, old_root, new_root, final_root32 bytes, a field element in little-endian0x hex string
encrypted_memo, encrypted_memos, change_encrypted_memo180 bytesUint8Array
depositor, recipient, validatorAccountId32SS58 string
amountBalance in planckbigint
leaf_index, leaf_indices, change_leaf_index, tree_size, tree_id, first_leaf_index, leaf_count, asset_idu32number
An empty Option—undefined
  • tree_size counts every leaf ever inserted, across the whole forest: after an insert it equals that leaf's index plus one.
  • NullifiersSpent always carries two entries. A zero nullifier (0x00…00) marks a dummy input: no note was spent and the runtime does not record it.
  • Shielded and Unshielded carry no asset id. The asset is an argument of the call; the pool accepts only ORB, asset 0.

Order within an extrinsic​

CallEvents, in order
shieldBalances.Transfer (depositor to pool account), MerkleRootUpdated, TreeSealed if the leaf filled a tree, Shielded
shield_batchThe shield sequence once per entry
private_transferMerkleRootUpdated twice, FeeCredited, NullifiersSpent, CommitmentsInserted
unshieldBalances.Transfer (pool account to recipient), MerkleRootUpdated for a change note, FeeCredited, Unshielded
claim_feesBalances.Transfer (pool account to validator), FeesClaimed

Every event of one extrinsic has the same phase, ApplyExtrinsic(index), which is how to group them. A failed extrinsic emits none of them. A private transfer, read from a block (hashes abbreviated):

2 MerkleRootUpdated { old_root: '0x4361…b32f', new_root: '0x3b27…2418', tree_size: 10434 }
2 MerkleRootUpdated { old_root: '0x3b27…2418', new_root: '0xce3c…fd04', tree_size: 10435 }
2 FeeCredited { validator: '5Grwva…utQY', asset_id: 0, amount: '1000000000000000' }
2 NullifiersSpent { nullifiers: [ '0x0bae…ab23', '0x0000…0000' ] }
2 CommitmentsInserted { commitments: [ '0x615a…fe2a', '0xd24f…750d' ], encrypted_memos: [ … ], leaf_indices: [ 10433, 10434 ] }

Reading one block​

client.substrate.queryBlockEvents(blockHash) decodes a block's events with the runtime of that block. Each record has phase, and event.section, event.method and event.data, whose toJSON() gives the fields with hashes and memos as hex and balances as decimal strings.

import { OrbinumClient } from '@orbinum/protocol';

const client = await OrbinumClient.connect({ substrateWs: 'wss://rpc-1.testnet.orbinum.io' });
try {
  const blockNumber = Number(process.argv[2]);
  const hash = await client.substrate.getBlockHash(blockNumber);
  if (hash === null) throw new Error(`no block ${blockNumber}`);

  const records = (await client.substrate.queryBlockEvents(hash)) ?? [];
  for (const { phase, event } of records) {
    if (event.section !== 'shieldedPool') continue;
    const extrinsic = phase.isApplyExtrinsic ? phase.asApplyExtrinsic.toNumber() : null;
    console.log(extrinsic, event.method, event.data.toJSON());
  }
} finally {
  client.destroy();
}

queryBlockEvents returns null when the events cannot be read, for example from a node that has pruned that block's state. Index from an archive node to read any height.

Following finalized blocks​

An indexer follows finalized blocks only, so it never records events from a block that is later replaced. This one uses polkadot-api directly, processes every block number in order even when finalization jumps several blocks at once, and stops on a gap in the leaf indices.

import { createClient } from 'polkadot-api';
import { getWsProvider } from 'polkadot-api/ws';
import { toHex } from '@orbinum/protocol';

// The shapes polkadot-api decodes these events into: hashes as 0x hex,
// memos as bytes, accounts as SS58, balances as bigint, None as undefined.
type PoolEvent =
  | { type: 'Shielded'; value: { commitment: string; encrypted_memo: Uint8Array; leaf_index: number } }
  | { type: 'CommitmentsInserted'; value: { commitments: string[]; encrypted_memos: Uint8Array[]; leaf_indices: number[] } }
  | { type: 'NullifiersSpent'; value: { nullifiers: string[] } }
  | {
      type: 'Unshielded';
      value: {
        nullifier: string;
        change_commitment?: string;
        change_encrypted_memo?: Uint8Array;
        change_leaf_index?: number;
      };
    }
  | { type: 'MerkleRootUpdated' | 'TreeSealed' | 'FeeCredited' | 'FeesClaimed'; value: unknown };
type EventRecord = { event: { type: string; value: PoolEvent } };

type Leaf = { leafIndex: number; commitment: string; memo: string; block: number };

const ZERO = `0x${'00'.repeat(32)}`;
const client = createClient(getWsProvider('wss://rpc-1.testnet.orbinum.io'));
const api = client.getUnsafeApi();

/** The leaves a block appended, in leaf order, and the nullifiers it spent. */
async function readBlock(hash: string, block: number) {
  const records = (await api.query.System.Events.getValue({ at: hash })) as EventRecord[];
  const leaves: Leaf[] = [];
  const spent: string[] = [];
  for (const { event } of records) {
    if (event.type !== 'ShieldedPool') continue;
    const e = event.value;
    switch (e.type) {
      case 'Shielded':
        leaves.push({
          leafIndex: e.value.leaf_index,
          commitment: e.value.commitment,
          memo: toHex(e.value.encrypted_memo),
          block,
        });
        break;
      case 'CommitmentsInserted':
        e.value.leaf_indices.forEach((leafIndex, i) =>
          leaves.push({
            leafIndex,
            commitment: e.value.commitments[i]!,
            memo: toHex(e.value.encrypted_memos[i]!),
            block,
          }),
        );
        break;
      case 'Unshielded':
        spent.push(e.value.nullifier);
        if (e.value.change_leaf_index !== undefined) {
          leaves.push({
            leafIndex: e.value.change_leaf_index,
            commitment: e.value.change_commitment!,
            memo: toHex(e.value.change_encrypted_memo!),
            block,
          });
        }
        break;
      case 'NullifiersSpent':
        // A zero nullifier is a dummy input: nothing was spent.
        spent.push(...e.value.nullifiers.filter((n) => n !== ZERO));
        break;
    }
  }
  return { leaves: leaves.sort((a, b) => a.leafIndex - b.leafIndex), spent };
}

let next = Number(process.env.FROM ?? 1); // first block not yet indexed
const stopAt = Number(process.env.TO ?? Infinity);
let expectedLeaf: number | null = null;
let running = false;

const subscription = client.finalizedBlock$.subscribe(async (finalized) => {
  if (running) return; // the next emission resumes from `next`
  running = true;
  try {
    for (; next <= Math.min(finalized.number, stopAt); next++) {
      const hash: string = await client._request('chain_getBlockHash', [next]);
      const { leaves, spent } = await readBlock(hash, next);
      for (const leaf of leaves) {
        if (expectedLeaf !== null && leaf.leafIndex !== expectedLeaf) {
          throw new Error(`leaf ${expectedLeaf} missing before block ${next}`);
        }
        expectedLeaf = leaf.leafIndex + 1;
      }
      if (leaves.length || spent.length) {
        console.log(`#${next}: ${leaves.length} leaves, ${spent.length} nullifiers`);
      }
    }
    if (next > stopAt) {
      subscription.unsubscribe();
      client.destroy();
    }
  } finally {
    running = false;
  }
});

FROM and TO bound the run; without TO it keeps following the chain. Store each block's leaves and nullifiers in one database transaction together with next, so a restart resumes exactly where it stopped.

Rebuilding the leaves​

No RPC method returns a leaf by its index, or the 64 leaves of a leaf block (the unit whose root is a level-6 subtree root). The events are the only source, and they are complete: every insert carries its leaf index.

Leaf sourceIndex field
A shield or each shield_batch entryShielded.leaf_index
A private transfer's two outputsCommitmentsInserted.leaf_indices, in the order of commitments
An unshield's change noteUnshielded.change_leaf_index

Leaf indices are global across the forest, dense, and strictly increasing in chain order, so a gap means a missed block. The last MerkleRootUpdated.tree_size of a block equals the next leaf index, and privacy_getPoolStats reports the same number as commitment_count. A leaf's tree is leafIndex >> 20 (treeIdOf({ leafIndex }) in @orbinum/protocol); a tree holds 1,048,576 leaves, in 16,384 leaf blocks of 64.

To check the leaves you store, fold a full leaf block into its subtree root and compare it with privacy_getSubtreeRoots; see Merkle Forest. A wallet uses these leaf blocks to build its Merkle paths without naming its note to a node.

Nullifiers and commitments in separate events​

A private transfer reports its nullifiers in NullifiersSpent and its new commitments in CommitmentsInserted. They are two feeds with two consumers: a wallet's scanner downloads every commitment and memo to find notes addressed to it, and checks spent state against the whole set of nullifiers. Serve them as separate feeds, each read whole by every client; see Note Discovery.

Both events come from the same extrinsic and share its phase, so they show that one transfer spent these nullifiers and created these commitments. What they do not reveal is which earlier commitment a nullifier spends: a nullifier is Poseidon2(commitment, spending_key), so linking it to its note takes the owner's spending key. See Notes.

Matching an EVM shield to its event​

A shield sent from an EVM wallet runs inside an Ethereum.transact extrinsic. The pallet sees the precompile, not the user, as the depositor:

WhereWhat it shows
Shielded.depositorThe precompile's mapped account: 0x…0801 followed by 12 zero bytes, 5C4hrfjw9DjXZTzV3MwzrrAr9P2LdXerjSYnseqkK8c9u9Am in SS58
Ethereum.Executedfrom (the user's EVM address), to (0x…0801), transaction_hash, exit_reason
Balances.TransferThe user's mapped account to the precompile account, then the precompile account to the pool account

The EVM block number is the Substrate block number, but the EVM transaction index is not the extrinsic index. Match on the transaction hash: find the Ethereum.Executed event whose transaction_hash is the EVM hash, then the Shielded event with the same phase.

import { OrbinumClient } from '@orbinum/protocol';

type ShieldedData = { depositor: string; amount: string; commitment: string; leaf_index: number };

/** The `Shielded` event an EVM `shield` transaction produced, or null. */
async function shieldedEventOf(client: OrbinumClient, txHash: string): Promise<ShieldedData | null> {
  const receipt = await client.evm!.request<{ blockNumber: string; status: string } | null>(
    'eth_getTransactionReceipt',
    [txHash],
  );
  if (receipt === null || receipt.status !== '0x1') return null;

  const blockHash = await client.substrate.getBlockHash(Number(receipt.blockNumber));
  const records = blockHash ? ((await client.substrate.queryBlockEvents(blockHash)) ?? []) : [];

  // The extrinsic that carried the EVM transaction emits Ethereum.Executed with its hash.
  const executed = records.find(
    ({ event }) =>
      event.section === 'ethereum' &&
      event.method === 'Executed' &&
      (event.data.toJSON() as { transaction_hash: string }).transaction_hash === txHash.toLowerCase(),
  );
  if (!executed?.phase.isApplyExtrinsic) return null;
  const extrinsic = executed.phase.asApplyExtrinsic.toNumber();

  const shielded = records.find(
    ({ phase, event }) =>
      phase.isApplyExtrinsic &&
      phase.asApplyExtrinsic.eq(extrinsic) &&
      event.section === 'shieldedPool' &&
      event.method === 'Shielded',
  );
  return shielded ? (shielded.event.data.toJSON() as ShieldedData) : null;
}

const client = await OrbinumClient.connect({
  substrateWs: 'wss://rpc-1.testnet.orbinum.io',
  evmRpc: 'https://rpc-1.testnet.orbinum.io',
});
try {
  console.log(await shieldedEventOf(client, process.argv[2]!));
} finally {
  client.destroy();
}

A reverted shield has receipt status 0x0 and no Shielded event. Index the user by Ethereum.Executed.from, never by depositor, which is the same for every EVM shield. The precompile's call rules are in ShieldedPool precompile.

Fee events​

Every spend credits its fee to the validator account of the block's author: one FeeCredited per spend, with no token movement. claim_fees pays part or all of it out with a Balances.Transfer from the pool account and a FeesClaimed. A validator's pending balance for an asset is the sum of its FeeCredited amounts minus the sum of its FeesClaimed amounts, which client.shieldedPool.pendingFees(account, assetId) reads directly. See Spend Fees.