Updates & Backups
Image updates
Watchtower ships in every node stack and polls every 5 minutes, rolling-restarting the node when a new image is published. On a normal day there is nothing to do.
To force an update immediately:
cd node-deploy/testnet/validator # or testnet/rpc
docker compose pull && docker compose up -d
Check what Watchtower has been doing:
docker compose logs watchtower
It runs with INCLUDE_STOPPED, REVIVE_STOPPED and INCLUDE_RESTARTING so a fix
still reaches a node stuck in created/restarting, which a default Watchtower
ignores.
Releases: binary first, runtime second
A release reaches the chain in two separate steps, always in this order:
| Step | What changes | Operator action |
|---|---|---|
| 1. Node release | A new image, <version>-testnet and testnet-latest. Binary only — chain state is untouched | None while Watchtower runs; it rolls the node onto the new image within ~5 minutes |
| 2. Runtime upgrade | The runtime Wasm in chain state, via a Root system.setCode | None. The node executes the new runtime from the block after the upgrade |
The runtime upgrade waits until the nodes run the matching binary. That order is what keeps a node from executing a runtime its binary does not fully support: a runtime that calls a host function the binary lacks makes that node stop importing at the upgrade block, for good. The runtime verifies Groth16 proofs through a native host function that node 0.4.0 and later register: a node older than 0.4.0 — RPC, archive or validator — cannot import current blocks.
A node that misses a release keeps running the old binary until it is updated. A validator can also lose its slots: see Minimum node version.
Current versions
| Component | Version |
|---|---|
| Node | orbinum-node 0.7.0 — image ghcr.io/orbinum/node:0.7.0-testnet |
| Runtime | spec_version 21, transaction_version 8 |
| Minimum author version | read validatorSet.minAuthorVersion — see Minimum node version |
A validator is its staking account and session keys only: there is no relay address and
no evmr key, and spend fees are credited to the block author — see
Spend fees. The node runs no relay and serves no
relayer_* or orbinum_relayShieldedCall RPC methods.
Do not run a node older than 0.3.1. Earlier binaries recorded storage proofs while importing but not while authoring, so they metered EVM proof-size gas differently from the author and could reject valid blocks. See The node rejects blocks the network accepted.
Check what a node runs
<node> is orbinum-validator or orbinum-rpc-node.
The binary:
docker exec <node> orbinum-node --version
The runtime it executes:
docker exec <node> curl -s -H 'Content-Type: application/json' \
-d '{"id":1,"jsonrpc":"2.0","method":"state_getRuntimeVersion"}' \
http://localhost:9944
Compare specVersion and transactionVersion with the table above.
Config changes need a recreate
A container's command line is fixed at creation. docker compose restart brings
the node back with its old arguments, so an .env edit appears to do nothing.
docker compose up -d --force-recreate
Verify the running command actually changed (<node> is orbinum-validator or
orbinum-rpc-node):
docker inspect <node> --format '{{join .Config.Cmd " "}}'
Back up your keystore
Chain data lives in a named Docker volume and can always be resynced. The keystore cannot. It holds the private halves of your session keys.
docker cp orbinum-validator:/data/chains/orbinum_testnet/keystore ./keystore-backup
tar -czf keystore-$(date +%Y%m%d).tar.gz keystore-backup
gpg --symmetric --cipher-algo AES256 keystore-*.tar.gz
rm -rf keystore-backup
Restoring this backup onto a second machine while the first is still running makes both sign with the same identity. Their conflicting finality votes are a GRANDPA equivocation: your account is banned, your node disabled, and your bond — and your nominators' — slashed. See Slashing.
If you are migrating, stop the old node first and confirm it is stopped.
Losing the keystore is recoverable:
generate new session keys
with author_rotateKeysWithOwner and
submit session.setKeys
again with the new keys and proof — while you are in the active or queued set that
needs governance, see Rotate session keys.
You miss blocks until the new keys take effect.
Database maintenance
Pruning is set by flag, not by a maintenance job — see
Node Flags. The public RPC runs
--state-pruning archive, so its disk grows without bound; size its storage for
that (see the sizing table).
A full reset (docker compose down -v) is a last resort — see
Troubleshooting.