Upgrading a Validator
How to perform an upgrade on a validator
Helm Chart details:
Validator chart version: 0.8.2 (app version 0.81.0)
Operators must install exactly this version, not the latest version published to the Helm repository.
Upgrading to chart 0.8.2: no values changes are needed. A values file that works with chart 0.6.x works unchanged with 0.8.2, and the upgrade moves both the ghost and ghost-vao deployments to ghcr.io/chronicleprotocol/ghost:0.81.0. If your values file sets an image tag (global.image.tag, ghost.image.tag or vao.image.tag), remove it, even if it is empty: a pinned tag keeps the release on the image you pinned, and an empty global.image.tag makes the chart use ghost:0.81.0 without the pinned digest.
Chart 0.7.0 added an optional startup probe, off by default. Enable it with global.startup.enabled: true if the liveness probe restarts your validator while it is still starting. The default budget is 30 checks, 10 seconds apart (5 minutes).
Chart 0.8.0 added ghost.enabled, vao.enabled, vao.rpcUrl and vao.ethConfig. Leave them unset unless the Chronicle team asks you to change them: both deployments keep running, and the ghost-vao deployment keeps using ghost.rpcUrl and ghost.ethConfig.
App 0.81.0 has no built-in fallback configuration. At start it reads the on-chain config registry through your Ethereum RPC (ghost.rpcUrl) and downloads its configuration from public IPFS gateways over HTTPS, so both must be reachable from the node. If you load your own configuration with -c ipfs://..., the URL must end with ?checksum=0x<keccak256 of the file>. Some metrics changed: musig_session_count is now chronicle_musig_session_count without the coordinator label, musig_session_suppressed_total is now chronicle_musig_session_rejected_total with different reason values, musig_session_limit was removed, and the WASM module memory gauges (chronicle_wasm_module_mallocs, _max_alloc, _total_alloc and _uptime_seconds) were replaced by new chronicle_wasm_module_* metrics. Update any alerts and dashboards built on them.
Do not use --reuse-values. When upgrading an existing release, always pass your values file with -f. --reuse-values keeps the defaults of the chart you are upgrading from, and with chart 0.8.x Helm then reports a successful upgrade while it deletes both validator deployments and their services. If that happened, run helm rollback $FEED_NAME -n $FEED_NAME straight away to return to the previous revision.
Upgrading from a chart older than 0.6.0: As of ChartVersion 0.6.0, the tor-controller and its associated CRDs have been removed from the chart. The chart upgrade will automatically remove tor-related pods, services, and secrets that were previously managed by Helm. After upgrading, remove any remaining tor resources manually:
# Remove the onionservice resource (if present)
kubectl delete onionservice ghost -n $FEED_NAME --ignore-not-found
# If the tor-controller namespace was deployed, remove it
kubectl delete namespace tor-controller-system --ignore-not-found
# Remove all CRD's IF NOT USED BY OTHER APPS
kubectl delete -f https://raw.githubusercontent.com/chronicleprotocol/charts/validator-0.3.24/charts/validator/crds/tor-controller.yaml --ignore-not-found
Sample config:
global:
logLevel: "warn"
ghost:
ethConfig:
ethFrom:
existingSecret: '<somesecret>'
key: "ethFrom"
ethKeys:
existingSecret: '<somesecret>'
key: "ethKeyStore"
ethPass:
existingSecret: '<somesecret>'
key: "ethPass"
ethRpcUrl: "https://MY_L1_RPC_URL"
rpcUrl: "https://MY_L1_RPC_URL"
env:
normal:
CFG_LIBP2P_EXTERNAL_ADDR: '/ip4/1.2.3.4' # public/reachable ip address of node. If DNS hostname set to `/dns/my.validator.com`
vao:
env:
normal:
CFG_LIBP2P_EXTERNAL_ADDR: '/ip4/1.2.3.4' # public/reachable ip address of node. If DNS hostname set to `/dns/my.validator.com`
Upgrading manually (helm upgrade)
Upgrading manually (helm upgrade)​
If you are upgrading between patch releases of the chart, simply updating the chart version will suffice:
ssh <SERVER_IP>
su - <FEED_USERNAME>
export FEED_NAME=my-feed
Prepare values​
The values.yaml file is used to configure the validator. The install script generates it as $HOME/$FEED_NAME/generated-values.yaml.
Chart version 0.8.2 needs no changes to a values.yaml / generated-values.yaml that already works with chart 0.6.x. For older files, compare against the structure below.
Please structure your helm values like this:
global:
logLevel: "warn"
ghost:
ethConfig:
ethFrom:
existingSecret: '<somesecret>'
key: "ethFrom"
ethKeys:
existingSecret: '<somesecret>'
key: "ethKeyStore"
ethPass:
existingSecret: '<somesecret>'
key: "ethPass"
ethRpcUrl: "https://MY_L1_RPC_URL"
rpcUrl: "https://MY_L1_RPC_URL"
env:
normal:
CFG_LIBP2P_EXTERNAL_ADDR: '/ip4/1.2.3.4' # public/reachable ip address of node
vao:
env:
normal:
CFG_LIBP2P_EXTERNAL_ADDR: '/ip4/1.2.3.4' # public/reachable ip address of node
Please ensure your values yaml file is updated to reflect the requirements of validator chart version 0.8.2, with the correct values for ethConfig and rpcUrl.
Always pass your values file with -f. Do not use --reuse-values: it keeps the defaults of the chart you are upgrading from, and with chart 0.8.x Helm then reports a successful upgrade while it deletes both validator deployments and their services. If that happened, run helm rollback $FEED_NAME -n $FEED_NAME straight away to return to the previous revision.
helm repo update
helm upgrade $FEED_NAME -n $FEED_NAME -f $HOME/$FEED_NAME/generated-values.yaml chronicle/validator --version 0.8.2
Upgrading using the helper script (upgrade.sh)
Upgrading using upgrade.sh​
To simplify the upgrade process, we have created a helper script that will upgrade your validator to the chart version pinned on this page (0.8.2).
This script runs helm upgrade <feedname> -n <feedname> -f $HOME/<feedname>/generated-values.yaml chronicle/validator --version 0.8.2 on your feed release. It runs a dry run first and asks for confirmation before it upgrades.
Please use the correct FEED_NAME, which should be the same as your helm release name, if deployed using the install.sh script previously
ssh <SERVER_IP>
su - <FEED_USERNAME>
export FEED_NAME=my-feed
Download upgrade.sh​
Get the upgrade.sh script pinned to a fixed commit. It upgrades to chart version 0.8.2:
wget -N https://raw.githubusercontent.com/chronicleprotocol/scripts/47ad1617ae4a13195ee331fd25619a359a80f5b7/feeds/k3s-install/upgrade.sh
chmod a+x upgrade.sh
./upgrade.sh
You can set the expected variables in the .env file, or export them as environment variables. If the script fails to find any of these values, it will prompt you for them when running the script.
Do not set CHART_VERSION in .env: when the script reads .env, a value there replaces the pinned chart version.
If kubectl/helm commands fail, please ensure you have $KUBECONFIG set correctly. Take a look here for more detail
Verify the helm release and version​
Verify the chart version has changed and matches the chart version pinned on this page (0.8.2):
helm list -n $FEED_NAME
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
validator demo 2 2026-10-09 12:56:31.070821 -0300 -03 deployed validator-0.8.2 0.81.0
Check that both deployments run the pinned image:
kubectl get deployments -n $FEED_NAME -o wide
The IMAGES column must show ghcr.io/chronicleprotocol/ghost:0.81.0@sha256:417b664eee72406bf1f36757b165135e0d71ce6321b9c0b463ec0d882a886de4 for both ghost and ghost-vao.
View all resources created in the namespace​
kubectl get pods,deployment,service,secrets -n $FEED_NAME
NAME READY STATUS RESTARTS AGE
pod/ghost-5c4cfb47bf-wvsvf 1/1 Running 0 14s
pod/ghost-vao-79d77454c7-l5qch 1/1 Running 0 14s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ghost 1/1 1 1 14s
deployment.apps/ghost-vao 1/1 1 1 14s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ghost ClusterIP 10.43.109.192 <none> 8000/TCP,8080/TCP 14s
service/ghost-vao ClusterIP 10.43.44.21 <none> 8001/TCP 14s
service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 287d
NAME TYPE DATA AGE
secret/somesecretname-eth-keys Opaque 3 30s
secret/sh.helm.release.v1.my-validator.v1 helm.sh/release.v1 1 14s
View pod logs:​
kubectl logs -n $FEED_NAME deployment/ghost
kubectl logs -n $FEED_NAME deployment/ghost-vao
and you're done!
If you encounter any issues please refer to the Trouble Shooting docs