Skip to main content

​
Operational notes

  • Alerts cover wallet balance thresholds, database usage, validator efficiency/blocks, synchronization, ADNL health, stake acceptance, slashes, and other key metrics. Review the alert keys in modules/alert_bot.py to understand the available triggers.
  • Each alert has an associated cooldown (timeout) to prevent spam. Info-level ok alerts reset state without sound notifications.
  • The bot requires network access to the Telegram API. Ensure outbound HTTPS is permitted from the server.
  • When validator mode is enabled, the alert bot automatically includes wallet and ADNL context in messages. In collator-only or other modes, some alerts may be skipped because prerequisites are missing.

​
setup_alert_bot

Purpose: Configure the alert bot with the Telegram bot token and chat ID, then start sending events. Syntax 
setup_alert_bot
Behavior
  • Launches the alert-bot configuration flow (interactive prompts) to store BotToken and ChatId in the local database.
  • Validates the token by sending test requests; fails fast if the bot or chat ID is incorrect.
  • Should be run after enabling alert-bot mode so the background scheduler picks up the alerts.

​
list_alerts

Purpose: Show all predefined alerts and whether they are currently enabled. Syntax 
list_alerts
Behavior
  • Lists every alert key (for example: low_wallet_balance, db_usage_80, out_of_sync) along with the enabled flag and the UNIX timestamp when it was last sent.
  • Helps you audit which alerts are muted and whether recent warnings have fired.

​
enable_alert

Purpose: Re-enable a previously muted alert. Syntax 
enable_alert <alert_name>
Behavior
  • Accepts any alert key defined in the alert module (for example: low_efficiency, service_down, validator_slashed).
  • Sets the alert’s enabled flag to true so future events can trigger notifications.
Example 
enable_alert low_wallet_balance

​
disable_alert

Purpose: Temporarily suppress a specific alert. Syntax 
disable_alert <alert_name>
Behavior
  • Marks the alert as disabled; the scheduler skips sending messages for it until re-enabled.
  • Use when you expect noisy conditions (e.g., during planned maintenance) but still want other alerts to deliver.
Example 
disable_alert service_down

​
test_alert

Purpose: Send a simple message through the configured alert channel to verify connectivity. Syntax 
test_alert
Behavior
  • Requires successful initialization (bot token and chat ID saved). If initialization hasn’t run yet, the command triggers it.
  • Sends Test alert with info severity so you can confirm the chat receives notifications.

​
Available alert names

  • low_wallet_balance: Validator wallet balance below 10 TON while the node is working and in sync.
  • low_wallet_balance_ok: Balance recovered to ≥10 TON after a low-balance alert.
  • db_usage_80: TON database usage exceeded 80% (but ≤95%).
  • db_usage_95: TON database usage exceeded 95%.
  • db_usage_ok: Database usage dropped back below 80% after a high-usage alert.
  • low_efficiency: Validator efficiency fell below 90% once ≥80% of the round elapsed.
  • out_of_sync: Node stayed more than 20 seconds behind the masterchain while otherwise running.
  • sync_ok: Node resynced to less than 20 seconds lag after an out-of-sync alert.
  • service_down: Validator service stopped reporting as working (outside of initial sync).
  • service_down_ok: Validator service resumed normal operation after downtime.
  • adnl_connection_failed: Remote ADNL connectivity checks failed for all probe hosts.
  • adnl_connection_ok: ADNL check succeeded again after a failure.
  • zero_block_created: No blocks produced in roughly the last half validation period (~8h on mainnet).
  • zero_block_created_ok: Block production resumed after a zero-block alert.
  • validator_slashed: Validator was slashed in the previous validation round.
  • stake_not_accepted: Election stake submission was rejected (validator missing from the current validator list).
  • stake_accepted: Election stake was accepted (validator present in the current validator list).
  • stake_returned: Elector returned the stake during the post-freeze payout window.
  • stake_not_returned: Stake was not returned during the expected post-freeze payout window.
  • voting: Governance offers with ≥50% approval remain unvoted by this validator.
  • voting_ok: All actionable governance offers have been voted on (no outstanding votes).
  • initial_sync_completed: Initial blockchain sync finished successfully.
  • shard_collators_offline: All registered collators for at least one shard are offline.
  • shard_collators_ok: Collators for previously offline shards reported back online.