Sonm Worker monitoring bot Guide

Telegram bot for monitoring Workers (Worker monitoring bot) is a component for Suppliers that helps to monitor the state of the hardware in real time. It receives and updates data every minute and provides it to you on request via Telegram. In the event of an emergency, you will immediately receive a notification.


  • Collect data about the state of your hardware every minute
  • Provide data on Telegram about the Worker at the request of the user
  • Send emergency notifications about the state of the Worker (or of all Workers of the specified Master) via Telegram

What it monitors

The state parameters of the Worker that the component monitors are as follows:

  • State (avalaibility of connection and related data)
  • Geolocation (country, state)
  • CPU load (%)
  • Disk space (total/free)
  • RAM (total/free)
  • for each GPU:
    • GPU fan speed (%)
    • GPU temperature (C)
    • GPU power (W)

How to use

The Worker-monitoring bot provides:

  • emergency notifications (if something unusual happens with the hardware)
  • a summary of the latest data received from the Worker during the last 24 hours, with a timestamp (upon request)

To start using the Worker-monitoring bot on Telegram, please update your Worker application to 0.4.20 version (see how to do that).

Next, subscribe to the Telegram bot @SONM_Workermon_bot.

The following bot commands are available:

  • /watch 'ethereum_address' (subscribe to notifications from the specified Ethereum address (a Worker, or all the Workers of the specified Master, if it is a Master address))

Note: If the Master has added a new Worker, you must renew the subscription to the Master address to receive emergency notifications about its parameters.

  • /status 'ethereum_address' (get the latest available data for the specified Worker address received in the last 24 hours)

Note: It is not necessary to be subscribed to the specified address to retrieve its status.

  • /remove 'ethereum_address' (remove notification subscription to the specified Worker address)
  • /list (view a list of subscribed Worker addresses)
  • /settings (show the notification settings)
  • /help (show the command descriptions)

Emergency notifications

Each subscriber receives Telegram notifications immediately in the event of a contingency. The notification is sent to everyone subscribed to the address of the Worker.

Notifications can be of two types: CRITICAL (parameter has gone out of bounds) and Recovered (parameter has returned to normal).

The notification settings are the same for everyone. In the future, users will be able to customize their own notification thresholds. The list of the notifications is given below.

  1. Connection error (connection state of the Worker in 60 seconds)

    CRITICAL: seems like Worker is turned on, but we can't connect to it

    Recovered: we’re now able to obtain updates from the Worker

  2. No data (no data received from the Worker in the last 90 seconds)

    CRITICAL: seems like we can't get any data from the Worker

    Recovered: we’re now able to obtain updates from the Worker

  3. CPU usage (the CPU utilization (all cores))

    CRITICAL: <more than 90%>

    Recovered: <less than or equal to 90%>

  4. Free RAM (free RAM space)

    CRITICAL: <less than 20%>

    Recovered: <more than or equal to 20%>

  5. Free disk space

    CRITICAL: <less than 20%>

    Recovered: <more than or equal to 20%>

  6. GPU temperature

    CRITICAL: <more than 90C>

    Recovered: <less than or equal to 90C>

  7. GPU fan RPMs (The GPU fan revs)

    CRITICAL: <less than 10%>

    Recovered: <more than or equal to 10%>