Setting up the Grafana dashboard using the RadixNode CLI

Introduction

RadixDLT uses Prometheus and Grafana to provide nodes with a real-time monitoring and alert system. A custom-built exporter extracts metrics from the node and sends it Prometheus which stores it as into the time series data. Grafana reads this time series information and uses it to build monitoring dashboards for displaying node information and alerts in real-time.

The Prometheus/Grafana installation has been packaged as part of the radixnode script. This will allow node runners to get a monitoring dashboard up and running by simply running the script and answering a few questions.

This setup runs the monitor on the same machine as the node. In the near future we will offer a cloud-based installation that will have less of an impact on the node in terms of maintenance and resource usage.

Prerequisites

It probably goes without saying that before you can install the node monitoring software, you must have a node up and running. If you haven’t, then run through one of our guides which will show you how to do it. We have separate instructions for installing a Docker node or a systemd node.

So before you start you must have the radixnode script in place, and an installed node.

1. Set up the Nginx Metrics password

In order to make sure the node is secure as possible, the services available are run from different endpoints, which require authorization rights to access them. The metrics service is runs from /metrics, so before we can begin, we need to set up the username/password for this endpoint.

If you are running a systemd node then you must first change to the radixdlt user:
Only run this command if you’re operating a systemd node.
```
sudo su - radixdlt
```
Run one of the following radixnode commands (depending on your installation type) to set up the endpoint’s authorization.
- docker
- systemd
radixnode auth set-metrics-password -m DOCKER
radixnode auth set-metrics-password -m SYSTEMD

2. Set up the monitoring service

The monitoring instance uses port 3000 on your server, so make sure that this port is open and available for Grafana to use.

If you are running a systemd node then you must first change to the radixdlt user:

Only run this command if you’re operating a systemd node.

sudo su - radixdlt

This is not required if you’re running a Docker node.

Now you can run the script to set up monitoring:

radixnode monitoring setup

And that’s pretty much it.

3. Accessing the dashboard.

You can view your node’s dashboard remotely using any browser, using this URL pattern:

http://<node-ip-address>:3000/d/radix_node_dashboard/radix-node-dashboard?orgId=1&refresh=5s

The node-ip-address is the external IP address of your node’s server.

Grafana will display a page asking for your username and password.

the Grafana welcome screen

Since this is the first time you’ve run the monitor, enter admin for the username and admin again for the password. Grafana will now display another dialog asking you to change the password for the admin user.

pick a new password

You will now see a blank page with a somewhat discouraging Not Found message at the top. Click the Search icon (the magnifying glass).

opening window

Navigate to Dashboards Radix Node Dashboard:

selecting the dashboard

Grafana will now the example dashboard:

The Grafana dashboard

4. Accessing the metrics endpoint

Now that the dashboard is up and running, you may want to add new elements yourself, or use the data from the node to build other applications for monitoring and gathering information. The same data that Grafana uses to build its screen is available on the metrics endpoint:

curl -X -k GET --location "https://localhost/prometheus/metrics" -H "Content-Type: application/json" --basic --user metrics:nginx-password  (1)

1	This is the password you set up in Section 1, “Set up the Nginx Metrics password”

5. Shut down Node monitoring

If you’re shutting down the monitor for a systemd node, then you’ll first need to sign in as the radixdlt user before issuing the shutdown command:

Only execute this command if you installed your node as a systemd instance.

sudo su - radixdlt

Then, to shut down the monitor, use the following command:

radixnode monitoring stop