Lab 8: Observability with Grafana: data sources, Kepler, dashboard creation.
This experiment showcases an example of running a KPM monitoring xApp on top of a 5G Standalone (SA) network using OpenAirInterface (OAI) RF Simulator gNB, OAI minimal 5GC and FlexRIC as the Near-RT RIC. Additionally, an overview of the available information at Grafana is provided, along with instructions on displaying metrics from the xApp.
apiVersion: athena.trirematics.io/v1
kind: Network
metadata:
name: bubbleran
namespace: trirematics
spec:
slices:
- plmn: "00101"
dnn: "internet"
network-mode: "IPv4"
service-type: eMBB
differentiator: 0x000000
ipv4-range: "12.1.1.0/24"
ipv6-range: "2001:db8:1::/64"
access:
- name: oai-gnb
stack: 5g-sa
model: oai-ran/monolithic-gnb
identity:
an-id: 50
radio:
device: rf-sim
cells:
- band: n78
arfcn: 641280
bandwidth: 40MHz
subcarrier-spacing: 30kHz
tdd-config:
period: 5ms
dl-slots: 7
dl-symbols: 6
ul-slots: 2
ul-symbols: 4
controller: flexric.bubbleran
core-networks:
- minimal.bubbleran
core:
- name: minimal
stack: 5g-sa
model: oai-cn/minimal
identity:
region: 0
cn-group: 4
cn-id: 5
dns:
ipv4:
default: 8.8.8.8
secondary: 8.8.4.4
edge:
- name: flexric
stack: 5g-sa
model: mosaic5g/flexric
- name: monitoring
stack: 5g-sa
model: mosaic5g/monitoring-c
profiles:
- kpm-sm
- database
- name: sdl
stack: 5g-sa
model: mosaic5g/xapps-sdl
---
apiVersion: athena.trirematics.io/v1
kind: Terminal
metadata:
name: ue1
namespace: trirematics
spec:
vendor: oai
stack: 5g-sa
model: terminal/nr-rfsim
preferred-access: oai-gnb.bubbleran
target-cores:
- minimal.bubbleran
identity:
imsi: "001010000000001"
pin: "1234"
opc: "0xc42449363bbad02b66d16bc975d77cc1"
key: "0xfec86ba6eb707ed08905757b1bb44b8f"
sqn: "0xff9bb4000001"
slice:
dnn: "internet"
network-mode: "IPv4"
service-type: eMBB
differentiator: 0x000000
radio:
bands:
- n78
readiness-check:
method: ping
target: google-ip
interface-name: oaitun_ue0
---
apiVersion: athena.trirematics.io/v1
kind: Terminal
metadata:
name: ue2
namespace: trirematics
spec:
vendor: oai
stack: 5g-sa
model: terminal/nr-rfsim
preferred-access: oai-gnb.bubbleran
target-cores:
- minimal.bubbleran
identity:
imsi: "001010000000002"
pin: "1234"
opc: "0xc42449363bbad02b66d16bc975d77cc1"
key: "0xfec86ba6eb707ed08905757b1bb44b8f"
sqn: "0xff9bb4000001"
slice:
dnn: "internet"
network-mode: "IPv4"
service-type: eMBB
differentiator: 0x000000
radio:
bands:
- n78
readiness-check:
method: ping
target: google-ip
interface-name: oaitun_ue0
Deployment
Use the command brc install network open-ran-xapp.yaml to deploy the network.
It should finish without errors and printout the three Kubernetes resource names that were created.
Check for the status of the deployment using the command brc observe.
Wait until all the Elements other than the UE are in the STATUS set to
1/1 Y state.
Grafana
Grafana is an open-source observability platform designed for visualizing, monitoring, and analyzing metrics, logs, and traces from various data sources. It enables users to create interactive dashboards that provide insights into system performance, infrastructure health, and application behavior.
It is compatible with a moltitute of data sources, among which MySQL and Prometheus. Particularly, these two sources are used in our monitoring stack to retrieve and display metrics coming the infrastructure (Kubernetes) and the RAN (xApp).
Grafana GUI can be accessed by running the following command:
kubectl port-forward -n tobs services/tobs-grafana 3000:80
This command will port-forward the Grafana service to port 3000 of the host machine.
Additionally, the service can be exposed to the external network by adding this flag to the command: --address 0.0.0.0.
Therefore, navigate to localhost:3000 (or [machine-ip]:3000 if exposed) to access the Grafana GUI from the browser.
In the case you are accessing your control-plane via SSH from a different subnet then you can do SSH with port-forwarding to be able to access grafana (e.g. ssh -L 3000:[machine-ip]:3000 [user]@[machine-ip]), then you can access on localhost:3000.
A login page, as shown in the figure, will be displayed.
By default, a randomly generated password is configured for accessing the GUI. It can be retrieved with the following command:
kubectl --namespace tobs get secrets tobs-grafana -o jsonpath="{.data.admin-password}" | base64 -d ; echo
The password can be used along with the admin username to login into the GUI.
Predefined dashboards
Grafana dashboards are intuitive collections of panels that display data from various sources. Each dashboard can be customized with different panels, layouts and settings to suit specific monitoring needs.
Our deployment offers a set of predefined dashboards, which can be accessed from the side menu Home -> Dashboards.
Each of these dashboards focuses on a particular aspect of the cluster. Their panels are configured to run queries against the data sources (MySQL and Prometheus). It is possible to clone and adapt these dashboards to match a specific use case or scenario, or to use them as a base to develop new ones.
Creating a dashboard for xApp metrics
To create a new dashboard, go to Home -> Dashboards -> New -> New dashboard.
First, let's add a new panel. Click on Add Visualization and select FlexRIC SDL as the data source. This data source is the MySQL database to which the deployed xApp is storing data in.
Under the Query panel, you can build your own query via SQL language, both as a code or with a GUI builder. In our case, the first is used. In the current configuration, the data will be stored in th dataset testdb, under the table KPM_IND_MEAS_DATA_INFO. The available parameters (see O-RAN specs) can be filtered by the meas_name column and plotted over time using the tstamp column.
Let's look at an example. In the following query, we will retrieve the UE download throughput.
SELECT meas_value_real AS "value", FROM_UNIXTIME(tstamp/1000000) AS "time" FROM testdb.KPM_IND_MEAS_DATA_INFO WHERE (meas_name = 'DRB.UEThpD') ORDER BY tstamp DESC LIMIT 100
Apply the query by clicking the Run Query and the panel should appear as in the figure.
It is possible to configure additional queries for the same panel, along with their visualisation details (e.g. panel title, axes). Once you have completed the process, you will find a Apply button in the top right corner of the panel.
You have the capability of arranging multiple panels within the same dashboard. Any changes to every panel or to the dasboard will be applied permanently by clicking the Save button.
Importing dashboards
Importing dashboards in Grafana allows users to quickly set up preconfigured visualizations without manually creating panels.
- Navigate to Dashboards in the main menu.
- Click New and select Import.
- Choose one of the following:
- Upload a JSON file containing the dashboard configuration.
- Paste a Grafana.com dashboard URL or ID.
- Directly paste JSON text into the provided field.
Kepler dashboard
Our deployment uses Kepler (Kubernetes Efficient Power Level Exporter), which employs eBPF to probe energy-related system statistics. These metrics are then exposed and stored in Prometheus, from which Grafana can retrieve data.
A Grafana dashboard is available for this metrics, and which can be downloaded at this link. The dashboard can be imported as a JSON file. Once uploaded, it will display the cluster's power statistics (e.g.; from pods) as shown in the figure.
