Lab 4: Storing and Exploring Data from the RAN
This experiment is to deploy a 5G Standalone (SA) network using OpenAirInterface (OAI) RF Simulator gNB and OAI minimal 5GC. We also deploy FlexRIC as the Near-RT RIC. For this lab, we are going to use VictoriaMetrics database to store RAN data. Finally a custom monitoring xApp with callbacks will be deployed on bare-metal connecting to the RIC and the database.
apiVersion: athena.trirematics.io/v1
kind: Network
metadata:
name: bubbleran
namespace: trirematics
spec:
slices:
- plmn: "00101"
dnn: "internet"
network-mode: "IPv4"
service-type: eMBB
differentiator: 0x000000
ipv4-range: "12.1.1.0/24"
ipv6-range: "2001:db8:1::/64"
access:
- name: oai-gnb
stack: 5g-sa
model: oai-ran/monolithic-gnb
identity:
an-id: 50
radio:
device: rf-sim
cells:
- band: n78
arfcn: 641280
bandwidth: 40MHz
subcarrier-spacing: 30kHz
tdd-config:
period: 5ms
dl-slots: 7
dl-symbols: 6
ul-slots: 2
ul-symbols: 4
controller: flexric.bubbleran
core-networks:
- minimal.bubbleran
core:
- name: minimal
stack: 5g-sa
model: oai-cn/minimal
identity:
region: 0
cn-group: 4
cn-id: 5
dns:
ipv4:
default: 8.8.8.8
secondary: 8.8.4.4
edge:
- name: flexric
stack: 5g-sa
model: mosaic5g/flexric
# - name: monitoring-xapp
# stack: 5g-sa
# model: mosaic5g/monitoring-c
# profiles:
# - rlc-sm
# - pdcp-sm
# - mac-sm
# - gtp-sm
# - slice-sm
# - kpm-sm
# - database
# annotations:
# extras.t9s.io/scenario: 'lab4'
---
apiVersion: athena.trirematics.io/v1
kind: Terminal
metadata:
name: ue1
namespace: trirematics
spec:
vendor: oai
stack: 5g-sa
model: terminal/nr-rfsim
preferred-access: oai-gnb.bubbleran
target-cores:
- minimal.bubbleran
identity:
imsi: "001010000000001"
pin: "1234"
opc: "0xc42449363bbad02b66d16bc975d77cc1"
key: "0xfec86ba6eb707ed08905757b1bb44b8f"
sqn: "0xff9bb4000001"
slice:
dnn: "internet"
network-mode: "IPv4"
service-type: eMBB
differentiator: 0x000000
radio:
bands:
- n78
readiness-check:
method: ping
target: google-ip
interface-name: oaitun_ue0
---
apiVersion: athena.trirematics.io/v1
kind: Terminal
metadata:
name: ue2
namespace: trirematics
spec:
vendor: oai
stack: 5g-sa
model: terminal/nr-rfsim
preferred-access: oai-gnb.bubbleran
target-cores:
- minimal.bubbleran
identity:
imsi: "001010000000002"
pin: "1234"
opc: "0xc42449363bbad02b66d16bc975d77cc1"
key: "0xfec86ba6eb707ed08905757b1bb44b8f"
sqn: "0xff9bb4000001"
slice:
dnn: "internet"
network-mode: "IPv4"
service-type: eMBB
differentiator: 0x000000
radio:
bands:
- n78
readiness-check:
method: ping
target: google-ip
interface-name: oaitun_ue0
Deployment
Use the command brc install network open-ran-db.yaml to deploy the network.
It should finish without errors and print out the three Kubernetes resource names that were created.
Check for the status of the deployment using the command brc observe.
Wait until all the Elements other than the UE are in the STATUS set to
1/1 Y state.
After deploying the network, before running the xApp, you should update the xApp configuration file with the IP of the deployed Near-RT RIC, the local source IP in the cluster subnet and the deployed database.
To do so you may execute the update_conf.py script which automatically extract the IPs from the cluster and updates the specified configuration file:
cd /path/to/xapp_sdk/conf
python3 update_conf.py xapp_cust_sm.yaml
Once the network is deployed, traffic can be generated between the UEs and the network with the following commands:
brc test throughput ue1 dl gateway -- -t 600
brc test throughput ue2 dl gateway -- -t 600
With the brc test throughput tool the user can generate iperf traffic from the UPF (gateway) to the UE for downlink traffic (dl) or vice-versa for uplink (ul).
For more details you may check the BubbleRAN CLI reference.
The script will prompt the user for selecting which Near-RT RIC the xApp should connect to (in case of multiple RICs and/or networks currently deployed).
Since xapp_all_sm.yaml also defines the database information, the script will prompt the user to specify which database in the deployed network to connect to.
After selecting a RIC and database to connect, an output as the following is obtained:
Select the RIC to configure:
1) flexric.flexric.bubbleran (10.244.1.75 - trirematics, pod)
Choice [1-1]: 1
Select the Database to configure:
1) vmsingle-vmks-victoria-metrics-k8s-stack (10.106.214.123 - monitoring, svc)
Choice [1-1]: 1
Port-forwarding enabled for vmsingle-vmks-victoria-metrics-k8s-stack on port 8428.
Enter the scenario name for VictoriaMetrics: lab4
Config 'xapp_all_sm.yaml' updated: ip_ric=10.244.1.75, ip_xapp=10.244.0.235, db.ip=localhost
If VictoriaMetrics is the selected database, the user should build the targets with VICTORIAMETRICS_XAPP CMakeList as shown in the Build and Install section.
For VictoriaMetrics, the script will automatically port-forward database to http://localhost:8428. It will also prompt for the scenario name to be configured for the xapp (lab4 in the example). Please, refer to xApp Generated Metrics to know more about it.
Once the configuration file has been updated accordingly, now the monitoring xApp can be run:
cd build
./src/dev/c/xapp_cust_moni ../conf/xapp_cust_sm.yaml
If run successfully an output like the following should be obtained:
13:09:04.660853 [INFO]: e42_xapp.c:161 NearRT-RIC Server IP Address = 10.244.1.75, PORT = 36422
13:09:04.660859 [INFO]: e42_xapp.c:162 xApp IP Address = 10.244.0.235
13:09:04.660879 [INFO]: endpoint_xapp.c:74 [xApp]: SCTP client bind to IP 10.244.0.235, Port 42094
13:09:04.660944 [INFO]: emb_sm_ag.c:95 Loaded SM(s) 11, custom SMs true
13:09:04.660959 [INFO]: emb_sm_ric.c:95 Loaded SM(s) 11, custom SMs true
13:09:04.661005 [INFO]: e42_xapp.c:194 DB_ENABLE = TRUE
13:09:04.661165 [INFO]: vm_wrapper.c:224 [VictoriaMetrics]: Successfully connected to VictoriaMetrics at localhost:8428
13:09:04.661809 [INFO]: msg_handler_xapp.c:553 E42 SETUP-REQUEST tx
13:09:04.662181 [INFO]: msg_handler_xapp.c:394 E42 SETUP-RESPONSE rx xApp ID 25
13:09:04.662194 [INFO]: msg_handler_xapp.c:410 Connected E2 Node(s) 1
13:09:05.662429 [INFO]: xapp_sub_cust_sm_conf.c:92 ---------------------------------------------------
Registered node 0 ran func id = 2
Registered node 0 ran func id = 3
Registered node 0 ran func id = 4
Registered node 0 ran func id = 5
Registered node 0 ran func id = 142
Registered node 0 ran func id = 143
Registered node 0 ran func id = 144
Registered node 0 ran func id = 145
Registered node 0 ran func id = 146
Registered node 0 ran func id = 147
Registered node 0 ran func id = 148
xApp subscribes RAN Func ID 142 in E2 node idx 0, nb_id 50
13:09:05.662560 [INFO]: msg_handler_xapp.c:588 RIC_SUBSCRIPTION_REQUEST tx RAN_FUNC_ID 142 RIC_REQ_ID 1
13:09:05.663033 [INFO]: msg_handler_xapp.c:155 RIC_SUBSCRIPTION_RESPONSE rx RAN_FUNC_ID 142 RIC_REQ_ID 1
xApp subscribes RAN Func ID 143 in E2 node idx 0, nb_id 50
13:09:05.663118 [INFO]: msg_handler_xapp.c:588 RIC_SUBSCRIPTION_REQUEST tx RAN_FUNC_ID 143 RIC_REQ_ID 2
13:09:05.663281 [INFO]: msg_handler_xapp.c:155 RIC_SUBSCRIPTION_RESPONSE rx RAN_FUNC_ID 143 RIC_REQ_ID 2
xApp subscribes RAN Func ID 144 in E2 node idx 0, nb_id 50
13:09:05.663342 [INFO]: msg_handler_xapp.c:588 RIC_SUBSCRIPTION_REQUEST tx RAN_FUNC_ID 144 RIC_REQ_ID 3
13:09:05.663607 [INFO]: msg_handler_xapp.c:155 RIC_SUBSCRIPTION_RESPONSE rx RAN_FUNC_ID 144 RIC_REQ_ID 3
MAC ind_msg latency = -591 from E2-node type 2 ID 50
RLC ind_msg latency = -683 from E2-node type 2 ID 50
PDCP ind_msg latency = -717 from E2-node type 2 ID 50
MAC ind_msg latency = -623 from E2-node type 2 ID 50
RLC ind_msg latency = -735 from E2-node type 2 ID 50
Accessing the database
In order to access the VictoriaMetrics database deployed within the cluster, the VictoriaMetrics database should be port-forwarded to localhost.
If you runned update_conf.py and selected VictoriaMetrics, then you can skip this step.
Run the following command:
# Port-forward VictoriaMetrics service port 8428 to localhost
kubectl port-forward -n monitoring services/vmsingle-vmks-victoria-metrics-k8s-stack 8428:8428
Now, we can access query data from the db. For example:
curl -G http://localhost:8428/api/v1/query \
--data-urlencode 'query=mac_dl_aggr_prb{job="raw_ran", sm="mac"}'
The query above retrieves the latest sample for the DL Aggregated PRBs for all the connected UEs.
This same data can be queried and visualized in Grafana. Check how at Visualizing Data with Grafana.
For the full parameter list, please refer to O-RAN Service Models and Custom Service Models.
Uninstall
To uninstall the network, use the following command:
brc remove network open-ran-db.yaml
Checking via the brc observe command, you should see that all the elements are removed.
💬 Lab Questions 💬
- Validate the MAC statistic (custom service model) in the database using SQL query.
- Use available tools to visualize the database.
- Construct a query to get the PLMN of the cell and the UE RNTI.
Advanced
This lab can also be performed leveraging an over-the-air deployment. In this section, we provide a sample deployment with a LITEON RU. Similarly to the previous deployment, this includes a 5G Standalone (SA) network with OpenAirInterface (OAI) gNB using a LITEON RU device, and Open5GS core network.
apiVersion: athena.trirematics.io/v1
kind: Network
metadata:
name: bubbleran
namespace: trirematics
spec:
slices:
- plmn: "00101"
dnn: internet
network-mode: IPv4
service-type: eMBB
differentiator: 0x000000
ipv4-range: 12.1.1.0/24
ipv6-range: 2001:db8:1::/64
access:
- name: oai-gnb
stack: 5g-sa
model: oai-ran/monolithic-gnb-ru #cu-du-ru
identity:
an-id: 30
tracking-area: 1
radio:
device: oran-7.2
antenna:
formation: 4x4
scheduling:
nodeName: bubble2 # Change with any other CU-DU node
annotations:
# PCI IDs obtained by running `dpdk-devbind.py --status` in the CU-DU node
extras.t9s.io/pci-ids: '["0000:01:11.0", "0000:01:11.1"]'
# VF MACs obtained for dpdk0 interface by running `ip link show` in CU-DU node
extras.t9s.io/du-macs: '["02:11:22:33:44:66", "02:11:22:33:44:67"]'
# RU MAC (repeated) obtained by logging into RU and running `show eth-info`
extras.t9s.io/ru-macs: '["e8:c7:4f:25:80:f9", "e8:c7:4f:25:80:f9"]'
extras.t9s.io/mtu: '9000'
# RU IP used for logging into it
extras.t9s.io/o1-remote-ipv4: 192.168.1.40
# Flag for reboot, after first use, it can be set to `false` if values in 'cells' is not changed
extras.t9s.io/liteon-ru-reboot: 'true' #'false'
cells:
- band: n78
arfcn: 643296
bandwidth: 100MHz
subcarrier-spacing: 30kHz
tdd-config:
period: 2.5ms
dl-slots: 3
dl-symbols: 6
ul-slots: 1
ul-symbols: 4
controller: flexric.bubbleran
core-networks:
- ogscore.bubbleran
core:
- name: ogscore
stack: 5g-sa
model: open5gs/5gc
profiles:
- debug
identity:
region: 128
cn-group: 4
cn-id: 5
edge:
- name: flexric
stack: 5g-sa
model: mosaic5g/flexric
dns:
ipv4:
default: 8.8.8.8
secondary: 8.8.4.4
Deployment
Use the command brc install network open-ran-liteon.yaml to deploy the network.
It should finish without errors and print out the three Kubernetes resource names that were created.
Check for the status of the deployment using the command brc observe.
Wait until all the Elements other than the UE are in the STATUS set to
1/1 Y state.
After the network has been successfully deployed, the monitoring xApp can be deployed following the exact same steps previously detailed.
Before deploying this network first make sure:
- The CU-DU node name specified in the deployment file (in this case
bubble2on the sectionscheduling) is correct. - The CU-DU node is able to reach the RU IP specified in the deployment file. If not sure about the IP, please check the Google Sheet (provided to all customers) with all IPs of your MX-PDK.
To do so you may try to ssh into the RU from the node with the command
ssh user@RU-IPreplacingRU-IPwith the actual IP. - The CU-DU node is properly synchronized with the PTP grandmaster switch. To do so you may check the status with
sudo systemctl status ptp4l.service
sudo systemctl status phc2sys.service
- The CU-DU node's DPDK has been setup with
sudo dpdk-setup