This week I setup the srsRAN Project and figured out how we can perform UL scheduling.
- srsRAN Project is a 5G CU/DU solution and does not include a UE application. However, srsRAN 4G does include a prototype 5G UE (srsUE) which can be used for testing. This application note shows how to create an end-to-end fully open-source 5G network with srsUE, the srsRAN Project gNodeB and Open5GS 5G core network.
- Following the ZeroMQ installation note (linked here) which requires me to install srsRAN 4G, Open5GS and the srsRAN Project.
Installing srsRAN 4G from source, similar to my previous work.
- Installing the 'libzmq' and 'czmq' packages
$ sudo apt-get update
$ sudo apt-get install libzmq3-dev libczmq-dev
- Building srsRAN 4G from source as described here
$ git clone https://github.com/srsRAN/srsRAN_4G.git
$ cd srsRAN_4G
$ mkdir build
$ cd build
$ cmake ../
$ make -j 20
Installing Open5GS and dependent packages according to this installation guide.
- Installing and Running MongoDB
- Proved to be slightly more difficult than expected considering I am using an Ubuntu 22.04 machine running on a Docker Container, so Tini is used as the init system (not systemcl). Figured out how to do the same with Tini.
# installing the public key used by the system
$ sudo apt update
$ sudo apt install gnupg
$ curl -fsSL https://pgp.mongodb.com/server-6.0.asc | sudo gpg -o /usr/share/keyrings/mongodb-server-6.0.gpg --dearmor
$ echo "deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-6.0.gpg] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-6.0.list
# installing the package and starting it
$ sudo apt update
$ sudo apt install -y mongodb-org
$ sudo tini -s -- /usr/bin/mongod --fork --config /etc/mongod.conf
-
To ensure I did not have to do this every single time I started the system, I created two scripts:
- 'start_mongo.sh' : This script starts MongoDB with Tini.
```bash #!/bin/bash /usr/bin/tini -s -- /usr/bin/mongod --fork --config /etc/mongod.conf ``` - start_mongo : an init script that calls our previous script, and runs on startup of the machine.#!/bin/sh case "$1" in start) echo "Starting MongoDB" /usr/local/bin/start_mongo.sh ;; stop) echo "Stopping MongoDB" ;; restart) echo "Restarting MongoDB" /usr/local/bin/start_mongo.sh ;; *) echo "Usage: /etc/init.d/start_mongo {start|stop|restart}" exit 1 ;; esac exit 0
Wrote these scripts and enabled their functionality using the following commands:
$ sudo nano /usr/local/bin/start_mongo.sh
$ sudo chmod +x /usr/local/bin/start_mongo.sh
$ sudo nano /etc/init.d/start_mongo
$ sudo chmod +x /etc/init.d/start_mongo
$ sudo update-rc.d start_mongo defaults
Installing Open5GS
- Install the Open5GS package
$ sudo add-apt-repository ppa:open5gs/latest
$ sudo apt update
$ sudo apt install open5gs
- Install the WebUI for Open5GS
# Download and import the Nodesource GPG key
$ sudo apt update
$ sudo apt install -y ca-certificates curl gnupg
$ sudo mkdir -p /etc/apt/keyrings
$ curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
# Create deb repository
$ NODE_MAJOR=20
$ echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | sudo tee /etc/apt/sources.list.d/nodesource.list
# Run Update and Install
$ sudo apt update
$ sudo apt install nodejs -y
# Install the WebUI
$ curl -fsSL https://open5gs.org/open5gs/assets/webui/install | sudo -E bash -
- Installing the srsRAN Project
$ git clone https://github.com/srsran/srsRAN_Project.git
$ cd srsRAN_Project
$ mkdir build
$ cd build
$ cmake ../ -DENABLE_EXPORT=ON -DENABLE_ZEROMQ=ON
$ make -j`nproc`
Ended up shifting to a new machine.
-
File:
cell_scheduler.cpp
- Location:
srsRAN_Project/lib/scheduler/cell_scheduler.cpp
- Key Function:
ue_sched.run_slot
- Location:
-
File:
ue_scheduler_impl.cpp
- Location:
srsRAN_Project/lib/scheduler/ue_scheduling/ue_scheduler_impl.cpp
- Key Function:
run_slot
- Details: In the
run_slot
definition,run_sched_strategy
is run every time.run_slot
is a part of the classue_scheduler_impl
.
- Location:
-
File:
ue_scheduler_impl.cpp
- Location:
srsRAN_Project/lib/scheduler/ue_scheduling/ue_scheduler_impl.cpp
- Key Function:
run_sched_strategy
- Details: For UL, executes a
ul_sched
method every runtime.ul_sched
is a member function of the object pointed to by thesched_strategy
pointer.
- Location:
-
File:
ue_scheduler_impl.cpp
- Location:
srsRAN_Project/lib/scheduler/ue_scheduling/ue_scheduler_impl.cpp
- Key Function:
create_scheduler_strategy(expert_cfg)
- Details:
run_sched_strategy
is populated withcreate_scheduler_strategy(expert_cfg)
.
- Location:
-
File:
scheduler_policy_factory.cpp
- Location:
srsRAN_Project/lib/scheduler/policy/scheduler_policy_factory.cpp
- Key Function:
create_scheduler_strategy
- Details: Scheduling strategy is decided using the object
expert_cfg_
of the classscheduler_ue_expert_config
.
- Location:
-
File:
scheduler_expert_config.h
- Location:
srsRAN_Project/include/srsran/scheduler/config/scheduler_expert_config.h
- Key Class:
scheduler_ue_expert_config
- Details:
rr
is defined as the default scheduling policy here (usingtime_rr_scheduler_expert_config
).
- Location:
-
File:
scheduler_expert_config.h
- Location:
srsRAN_Project/include/srsran/scheduler/config/scheduler_expert_config.h
- Key Classes:
time_pf_scheduler_expert_config
andtime_pf_scheduler_expert_config
- Details: Both are defined here.
- Location:
-
Round Robin
A simple round robin implementation, where UE gets a chance to transmit one after the other.
- Functions of interest:
-
ul_sched
- Class: scheduler_time_rr
- Defined File: scheduler_time_rr.cpp
- Logic:
- If there are no UEs to schedule, the function terminates.
- If there exist UEs to schedule, the scheduling logic has three parts (in order):
- retransmitting old UL data
- pending scheduling requests (Find how this is done)
- transmitting new UL data
- Each of these three logics is implemented within the 'ul_sched' function using Lambda Expressions ('data_retx_ue_function', 'sr_ue_function' and 'data_tx_ue_function'). The one thing in common each of these logics have is that they call the 'round_robin_apply' and 'alloc_ul_ue' functions.
-
round_robin_apply
- Function is defined as a template. Allows the function to accept any callable object (in the case, a lambda expression) to allocate resources to a UE.
- Loops through all UEs, don't see how a weight based scheduling can be implemented here.
-
-
Proportional Fair
A proportional fair implementation also exists, which can be found in the file 'scheduler_time_pf.cpp'. This is more of an implementation we can play around with, rather than the Round Robin one. Defined in 'scheduler_time_pf.cpp' file (srsRAN_Project/lib/scheduler/policy/scheduler_time_pf.cpp).
#### Basic Logic:
- The logic is based on a queue system.
- The queues are determined using the `compute_ul_prio` function.
- Information from the UE at the top of the queue is taken, and then the `try_ue_alloc` function is called.
- UE allocation details are decided in the `compute_ul_prio` function. Here, other scheduling implementations, such as Max CQI, Max Weight, etc, can also be implemented. A priority function is involved, which seems to be another name for the weight variable we have been using.
#### Priority Computation:
1. Estimates the instantaneous UL rate using the `get_estimated_ul_rate` function.
2. Retrieves the current average UL rate using the `ul_avg_rate` function.
3. Calculates the priority using a proportional fair formula:
- priority = estimated_rate / (current_avg_rate ^ fairness_coefficient)
4. If the current average rate is zero, sets priority to maximum if estimated rate is non-zero, or to zero otherwise.
5. The fairness coefficient is defined using the `pf_sched_fairness_coeff` function. It is defined in the `scheduler_expert_config.h` file.
#### Functions of Interest:
1. `get_estimated_ul_rate`: Defined in `ue_cell.cpp` and used in `tbs_calculator.cpp`.
2. `ul_avg_rate`
-
Few Definitions:
- UL : UE to N/W
- DL : N/W to UE
- PUCCH (Physical Uplink Control Channel) is a UL Physical channel that is designed to carry UCI (Uplink Control Information). UCI messages support the scheduling of uplink shared channel (UL-SCH) transmissions. UCI can be carried either by PUCCH or PUSCH depending on the situation.
- PUSCH (Physical Uplink Shared Channel) is responsible for carrying user data from the UE to the network (gNB). Since it is a shared channel, multiple UEs can use the same physical resources for transmission, which are allocated dynamically by the gNB based on the network conditions and demand. This allocation is typically signaled through Downlink Control Information (DCI) sent over the PDCCH (Physical Downlink Control Channel).
- MCS (Modulation and Coding Scheme) decides both the modulation scheme and the coding rate.
-
Scheduling Request is a way for a UE to tell the network that it has some data in its MAC buffer and it needs PHY resources in the Uplink direction to send that data. A SR is only triggered when the UE does not any have any UL resources scheduled, i.e, PUSCH resources. Refer to this link.
-
There are roughly two types of situations where SR is triggered. The use cases are:
- Necessity-Based Use Case: A UE triggers a SR when it has some data to transmit but has no UL grant from the network.
- Periodicity-Based Use Case: In this case, the UE triggers SR with a certain periodicity (specified in RRC) regardless of whether it has any data to send or not.
-
Even though the SR message itself is a kind of physical layer message, it is controlled by a MAC layer process.
- Negative vs Positive SR:
- Positive SRs are sent when a UE requires an uplink transmission opportunity to send its buffered data to the network. Positive SR is crucial for efficient resource utilization, as it helps the gNB identify UEs with pending data and allocate resources accordingly.
- Negative SRs occur when a UE does not have sufficient data in its buffers to transmit or when it determines the network conditions are not optimum for uplink transmission. An example of these types of SRs are SRs sent periodically, often ignoring the environment in which transmission occurs.
Why am I reading about this?: We need to establish a benchmark of time taken from the start of the transmission across the PUCCH to the end of the transmission across the PUSCH.
Ran a 2 UE system and performed a few experiments to better understand scheduling and SRs in the srsRAN Project stack.
- To connect multiple UEs to a single gNB using ZMQ-based RF devices, we require a broker that can both receive and process UL and DL signals.
- We use GNU-Radio Companion to implement such a broker, as it allows easy integration with our ZMQ-based RF devices. More information about the setup can be found here.
Each step is executed in a seperate terminal. The Open5GS Core is already running. Kill and start the flowgraph each time you want to run the system.
- Run the GRC Flowgraph
$ cd /home/wcsng-32/gitrepos/EdgeRIC-Pulak
$ sudo -E python3 2ue-zmq-mode-23.04Mhz.py
- Run gNB
$ cd /home/wcsng-32/gitrepos/EdgeRIC-Pulak/build/apps/gnb
$ sudo ./gnb -c zmq-mode-multi-ue.yml
- Create network namespaces for both the UEs and run UE 1
$ cd /home/wcsng-32/gitrepos/EdgeRIC-5G/srsRAN_4G/build/srsue/src
$ sudo ./srsue ../../../.config/ue1-4g-zmq.conf
- Run UE 2
$ cd /home/wcsng-32/gitrepos/EdgeRIC-5G/srsRAN_4G/build/srsue/src
$ sudo ./srsue ../../../.config/ue2-4g-zmq.conf
Note: The Core Network runs using Open5GS at the destination IP Address: 10.45.0.1. We can just ping back and forth from that location.
- Traffic Server
$ cd /home/wcsng-32/gitrepos/EdgeRIC-Pulak
$ ./traffic_2ues_ul_server.sh
- Traffic Client
$ cd /home/wcsng-32/gitrepos/EdgeRIC-Pulak
$ ./traffic_2ues_ul_client.sh
-
ul_avg_rate()
function:- located in
srsRAN_Project/lib/scheduler/policy/scheduler_time_pf.cpp
- scheduler_time_pf.cpp (used) --> scheduler_time_pf.h (defined here) --> ul_avg_rate_ (after a checkpoint)
- located in
-
ul_avg_rate_
variable:- located in
srsRAN_Project/lib/scheduler/policy/scheduler_time_pf.cpp
- initialised as 0 in
scheduler_time_pf.h
- later updated below. clearly,
alloc_bytes
is the variable we need
void scheduler_time_pf::ue_ctxt::save_ul_alloc(uint32_t alloc_bytes) { if (ul_nof_samples < 1 / parent->exp_avg_alpha) { // Fast start before transitioning to exponential average. ul_avg_rate_ = ul_avg_rate_ + (alloc_bytes - ul_avg_rate_) / (ul_nof_samples + 1); } else { ul_avg_rate_ = (1 - parent->exp_avg_alpha) * ul_avg_rate_ + (parent->exp_avg_alpha) * alloc_bytes; } ul_nof_samples++; }
- located in
-
alloc_bytes
variable:- is actually just a local placeholder variable as are present in function definitions
- stores data from
alloc_results.alloc_bytes
alloc_results
is defined below, and is dependent onpusch_alloc.allocate_ul_grant(grant)
as defined in thetry_ul_alloc
function (located in the same file)
if (ctxt.ul_newtx_h != nullptr) { grant.h_id = ctxt.ul_newtx_h->id; grant.recommended_nof_bytes = ues[ctxt.ue_index].pending_ul_newtx_bytes(); alloc_result = pusch_alloc.allocate_ul_grant(grant); if (alloc_result.status == alloc_status::success) { ctxt.ul_newtx_h = nullptr; } return alloc_result; } if (ctxt.ul_newtx_h != nullptr) { grant.h_id = ctxt.ul_newtx_h->id; grant.recommended_nof_bytes = ues[ctxt.ue_index].pending_ul_newtx_bytes(); alloc_result = pusch_alloc.allocate_ul_grant(grant); if (alloc_result.status == alloc_status::success) { ctxt.ul_newtx_h = nullptr; } return alloc_result; }
- The
try_ul_alloc
function ensures thealloc_results
class is populated with the output ofpusch_alloc.allocate_ul_grant(grant)
-
pusch_alloc.allocate_ul_grant(grant)
function:- located in
srsRAN_Project/lib/scheduler/ue_scheduling/ue_cell_grid_allocator.cpp
- uses a function
compute_ul_mcs_tbs
to calculate the UL throughput per slot.
- located in
-
compute_ul_mcs_tbs
function:- the function definition is located in
srsRAN_Project/lib/scheduler/support/mcs_tbs_calculator.cpp
- the function definition is located in
Edit accordingly.
alloc_result alloc_result = {alloc_status::invalid_params};
while (not ul_queue.empty()) {
ue_ctxt& ue = *ul_queue.top();
if (alloc_result.status != alloc_status::skip_slot) {
alloc_result = try_ul_alloc(ue, ues, pusch_alloc);
}
//[PULAK] log using the rnti of this UE along with the alloc_bytes variable
ue.save_ul_alloc(alloc_result.alloc_bytes);
// Re-add the UE to the queue if scheduling of re-transmission fails so that scheduling of retransmission are
// attempted again before scheduling new transmissions.
if (ue.ul_retx_h != nullptr and alloc_result.status == alloc_status::invalid_params) {
ul_queue.push(&ue);
}
ul_queue.pop();
-
get_effective_snr()
- located in
srsRAN_Project/lib/scheduler/ue_scheduling/ue_link_adaptation_controller.cpp
- defined as below, uses the
ue_ch_st.get_pusch_snr()
as the base
float ue_link_adaptation_controller::get_effective_snr() const { return ue_ch_st.get_pusch_snr() + (ul_olla.has_value() ? ul_olla.value().offset_db() : 0.0f); }
- located in
-
ue_ch_st.get_pusch_snr()
- located in the
srsRAN_Project/lib/scheduler/ue_scheduling/ue_link_adaptation_controller.cpp
- simply returns another variable
pusch_snr_db
as the result - as we see here,
pusch_snr_db
is simply populated with the results ofexpert_cfg.initial_ul_sinr
ue_channel_state_manager::ue_channel_state_manager(const scheduler_ue_expert_config& expert_cfg, unsigned nof_dl_ports_) : // Initialization nof_dl_ports(nof_dl_ports_), pusch_snr_db(expert_cfg.initial_ul_sinr), wideband_cqi(expert_cfg.initial_cqi)
- located in the
-
scheduler_ue_expert_config.initial_ul_sinr
- private variable part of the
scheduler_ue_expert_config
- located in
srsRAN_Project/include/srsran/config/scheduler_expert_config.h
- seems to be defined somewhere, can't find how this value is set/defined. [USHASI'S INPUT NEEDED]
- private variable part of the
Define a function which creates an object of class ue_link_adaptation_controller
using which we can call the function get_effective_snr()
. How will this work slot-wise and UE-wise? Not sure if this is the right one.
Spent some time understanding C++ code and debugging C++ using VSCode. Also, tried to understand the variables and functions in use a little better. Helped me log metrics and decide how to implement further scheduling schemes.
-
ue_history_db
- Data Type: std::map<uint16_t, ue_ctxt>
- Key is a uint16_t (
ue_index
) representing the unique index or ID of a UE, and the value is an instance of ue_ctxt, which holds the context information for that UE. ue_history_db
keeps track of all UEs that have been scheduled.
-
it
- iterator used to access every member of
ue_history_db
it->ue_index
is used to access the key
for (auto it = ue_history_db.begin(); it != ue_history_db.end();) { if (not ues.contains(it->ue_index)) { ue_history_db.erase(it->ue_index); } ++it; }
- iterator used to access every member of
-
ues
- likely contains all currently connected UEs
-
ctxt
- reference of the
ue_ctxt
object, contains information about a specific UE (chosen using the UE Index).
- reference of the
-
try_ul_alloc
- how do you find the final UE which transmits in a slot finally?
-
ue.save_ul_alloc(alloc_result.alloc_bytes)
- effectively updates the estioated rate
-
ue_ctxt
- contains information about the UE
-
compute_ul_prio
- This is the function to be edited to implement different types of scheduling algorithms.
- how often is ul_prio used? - in a slot, once for every UE to be scheduled in that slot
- Functions/Files of Interest:
- Class
ue_cell
:- defined in
srsRAN_Project/lib/scheduler/ue_scheduling/ue_cell.h
. Check comments for more information. - used in
srsRAN_Project/lib/scheduler/ue_scheduling/ue_cell.cpp
- has a attribute called
rnti
--> access usingrnti_t my_rnti = ue_cell_object.rnti();
. - has
- defined in
- File
srsRAN_Project/lib/scheduler/config/ue_configuration.h
:- has two classes
ue_cell_configiration
andue_configiration
that contains information about the UE (especially the cell and realted information) for the given scheduling round.
- has two classes
- Class
-
ue_scheduler.run_slot(...)
:- called in
srsRAN_Project/lib/scheduler/cell_scheduler.cpp
- defined in
srsRAN_Project/lib/scheduler/ue_scheduling/ue_scheduler.h
andsrsRAN_Project/lib/scheduler/ue_scheduling/ue_scheduler_impl.cpp
- called in
-
run_slot(slot_point slot_tx, du_cell_index_t cell_index)
:- located in
srsRAN_Project/lib/scheduler/ue_scheduling/ue_scheduler_impl.cpp
- calls
run_sched_strategy(...)
which is located in the same file. This function eventually callsul_sched
- located in
-
ul_sched(ue_alloc, ue_res_grid_view, ue_db)
:- located in
srsRAN_Project/lib/scheduler/policy/scheduler_time_pf.cpp
. - finally!!!
- located in
For some reason, initially my setup didn't seem to be working. Rebuilding the srsRAN Project with the appropriate ZMQ and Debugging hooks. Using the below commands to rebuild my system.
rm -rf build
make clean
cmake -DENABLE_EXPORT=ON -DENABLE_ZEROMQ=ON -DCMAKE_BUILD_TYPE=Debug ..
make -j1