Introduction

This is the implementation of Puppis, a framework that adapts CPU core configuration and packet steering based on traffic loads.

Components

• xdp/: contains the main puppis program, to run on DUT. Run the Python control plane which performs dynamic CPU scaling, and loads the XDP program to perform packet steering between CPU cores.
• p4prog/: contains the P4 program, kaypou, to run on a programmable switch. kaypou generates packets based on a traffic distribution, timestamps egress and ingress packets to obtain latency, and sends the resultant latency over ZeroMQ to device under test.

Install necessary tools for BCC and eBPF

Run

sudo apt install bpfcc-tools linux-headers-$(uname -r)

Note that pip3 install bcc may not work according to this.

Then if there is any other module not found, just install with pip3.

If there are issues after installation, another option is to build from source. Follow this comment.

Running the program

With the existing setup, the most straightforward way to run puppis is as follows:

cd xdp
../setup.sh
sudo python3 xdp_prog_user.py --enable-c-state [--enable-lat-resp]

You can simply quit execution using ctrl + C, it would detach xdp program from NIC then.

Run the setup script to return the system to original state.

../setup.sh

System design

Control loop

Here data plane refers to c xdp programs in kernel space, while control plane means python controller in user space.

For now, there are 2 shared table between the 2 planes to achieve control loop:

• traffic stats
  • Written by data plane
  • Read by control plane
  • Data plane keep track of network stats when receiving packets, for now it is a coarse-grained measurement that doesn't distinguish between flows.
  • Control plane periodically fetch the table to know what is happening in the network, helping it to come up with better rss plans.
• rss distr
  • Written by control plane
  • Read by data plane
  • Data plane chooses responsible CPU core for packets based on the distr table, using the 4 tuple hash as key.
  • Control plane dynamically adjust the distr table, rearrange the forwarding ratio for each CPU core.

Reference codes

General structure

• XDP tutorials
• bcc examples
• learning ebpf

BCC related

• How to read C struct with bcc in here
• Useful manual

Developing

Debug

To debug the data plane program, the XDP steering part, one might find bpf\_Printk useful, however, you need to set the debug flag to True when loading the XDP program in control plane, and observe it through the tracing\_pipe.

Use bpftool to read data structures

# Get program ID
sudo bpftool prog list

# Get program map ID
sudo bpftool map list

# Dump contents of the map
sudo bpftool map dump id [id]

Use bpf_trace_printk to print statements

You can write bpf_trace_printk in the C code (ref). Read the printout with

sudo cat /sys/kernel/debug/tracing/trace_pipe