Skip to content

Latest commit

 

History

History
92 lines (68 loc) · 3.45 KB

XDMA.org

File metadata and controls

92 lines (68 loc) · 3.45 KB
Raw

XDMA

XDMA

We are using the Xilinx PCIE XDMA driver for register access to the KCU105 board. It is configured for 8 GT/s, 1 lane PCIE (we can expand the number of lanes in the future if needed).

The open source XDMA driver must be installed on a Linux PC in order to use the board.

Installation

1. Install the XDMA Kernel Driver

Based on instructions from https://www.xilinx.com/support/answers/65444.html

XDMA driver should be compiled from source. Code is at: https://xilinx.github.io/dma_ip_drivers/2019.2/linux-kernel/html/build.html

A working recipe tested on Ubuntu 16.04 is:

sudo apt-get install libaio1 libaio-dev
git clone https://github.com/Xilinx/dma_ip_drivers
#cd dma_ip_drivers/QDMA/linux-kernel
#make clean && make
#sudo make install
#sudo insmod bin/qdma-pf.ko

cd dma_ip_drivers/XDMA/linux-kernel/xdma
sudo make
sudo make install
sudo depmod

add xdma to ~/etc/modules~

2. Set up XDMA uDEV Rules

For non-root access to the XDMA driver you should add a udev rule:

sudo wget https://github.com/RHSResearchLLC/XilinxAR65444/raw/master/Linux/Xilinx_Answer_65444_Linux_Files/etc/udev/rules.d/xdma-udev-command.sh https://github.com/RHSResearchLLC/XilinxAR65444/raw/master/Linux/Xilinx_Answer_65444_Linux_Files/etc/udev/rules.d/60-xdma.rules -P /etc/udev/rules.d/
sudo udevadm control --reload-rules
sudo udevadm trigger

3. Reboot

n.b you may need to reboot after installation before this works

  • try rebooting with a bitstream loaded already

4. Check for the XDMA Device

The lspci command in Unix can be used to list and display information about devices connected to the system’s PCI(e) buses. After running /sys/bus/pci/rescan, you should run lspci to check that the Xilinx device has been detected correctly - if it is, the output of lspci will contain a line like:

01:00.0 Serial controller: Xilinx Corporation Device 8031

You should also now see xdma devices in /dev/, e.g.

$ ls /dev/x*
/dev/xdma0_c2h_0     /dev/xdma0_events_10  /dev/xdma0_events_14  /dev/xdma0_events_4  /dev/xdma0_events_8
/dev/xdma0_control   /dev/xdma0_events_11  /dev/xdma0_events_15  /dev/xdma0_events_5  /dev/xdma0_events_9
/dev/xdma0_events_0  /dev/xdma0_events_12  /dev/xdma0_events_2   /dev/xdma0_events_6  /dev/xdma0_h2c_0
/dev/xdma0_events_1  /dev/xdma0_events_13  /dev/xdma0_events_3   /dev/xdma0_events_7

Testing the XDMA Link

python3 read_write_single_register_without_connection_file.py ipbuspcie-2.0:///dev/xdma0_h2c_0,/dev/xdma0_c2h_0 ipbus_example.xml reg

FAQ

Reloading the bus

After loading a new bitfile onto the FPGA, in order to refresh the computer’s PCI tree, you should run the rescan command /sys/bus/pci/rescan. e.g.

echo 1 | sudo tee /sys/bus/pci/rescan
https://alexforencich.com/wiki/en/pcie/hot-reset-linux

https://www.origin.xilinx.com/support/answers/37406.html

cp /sys/bus/pci/devices/0000\:02\:00.0/config pcie_config

Where 03:00:0 is your own domain:bus:dev.func number that the BIOS chooses for you.
This will leave you with a file called myConfig with all the configuration data.

You can take a look at the content with the following command:
% od -x myConfig

Now the FPGA can be reconfigured using JTAG or some other method. To reinstate the PCIe configuration space in the endpoint, run the following command:

cp pcie_config /sys/bus/pci/devices/0000\:02\:00.0/config