rfnocmodtool replacement, Unable to build Custom OOT module with deprecated rfnocmodtool for UHD 4.6 and GNURadio 3.10

[vahidrezaee]

Issue Description

This issue concerns the transition from the deprecated rfnocmodtool to building custom OOT modules for newer versions of UHD and GNURadio. Previously (UHD v4.4 , gnuradio v3.8, grr-ettus version "maint3.8-UHD4.4"), I successfully built a custom OOT module using rfnocmodtool and integrated it with GNU Radio Companion with the help of gr-ettus. However, both tools are now deprecated with UHD 4.6 and GNURadio 3.10.

Specific Challenges

Lack of replacement for rfnocmodtool: While understanding that rfnocmodtool is no longer supported, I'm unsure of the recommended alternative to build custom OOT modules in the new workflow.
Integrating modules with GNU Radio Companion: With gr-ettus being deprecated, the process of integrating OOT modules into GNU Radio Companion remains unclear.

Desired Outcome:

Understand the proper workflow for building custom OOT modules compatible with UHD 4.6 and GNURadio 3.10.
Obtain detailed instructions or resources on integrating these modules with GNU Radio Companion, replacing the functionality previously achieved with gr-ettus

Additional Information

Platform: Ubuntu 20.04
Hardware: X410 usrp
UHD version: 4.6
GNURadio version: 3.10
Previous module development approach: Used rfnocmodtool

Request

I kindly request assistance from the community or developers in understanding the current best practices for building and integrating custom OOT modules with newer versions of UHD and GNURadio. Guidance on specific tools, resources, or steps involved would be greatly appreciated.

[lrdrive]

Has any progress been made in that regard? Are we close to having a rfnocmodtool replacement by any chance?

[mbr0wn]

There is some progress, but we are still a way out from a new modtool. For the upcoming release, we are first improving the image builder, as a foundational tool. We are planning on working on modtool later this year.

[zacaric]

I am also trying to get an OOT module working with GNURadio 3.10 and I noticed that the master branch has some extra files for GNURadio that the UHD 4.7 branch doesn't have and that these files match up with the files talked about in this video. Does the current sudo make install command in the main branch also install the files needed to connect your OOT block to GNURadio or is there another command that needs to be ran after building your OOT module?

UPDATE: I was able to install the main branch on my computer and get it all setup and use the new rfnoc_modtool just fine. It seems there is still not a good solution for the GNURadio layer though. I tried getting the rfnoc_gain example project to build but it seems building gr_rfnoc_gain still doesn't work (error below) and I can't see anyway to create a gr_rfnoc_<my_block> folder unless I copy and modify a bunch of files.

Here is the error when trying to build gr_rfnoc_gain:

[ 54%] /home/zacaric/git/rfnoc/rfnoc-project-example/src/oot/rfnoc-gain/gr-rfnoc_gain/python/rfnoc_gain/bindings/../../../include/gnuradio/rfnoc_gain/rfnoc_gain.h
[REGISTRY] WARNING: Attempting to overwrite previously registered RFNoC block with noc_id,device_id: 0xb16, 0xffff
[REGISTRY] WARNING: Attempting to overwrite previously registered RFNoC block with noc_id,device_id: 0xb16, 0xffff
usage: bind_oot_file.py [-h] [--module MODULE] [--output_dir OUTPUT_DIR] [--prefix PREFIX] [--filename FILENAME] [--defines [DEFINES ...]]
                        [--include [INCLUDE ...]] [--status STATUS] [--flag_automatic FLAG_AUTOMATIC] [--flag_pygccxml FLAG_PYGCCXML]
bind_oot_file.py: error: unrecognized arguments: --src /home/zacaric/git/rfnoc/rfnoc-project-example/src/oot/rfnoc-gain/gr-rfnoc_gain
make[2]: *** [python/rfnoc_gain/bindings/CMakeFiles/rfnoc_gain_python.cc_regen_bindings.dir/build.make:73: /home/zacaric/git/rfnoc/rfnoc-project-example/src/oot/rfnoc-gain/gr-rfnoc_gain/python/rfnoc_gain/bindings}/rfnoc_gain_python.cc] Error 2
make[1]: *** [CMakeFiles/Makefile2:403: python/rfnoc_gain/bindings/CMakeFiles/rfnoc_gain_python.cc_regen_bindings.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

This seems to be just the last thing that needs to be done as I am able to build the gain OOT block, install it, and put it on my x310 using the rfnoc_image_builder

[zacaric]

Another Update.

After installing GNURadio 2.10.11.x the build is able to get much farther. Now instead it fails with the following error:

[ 50%] Built target rfnoc_gain_python.cc_regen_bindings
[ 60%] Built target extracted_docstrings
[ 70%] Built target rfnoc_gain_docstrings
[ 80%] Building CXX object python/rfnoc_gain/bindings/CMakeFiles/rfnoc_gain_python.dir/rfnoc_gain_python.cc.o
/home/zacaric/git/rfnoc/rfnoc-project-example/src/oot/rfnoc-gain/gr-rfnoc_gain/python/rfnoc_gain/bindings/rfnoc_gain_python.cc:26:10: fatal error: rfnoc_gain/rfnoc_gain.h: No such file or directory
   26 | #include <rfnoc_gain/rfnoc_gain.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [python/rfnoc_gain/bindings/CMakeFiles/rfnoc_gain_python.dir/build.make:76: python/rfnoc_gain/bindings/CMakeFiles/rfnoc_gain_python.dir/rfnoc_gain_python.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:405: python/rfnoc_gain/bindings/CMakeFiles/rfnoc_gain_python.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

So it is pretty close to working building!

[mbr0wn]

@zacaric For getting started with the rfnoc-gain example, I suggest:

1. Install the latest GNU Radio (looks like you already did this)
2. Install UHD from master branch from source (this is advanced, but seems you're capable)
3. Then, install host/examples/rfnoc-gain
4. You need to manually install gr-rfnoc_gain as well (host/examples/rfnoc-gain/gr-rfnoc_gain).

Now you can run the GNU Radio Examples for the RFNoC gain block. This is a good starting point.

You're right -- creating your own GNU Radio Blocks for RFNoC blocks is not (yet) super convenient. But these steps show you what you need and where to put it.

[zacaric]

Thanks so much for the feedback and response!

I am trying to install gr-rfnoc_gain, but unfortunately it is getting an error when trying to link to #include <rfnoc_gain/rfnoc_gain.h>. Inside of the gr-rfnoc_gain directory, I did the following:

mkdir build
cd build
cmake ../
make

It then fails to run through the make command with the same errors that I mentioned here. I even ran it from the host/examples/rfnoc-gain/gr-rfnoc_gain directory as well.

[mbr0wn]

Did you make install the rfnoc-gain OOT first? If so, where did it go?

[zacaric]

I did run sudo make install on the rfnoc_gain block first. It looks like it installs in /usr/local/share/uhd/rfnoc/ according to the output below:

Install the project...
-- Install configuration: ""
-- Up-to-date: /usr/local/share/uhd/rfnoc/blocks/gain.yml
-- Up-to-date: /usr/local/share/uhd/rfnoc/blocks/my_block.yml
-- Up-to-date: /usr/local/share/uhd/rfnoc/dts/gain_block.dtsi
-- Up-to-date: /usr/local/share/uhd/rfnoc/fpga/gain/ip
-- Up-to-date: /usr/local/share/uhd/rfnoc/fpga/gain/ip/cmplx_mul
-- Up-to-date: /usr/local/share/uhd/rfnoc/fpga/gain/ip/cmplx_mul/Makefile.inc
-- Up-to-date: /usr/local/share/uhd/rfnoc/fpga/gain/ip/cmplx_mul/cmplx_mul.xci
-- Up-to-date: /usr/local/share/uhd/rfnoc/fpga/gain/rfnoc_block_gain/Makefile.srcs
-- Up-to-date: /usr/local/share/uhd/rfnoc/fpga/gain/rfnoc_block_gain/rfnoc_block_gain.v
-- Up-to-date: /usr/local/share/uhd/rfnoc/fpga/gain/rfnoc_block_gain/noc_shell_gain.v
-- Up-to-date: /usr/local/include/rfnoc/gain/my_block_block_control.hpp
-- Up-to-date: /usr/local/include/rfnoc/gain/my_block_block_control.hpp
-- Up-to-date: /usr/local/include/rfnoc/gain/my_block_block_control.hpp
-- Up-to-date: /usr/local/include/rfnoc/gain/gain_block_control.hpp
-- Up-to-date: /usr/local/lib/librfnoc-gain.so
-- Up-to-date: /usr/local/share/uhd/modules.d/rfnoc-gain
-- Up-to-date: /usr/local/bin/probe_gain_block
-- Up-to-date: /usr/local/share/uhd/examples/rfnoc-gain/rx_gain_estimate_power.py
-- Up-to-date: /usr/local/lib/python3.10/site-packages/rfnoc_gain
-- Up-to-date: /usr/local/lib/python3.10/site-packages/rfnoc_gain/__init__.py
-- Up-to-date: /usr/local/lib/python3.10/site-packages/rfnoc_gain/rfnoc_gain_python.cpython-310-x86_64-linux-gnu.so
-- Up-to-date: /usr/local/lib/python3.10/site-packages/rfnoc_gain/rfnoc_gain_python.cpython-310-x86_64-linux-gnu.so

[mbr0wn]

OK that's good. But this is the line from gr-rfnoc_gain:

https://github.com/EttusResearch/uhd/blob/master/host/examples/rfnoc-gain/gr-rfnoc_gain/python/rfnoc_gain/bindings/rfnoc_gain_python.cc#L26

It's different from the one you are referencing (see the missing gnuradio/). Sure you're up to date?

[zacaric]

Hmm.... I should be since I cloned from the master branch last week. I will reclone and try again. Thanks so much for the help

[zacaric]

Even after cloning the master branch, I had to go into /host/examples/rfnoc-gain/gr-rfnoc_gain/python/rfnoc_gain/bindings/rfnoc_gain_python.cc and change #include <rfnoc_gain/rfnoc_gain.h> to #include <gnuradio/rfnoc_gain/rfnoc_gain.h>. After I did that, it was able to build just fine byt running make.

Now I am getting the following error when I run sudo make install:

[ 22%] Built target gnuradio-rfnoc_gain
[ 33%] Built target doxygen_target
[ 44%] Built target pygen_python_rfnoc_gain_259f5b3729c9866ce8cd3a01da45df64
[ 44%] Built target copy_module_for_tests
[ 55%] Built target extracted_docstrings
[ 66%] Built target rfnoc_gain_docstrings
[100%] Built target rfnoc_gain_python
Install the project...
-- Install configuration: "Release"
-- Installing: /usr/local/include/gnuradio/rfnoc_gain/api.h
-- Installing: /usr/local/include/gnuradio/rfnoc_gain/rfnoc_gain.h
-- Installing: /usr/local/lib/libgnuradio-rfnoc_gain.so.4.7.0.0
-- Up-to-date: /usr/local/lib/libgnuradio-rfnoc_gain.so.4.7.0
-- Set non-toolchain portion of runtime path of "/usr/local/lib/libgnuradio-rfnoc_gain.so.4.7.0.0" to ""
-- Up-to-date: /usr/local/lib/libgnuradio-rfnoc_gain.so
-- Installing: /usr/local/lib/cmake/gnuradio-rfnoc_gain/gnuradio-rfnoc_gainTargets.cmake
-- Installing: /usr/local/lib/cmake/gnuradio-rfnoc_gain/gnuradio-rfnoc_gainTargets-release.cmake
-- Installing: /usr/local/lib/cmake/gnuradio-rfnoc_gain/gnuradio-rfnoc_gainConfig.cmake
CMake Error at examples/cmake_install.cmake:46 (file):
  file INSTALL cannot find
  "/home/zacaric/git/rfnoc/rfnoc-project-example/src/oot/rfnoc-gain/gr-rfnoc_gain/examples/grc":
  No such file or directory.
Call Stack (most recent call first):
  cmake_install.cmake:62 (include)


make: *** [Makefile:110: install] Error 1

UPDATE: It turned out I had to move host/examples/rfnoc-gain/gr-rfnoc_gain/grc/ into host/examples/rfnoc-gain/gr-rfnoc_gain/examples/. After that it was able to to install

[zacaric]

Okay, we are really close here.

The issue I am having now is when I implement the gain block I get the following error:

Traceback (most recent call last):
  File "/home/zacaric/Documents/gnuradio/rfnoc_test/gain_example.py", line 213, in <module>
    main()
  File "/home/zacaric/Documents/gnuradio/rfnoc_test/gain_example.py", line 190, in main
    tb = top_block_cls()
  File "/home/zacaric/Documents/gnuradio/rfnoc_test/gain_example.py", line 149, in __init__
    self.rfnoc_graph.connect(self.rfnoc_gain_gain_native_0.get_unique_id(), 0, self.uhd_rfnoc_rx_streamer_1.get_unique_id(), 0, False)
AttributeError: 'gnuradio.rfnoc_gain.rfnoc_gain_python.rfnoc_gain' object has no attribute 'get_unique_id'. Did you mean: 'unique_id'?

Looking into the files in gr-rfnoc_gain, I can't seem to find a function called unique_id(). And when I modify the python file to use unique_id(). I get the following error when running python3 gain_example.py:

Traceback (most recent call last):
  File "/home/zacaric/Documents/gnuradio/rfnoc_test/gain_example.py", line 213, in <module>
    main()
  File "/home/zacaric/Documents/gnuradio/rfnoc_test/gain_example.py", line 190, in main
    tb = top_block_cls()
  File "/home/zacaric/Documents/gnuradio/rfnoc_test/gain_example.py", line 149, in __init__
    self.rfnoc_graph.connect(self.rfnoc_gain_gain_native_0.unique_id(), 0, self.uhd_rfnoc_rx_streamer_1.get_unique_id(), 0, False)
TypeError: connect(): incompatible function arguments. The following argument types are supported:
    1. (self: gnuradio.uhd.uhd_python.rfnoc_graph, arg0: str, arg1: int, arg2: str, arg3: int, arg4: bool) -> None
    2. (self: gnuradio.uhd.uhd_python.rfnoc_graph, arg0: str, arg1: str, arg2: bool) -> None

Invoked with: <gnuradio.uhd.uhd_python.rfnoc_graph object at 0x763c19b85730>, 3, 0, 'RxStreamer#0', 0, False

[zacaric]

UPDATE: After messing around some more, I found a few things:
1. Two Gain blocks get made. Generic and Native C++
2. Neither of them will work unless you use an RFNoC DDC block first
3. The Native C++ block doesn't work even after adding the the RFNoC DDR block before the gain

Now I have an RFNoC gain block working in GNURadio 3.10.11.x

Thanks so much for the help!

[mbr0wn]

The new rfnoc_modtool isn't perfect, and doesn't generate GNU Radio bindings yet. But it's a big step in the right direction. I'm going to close this. Please open new issues if things come up. Note that we're still working on rfnoc_modtool, we're not finished!