Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VHDL-embedded Python code #986

Draft
wants to merge 8 commits into
base: master
Choose a base branch
from
Draft

VHDL-embedded Python code #986

wants to merge 8 commits into from

Conversation

LarsAsplund
Copy link
Collaborator

@LarsAsplund LarsAsplund commented Jan 16, 2024

This PR contains work toward enabling embedded Python code within VHDL

For the first iteration I've focused on providing the lowest possible threshold for embedded Python and I found that to be the "eval and exec model" where the user can create any valid Python string and have that evaluated as an expression or executed as code by a Python interpreter.

The main limitation to what can be done is basically that variables passed between Python and VHDL must have a type that can be represented in both domains. For the first set of examples I also limited myself to a small set of scalar and vector/list types. Supporting remaining types is just more of the same and will be added as I move along.

Simplicity also means:

  • Good error messages when the code strings created in VHDL fail when being executed or evaluated in Python. Obviously such a test must also fail.
  • The user should not need any knowledge of C, C compilers, compiler versions,  foreign language interfaces etc. In fact, they should not even have to know that C and foreign language interfaces are the means by which this is achieved. Knowing VHDL and Python should be sufficient. In reality I may not reach that goal fully but for Riviera-Pro and Active-HDL, the only thing the user has to remember is to tick the "include C stuff" box during installation of the tool.

Riviera-PRO and Active-HDL use VHPI but it is important that the user interfaces do not leak VHPI specific stuff. The same API should be possible to use on all simulators regardless of what foreign language interface being used in the background
So far I've only worked on VHPI and on Windows.

Some work has also been done on running the Python code in separate process(es) (as in separate applications). Rather than passing code strings, the VHDL code would communicate with the concurrent Python actors using message passing or simple data queues. This work will be completed in a second iteration and in a separate PR. The goal of that use model is more towards performance.

Feedback on the eval and exec model is appreciated and a good starting point is to review the provided example tests. If you have an Riviera-PRO or Active-HDL license you can run the examples. If not, you should get a good feel for the concepts by just looking at the example code.

I would also appreciate more use case examples. So far I have these (note that some are expected to fail)

image

For you who don't have any of the currently supported simulators, here is a screenshot of what the user interactions provided by some tests may look like. I use PySimpleGUI which creates these types of interactions from one-liners

image

Currently tested simulators

Riviera-PRO

  • Version 2023.10, 64-bit
  • Windows 10
  • Python 3.12
  • GCC shipped with simulator (gcc.exe (MinGW-W64 x86_64-ucrt-posix-seh, built by Brecht Sanders) 12.3.0). Note that it is an option that must be selected during installation

Active-HDL

  • Version 14, 64-bit
  • Windows 10
  • Python 3.12
  • GCC shipped with simulator (gcc.exe (x86_64-posix-seh-rev0, Built by MinGW-W64 project) 4.8.4) doesn't work with the VHPI application. Not investigated further but that GCC version is more than 10 years old. In-place replacement with a newer version such that the GCC provided by Riviera-PRO works

Questa

  • Version 2023.3, 64-bit
  • Windows 10
  • Python 3.12
  • GCC provided by Siemens as a zip file to be unzipped in the installation directory (gcc.exe (GCC) 7.4.0)

nvc

  • Version 1.11.2
  • Windows 10/MSYS2
  • Python 3.11.7
  • GCC 13.2.0

GHDL

  • GHDL 4.0.0-dev (3.0.0.r750.g2135cbf14) [Dunoon edition]. Compiled with GNAT Version: 13.2.0. llvm 17.0.6 code generator
  • Windows 10/MSYS2
  • Python 3.11.7
  • GCC 13.2.0

@joshrsmith
Copy link
Contributor

Very interesting. This is like cocotb in reverse. 😆

@crdavis12

@LarsAsplund
Copy link
Collaborator Author

@joshrsmith Yes, it is reversed in the sense that the simulation is driven from VHDL rather than from Python. The reason is that we want to use VHDL for what is good for and only use Python when VHDL runs out of steam. Doing pin-wiggling from Python is a bit inefficient and that is not a price I want to pay for getting access to the areas where Python is a good fit.

@javValverde
Copy link

Great effort, this certainly opens up a lot of possibilities!

I'm still really interested in communication between python and VHDL with messages.
I would love for python to send a message to VHDL, and block until VHDL answers to the message.

@LarsAsplund
Copy link
Collaborator Author

@javValverde Yes, that will be step 2. I thought I should start with something simpler to find all the different quirks in VHPI, FLI, and the others. I also got FLI working so now the same examples can be run with no modification under Questa. Will do some refactoring before pushing that.

@cmarqu
Copy link
Contributor

cmarqu commented Jan 23, 2024

@LarsAsplund Not sure if you are aware, but Questa also supports VHPI since about two years.

@LarsAsplund
Copy link
Collaborator Author

@cmarqu Yes, I discovered that after I started with FLI. It was a bit hidden. However, very few VHPI functions are supported so currently it is not possible to use it for our purposes.

@LarsAsplund LarsAsplund force-pushed the python_pkg branch 2 times, most recently from c33ee57 to 958b275 Compare January 27, 2024 10:26
@LudvigVidlid
Copy link
Contributor

This looks so nice :D

@LarsAsplund LarsAsplund force-pushed the python_pkg branch 2 times, most recently from 25dcd0e to 90056c9 Compare April 1, 2024 19:22
Copy link
Contributor

@nickg nickg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The master branch of NVC now has some very basic support for VHPI foreign sub-programs. There's some examples here:

test/regress/vhpi14.vhd
test/vhpi/vhpi14.c

Although at the moment I don't support Aldec's format for the foreign attribute.


for c_file_path in c_file_paths:
args = [
"gcc",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs -fPIC too on Linux.

if (vhpi_get_value(parameter_handle, &parameter)) {
ffi_error_handler("getting VHDL parameter value", true);
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs a call to vhpi_release_handle(parameter_handle) here or a it leaks a handle on each invocation.

procedure python_setup;
-- TODO: Looks like Riviera-PRO requires the path to the shared library to be fixed at compile time
-- and that may become a bit limited. VHDL standard allow for expressions.
attribute foreign of python_setup : procedure is "VHPI libraries/python; python_setup";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does "VHPI python python_setup" work? ("python" here is the libraryName argument passed to vhpi_register_foreignf().) I think passing the path to the native library here and the semicolon separator is an Aldec extension that's not present in the VHDL LRM.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The semicolon is something I took from Aldec's examples. However, it works well without it so I will have it removed.

It seems like just python should be enough to identify the shared library if I put the libraries directory in the PATH environment variable. Are you also using that approach to for finding the libraries?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, what would be the benefits of using VHPI over VHPIDIRECT in this case?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like just python should be enough to identify the shared library if I put the libraries directory in the PATH environment variable. Are you also using that approach to for finding the libraries?

No, I'm rely on the user loading the library explicitly with --load and then the "VHPI libraryName modelName" should match the corresponding libraryName and modelName arguments passed to vhpi_register_foreignf(). I'm actually having a hard time understanding how the LRM intended these to be used. Does Aldec treat the first libraryName as the path to the shared library?

Btw, what would be the benefits of using VHPI over VHPIDIRECT in this case?

Honestly probably not many benefits. In theory if every simulator implemented the VHPI standard you'd only need one backend here rather than one per simulator.

@willkamp
Copy link

willkamp commented Sep 4, 2024

@LarsAsplund , Thanks for your efforts, this looks like a really powerful addition to the VUnit ecosystem. I look forward to this PR making it into the main branch and a release.

I've had a hack at compile_fli_application() in python_pkg.py to make it work in my environment - Ubuntu + python 3.10 + Intel Questa (from Quartus 24.1). I have the tests running :) 18 of 28 pass, and 10 error cases failing as expected.

We're quite interested is using something like this to build python based predictors of our signal processing in the Square Kilometre Array Telescope Correlator and Beamformer.

@LarsAsplund
Copy link
Collaborator Author

@willkamp Do you have any details on the hacks you had to do?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants