Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement RPC support for the LSVD bdev backend #40

Closed
knikolla opened this issue Jul 31, 2024 · 1 comment
Closed

Implement RPC support for the LSVD bdev backend #40

knikolla opened this issue Jul 31, 2024 · 1 comment

Comments

@knikolla
Copy link
Contributor

Currently, the lsvd binary starts up SPDK, registers a LSVD bdev backend and a single bdev, and creates an NVMe over fabrics TCP transport. It is not possible to create others because there is no support for issuing RPC commands to register them.

From an email from Isaac

We were talking about the canonical way to use LSVD going forward, and it seems like the best way forward is to integrate LSVD into the rpc framework.

The implementation shouldn’t be difficult, just tedious. Take a look at https://github.com/spdk/spdk/blob/master/module/bdev/rbd/bdev_rbd_rpc.c for reference on how it’s supposed to be done. The equivalent of bdev_rbd.c is for lsvd is https://github.com/CCI-MOC/lsvd-rbd/blob/main/src/bdev_lsvd.cc.

The main idea here is that the file itself defines SPDK_RPC_REGISTER sections, which defines a new rpc endpoint. At startup SPDK will scan through the binary and libraries for defined RPC endpoints and do the actual registration.The registration takes a function pointer for the entry point, and since the commands are passed in as json it’s probably easiest to just use their json decoding framework (if you want to pull in a cpp json framework that’s nicer feel free).

The main thing to note is that the RBD bdev rpc implementation is very async-driven; we can avoid the c-style callbackfn hell to a certain extent because the LSVD api is designed with some of this in mind. Just keep lifetimes in mind.

LSVD currently creates its own RBD cluster on image creation if one is not passed in iirc, so we don’t need to implement or integrate any of the RBD cluster functions. The only endpoints we need to register should be the bdev_lsvd_create and delete; everything else should be fine being left out of the implementation. We can always add them later if the need arises.

  • Isaac Khor
@knikolla
Copy link
Contributor Author

knikolla commented Sep 4, 2024

Closed by #41

@knikolla knikolla closed this as completed Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant