Backport DMABUF Support From Upstream To Ubuntu Jammy #3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The pin callback does not necessarily have to move the memory to system memory, remove the sentence from the comment. Link: https://lore.kernel.org/r/20211012120903.96933-2-galpress@amazon.com Signed-off-by: Gal Pressman <galpress@amazon.com> Reviewed-by: Christian König <christian.koenig@amd.com> (cherry picked from commit 5460601) Signed-off-by: Tushar Dave <tdave@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Introduce ib_umem_dmabuf_get_pinned() which allows the driver to get a dmabuf umem which is pinned and does not require move_notify callback implementation. The returned umem is pinned and DMA mapped like standard cpu umems, and is released through ib_umem_release() (incl. unpinning and unmapping). Link: https://lore.kernel.org/r/20211012120903.96933-3-galpress@amazon.com Signed-off-by: Gal Pressman <galpress@amazon.com> (cherry picked from commit 1e4df4a) Signed-off-by: Tushar Dave <tdave@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Implement a dmabuf importer for the EFA driver. As ODP is not supported, the pinned dmabuf are used to prevent the move_notify callback from being called. Link: https://lore.kernel.org/r/20211012120903.96933-4-galpress@amazon.com Signed-off-by: Gal Pressman <galpress@amazon.com> (cherry picked from commit 66f4817) Signed-off-by: Tushar Dave <tdave@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
This uses the same passing protocol as UVERBS_ATTR_FD (eg len = 0 data_s64 = fd), except that the FD is not required to be a uverbs object and the core code does not covert the FD to an object handle automatically. Access to the int fd is provided by uverbs_get_raw_fd(). Link: https://lore.kernel.org/r/2-v1-bd147097458e+ede-umem_dmabuf_jgg@nvidia.com (cherry picked from commit 015bda8) Signed-off-by: Tushar Dave <tdave@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
This is modeled after the similar EFA enablement in commit 66f4817 ("RDMA/efa: Add support for dmabuf memory regions"). Like EFA there is no support for revocation so we simply call the ib_umem_dmabuf_get_pinned() to obtain a umem instead of the normal ib_umem_get(). Everything else stays the same. Link: https://lore.kernel.org/r/3-v1-bd147097458e+ede-umem_dmabuf_jgg@nvidia.com (backported from commit 9af859c) [tdave: conflict fixup plus use ib_umem_get_peer() in drivers/infiniband/hw/mlx5/devx.c] Signed-off-by: Tushar Dave <tdave@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
For mlx5 if ATS is enabled in the PCI config then the device will use ATS requests for only certain DMA operations. This has to be opted in by the SW side based on the mkey or umem settings. ATS slows down the PCI performance, so it should only be set in cases when it is needed. All of these cases revolve around optimizing PCI P2P transfers and avoiding bad cases where the bus just doesn't work. Link: https://lore.kernel.org/r/4-v1-bd147097458e+ede-umem_dmabuf_jgg@nvidia.com (backported from commit 72b2f76) [tdave: conflict fixup in 2 files drivers/infiniband/hw/mlx5/devx and mr.c] Signed-off-by: Tushar Dave <tdave@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DAMBUF is backported using the two series of patches:
First, we need the series of 3 patches at https://lore.kernel.org/all/20211012120903.96933-1-galpress@amazon.com/
Second, we need series of 4 patches at https://lore.kernel.org/all/0-v1-bd147097458e+ede-umem_dmabuf_jgg@nvidia.com/
Note: The changes made by first patch in series "net/mlx5: Add IFC bits for mkey ATS" had already been there in Ubuntu jammy master-next.