Skip to content

Commit

Permalink
Merge pull request #310 from mregmi/documentation
Browse files Browse the repository at this point in the history
Documentation: add DSA device plugin documentation
  • Loading branch information
uMartinXu authored Sep 30, 2024
2 parents 95ebfdc + 5d4ba44 commit bae8997
Show file tree
Hide file tree
Showing 4 changed files with 76 additions and 1 deletion.
2 changes: 2 additions & 0 deletions device_plugins/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,11 @@ The resources are the user interface for customers to claim and consume the hard
| Intel® SGX | `sgx.intel.com/epc` | Intel SGX EPC memory for user pod to claim | [Link](https://github.com/intel/intel-technology-enabling-for-openshift/blob/64a6c86f3be25459c14ea988e892f9f5d873a8ca/tests/l2/sgx/sgx_job.yaml#L21) |
| Intel® Data Center GPU Flex Series </br> Intel® Data Center GPU Max Series | `gpu.intel.com/i915 ` | Intel Data Center GPU Card for user pod to claim | [Link](https://github.com/intel/intel-technology-enabling-for-openshift/blob/main/device_plugins/deploy_gpu.md#using-intel-data-center-gpu-resource-exclusively) |
| Intel® QAT | `qat.intel.com/cy` </br> `qat.intel.com/dc` | `cy`: Intel QAT VFIO Virtual Function device configured for cryptography for user pod to claim </br> `dc`: Intel QAT VFIO Virtual Function device configured for cryptography for user pod to claim | [Link](https://github.com/intel/intel-technology-enabling-for-openshift/blob/main/tests/l2/qat/qatlib_job.yaml#L24) </br> [Link](https://github.com/intel/intel-technology-enabling-for-openshift/blob/main/tests/l2/qat/qatlib_job.yaml#L28) |
| Intel® DSA | `dsa.intel.com/wq-user-shared` </br> `dsa.intel.com/wq-user-dedicated` | Intel DSA Work Queue for user pod to claim | [Link](https://github.com/intel/intel-technology-enabling-for-openshift/blob/main/tests/l2/dsa/dsa_job.yaml#L27) |


## Creating Intel Device Plugin custom resource (CR)
- To create an Intel SGX device plugin CR, follow this [link](/device_plugins/deploy_sgx.md).
- To create an Intel GPU device plugin CR, follow this [link](/device_plugins/deploy_gpu.md).
- To create an Intel QAT device plugin CR, follow this [link](/device_plugins/deploy_qat.md).
- To create an Intel DSA device plugin CR, follow this [link](/device_plugins/deploy_dsa.md).
56 changes: 56 additions & 0 deletions device_plugins/deploy_dsa.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Create Intel DSA Device Plugin CR

## Create a CR via web console
1. Go to **Operator** -> **Installed Operators**.
2. Open **Intel Device Plugins Operator**.
3. Navigate to tab **Intel DSA Device Plugin**.
4. Click **Create DSADevicePlugin** -> set correct parameters -> Click **Create**
5. Optional: If you want to make any customizations, select YAML view and edit the details. When you are done, click **Create**.

## Verify via web console
1. Verify CR by checking the status of **Workloads** -> **DaemonSet** -> **intel-dsa-plugin**.
2. Now `DsaDevicePlugin` is created.

## Create CR via CLI
Apply the CR yaml file:
```
$ oc apply -f https://raw.githubusercontent.com/intel/intel-technology-enabling-for-openshift/main/device_plugins/dsa_device_plugin.yaml
```

## Verify via CLI
Verify that the device plugin CR is ready:
```
$ oc get DsaDevicePlugin
```
Output:
```
NAME DESIRED READY NODE SELECTOR AGE
dsadeviceplugin-sample 3 3 {"intel.feature.node.kubernetes.io/dsa":"true"} 98m
```

# Verify DSA Device Plugin
After the plugin is deployed, use below command to verify DSA resources:
```
$ oc describe node srf-2 | grep dsa.intel.com
dsa.intel.com/wq-user-dedicated: 0
dsa.intel.com/wq-user-shared: 160
dsa.intel.com/wq-user-dedicated: 0
dsa.intel.com/wq-user-shared: 160
```

## DSA Resource Configuration
By default the DSA plugin uses [this configuration file](https://github.com/intel/intel-device-plugins-for-kubernetes/blob/main/demo/dsa.conf).
The dsa init container comes with a utility called `accel-config` which takes file as input and configures the DSA hardwares based on that.
The default configuration has creates dedicated WQs for each DSA device so that it's four groups per device where each groups is with 1 WQ linked to 1 engine.
Users can customise the config and can use the pre-customised config for their specific use case from [here](https://github.com/intel/idxd-config/tree/stable/contrib/configs)
There's also a possibility for a node specific configuration by passing a node specific profile via configMap volume mount.
Users can use the steps below to customize the DSA resource configuration:
1. Create the configmap for DSA resource configuration
```
$ oc create configmap --namespace=openshift-operators intel-dsa-config --from-file=dsa[-$NODE_NAME].conf
2. Create DSA device plugin CR with -provisioning-config set as the name of the ConfigMap (created in step 1) in the dsa_device_plugin.yaml file or set ConfigMap name in the provisioning-config option from web console.
# Run Intel DSA based workloads on RHOCP
To run the Intel DSA based workloads as an unprivileged pod, you need to use a customised SCC. The customized `dsa-scc` Security Context Constraint (SCC) is provided to bind with service account and run the DSA based workload.
See [Verify Intel DSA Provisioning](/tests/l2/dsa/README.md) for the detailed steps.
14 changes: 14 additions & 0 deletions device_plugins/dsa_device_plugin.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Copyright (c) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

kind: DsaDevicePlugin
apiVersion: deviceplugin.intel.com/v1
metadata:
name: dsadeviceplugin-sample
spec:
image: 'quay.io/ocpeng/intel-dsa-plugin:0.29.0'
initImage: 'quay.io/ocpeng/intel-idxd-config-initcontainer:0.29.0'
logLevel: 4
nodeSelector:
intel.feature.node.kubernetes.io/dsa: 'true'
sharedDevNum: 10
5 changes: 4 additions & 1 deletion docs/supported_platforms.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ The following Intel feature technologies are supported in the 1.2.1 release.
| Intel® Software Guard Extensions (Intel® SGX) | [5th Gen Intel® Xeon® Scalable processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon/5th-gen-xeon-scalable-processors.html) <br/> [4th Gen Intel® Xeon® Scalable processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon-accelerated/4th-gen-xeon-scalable-processors.html) <br/> [3rd Gen Intel® Xeon® Scalable processors ](https://www.intel.com/content/www/us/en/products/docs/processors/xeon-accelerated/3rd-gen-xeon-scalable-processors.html) |
| Intel® Data Center GPU Flex Series | [Intel Data Center GPU Flex 140](https://www.intel.com/content/www/us/en/products/sku/230020/intel-data-center-gpu-flex-140/specifications.html) <br/> [Intel Data Center GPU Flex 170](https://www.intel.com/content/www/us/en/products/sku/230019/intel-data-center-gpu-flex-170/specifications.html) |
| Intel® Data Center GPU Max Series | [Intel Data Center GPU Max 1100](https://www.intel.com/content/www/us/en/products/sku/232876/intel-data-center-gpu-max-1100/specifications.html) <br/> [Intel Data Center GPU Max 1550](https://www.intel.com/content/www/us/en/products/sku/232873/intel-data-center-gpu-max-1550/specifications.html) |
| Intel® QuickAssist Technology (Intel® QAT) | [5th Gen Intel® Xeon® Scalable processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon/5th-gen-xeon-scalable-processors.html) <br/> [4th Gen Intel® Xeon® Scalable processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon-accelerated/4th-gen-xeon-scalable-processors.html) |
| Intel® QuickAssist Technology (Intel® QAT) | [6th Gen Intel® Xeon® Scalable processors](https://www.intel.com/content/www/us/en/products/details/processors/xeon/xeon6-product-brief.html) <br/> [5th Gen Intel® Xeon® Scalable processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon/5th-gen-xeon-scalable-processors.html) <br/> [4th Gen Intel® Xeon® Scalable processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon-accelerated/4th-gen-xeon-scalable-processors.html) |
| Intel® Data Streaming Accelerator (Intel® DSA) | [6th Gen Intel® Xeon® Scalable processors](https://www.intel.com/content/www/us/en/products/details/processors/xeon/xeon6-product-brief.html) <br/> [5th Gen Intel® Xeon® Scalable processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon/5th-gen-xeon-scalable-processors.html) <br/> [4th Gen Intel® Xeon® Scalable processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon-accelerated/4th-gen-xeon-scalable-processors.html) |

## Setting up Intel Hardware Features

Expand All @@ -41,6 +42,7 @@ Note: Please refer to your BIOS vendor for specific instructions. This is only a
| Intel Data Center GPU Flex Series | [Link](https://www.intel.com/content/www/us/en/content-details/774119/virtualization-guide-for-intel-data-center-gpu-flex-series.html?wapkw=gpu%20flex%20series%20setup%20guide) |
| Intel Data Center GPU Max Series | [Link](https://dgpu-docs.intel.com/system-user-guides/DNP-Max-1100-userguide/DNP-Max-1100-userguide.html#bios-setup) |
| Intel QAT | [Link](https://github.com/intel/qatlib/blob/7429ee2b7c837137ed11959a3c2cc3729dc15739/INSTALL#L104) |
| Intel DSA | [Link](https://cdrdv2-public.intel.com/759709/353216-data-streaming-accelerator-user-guide-003.pdf) |

## Supported RHOCP Versions

Expand All @@ -50,6 +52,7 @@ Note: Please refer to your BIOS vendor for specific instructions. This is only a
| Intel Data Center GPU Flex Series | RHOCP 4.14.11 and above (support is only for z streams in 4.14.z where z >= 11) |
| Intel Data Center GPU Max Series | RHOCP 4.14.11 and above (support is only for z streams in 4.14.z where z >= 11) |
| Intel QAT | RHOCP 4.14.11 and above (support is only for z streams in 4.14.z where z >= 11) |
| Intel DSA | RHOCP 4.16.3 and above (support is only for z streams in 4.16.z where z >= 3) |

## Support

Expand Down

0 comments on commit bae8997

Please sign in to comment.