-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #33 from OneIdentity/wip/ddavid
sps: Update connector documentation and name
- Loading branch information
Showing
12 changed files
with
279 additions
and
81 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| ## Development guide | ||
|
|
||
| ### Set up the development environment | ||
|
|
||
| To develop a Power BI custom connector the following code editors can be used: | ||
|
|
||
| - Visual Studio | ||
| - Visual Studio Code | ||
|
|
||
| Both require a dedicated [Power Query SDK] extension to be able to create and build data connector projects. However, Microsoft suggests to use Visual Studio Code as the Visual Studio Code Power Query SDK will eventually become the default and maintained SDK. | ||
|
|
||
| The following steps are required to set up a Visual Studio Code development environment: | ||
|
|
||
| - Install [Visual Studio Code] | ||
| - Install [Visual Studio Code Power Query SDK] (This will also install the Power Query / M Language extension.) | ||
|
|
||
| To make development easier you can use `make` commands for building the connector and running the tests. In order to install `make` on Windows, do the following: | ||
|
|
||
| - Install [Chocolatey] | ||
| - Install make via Chocolatey by running the following command: `choco install make` | ||
|
|
||
| ### Build the connector from source code | ||
|
|
||
| 1. Obtain the source from the GitHub repository | ||
| 2. Install [MSBuild] for Windows | ||
| 3. Ensure MSBuild is included in your `PATH` variable. | ||
| 4. Build the source code with one of the options: | ||
| - In Visual Studio Code: Go to **Terminal > Run Build Task...** and select **Build connenctor project using MSBuild** | ||
| - From the command line: use the `make build` command | ||
|
|
||
| ### Testing | ||
|
|
||
| Running tests is only available from the command line as the test framework has been decoupled from the connector code. The tests and the unit test framework can be found under the test subfolder. This is a separate Power Query SDK project, so it is also necessary to be built. The `make build` command also builds the test project. | ||
|
|
||
| You can run the tests with `PQTest.exe` which comes with the Power Query SDK. By default, it is located in the SDK's extension folder, for example: | ||
|
|
||
| ``` | ||
| %USERPROFILE%\.vscode\extensions\powerquery.vscode-powerquery-sdk-0.2.2-win32-x64\.nuget\Microsoft.PowerQuery.SdkTools.2.114.4\tools\ | ||
| ``` | ||
|
|
||
| To run tests using **make** targets, you need to include the path of the `PQTest.exe` in your `PATH` variable. You have the following options: | ||
|
|
||
| - Run all tests using `make unit-tests` | ||
| - Run an individual test using `make unit-test test_file=<Path-To-The-Query-Pq-File>` | ||
|
|
||
| ### Coding style | ||
|
|
||
| To adhere with our coding style, consider the following when **writing code**: | ||
|
|
||
| - Folders should be named using **PascalCase** | ||
| - Functions should be named using **PascalCase** | ||
| - Variables should be named using **camelCase** | ||
|
|
||
| When writing **commit messages**, consider the following: | ||
|
|
||
| - Commit message header should be at most 50 characters long | ||
| - Start the commit message header by defining which Safeguard product does the modification addresses | ||
| - Insert and empty line between commit message header and body | ||
| - Define the scope of the modification | ||
| - It should highlight the relative path to the file/folder in the repository that contains the most essential modification of the commit | ||
| - Do not forget to describe the why-s in the commit message body | ||
| - If the modification corresponds to an issue in the issue tracking system, include its ID at the end of the commit message | ||
| - If it only has a work item in Azure DevOps: `References: azure #<issue-id>` | ||
| - If it only has a GitHub issue: `References: #<issue-id>` | ||
| - If it both has a work item in Azure DevOps and a GitHub issue: `References: azure #<azure-id>, #<github-id>` | ||
| - For examples, view the [Commit history of the main branch] | ||
|
|
||
| ## Contribution | ||
|
|
||
| 1. Fork the [GitHub project] | ||
| 2. Create a branch in git | ||
| 3. Commit your changes | ||
| 4. Push to the branch | ||
| 5. Open a [Pull Request] | ||
|
|
||
| Make sure you respect the [Coding style](#coding-style) and know the essentials of [Creating good commits]. | ||
|
|
||
| ## Useful documents | ||
|
|
||
| - [Power Query SDK] | ||
| - [Visual Studio Code Power Query SDK GitHub] | ||
| - [Power Query M formula lanaguage] | ||
| - [In-depth tutorial on Power Query M] | ||
| - [Creating good commits] | ||
|
|
||
| <!-- Links --> | ||
|
|
||
| [GitHub project]: https://github.com/OneIdentity/SafeguardPowerBI | ||
| [Commit history of the main branch]: https://github.com/OneIdentity/SafeguardPowerBI/commits/main | ||
| [Pull Request]: https://github.com/OneIdentity/SafeguardPowerBI/pulls | ||
|
|
||
| [Power Query SDK]: https://learn.microsoft.com/en-us/power-query/installingsdk | ||
| [Visual Studio Code Power Query SDK GitHub]: https://github.com/microsoft/vscode-powerquery-sdk/ | ||
| [Power Query M formula lanaguage]: https://learn.microsoft.com/en-us/powerquery-m/ | ||
| [In-depth tutorial on Power Query M]: https://bengribaudo.com/blog/2017/11/17/4107 | ||
|
|
||
| [Visual Studio Code]: https://code.visualstudio.com/ | ||
| [Visual Studio Code Power Query SDK]: https://marketplace.visualstudio.com/items?itemName=PowerQuery.vscode-powerquery-sdk | ||
|
|
||
| [Chocolatey]: https://chocolatey.org/install | ||
| [MSBuild]: https://github.com/microsoft/vscode-powerquery-sdk/issues/192#issuecomment-1311882460 | ||
|
|
||
| [Creating good commits]: https://google.github.io/eng-practices/review/developer/ | ||
|
|
||
| <!-- Links END --> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,78 +1,176 @@ | ||
| # Safeguard for Previleged Sessions Power BI Connector developer documentation | ||
| # One Identity Safeguard Power BI Connector | ||
|
|
||
| This document describes the steps to be aware of to use and develop the Safeguard for Privileged Session Power BI connector code. | ||
| ## Table of Contents | ||
|
|
||
| The current functionality of the connector is as follows after the user has entered an SPS Url and logged in with baisc athentication: | ||
| - [About the project](#about-the-project) | ||
| - [Installation and usage](#installation-and-usage) | ||
| - [Common errors](#common-errors) | ||
| - [Troubleshooting](#troubleshooting) | ||
| - [Release policy](#release-policy) | ||
| - [Contribution](#contribution) | ||
| - [License](#license) | ||
| - [References](#references) | ||
|
|
||
| - Displays the details of the REST API info endpoint in a table. | ||
| ## About the project | ||
|
|
||
| ## Development environment | ||
| The aim of One Identity Safeguard Power BI Connector (Connector) is to provide a solution for customers to visualize their audit data captured by [One Identity Safeguard for Priviledged Sessions] (SPS) in a highly configurable way compared to the on-box reporting system of SPS. The Connector uses basic authentication to connect to SPS and imports sessions metadata that matches the criteria given by the Connector's input parameters. After successful data retrieval, the Connector returns with two tables: | ||
|
|
||
| Two supported development environments are currently available for Power BI custom connector creation. These are: | ||
| - **Info**: contains information about the data fetch process for debugging purposes | ||
| - **Sessions**: contains the actual sessions metadata | ||
|
|
||
| - **Visual Studio 2017 and 2019** | ||
| - **Visual Studio Code** | ||
| The project also a includes a Power BI report template to quickstart report creation and visualize your audit data. | ||
|
|
||
| In themselves, development environments do not support connector development. Both require the **Power Query SDK** extension. It was released for Visual Studio in 2017, and for Visual Studio Code (later VS Code) in 2022, currently in preview. Microsoft encourages all developers to install and use the newly released Visual Studio Code Power Query SDK (Preview) as this version will eventually be the default SDK going forward. Therefore, this was used to develop the current Safeguard for Previleged Session Power BI connector. | ||
| ## Installation and usage | ||
|
|
||
| The following steps are required to set up a VS Code development environment: | ||
| To use the connector, follow these steps: | ||
|
|
||
| - [Install Visual Studio Code] | ||
| - After starting Visual Studio Code, you can install the Power Query SDK via the Extensions tab. (This will also install the Power Query / M Language extension.) (At the bottom of the Explorer tab, you will see a Power Query SDK drop-down menu.) | ||
| 1. Get a connector MEZ file compatible with your SPS version from the [Releases] page or build it from source code following the steps described in the [Contribution] guide. | ||
| 2. Copy the file to `%USERPROFILE%\Documents\Power BI Desktop\Custom Connectors` folder. | ||
| 3. Login to your SPS appliance and go to **Basic Settings > Management**. | ||
| 4. Open the SSL tab and click on the link next to the "CA X.509 certificate" text. | ||
| 5. Download the DER version. (This is recognised by default by Windows.) | ||
| 6. Import the downloaded certificate to your local computer's certificate store as trusted root certification authority. For help, visit: [Install imported certificates]. | ||
| 7. In Power BI Desktop, go to **File > Options and settings > Options > Security > Data Extensions** and select the **(Not Recommended) Allow any extension to load without validation or warning** option since the Connector is not signed. | ||
| 8. Restart Power BI Desktop. | ||
| 9. In Power BI Desktop, click **Home > Get data** and search for **One Identity Safeguard**, then click **Connect**. | ||
| 10. Enter the input parameters for the Connector and click **OK**. | ||
| 11. Enter the credentials that will be used to authenticate to the SPS appliance and click **Connect**. | ||
| 12. Select both the **Info** and **Sessions** tables then click **Load**. | ||
| 13. Once data is imported into Power BI, you can start creating your reports by either creating it from scratch or using the One Idenity Safeguard Power BI Report Template. | ||
|
|
||
| ## Building | ||
| You can read a more comprehensive guide on using the Connector and the Report Template in the One Idenity Safeguard Power BI Tutorial that can be found on the [Technical documents for One Identity Safeguard for Priviledged Sessions] page. | ||
|
|
||
| The connector can be accessed locally by running the following git command. (It is necessary to have [git installed].) | ||
| ## Common errors | ||
|
|
||
| I cannot find One Identity Safeguard under **Home > Get data**. | ||
|
|
||
| A) You might have forgotten to enable loading non-certified connectors. Make sure to enable this option described in step 7 in the [Installation and usage](#installation-and-usage) section. | ||
|
|
||
| B) You have built your connector from source code but it contains error(s). | ||
|
|
||
| --- | ||
|
|
||
| I get the following error when trying to connect to SPS: | ||
|
|
||
| ```sh | ||
| git clone https://github.com/OneIdentity/SafeguardPowerBI.git | ||
| ``` | ||
| Unable to connect | ||
| We encountered an error while trying to connect. Details: "The underlying | ||
| connection was closed: Could not establish trust relationship for the SSL/TLS | ||
| secure channel." | ||
| ``` | ||
|
|
||
| You might have forgotten to import the CA X509 certificate of your SPS appliance described in steps 2-6 in the [Installation and usage](#installation-and-usage) section. | ||
|
|
||
| --- | ||
|
|
||
| I see "Error" in the Status column of the Info table and the Message column contains the following: | ||
|
|
||
| **A) SPS returned response with missing fields** | ||
|
|
||
| A response returned by SPS does not contain a field that Connector needs for processing data. | ||
| Reproduce the error with trace logging enabled and create a technical case described in the [Troubleshooting](#troubleshooting) section. | ||
| Attach the mashup trace logs to the issue. | ||
|
|
||
| **B) SPS interpreted a malformed request** | ||
|
|
||
| Your filter might be invalid. Make sure you input your filter parameters correctly. For more information on the available search fields, see **List of available search queries** in the **One Identity Safeguard for Privileged Sessions Administration Guide** on the [Technical documents for One Identity Safeguard for Priviledged Sessions] page. | ||
|
|
||
| **C) The username or password is invalid** | ||
|
|
||
| Make sure to enter your username and password you use to access SPS for viewing audit data. | ||
|
|
||
| **D) You are not authorized to access the given resource** | ||
|
|
||
| In order to access audit data, the user you use to access SPS needs to have search access rights. | ||
|
|
||
| **E) The requested resource cannot be found** | ||
|
|
||
| In order to fetch audit data from SPS, the Connector relies on advanced search that uses database snapshots to ensure data consistency throughout a data fetch. If for some reason the database snapshot disappears, this error can occur. Try to initiate the data fetch process again. | ||
|
|
||
| **F) Snapshot quota exceeded** | ||
|
|
||
| The connector is located in the SafeguardSession folder, which should be opened in VS Code as well. | ||
| This happens if multiple users want to use advanced search relying on database snapshots at the same time. In this case, try initiating the data fetch at least 5 minutes later so that an existing snapshot may expire in the system and a new one can be opened. | ||
|
|
||
| The connector code can be built with two different tools: | ||
| **G) SPS responded with server error** | ||
|
|
||
| - **MakePQX:** This is the default. A .mez file will be created during the build using all of the files at the top level of the project directory, with a few exceptions (these are not user configurable). You may want to specify the directory of MakePQX.exe as a system path variable, so you can easily build from the terminal. | ||
| - **MSBuild:** This is not available by default. It must be [installed] separately, and the folder added to the system variable path where msbuild.exe is located. Here you also have the option to manually specify which files you want to include in the .mez file. This requires modifying the .proj file. (The .proj file is not used by makepqx.) | ||
| This error indicates problems with the SPS appliance. Try the suggestions written in the [Troubleshooting](#troubleshooting) section. | ||
|
|
||
| Building is available in VS Code under **Terminal > Run Build Task...**, where you can choose from the build options listed above if you have msbuild on your machine. | ||
| ## Troubleshooting | ||
|
|
||
| ## Testing | ||
| If an error occurs during data fetch, you can check the Info table to see a descriptive message about the error type. In this case, the Sessions table will contain the exact error record that you can inspect if you go to **Home > Transform data** and select the Sessions table. | ||
|
|
||
| Currently, running tests is only available from the command line, as the test framework has been decoupled from the connector code. The tests and the unit test framework can be found under the test subfolder. This is actually a separate Power Query SDK project. So it is also necessary to build here. | ||
| If you cannot resolve the issue, you can collect the Power BI mashup trace logs as described in the [Collecting Power BI mashup trace logs] documentation and create a [One Identity Technical Case]. Attach the collected trace logs for debugging purposes. | ||
|
|
||
| Testing is done by running PQTest.exe. You may also want to include the exe file directory in the system path variables. PQTest.exe needs an extension (.mez file) and a .query.pq file (which contains the tests). Because the test framework, which is based on Microsoft's Fact unit test framework, and its connector functionality are in two different places, both extensions must be given to PQTest, which is not possible from the VS Code interface. | ||
| ## Release policy | ||
|
|
||
| The following command is an example of a unit test file run: | ||
| As of now, One Identity releases a new connector with each feature release of SPS until 8 LTS. | ||
|
|
||
| ```sh | ||
| PQTest.exe --extension ".\bin\SafeguardSessions.mez" --extension ".\test\bin\UnitTestFramework.mez" run-test -q ".\test\TestBasicAuthentication.query.pq" --prettyPrint | ||
| Releases have the following rules: | ||
|
|
||
| - Release versioning follows the `v<connector-version>+sps-<sps-version>` | ||
| - Connector versioning follows the `v<major.minor.patch(pre)>` convention | ||
| - `major`: new version includes breaking change compared to previous version | ||
| - `minor`: new version includes new functionality without breaking change compared to previous version | ||
| - `patch`: new version includes small, non-breaking change compared to previous | ||
| - `pre`: the new version is a pre-release version | ||
| - Release title should be the same as the tag | ||
| - Examples | ||
| - v1.0.0+sps-7.3.0 | ||
| - v1.0.0pre1+sps7.3.0 | ||
| - v1.0.0+sps-8.0.3 | ||
| - A Report Template of a release is not only compatible with the connector from the same release but also with other connectors having the same major version | ||
|
|
||
| When writing a release note, use the following template: | ||
|
|
||
| ``` | ||
| # Compatible SPS versions | ||
| ... | ||
| # Breaking changes | ||
| ... | ||
| # New features | ||
| ... | ||
| # Bug fixes | ||
| ... | ||
| # Other changes | ||
| ... | ||
| ``` | ||
|
|
||
| ## Installation and Deployment | ||
|
|
||
| In order to use the newly built connector in Power BI, the following steps are required: | ||
|
|
||
| 1. Login to your SPS. | ||
| 2. Go to Basic Settings > Management. | ||
| 3. Open the SSL tab and clcik on the link next to the "CA X.509 certificate" text. | ||
| 4. Download the DER version. (This is recognised by default by Windows.) | ||
| 5. [Install the downloaded certificate]. | ||
| 6. Create a Custom Connectors folder under Documents > Power BI Desktop. | ||
| 7. Copy the connector's .mez file here. (Not the test's!) | ||
| 8. In Power BI Desktop, select the (Not Recommended) Allow any extension to load without validation or warning option under File > Options and settings > Options > Security > Data Extensions. | ||
| 9. Restart Power BI Desktop. | ||
|
|
||
| ## Useful documentations | ||
|
|
||
| - [Power Query SDK] | ||
| - [VS Code Power Query SDK Git Hub page] | ||
| - [Power Query M formula lanaguage] | ||
|
|
||
| [Install Visual Studio Code]: <https://code.visualstudio.com/> | ||
| [git installed]: <https://git-scm.com/downloads> | ||
| [installed]: <https://github.com/microsoft/vscode-powerquery-sdk/issues/192#issuecomment-1311882460> | ||
| [Install the downloaded certificate]: <https://learn.microsoft.com/en-us/troubleshoot/windows-server/windows-security/install-imported-certificates> | ||
| [Power Query SDK]: <https://learn.microsoft.com/en-us/power-query/installingsdk> | ||
| [VS Code Power Query SDK Git Hub page]: <https://github.com/microsoft/vscode-powerquery-sdk/> | ||
| [Power Query M formula lanaguage]: <https://learn.microsoft.com/en-us/powerquery-m/> | ||
| If a section does not have related content, it can be left out. | ||
|
|
||
| Modifications to the Report Template must be listed in the appropriate sections, since it is part of the release. | ||
|
|
||
| ## Contribution | ||
|
|
||
| For guidance on contribution and development quickstart, see [Contribution]. | ||
|
|
||
| ## License | ||
|
|
||
| Distributed under the One Identity - Open Source License. See [License] for more information. | ||
|
|
||
| ## References | ||
|
|
||
| - [One Identity Safeguard for Priviledged Sessions] | ||
| - [Technical documents for One Identity Safeguard for Priviledged Sessions] | ||
| - Create a [One Identity Technical Case] | ||
| - [Releases] of the Power BI Connector | ||
| - [License] of the Power BI Connector | ||
| - [Contribution] to the Power BI Connector | ||
|
|
||
| <!-- Links --> | ||
|
|
||
| [One Identity Safeguard for Priviledged Sessions]: https://www.oneidentity.com/products/one-identity-safeguard-for-privileged-sessions/ | ||
| [Technical documents for One Identity Safeguard for Priviledged Sessions]: https://support.oneidentity.com/one-identity-safeguard-for-privileged-sessions/technical-documents | ||
| [One Identity Technical Case]: https://support.oneidentity.com/create-service-request | ||
|
|
||
| [Releases]: https://github.com/OneIdentity/SafeguardPowerBI/releases | ||
| [License]: https://github.com/OneIdentity/SafeguardPowerBI/blob/main/LICENSE | ||
| [Contribution]: CONTRIBUTION.md | ||
|
|
||
| [Install imported certificates]: https://learn.microsoft.com/en-us/troubleshoot/windows-server/windows-security/install-imported-certificates | ||
| [Collecting Power BI mashup trace logs]: https://learn.microsoft.com/en-us/power-bi/fundamentals/desktop-diagnostics#collecting-mashup-traces | ||
|
|
||
| <!-- Links END --> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.