From 8e8c034d02f8bea7c9ec32eb4dcd7512da4a2052 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Tue, 25 Oct 2022 12:45:12 +0100 Subject: [PATCH 01/52] add workflows for auto-assigning projects --- .github/workflows/label-issue-project.yml | 25 +++++++++++++++++++++++ .github/workflows/open-issue-project.yml | 18 ++++++++++++++++ README.md | 2 +- docs/roadmap.md | 2 +- 4 files changed, 45 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/label-issue-project.yml create mode 100644 .github/workflows/open-issue-project.yml diff --git a/.github/workflows/label-issue-project.yml b/.github/workflows/label-issue-project.yml new file mode 100644 index 000000000..9cf32d521 --- /dev/null +++ b/.github/workflows/label-issue-project.yml @@ -0,0 +1,25 @@ +name: Add labeled issues to project + +on: + issues: + types: + - labeled + +jobs: + add-to-project: + name: Add issue to project + runs-on: ubuntu-latest + steps: + - uses: actions/add-to-project@RELEASE_VERSION + with: + project-url: https://github.com/users/Badgerati/projects/2 + github-token: ${{ secrets.PROJECT_TOKEN }} + labeled: 'planned :calendar:, roadmap :rocket:, backlog :scroll:' + label-operator: OR + + - uses: actions/add-to-project@RELEASE_VERSION + with: + project-url: https://github.com/users/Badgerati/projects/4 + github-token: ${{ secrets.PROJECT_TOKEN }} + labeled: 'design :art:, draft :pencil2:, idea :bulb:' + label-operator: OR \ No newline at end of file diff --git a/.github/workflows/open-issue-project.yml b/.github/workflows/open-issue-project.yml new file mode 100644 index 000000000..860683b69 --- /dev/null +++ b/.github/workflows/open-issue-project.yml @@ -0,0 +1,18 @@ +name: Add opened issues to project + +on: + issues: + types: + - opened + +jobs: + add-to-project: + name: Add issue to project + runs-on: ubuntu-latest + steps: + - uses: actions/add-to-project@RELEASE_VERSION + with: + project-url: https://github.com/users/Badgerati/projects/2 + github-token: ${{ secrets.PROJECT_TOKEN }} + labeled: 'bug :bug:, documentation :book:, packaging :package:' + label-operator: OR \ No newline at end of file diff --git a/README.md b/README.md index 04e8fdbfe..1b8b69596 100644 --- a/README.md +++ b/README.md @@ -110,4 +110,4 @@ To work on issues you can fork Pode, and then open a Pull Request for approval. You can find a list of the features, enhancements and ideas that will hopefully one day make it into Pode [here in the documentation](https://badgerati.github.io/Pode/roadmap/). -There is also a [Project](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues. Plus, there is a [Draft Board](https://github.com/users/Badgerati/projects/2/views/3) which contains a range of ideas for Pode features/enhancements which are either brilliant, ludicrous, or down right insane! Draft Issues are purely ideas, and any in the design stage might one day make it in! If you see a Draft Issue you which to discuss, or have an idea for one, please dicuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. +There is also a [Project](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues. Plus, there is a [Draft Board](https://github.com/users/Badgerati/projects/4) which contains a range of ideas for Pode features/enhancements which are either brilliant, ludicrous, or down right insane! Draft Issues are purely ideas, and any in the design stage might one day make it in! If you see a Draft Issue you which to discuss, or have an idea for one, please dicuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. diff --git a/docs/roadmap.md b/docs/roadmap.md index 4beecaf38..e33bf4e69 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -4,7 +4,7 @@ This page lists the planned features and enhancements that will, hopefully, one Where possible items listed here will have a link to any relevant issues in GitHub. -There is also a [Project](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues. Plus, there is a [Draft Board](https://github.com/users/Badgerati/projects/2/views/3) which contains a range of ideas for Pode features/enhancements which are either brilliant, ludicrous, or down right insane! Draft Issues are purely ideas, and any in the design stage might one day make it in! If you see a Draft Issue you which to discuss, or have an idea for one, please dicuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. +There is also a [Project](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues. Plus, there is a [Draft Board](https://github.com/users/Badgerati/projects/4) which contains a range of ideas for Pode features/enhancements which are either brilliant, ludicrous, or down right insane! Draft Issues are purely ideas, and any in the design stage might one day make it in! If you see a Draft Issue you which to discuss, or have an idea for one, please dicuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. ## 🎯 Goal From 2de81030f6e628774c05b3d73e15f614c42e87a9 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Tue, 25 Oct 2022 19:25:25 +0100 Subject: [PATCH 02/52] helps if you add the action version! --- .github/workflows/label-issue-project.yml | 4 ++-- .github/workflows/open-issue-project.yml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/label-issue-project.yml b/.github/workflows/label-issue-project.yml index 9cf32d521..ced6ccf81 100644 --- a/.github/workflows/label-issue-project.yml +++ b/.github/workflows/label-issue-project.yml @@ -10,14 +10,14 @@ jobs: name: Add issue to project runs-on: ubuntu-latest steps: - - uses: actions/add-to-project@RELEASE_VERSION + - uses: actions/add-to-project@v0.3.0 with: project-url: https://github.com/users/Badgerati/projects/2 github-token: ${{ secrets.PROJECT_TOKEN }} labeled: 'planned :calendar:, roadmap :rocket:, backlog :scroll:' label-operator: OR - - uses: actions/add-to-project@RELEASE_VERSION + - uses: actions/add-to-project@v0.3.0 with: project-url: https://github.com/users/Badgerati/projects/4 github-token: ${{ secrets.PROJECT_TOKEN }} diff --git a/.github/workflows/open-issue-project.yml b/.github/workflows/open-issue-project.yml index 860683b69..7cbf8296d 100644 --- a/.github/workflows/open-issue-project.yml +++ b/.github/workflows/open-issue-project.yml @@ -10,7 +10,7 @@ jobs: name: Add issue to project runs-on: ubuntu-latest steps: - - uses: actions/add-to-project@RELEASE_VERSION + - uses: actions/add-to-project@v0.3.0 with: project-url: https://github.com/users/Badgerati/projects/2 github-token: ${{ secrets.PROJECT_TOKEN }} From 45184b2c4fd4459c163d0b93c6dad9c5141c42b5 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Tue, 29 Nov 2022 21:44:17 +0000 Subject: [PATCH 03/52] condense project boards into 1 board --- .github/workflows/label-issue-project.yml | 9 +-------- .github/workflows/open-issue-project.yml | 2 +- README.md | 2 +- docs/roadmap.md | 2 +- 4 files changed, 4 insertions(+), 11 deletions(-) diff --git a/.github/workflows/label-issue-project.yml b/.github/workflows/label-issue-project.yml index ced6ccf81..eb554620c 100644 --- a/.github/workflows/label-issue-project.yml +++ b/.github/workflows/label-issue-project.yml @@ -14,12 +14,5 @@ jobs: with: project-url: https://github.com/users/Badgerati/projects/2 github-token: ${{ secrets.PROJECT_TOKEN }} - labeled: 'planned :calendar:, roadmap :rocket:, backlog :scroll:' - label-operator: OR - - - uses: actions/add-to-project@v0.3.0 - with: - project-url: https://github.com/users/Badgerati/projects/4 - github-token: ${{ secrets.PROJECT_TOKEN }} - labeled: 'design :art:, draft :pencil2:, idea :bulb:' + labeled: 'planned :calendar:, roadmap :rocket:, backlog :scroll:, draft :pencil2:, idea :bulb:' label-operator: OR \ No newline at end of file diff --git a/.github/workflows/open-issue-project.yml b/.github/workflows/open-issue-project.yml index 7cbf8296d..84cd5fb87 100644 --- a/.github/workflows/open-issue-project.yml +++ b/.github/workflows/open-issue-project.yml @@ -14,5 +14,5 @@ jobs: with: project-url: https://github.com/users/Badgerati/projects/2 github-token: ${{ secrets.PROJECT_TOKEN }} - labeled: 'bug :bug:, documentation :book:, packaging :package:' + labeled: 'bug :bug:, documentation :book:, packaging :package:, enhancement :arrow_up:, feature :sunny:' label-operator: OR \ No newline at end of file diff --git a/README.md b/README.md index 1b8b69596..48017d4e0 100644 --- a/README.md +++ b/README.md @@ -110,4 +110,4 @@ To work on issues you can fork Pode, and then open a Pull Request for approval. You can find a list of the features, enhancements and ideas that will hopefully one day make it into Pode [here in the documentation](https://badgerati.github.io/Pode/roadmap/). -There is also a [Project](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues. Plus, there is a [Draft Board](https://github.com/users/Badgerati/projects/4) which contains a range of ideas for Pode features/enhancements which are either brilliant, ludicrous, or down right insane! Draft Issues are purely ideas, and any in the design stage might one day make it in! If you see a Draft Issue you which to discuss, or have an idea for one, please dicuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. +There is also a [Project Board](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues and ideas. If you see any draft issues you wish to discuss, or have an idea for one, please dicuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. diff --git a/docs/roadmap.md b/docs/roadmap.md index e33bf4e69..8057313ce 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -4,7 +4,7 @@ This page lists the planned features and enhancements that will, hopefully, one Where possible items listed here will have a link to any relevant issues in GitHub. -There is also a [Project](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues. Plus, there is a [Draft Board](https://github.com/users/Badgerati/projects/4) which contains a range of ideas for Pode features/enhancements which are either brilliant, ludicrous, or down right insane! Draft Issues are purely ideas, and any in the design stage might one day make it in! If you see a Draft Issue you which to discuss, or have an idea for one, please dicuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. +There is also a [Project Board](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues and ideas. If you see any draft issues you wish to discuss, or have an idea for one, please dicuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. ## 🎯 Goal From 18804fb89a8e3ec8b98edc6f195d34d7248bcdd6 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sun, 4 Dec 2022 09:56:10 +0000 Subject: [PATCH 04/52] #1028: fix missing 221 response on smtp quit --- src/Listener/PodeSmtpRequest.cs | 1 + 1 file changed, 1 insertion(+) diff --git a/src/Listener/PodeSmtpRequest.cs b/src/Listener/PodeSmtpRequest.cs index 1d5e416e5..56582c47d 100644 --- a/src/Listener/PodeSmtpRequest.cs +++ b/src/Listener/PodeSmtpRequest.cs @@ -115,6 +115,7 @@ protected override bool Parse(byte[] bytes) if (IsCommand(content, "QUIT")) { Command = PodeSmtpCommand.Quit; + Context.Response.WriteLine("221 OK", true); return true; } From 06f5686ef9396491ee38a30a5c9e61127dfcf392 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sat, 10 Dec 2022 10:38:06 +0000 Subject: [PATCH 05/52] #964: add new -IfExists parameter for Routes - to either Error, Overwrite, or Skip --- src/Pode.psd1 | 4 + src/Private/Context.ps1 | 7 + src/Private/Routes.ps1 | 54 +++++-- src/Public/Routes.ps1 | 319 ++++++++++++++++++++++++++++++++++++++-- 4 files changed, 360 insertions(+), 24 deletions(-) diff --git a/src/Pode.psd1 b/src/Pode.psd1 index 25f117a4b..40549fb62 100644 --- a/src/Pode.psd1 +++ b/src/Pode.psd1 @@ -139,6 +139,10 @@ 'Add-PodeRouteGroup', 'Add-PodeStaticRouteGroup', 'Add-PodeSignalRouteGroup', + 'Set-PodeRouteIfExistsPreference', + 'Test-PodeRoute', + 'Test-PodeStaticRoute', + 'Test-PodeSignalRoute', # handlers 'Add-PodeHandler', diff --git a/src/Private/Context.ps1 b/src/Private/Context.ps1 index fdb83c296..88eedf7fa 100644 --- a/src/Private/Context.ps1 +++ b/src/Private/Context.ps1 @@ -276,6 +276,13 @@ function New-PodeContext IsDynamic = $false } + # pode default preferences + $ctx.Server.Preferences = @{ + Routes = @{ + IfExists = $null + } + } + # routes for pages and api $ctx.Server.Routes = @{ 'delete' = [ordered]@{} diff --git a/src/Private/Routes.ps1 b/src/Private/Routes.ps1 index d4d20ac78..a62a847c3 100644 --- a/src/Private/Routes.ps1 +++ b/src/Private/Routes.ps1 @@ -1,4 +1,4 @@ -function Test-PodeRoute +function Test-PodeRouteFromRequest { param ( [Parameter(Mandatory=$true)] @@ -395,9 +395,9 @@ function Get-PodeStaticRouteDefaults ) } -function Test-PodeRouteAndError +function Test-PodeRouteInternal { - param ( + param( [Parameter(Mandatory=$true)] [string] $Method, @@ -412,15 +412,34 @@ function Test-PodeRouteAndError [Parameter()] [string] - $Address + $Address, + + [switch] + $ThrowError ) - $found = @($PodeContext.Server.Routes[$Method][$Path]) + # check the routes + $found = $false + $routes = @($PodeContext.Server.Routes[$Method][$Path]) + + foreach ($route in $routes) { + if (($route.Endpoint.Protocol -ieq $Protocol) -and ($route.Endpoint.Address -ieq $Address)) { + $found = $true + break + } + } + + # skip if not found + if (!$found) { + return $false + } - if (($found | Where-Object { ($_.Endpoint.Protocol -ieq $Protocol) -and ($_.Endpoint.Address -ieq $Address) } | Measure-Object).Count -eq 0) { - return + # do we want to throw an error if found, or skip? + if (!$ThrowError) { + return $true } + # throw error $_url = $Protocol if (![string]::IsNullOrEmpty($_url) -and ![string]::IsNullOrWhiteSpace($Address)) { $_url = "$($_url)://$($Address)" @@ -432,9 +451,8 @@ function Test-PodeRouteAndError if ([string]::IsNullOrEmpty($_url)) { throw "[$($Method)] $($Path): Already defined" } - else { - throw "[$($Method)] $($Path): Already defined for $($_url)" - } + + throw "[$($Method)] $($Path): Already defined for $($_url)" } function Convert-PodeFunctionVerbToHttpMethod @@ -587,4 +605,20 @@ function ConvertTo-PodeMiddleware }) return $converted +} + +function Get-PodeRouteIfExistsPreference +{ + $interalPref = $RouteIfExists + $globalPref = $PodeContext.Server.Preferences.Routes.IfExists + + if ([string]::IsNullOrWhiteSpace($interalPref) -or ($interalPref -ieq 'default')) { + if ([string]::IsNullOrWhiteSpace($globalPref) -or ($globalPref -ieq 'default')) { + return 'Error' + } + + return $globalPref + } + + return $interalPref } \ No newline at end of file diff --git a/src/Public/Routes.ps1 b/src/Public/Routes.ps1 index aedd39449..e9b7c76a5 100644 --- a/src/Public/Routes.ps1 +++ b/src/Public/Routes.ps1 @@ -50,6 +50,9 @@ If supplied, the Route will be flagged to Authentication as being a Route that h .PARAMETER PassThru If supplied, the route created will be returned so it can be passed through a pipe. +.PARAMETER IfExists +Specifies what action to take when a Route already exists. (Default: Default) + .EXAMPLE Add-PodeRoute -Method Get -Path '/' -ScriptBlock { /* logic */ } @@ -119,6 +122,10 @@ function Add-PodeRoute [string] $Authentication, + [Parameter()] + [ValidateSet('Default', 'Error', 'Overwrite', 'Skip')] + $IfExists = 'Default', + [switch] $AllowAnon, @@ -167,6 +174,9 @@ function Add-PodeRoute } } + # store the original path + $origPath = $Path + # split route on '?' for query $Path = Split-PodeRouteQuery -Path $Path if ([string]::IsNullOrWhiteSpace($Path)) { @@ -185,9 +195,30 @@ function Add-PodeRoute $endpoints = Find-PodeEndpoints -EndpointName $EndpointName + # get default route IfExists state + if ($IfExists -ieq 'Default') { + $IfExists = Get-PodeRouteIfExistsPreference + } + # ensure the route doesn't already exist for each endpoint - foreach ($_endpoint in $endpoints) { - Test-PodeRouteAndError -Method $Method -Path $Path -Protocol $_endpoint.Protocol -Address $_endpoint.Address + $endpoints = @(foreach ($_endpoint in $endpoints) { + $found = Test-PodeRouteInternal -Method $Method -Path $Path -Protocol $_endpoint.Protocol -Address $_endpoint.Address -ThrowError:($IfExists -ieq 'Error') + + if ($found) { + if ($IfExists -ieq 'Overwrite') { + Remove-PodeRoute -Method $Method -Path $origPath -EndpointName $_endpoint.Name + } + + if ($IfExists -ieq 'Skip') { + continue + } + } + + $_endpoint + }) + + if (($null -eq $endpoints) -or ($endpoints.Length -eq 0)) { + return } # if middleware, scriptblock and file path are all null/empty, error @@ -326,6 +357,9 @@ When supplied, all static content on this Route will be attached as downloads - .PARAMETER PassThru If supplied, the static route created will be returned so it can be passed through a pipe. +.PARAMETER IfExists +Specifies what action to take when a Static Route already exists. (Default: Default) + .EXAMPLE Add-PodeStaticRoute -Path '/assets' -Source './assets' @@ -338,7 +372,7 @@ Add-PodeStaticRoute -Path '/installers' -Source './exes' -DownloadOnly function Add-PodeStaticRoute { [CmdletBinding()] - param ( + param( [Parameter(Mandatory=$true)] [string] $Path, @@ -377,6 +411,10 @@ function Add-PodeStaticRoute [string] $Authentication, + [Parameter()] + [ValidateSet('Default', 'Error', 'Overwrite', 'Skip')] + $IfExists = 'Default', + [switch] $AllowAnon, @@ -437,6 +475,9 @@ function Add-PodeStaticRoute # store the route method $Method = 'Static' + # store the original path + $origPath = $Path + # split route on '?' for query $Path = Split-PodeRouteQuery -Path $Path if ([string]::IsNullOrWhiteSpace($Path)) { @@ -455,9 +496,30 @@ function Add-PodeStaticRoute $endpoints = Find-PodeEndpoints -EndpointName $EndpointName + # get default route IfExists state + if ($IfExists -ieq 'Default') { + $IfExists = Get-PodeRouteIfExistsPreference + } + # ensure the route doesn't already exist for each endpoint - foreach ($_endpoint in $endpoints) { - Test-PodeRouteAndError -Method $Method -Path $Path -Protocol $_endpoint.Protocol -Address $_endpoint.Address + $endpoints = @(foreach ($_endpoint in $endpoints) { + $found = Test-PodeRouteInternal -Method $Method -Path $Path -Protocol $_endpoint.Protocol -Address $_endpoint.Address -ThrowError:($IfExists -ieq 'Error') + + if ($found) { + if ($IfExists -ieq 'Overwrite') { + Remove-PodeStaticRoute -Path $origPath -EndpointName $_endpoint.Name + } + + if ($IfExists -ieq 'Skip') { + continue + } + } + + $_endpoint + }) + + if (($null -eq $endpoints) -or ($endpoints.Length -eq 0)) { + return } # if static, ensure the path exists at server root @@ -569,6 +631,9 @@ A literal, or relative, path to a file containing a ScriptBlock for the Signal R .PARAMETER ArgumentList An array of arguments to supply to the Signal Route's ScriptBlock. +.PARAMETER IfExists +Specifies what action to take when a Signal Route already exists. (Default: Default) + .EXAMPLE Add-PodeSignalRoute -Path '/message' -ScriptBlock { /* logic */ } @@ -597,7 +662,11 @@ function Add-PodeSignalRoute [Parameter()] [object[]] - $ArgumentList + $ArgumentList, + + [Parameter()] + [ValidateSet('Default', 'Error', 'Overwrite', 'Skip')] + $IfExists = 'Default' ) # check if we have any route group info defined @@ -613,6 +682,9 @@ function Add-PodeSignalRoute $Method = 'Signal' + # store the original path + $origPath = $Path + # ensure the route has appropriate slashes $Path = Update-PodeRouteSlashes -Path $Path @@ -623,9 +695,30 @@ function Add-PodeSignalRoute $endpoints = Find-PodeEndpoints -EndpointName $EndpointName + # get default route IfExists state + if ($IfExists -ieq 'Default') { + $IfExists = Get-PodeRouteIfExistsPreference + } + # ensure the route doesn't already exist for each endpoint - foreach ($_endpoint in $endpoints) { - Test-PodeRouteAndError -Method $Method -Path $Path -Protocol $_endpoint.Protocol -Address $_endpoint.Address + $endpoints = @(foreach ($_endpoint in $endpoints) { + $found = Test-PodeRouteInternal -Method $Method -Path $Path -Protocol $_endpoint.Protocol -Address $_endpoint.Address -ThrowError:($IfExists -ieq 'Error') + + if ($found) { + if ($IfExists -ieq 'Overwrite') { + Remove-PodeSignalRoute -Path $origPath -EndpointName $_endpoint.Name + } + + if ($IfExists -ieq 'Skip') { + continue + } + } + + $_endpoint + }) + + if (($null -eq $endpoints) -or ($endpoints.Length -eq 0)) { + return } # if scriptblock and file path are all null/empty, error @@ -1154,7 +1247,7 @@ Remove-PodeStaticRoute -Path '/assets' function Remove-PodeStaticRoute { [CmdletBinding()] - param ( + param( [Parameter(Mandatory=$true)] [string] $Path, @@ -1356,7 +1449,7 @@ ConvertTo-PodeRoute -Commands @('Invoke-Pester') -Module Pester function ConvertTo-PodeRoute { [CmdletBinding()] - param ( + param( [Parameter(ValueFromPipeline=$true)] [string[]] $Commands, @@ -1554,7 +1647,7 @@ Add-PodePage -Name About -FilePath '.\views\about.pode' -Data @{ Date = [DateTim function Add-PodePage { [CmdletBinding(DefaultParameterSetName='ScriptBlock')] - param ( + param( [Parameter(Mandatory=$true)] [ValidateNotNullOrEmpty()] [string] @@ -1779,7 +1872,7 @@ Get-PodeStaticRoute -Path '/assets' -EndpointName User function Get-PodeStaticRoute { [CmdletBinding()] - param ( + param( [Parameter()] [string] $Path, @@ -1898,11 +1991,14 @@ Automatically loads route ps1 files from either a /routes folder, or a custom fo .PARAMETER Path Optional Path to a folder containing ps1 files, can be relative or literal. +.PARAMETER IfExists +Specifies what action to take when a Route already exists. (Default: Default) + .EXAMPLE Use-PodeRoutes .EXAMPLE -Use-PodeRoutes -Path './my-routes' +Use-PodeRoutes -Path './my-routes' -IfExists Skip #> function Use-PodeRoutes { @@ -1910,8 +2006,203 @@ function Use-PodeRoutes param( [Parameter()] [string] - $Path + $Path, + + [Parameter()] + [ValidateSet('Default', 'Error', 'Overwrite', 'Skip')] + $IfExists = 'Default' ) + if ($IfExists -ieq 'Default') { + $IfExists = Get-PodeRouteIfExistsPreference + } + + $RouteIfExists = $IfExists Use-PodeFolder -Path $Path -DefaultPath 'routes' +} + +<# +.SYNOPSIS +Set the default IfExists preference for Routes. + +.DESCRIPTION +Set the default IfExists preference for Routes. + +.PARAMETER Value +Specifies what action to take when a Route already exists. (Default: Default) + +.EXAMPLE +Set-PodeRouteIfExistsPreference -Value Overwrite +#> +function Set-PodeRouteIfExistsPreference +{ + [CmdletBinding()] + param( + [Parameter()] + [ValidateSet('Default', 'Error', 'Overwrite', 'Skip')] + $Value = 'Default' + ) + + $PodeContext.Server.Preferences.Routes.IfExists = $Value +} + +<# +.SYNOPSIS +Test if a Route already exists. + +.DESCRIPTION +Test if a Route already exists for a given Method and Path. + +.PARAMETER Method +The HTTP Method of the Route. + +.PARAMETER Path +The URI path of the Route. + +.PARAMETER EndpointName +The EndpointName of an Endpoint the Route is bound against. + +.PARAMETER CheckWildcard +If supplied, Pode will check for the Route on the Method first, and then check for the Route on the '*' Method. + +.EXAMPLE +Test-PodeRoute -Method Post -Path '/example' + +.EXAMPLE +Test-PodeRoute -Method Post -Path '/example' -CheckWildcard + +.EXAMPLE +Test-PodeRoute -Method Get -Path '/example/:exampleId' -CheckWildcard +#> +function Test-PodeRoute +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [ValidateSet('Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] + [string] + $Method, + + [Parameter(Mandatory=$true)] + [string] + $Path, + + [Parameter()] + [string] + $EndpointName, + + [switch] + $CheckWildcard + ) + + # split route on '?' for query + $Path = Split-PodeRouteQuery -Path $Path + if ([string]::IsNullOrWhiteSpace($Path)) { + throw "No Path supplied for testing Route" + } + + # ensure the route has appropriate slashes + $Path = Update-PodeRouteSlashes -Path $Path + $Path = Update-PodeRoutePlaceholders -Path $Path + + # get endpoint from name + $endpoint = @(Find-PodeEndpoints -EndpointName $EndpointName)[0] + + # check for routes + $found = (Test-PodeRouteInternal -Method $Method -Path $Path -Protocol $endpoint.Protocol -Address $endpoint.Address) + if (!$found -and $CheckWildcard) { + $found = (Test-PodeRouteInternal -Method '*' -Path $Path -Protocol $endpoint.Protocol -Address $endpoint.Address) + } + + return $found +} + +<# +.SYNOPSIS +Test if a Static Route already exists. + +.DESCRIPTION +Test if a Static Route already exists for a given Path. + +.PARAMETER Path +The URI path of the Static Route. + +.PARAMETER EndpointName +The EndpointName of an Endpoint the Static Route is bound against. + +.EXAMPLE +Test-PodeStaticRoute -Path '/assets' +#> +function Test-PodeStaticRoute +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Path, + + [Parameter()] + [string] + $EndpointName + ) + + # store the route method + $Method = 'Static' + + # split route on '?' for query + $Path = Split-PodeRouteQuery -Path $Path + if ([string]::IsNullOrWhiteSpace($Path)) { + throw "No Path supplied for testing Static Route" + } + + # ensure the route has appropriate slashes + $Path = Update-PodeRouteSlashes -Path $Path -Static + $Path = Update-PodeRoutePlaceholders -Path $Path + + # get endpoint from name + $endpoint = @(Find-PodeEndpoints -EndpointName $EndpointName)[0] + + # check for routes + return (Test-PodeRouteInternal -Method $Method -Path $Path -Protocol $endpoint.Protocol -Address $endpoint.Address) +} + +<# +.SYNOPSIS +Test if a Signal Route already exists. + +.DESCRIPTION +Test if a Signal Route already exists for a given Path. + +.PARAMETER Path +The URI path of the Signal Route. + +.PARAMETER EndpointName +The EndpointName of an Endpoint the Signal Route is bound against. + +.EXAMPLE +Test-PodeSignalRoute -Path '/message' +#> +function Test-PodeSignalRoute +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Path, + + [Parameter()] + [string] + $EndpointName + ) + + $Method = 'Signal' + + # ensure the route has appropriate slashes + $Path = Update-PodeRouteSlashes -Path $Path + + # get endpoint from name + $endpoint = @(Find-PodeEndpoints -EndpointName $EndpointName)[0] + + # check for routes + return (Test-PodeRouteInternal -Method $Method -Path $Path -Protocol $endpoint.Protocol -Address $endpoint.Address) } \ No newline at end of file From ca814248925d0b0e0d9831059929807bbb4814d8 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sat, 10 Dec 2022 18:48:19 +0000 Subject: [PATCH 06/52] #964: add IfExists for Route Groups, and adds docs --- docs/Tutorials/Routes/Overview.md | 65 +++++++++++++++++++++++++++++++ src/Private/Routes.ps1 | 21 ++++++---- src/Public/Routes.ps1 | 38 +++++++++++++++++- 3 files changed, 116 insertions(+), 8 deletions(-) diff --git a/docs/Tutorials/Routes/Overview.md b/docs/Tutorials/Routes/Overview.md index 779eb013f..cc31a23a2 100644 --- a/docs/Tutorials/Routes/Overview.md +++ b/docs/Tutorials/Routes/Overview.md @@ -166,6 +166,71 @@ Get-PodeRoute -EndpointName Admin The [`Get-PodeStaticRoute`](../../../Functions/Routes/Get-PodeStaticRoute) function works in the same way as above - but with no `-Method` parameter. +## If Exists Preference + +By default when you try and add a Route with the same Method and Path twice, Pode will throw an error when attempting to add the second Route. + +You can alter this behaviour by using the `-IfExists` parameter on several of the Route functions: + +* [`Add-PodeRoute`](../../../Functions/Routes/Add-PodeRoute) +* [`Add-PodeStaticRoute`](../../../Functions/Routes/Add-PodeStaticRoute) +* [`Add-PodeSignalRoute`](../../../Functions/Routes/Add-PodeSignalRoute) +* [`Add-PodeRouteGroup`](../../../Functions/Routes/Add-PodeRouteGroup) +* [`Add-PodeStaticRouteGroup`](../../../Functions/Routes/Add-PodeStaticRouteGroup) +* [`Add-PodeSignalRouteGroup`](../../../Functions/Routes/Add-PodeSignalRouteGroup) +* [`Use-PodeRoutes`](../../../Functions/Routes/Use-PodeRoutes) + +Or you can alter the global default preference for all Routes using [`Set-PodeRouteIfExistsPreference`](../../../Functions/Routes/Set-PodeRouteIfExistsPreference). + +This parameter accepts the following options: + +| Option | Description | +| ------ | ----------- | +| Default | This will use the `-IfExists` value from higher up the hierarchy (as defined see below) - if none defined, Error is the final default | +| Error | Throw an error if the Route already exists | +| Overwrite | Delete the existing Route if one exists, and then recreate the Route with the new definition | +| Skip | Skip over adding the Route if it already exists | + +and the following hierarchy is used when deciding which behaviour to use. At each step if the value defined is `Default` then check the next value in the hierarchy: + +1. Use the value defined directly on the Route, such as [`Add-PodeRoute`](../../../Functions/Routes/Add-PodeRoute) +2. Use the value defined on a Route Group, such as [`Add-PodeRouteGroup`](../../../Functions/Routes/Add-PodeRouteGroup) +3. Use the value defined on [`Use-PodeRoutes`](../../../Functions/Routes/Use-PodeRoutes) +4. Use the value defined from [`Set-PodeRouteIfExistsPreference`](../../../Functions/Routes/Set-PodeRouteIfExistsPreference) +5. Throw an error if the Route already exists + +For example, the following will now skip attempting to add the second Route because it already exists; meaning the value returned from `http://localhost:8080` is `1` not `2`: + +```powershell +Start-PodeServer { + Add-PodeEndpoint -Address * -Port 8080 -Protocol Http + + Add-PodeRoute -Method Get -Path '/' -ScriptBlock { + Write-PodeJsonResponse -Value @{ Result = 1 } + } + + Add-PodeRoute -Method Get -Path '/' -IfExists Skip -ScriptBlock { + Write-PodeJsonResponse -Value @{ Result = 2 } + } +} +``` + +Or, we could use Overwrite and the value returned will now be `2` not `1`: + +```powershell +Start-PodeServer { + Add-PodeEndpoint -Address * -Port 8080 -Protocol Http + + Add-PodeRoute -Method Get -Path '/' -ScriptBlock { + Write-PodeJsonResponse -Value @{ Result = 1 } + } + + Add-PodeRoute -Method Get -Path '/' -IfExists Overwrite -ScriptBlock { + Write-PodeJsonResponse -Value @{ Result = 2 } + } +} +``` + ## Grouping If you have a number of Routes that all share the same base path, middleware, authentication, or other parameters, then you can add these Routes within a Route Group (via [`Add-PodeRouteGroup`](../../../Functions/Routes/Add-PodeRouteGroup)) to share the parameter values: diff --git a/src/Private/Routes.ps1 b/src/Private/Routes.ps1 index a62a847c3..1136cc5dc 100644 --- a/src/Private/Routes.ps1 +++ b/src/Private/Routes.ps1 @@ -609,16 +609,23 @@ function ConvertTo-PodeMiddleware function Get-PodeRouteIfExistsPreference { - $interalPref = $RouteIfExists - $globalPref = $PodeContext.Server.Preferences.Routes.IfExists + # from route groups + $groupPref = $RouteGroup.IfExists + if (![string]::IsNullOrWhiteSpace($groupPref) -and ($groupPref -ine 'default')) { + return $groupPref + } - if ([string]::IsNullOrWhiteSpace($interalPref) -or ($interalPref -ieq 'default')) { - if ([string]::IsNullOrWhiteSpace($globalPref) -or ($globalPref -ieq 'default')) { - return 'Error' - } + # from Use-PodeRoute + if (![string]::IsNullOrWhiteSpace($RouteIfExists) -and ($RouteIfExists -ine 'default')) { + return $RouteIfExists + } + # global preference + $globalPref = $PodeContext.Server.Preferences.Routes.IfExists + if (![string]::IsNullOrWhiteSpace($globalPref) -and ($globalPref -ine 'default')) { return $globalPref } - return $interalPref + # final global default + return 'Error' } \ No newline at end of file diff --git a/src/Public/Routes.ps1 b/src/Public/Routes.ps1 index e9b7c76a5..c004d3287 100644 --- a/src/Public/Routes.ps1 +++ b/src/Public/Routes.ps1 @@ -795,6 +795,9 @@ The content type of any error pages that may get returned. .PARAMETER Authentication The name of an Authentication method which should be used as middleware on the Routes. +.PARAMETER IfExists +Specifies what action to take when a Route already exists. (Default: Default) + .PARAMETER AllowAnon If supplied, the Routes will allow anonymous access for non-authenticated users. @@ -839,6 +842,10 @@ function Add-PodeRouteGroup [string] $Authentication, + [Parameter()] + [ValidateSet('Default', 'Error', 'Overwrite', 'Skip')] + $IfExists = 'Default', + [switch] $AllowAnon ) @@ -891,6 +898,10 @@ function Add-PodeRouteGroup if ($RouteGroup.AllowAnon) { $AllowAnon = $RouteGroup.AllowAnon } + + if ($IfExists -ieq 'default') { + $IfExists = Get-PodeRouteIfExistsPreference + } } $RouteGroup = @{ @@ -902,6 +913,7 @@ function Add-PodeRouteGroup ErrorContentType = $ErrorContentType Authentication = $Authentication AllowAnon = $AllowAnon + IfExists = $IfExists } # add routes @@ -946,6 +958,9 @@ The content type of any error pages that may get returned. .PARAMETER Authentication The name of an Authentication method which should be used as middleware on the Static Routes. +.PARAMETER IfExists +Specifies what action to take when a Static Route already exists. (Default: Default) + .PARAMETER AllowAnon If supplied, the Static Routes will allow anonymous access for non-authenticated users. @@ -1001,6 +1016,10 @@ function Add-PodeStaticRouteGroup [string] $Authentication, + [Parameter()] + [ValidateSet('Default', 'Error', 'Overwrite', 'Skip')] + $IfExists = 'Default', + [switch] $AllowAnon, @@ -1068,6 +1087,10 @@ function Add-PodeStaticRouteGroup if ($RouteGroup.DownloadOnly) { $DownloadOnly = $RouteGroup.DownloadOnly } + + if ($IfExists -ieq 'default') { + $IfExists = Get-PodeRouteIfExistsPreference + } } $RouteGroup = @{ @@ -1082,6 +1105,7 @@ function Add-PodeStaticRouteGroup Authentication = $Authentication AllowAnon = $AllowAnon DownloadOnly = $DownloadOnly + IfExists = $IfExists } # add routes @@ -1106,6 +1130,9 @@ A ScriptBlock for adding Signal Routes. .PARAMETER EndpointName The EndpointName of an Endpoint(s) to use for the Signal Routes. +.PARAMETER IfExists +Specifies what action to take when a Signal Route already exists. (Default: Default) + .EXAMPLE Add-PodeSignalRouteGroup -Path '/signals' -Routes { Add-PodeSignalRoute -Path '/signal1' -Etc } #> @@ -1123,7 +1150,11 @@ function Add-PodeSignalRouteGroup [Parameter()] [string[]] - $EndpointName + $EndpointName, + + [Parameter()] + [ValidateSet('Default', 'Error', 'Overwrite', 'Skip')] + $IfExists = 'Default' ) if (Test-PodeIsEmpty $Routes) { @@ -1150,11 +1181,16 @@ function Add-PodeSignalRouteGroup if ([string]::IsNullOrWhiteSpace($EndpointName)) { $EndpointName = $RouteGroup.EndpointName } + + if ($IfExists -ieq 'default') { + $IfExists = Get-PodeRouteIfExistsPreference + } } $RouteGroup = @{ Path = $Path EndpointName = $EndpointName + IfExists = $IfExists } # add routes From 3baba973ab8f3a43dcc6f5ac55703599548c8553 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sun, 11 Dec 2022 12:11:33 +0000 Subject: [PATCH 07/52] #647: add New-PodeCron helper function for aiding with building cron expressions --- docs/Tutorials/Misc/CronExpressions.md | 52 ++++- docs/Tutorials/Schedules.md | 9 +- examples/schedules-cron-helper.ps1 | 20 ++ src/Pode.psd1 | 1 + src/Private/Schedules.ps1 | 36 ++++ src/Public/Utilities.ps1 | 250 +++++++++++++++++++++++++ tests/unit/Helpers.Tests.ps1 | 62 ++++++ 7 files changed, 427 insertions(+), 3 deletions(-) create mode 100644 examples/schedules-cron-helper.ps1 diff --git a/docs/Tutorials/Misc/CronExpressions.md b/docs/Tutorials/Misc/CronExpressions.md index cc564cae2..f1e66e19f 100644 --- a/docs/Tutorials/Misc/CronExpressions.md +++ b/docs/Tutorials/Misc/CronExpressions.md @@ -1,6 +1,6 @@ # Cron Expressions -Schedules and [`Auto Server Restarting`](../../Restarting/AutoRestarting) in Pode use cron expressions to define when they trigger. This page is a brief overview of the expressions supported by Pode. +Schedules and [`Auto Server Restarting`](../../Restarting/Types/AutoRestarting) in Pode use cron expressions to define when they trigger. This page is a brief overview of the expressions supported by Pode. ## Basic @@ -33,7 +33,7 @@ The following table outlines some of the predefined cron expressions supported b | @daily | `0 0 * * *` | | @weekly | `0 0 * * 0` | | @monthly | `0 0 1 * *` | -| @quarterly | `0 0 1 1,4,8,7,10 *` | +| @quarterly | `0 0 1 1,4,7,10 *` | | @yearly | `0 0 1 1 *` | | @annually | `0 0 1 1 *` | | @twice-hourly | `0,30 * * * *` | @@ -48,3 +48,51 @@ The following table outlines some of the predefined cron expressions supported b Pode does have some support for advanced cron features, including its own placeholder: `R`. * `R`: using this on an atom will use a random value between that atom's constraints, and when the expression is triggered the atom is re-randomised - you can force an initial trigger value using `/R`. For example: `30/R * * * *` will trigger on 30mins, then a random minute afterwards; whereas using `R * * * *` will always trigger on a random minute between 0-59. + +## Helper + +Pode has an inbuilt helper function, [`New-PodeCron`](../../../Functions/Utilities/New-PodeCron), which can be used to generate cron expressions more easily. These cron expressions can then be used in [Schedules](../../Schedules) and other Pode functions that use cron expressions. + +The main way to use [`New-PodeCron`](../../../Functions/Utilities/New-PodeCron) is to start with the `-Every` parameter, such as `-Every Hour` or `-Every Day`. From this, you can customise the expression to run at specific times/days, or apply a recurring `-Interval`: + +```powershell +# Everyday, at 00:00 +New-PodeCron -Every Day + +# Every Tuesday and Friday, at 01:00 +New-PodeCron -Every Day -Day Tuesday, Friday -Hour 1 + +# Every 15th of the month at 00:00 +New-PodeCron -Every Month -Date 15 + +# Every other day, starting from the 2nd of each month, at 00:00 +New-PodeCron -Every Date -Interval 2 -Date 2 + +# Every 1st June, at 00:00 +New-PodeCron -Every Year -Month June + +# Every hour, starting at 01:00 +New-PodeCron -Every Hour -Hour 1 -Interval 1 + +# Every 15 minutes, between 01:00 and 05:00 +New-PodeCron -Every Minute -Hour 1, 2, 3, 4, 5 -Interval 15 + +# Every hour of every Monday (ie: 00:00, 01:00, 02:00, etc.) +New-PodeCron -Every Hour -Day Monday + +# Every 1st January, April, July, and October, at 00:00 +New-PodeCron -Every Quarter + +# Everyday at 05:15 +New-PodeCron -Every Day -Hour 5 -Minute 15 +``` + +You can also use [`New-PodeCron`](../../../Functions/Utilities/New-PodeCron) without using the `-Every` parameter. In this state, every part of the cron expression will be wildcarded by default - such as every minute, every hour, every day, etc. - unless you specify the parameter explicitly: + +```powershell +# Every 10 minutes on Tuesdays +New-PodeCron -Day Tuesday -Minute 0, 10, 20, 30, 40, 50 + +# Every minute on Tuesdays +New-PodeCron -Day Tuesday +``` diff --git a/docs/Tutorials/Schedules.md b/docs/Tutorials/Schedules.md index 6a70b4fbf..861f3da06 100644 --- a/docs/Tutorials/Schedules.md +++ b/docs/Tutorials/Schedules.md @@ -2,7 +2,7 @@ A Schedule in Pode is a long-running async task, and unlike timers, when they trigger they are run in their own separate runspace - so they don't affect each other if they take a while to process. By default up to a maximum of 10 schedules can run concurrently, but this can be changed by using [`Set-PodeScheduleConcurrency`](../../Functions/Schedules/Set-PodeScheduleConcurrency). -Schedule triggers are defined using [`cron expressions`](../Misc/CronExpressions), basic syntax is supported as well as some predefined expressions. Schedules can start immediately, have a delayed start time, and also have a defined end time. +Schedule triggers are defined using [`cron expressions`](../Misc/CronExpressions), basic syntax is supported as well as some predefined expressions. Pode also has an inbuilt helper, [`New-PodeCron`](../../Functions/Utilities/New-PodeCron), to help with building cron expressions - as [described here](../Misc/CronExpressions#helper). Schedules can start immediately, have a delayed start time, and also have a defined end time. ## Create a Schedule @@ -12,6 +12,13 @@ You can create a new schedule using [`Add-PodeSchedule`](../../Functions/Schedul Add-PodeSchedule -Name 'date' -Cron '5 0 * * TUE' -ScriptBlock { Write-Host "$([DateTime]::Now)" } + +# or, using Pode's helper +$cron = New-PodeCron -Day Tuesday -Hour 0 -Minute 5 + +Add-PodeSchedule -Name 'date' -Cron $cron -ScriptBlock { + Write-Host "$([DateTime]::Now)" +} ``` Whereas the following will create the same schedule, but will only trigger the schedule 4 times due to the `-Limit` value supplied: diff --git a/examples/schedules-cron-helper.ps1 b/examples/schedules-cron-helper.ps1 new file mode 100644 index 000000000..df04b9a4c --- /dev/null +++ b/examples/schedules-cron-helper.ps1 @@ -0,0 +1,20 @@ +$path = Split-Path -Parent -Path (Split-Path -Parent -Path $MyInvocation.MyCommand.Path) +Import-Module "$($path)/src/Pode.psm1" -Force -ErrorAction Stop + +# or just: +# Import-Module Pode + +Start-PodeServer { + + Add-PodeEndpoint -Address * -Port 8085 -Protocol Http + + $cron = New-PodeCron -Every Minute -Interval 2 + Add-PodeSchedule -Name 'example' -Cron $cron -ScriptBlock { + 'Hi there!' | Out-Default + } + + Add-PodeRoute -Method Get -Path '/' -ScriptBlock { + Write-PodeJsonResponse -Value @{ Result = 1 } + } + +} \ No newline at end of file diff --git a/src/Pode.psd1 b/src/Pode.psd1 index 25f117a4b..6d509ff0c 100644 --- a/src/Pode.psd1 +++ b/src/Pode.psd1 @@ -156,6 +156,7 @@ 'Get-PodeSchedule', 'Get-PodeScheduleNextTrigger', 'Use-PodeSchedules', + 'New-PodeCron', # timers 'Add-PodeTimer', diff --git a/src/Private/Schedules.ps1 b/src/Private/Schedules.ps1 index 53cde8b7b..a0bd193ee 100644 --- a/src/Private/Schedules.ps1 +++ b/src/Private/Schedules.ps1 @@ -211,4 +211,40 @@ function Invoke-PodeInternalScheduleLogic catch { $_ | Write-PodeErrorLog } +} + +function Set-PodeScheduleCronInterval +{ + param( + [Parameter()] + [hashtable] + $Cron, + + [Parameter()] + [string] + $Type, + + [Parameter()] + [int[]] + $Value, + + [Parameter()] + [int] + $Interval + ) + + if ($Interval -le 0) { + return $false + } + + if ($Value.Length -gt 1) { + throw "You can only supply a single $($Type) value when using intervals" + } + + if ($Value.Length -eq 1) { + $Cron[$Type] = "$(@($Value)[0])" + } + + $Cron[$Type] += "/$($Interval)" + return ($Value.Length -eq 1) } \ No newline at end of file diff --git a/src/Public/Utilities.ps1 b/src/Public/Utilities.ps1 index 364df3a36..e503f8e09 100644 --- a/src/Public/Utilities.ps1 +++ b/src/Public/Utilities.ps1 @@ -1068,4 +1068,254 @@ function Out-PodeVariable ) $PodeContext.Server.Output.Variables[$Name] = $Value +} + +<# +.SYNOPSIS +A helper function to generate cron expressions. + +.DESCRIPTION +A helper function to generate cron expressions, which can be used for Schedules and other functions that use cron expressions. +This helper function only covers simple cron use-cases, with some advanced use-cases. If you need further advanced cron +expressions it would be best to write the expression by hand. + +.PARAMETER Minute +This is an array of Minutes that the expression should use between 0-59. + +.PARAMETER Hour +This is an array of Hours that the expression should use between 0-23. + +.PARAMETER Date +This is an array of Dates in the monnth that the expression should use between 1-31. + +.PARAMETER Month +This is an array of Months that the expression should use between January-December. + +.PARAMETER Day +This is an array of Days in the week that the expression should use between Monday-Sunday. + +.PARAMETER Every +This can be used to more easily specify "Every Hour" than writing out all the hours. + +.PARAMETER Interval +This can only be used when using the Every parameter, and will setup an interval on the "every" used. +If you want "every 2 hours" then Every should be set to Hour and Interval to 2. + +.EXAMPLE +New-PodeCron -Every Day # every 00:00 + +.EXAMPLE +New-PodeCron -Every Day -Day Tuesday, Friday -Hour 1 # every tuesday and friday at 01:00 + +.EXAMPLE +New-PodeCron -Every Month -Date 15 # every 15th of the month at 00:00 + +.EXAMPLE +New-PodeCron -Every Date -Interval 2 -Date 2 # every month, every other day from 2nd, at 00:00 + +.EXAMPLE +New-PodeCron -Every Year -Month June # every 1st june, at 00:00 + +.EXAMPLE +New-PodeCron -Every Hour -Hour 1 -Interval 1 # every hour, starting at 01:00 + +.EXAMPLE +New-PodeCron -Every Minute -Hour 1, 2, 3, 4, 5 -Interval 15 # every 15mins, starting at 01:00 until 05:00 + +.EXAMPLE +New-PodeCron -Every Hour -Day Monday # every hour of every monday + +.EXAMPLE +New-PodeCron -Every Quarter # every 1st jan, apr, jul, oct, at 00:00 +#> +function New-PodeCron +{ + [CmdletBinding()] + param( + [Parameter()] + [ValidateRange(0, 59)] + [int[]] + $Minute = $null, + + [Parameter()] + [ValidateRange(0, 23)] + [int[]] + $Hour = $null, + + [Parameter()] + [ValidateRange(1, 31)] + [int[]] + $Date = $null, + + [Parameter()] + [ValidateSet('January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December')] + [string[]] + $Month = $null, + + [Parameter()] + [ValidateSet('Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday')] + [string[]] + $Day = $null, + + [Parameter()] + [ValidateSet('Minute', 'Hour', 'Day', 'Date', 'Month', 'Quarter', 'Year', 'None')] + [string] + $Every = 'None', + + [Parameter()] + [int] + $Interval = 0 + ) + + # New-PodeCron -Every Day # every 00:00 + # New-PodeCron -Every Day -Day Tuesday, Friday -Hour 1 # every tuesday and friday at 01:00 + # New-PodeCron -Every Month -Date 15 # every 15th of the month at 00:00 + # New-PodeCron -Every Date -Interval 2 -Date 2 # every month, every other day from 2nd, at 00:00 + # New-PodeCron -Every Year -Month June # every 1st june, at 00:00 + # New-PodeCron -Every Hour -Hour 1 -Interval 1 # every hour, starting at 01:00 + # New-PodeCron -Every Minute -Hour 1, 2, 3, 4, 5 -Interval 15 # every 15mins, starting at 01:00 until 05:00 + # New-PodeCron -Every Hour -Day Monday # every hour of every monday + # New-PodeCron -Every Quarter # every 1st jan, apr, jul, oct, at 00:00 + + # cant have None and Interval + if (($Every -ieq 'none') -and ($Interval -gt 0)) { + throw "Cannot supply an interval when -Every is set to None" + } + + # base cron + $cron = @{ + Minute = '*' + Hour = '*' + Date = '*' + Month = '*' + Day = '*' + } + + # convert month/day to numbers + if ($Month.Length -gt 0) { + $MonthInts = @(foreach ($item in $Month) { + (@{ + January = 1 + February = 2 + March = 3 + April = 4 + May = 5 + June = 6 + July = 7 + August = 8 + September = 9 + October = 10 + November = 11 + December = 12 + })[$item] + }) + } + + if ($Day.Length -gt 0) { + $DayInts = @(foreach ($item in $Day) { + (@{ + Sunday = 0 + Monday = 1 + Tuesday = 2 + Wednesday = 3 + Thursday = 4 + Friday = 5 + Saturday = 6 + })[$item] + }) + } + + # set "every" defaults + switch ($Every.ToUpperInvariant()) { + 'MINUTE' { + if (Set-PodeScheduleCronInterval -Cron $cron -Type 'Minute' -Value $Minute -Interval $Interval) { + $Minute = @() + } + } + + 'HOUR' { + $cron.Minute = '0' + + if (Set-PodeScheduleCronInterval -Cron $cron -Type 'Hour' -Value $Hour -Interval $Interval) { + $Hour = @() + } + } + + 'DAY' { + $cron.Minute = '0' + $cron.Hour = '0' + + if (Set-PodeScheduleCronInterval -Cron $cron -Type 'Day' -Value $DayInts -Interval $Interval) { + $DayInts = @() + } + } + + 'DATE' { + $cron.Minute = '0' + $cron.Hour = '0' + + if (Set-PodeScheduleCronInterval -Cron $cron -Type 'Date' -Value $Date -Interval $Interval) { + $Date = @() + } + } + + 'MONTH' { + $cron.Minute = '0' + $cron.Hour = '0' + + if ($DayInts.Length -eq 0) { + $cron.Date = '1' + } + + if (Set-PodeScheduleCronInterval -Cron $cron -Type 'Month' -Value $MonthInts -Interval $Interval) { + $MonthInts = @() + } + } + + 'QUARTER' { + $cron.Minute = '0' + $cron.Hour = '0' + $cron.Date = '1' + $cron.Month = '1,4,7,10' + + if ($Interval -gt 0) { + throw "Cannot supply interval value for every quarter" + } + } + + 'YEAR' { + $cron.Minute = '0' + $cron.Hour = '0' + $cron.Date = '1' + $cron.Month = '1' + + if ($Interval -gt 0) { + throw "Cannot supply interval value for every year" + } + } + } + + # set any custom overrides + if ($Minute.Length -gt 0) { + $cron.Minute = $Minute -join ',' + } + + if ($Hour.Length -gt 0) { + $cron.Hour = $Hour -join ',' + } + + if ($DayInts.Length -gt 0) { + $cron.Day = $DayInts -join ',' + } + + if ($Date.Length -gt 0) { + $cron.Date = $Date -join ',' + } + + if ($MonthInts.Length -gt 0) { + $cron.Month = $MonthInts -join ',' + } + + # build and return + return "$($cron.Minute) $($cron.Hour) $($cron.Date) $($cron.Month) $($cron.Day)" } \ No newline at end of file diff --git a/tests/unit/Helpers.Tests.ps1 b/tests/unit/Helpers.Tests.ps1 index bad84ae88..67380dade 100644 --- a/tests/unit/Helpers.Tests.ps1 +++ b/tests/unit/Helpers.Tests.ps1 @@ -1642,4 +1642,66 @@ Describe 'Get-PodeEncodingFromContentType' { $enc = Get-PodeEncodingFromContentType -ContentType 'application/json;charset=utf-8' $enc.EncodingName | Should Be 'Unicode (UTF-8)' } +} + +Describe 'New-PodeCron' { + It 'Returns a minutely expression' { + New-PodeCron -Every Minute | Should Be '* * * * *' + } + + It 'Returns an hourly expression' { + New-PodeCron -Every Hour | Should Be '0 * * * *' + } + + It 'Returns a daily expression (by day)' { + New-PodeCron -Every Day | Should Be '0 0 * * *' + } + + It 'Returns a daily expression (by date)' { + New-PodeCron -Every Date | Should Be '0 0 * * *' + } + + It 'Returns a monthly expression' { + New-PodeCron -Every Month | Should Be '0 0 1 * *' + } + + It 'Returns a quarterly expression' { + New-PodeCron -Every Quarter | Should Be '0 0 1 1,4,7,10 *' + } + + It 'Returns a yearly expression' { + New-PodeCron -Every Year | Should Be '0 0 1 1 *' + } + + It 'Returns an expression for every 15mins' { + New-PodeCron -Every Minute -Interval 15 | Should Be '*/15 * * * *' + } + + It 'Returns an expression for every tues/fri at 1am' { + New-PodeCron -Every Day -Day Tuesday, Friday -Hour 1 | Should Be '0 1 * * 2,5' + } + + It 'Returns an expression for every 15th of the month' { + New-PodeCron -Every Month -Date 15 | Should Be '0 0 15 * *' + } + + It 'Returns an expression for every other day, from the 2nd' { + New-PodeCron -Every Date -Interval 2 -Date 2 | Should Be '0 0 2/2 * *' + } + + It 'Returns an expression for every june 1st' { + New-PodeCron -Every Year -Month June | Should Be '0 0 1 6 *' + } + + It 'Returns an expression for every 15mins between 1am-5am' { + New-PodeCron -Every Minute -Interval 15 -Hour 1, 2, 3, 4, 5 | Should Be '*/15 1,2,3,4,5 * * *' + } + + It 'Returns an expression for every hour of every monday' { + New-PodeCron -Every Hour -Day Monday | Should Be '0 * * * 1' + } + + It 'Returns an expression for everyday at 5:15am' { + New-PodeCron -Every Day -Hour 5 -Minute 15 | Should Be '15 5 * * *' + } } \ No newline at end of file From 35ec25db2cd76568bad0ac7c167793ce73a2a0da Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sun, 11 Dec 2022 12:18:23 +0000 Subject: [PATCH 08/52] #647: tweak to docs --- docs/Tutorials/Misc/CronExpressions.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/Tutorials/Misc/CronExpressions.md b/docs/Tutorials/Misc/CronExpressions.md index f1e66e19f..d77ee0c3d 100644 --- a/docs/Tutorials/Misc/CronExpressions.md +++ b/docs/Tutorials/Misc/CronExpressions.md @@ -96,3 +96,14 @@ New-PodeCron -Day Tuesday -Minute 0, 10, 20, 30, 40, 50 # Every minute on Tuesdays New-PodeCron -Day Tuesday ``` + +The value returned by [`New-PodeCron`](../../../Functions/Utilities/New-PodeCron) is a valid cron expression, that can then be used when creating Schedules: + +```powershell +# Every Tuesday, at 00:05 +$cron = New-PodeCron -Day Tuesday -Hour 0 -Minute 5 + +Add-PodeSchedule -Name 'date' -Cron $cron -ScriptBlock { + Write-Host "$([DateTime]::Now)" +} +``` From f2f818c38a657992eb85b8977863c5fa3d1b99d2 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sun, 11 Dec 2022 12:27:18 +0000 Subject: [PATCH 09/52] #647: change private func name, add extra tests, clean-up comments --- src/Pode.psd1 | 2 +- src/Private/Helpers.ps1 | 36 ++++++++++++++++++++++++++++++++++++ src/Private/Schedules.ps1 | 36 ------------------------------------ src/Public/Utilities.ps1 | 20 +++++--------------- tests/unit/Helpers.Tests.ps1 | 20 ++++++++++++++++++++ 5 files changed, 62 insertions(+), 52 deletions(-) diff --git a/src/Pode.psd1 b/src/Pode.psd1 index 6d509ff0c..c84ff17c0 100644 --- a/src/Pode.psd1 +++ b/src/Pode.psd1 @@ -119,6 +119,7 @@ 'Test-PodeLockable', 'Out-PodeVariable', 'Test-PodeIsHosted', + 'New-PodeCron', # routes 'Add-PodeRoute', @@ -156,7 +157,6 @@ 'Get-PodeSchedule', 'Get-PodeScheduleNextTrigger', 'Use-PodeSchedules', - 'New-PodeCron', # timers 'Add-PodeTimer', diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 93bab9edd..daeb5687e 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -2941,4 +2941,40 @@ function Clear-PodeHashtableInnerKeys $InputObject.Keys.Clone() | ForEach-Object { $InputObject[$_].Clear() } +} + +function Set-PodeCronInterval +{ + param( + [Parameter()] + [hashtable] + $Cron, + + [Parameter()] + [string] + $Type, + + [Parameter()] + [int[]] + $Value, + + [Parameter()] + [int] + $Interval + ) + + if ($Interval -le 0) { + return $false + } + + if ($Value.Length -gt 1) { + throw "You can only supply a single $($Type) value when using intervals" + } + + if ($Value.Length -eq 1) { + $Cron[$Type] = "$(@($Value)[0])" + } + + $Cron[$Type] += "/$($Interval)" + return ($Value.Length -eq 1) } \ No newline at end of file diff --git a/src/Private/Schedules.ps1 b/src/Private/Schedules.ps1 index a0bd193ee..53cde8b7b 100644 --- a/src/Private/Schedules.ps1 +++ b/src/Private/Schedules.ps1 @@ -211,40 +211,4 @@ function Invoke-PodeInternalScheduleLogic catch { $_ | Write-PodeErrorLog } -} - -function Set-PodeScheduleCronInterval -{ - param( - [Parameter()] - [hashtable] - $Cron, - - [Parameter()] - [string] - $Type, - - [Parameter()] - [int[]] - $Value, - - [Parameter()] - [int] - $Interval - ) - - if ($Interval -le 0) { - return $false - } - - if ($Value.Length -gt 1) { - throw "You can only supply a single $($Type) value when using intervals" - } - - if ($Value.Length -eq 1) { - $Cron[$Type] = "$(@($Value)[0])" - } - - $Cron[$Type] += "/$($Interval)" - return ($Value.Length -eq 1) } \ No newline at end of file diff --git a/src/Public/Utilities.ps1 b/src/Public/Utilities.ps1 index e503f8e09..abd04204c 100644 --- a/src/Public/Utilities.ps1 +++ b/src/Public/Utilities.ps1 @@ -1167,16 +1167,6 @@ function New-PodeCron $Interval = 0 ) - # New-PodeCron -Every Day # every 00:00 - # New-PodeCron -Every Day -Day Tuesday, Friday -Hour 1 # every tuesday and friday at 01:00 - # New-PodeCron -Every Month -Date 15 # every 15th of the month at 00:00 - # New-PodeCron -Every Date -Interval 2 -Date 2 # every month, every other day from 2nd, at 00:00 - # New-PodeCron -Every Year -Month June # every 1st june, at 00:00 - # New-PodeCron -Every Hour -Hour 1 -Interval 1 # every hour, starting at 01:00 - # New-PodeCron -Every Minute -Hour 1, 2, 3, 4, 5 -Interval 15 # every 15mins, starting at 01:00 until 05:00 - # New-PodeCron -Every Hour -Day Monday # every hour of every monday - # New-PodeCron -Every Quarter # every 1st jan, apr, jul, oct, at 00:00 - # cant have None and Interval if (($Every -ieq 'none') -and ($Interval -gt 0)) { throw "Cannot supply an interval when -Every is set to None" @@ -1228,7 +1218,7 @@ function New-PodeCron # set "every" defaults switch ($Every.ToUpperInvariant()) { 'MINUTE' { - if (Set-PodeScheduleCronInterval -Cron $cron -Type 'Minute' -Value $Minute -Interval $Interval) { + if (Set-PodeCronInterval -Cron $cron -Type 'Minute' -Value $Minute -Interval $Interval) { $Minute = @() } } @@ -1236,7 +1226,7 @@ function New-PodeCron 'HOUR' { $cron.Minute = '0' - if (Set-PodeScheduleCronInterval -Cron $cron -Type 'Hour' -Value $Hour -Interval $Interval) { + if (Set-PodeCronInterval -Cron $cron -Type 'Hour' -Value $Hour -Interval $Interval) { $Hour = @() } } @@ -1245,7 +1235,7 @@ function New-PodeCron $cron.Minute = '0' $cron.Hour = '0' - if (Set-PodeScheduleCronInterval -Cron $cron -Type 'Day' -Value $DayInts -Interval $Interval) { + if (Set-PodeCronInterval -Cron $cron -Type 'Day' -Value $DayInts -Interval $Interval) { $DayInts = @() } } @@ -1254,7 +1244,7 @@ function New-PodeCron $cron.Minute = '0' $cron.Hour = '0' - if (Set-PodeScheduleCronInterval -Cron $cron -Type 'Date' -Value $Date -Interval $Interval) { + if (Set-PodeCronInterval -Cron $cron -Type 'Date' -Value $Date -Interval $Interval) { $Date = @() } } @@ -1267,7 +1257,7 @@ function New-PodeCron $cron.Date = '1' } - if (Set-PodeScheduleCronInterval -Cron $cron -Type 'Month' -Value $MonthInts -Interval $Interval) { + if (Set-PodeCronInterval -Cron $cron -Type 'Month' -Value $MonthInts -Interval $Interval) { $MonthInts = @() } } diff --git a/tests/unit/Helpers.Tests.ps1 b/tests/unit/Helpers.Tests.ps1 index 67380dade..9bb8715b5 100644 --- a/tests/unit/Helpers.Tests.ps1 +++ b/tests/unit/Helpers.Tests.ps1 @@ -1704,4 +1704,24 @@ Describe 'New-PodeCron' { It 'Returns an expression for everyday at 5:15am' { New-PodeCron -Every Day -Hour 5 -Minute 15 | Should Be '15 5 * * *' } + + It 'Throws an error for multiple Hours when using Interval' { + { New-PodeCron -Every Hour -Hour 2, 4 -Interval 3 } | Should Throw 'only supply a single' + } + + It 'Throws an error for multiple Minutes when using Interval' { + { New-PodeCron -Every Minute -Minute 2, 4 -Interval 15 } | Should Throw 'only supply a single' + } + + It 'Throws an error when using Interval without Every' { + { New-PodeCron -Interval 3 } | Should Throw 'Cannot supply an interval' + } + + It 'Throws an error when using Interval for Every Quarter' { + { New-PodeCron -Every Quarter -Interval 3 } | Should Throw 'Cannot supply interval value for every quarter' + } + + It 'Throws an error when using Interval for Every Year' { + { New-PodeCron -Every Year -Interval 3 } | Should Throw 'Cannot supply interval value for every year' + } } \ No newline at end of file From c37cce077a9a6131ec333776b95a57ed196d034d Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Fri, 16 Dec 2022 11:05:32 +0000 Subject: [PATCH 10/52] #1036: add ways to reset/get session expiry, and clean-up the code a bit around session cookies --- docs/Tutorials/Cookies.md | 4 +- docs/Tutorials/Middleware/Types/Sessions.md | 100 +++++- examples/views/auth-home.pode | 3 + examples/web-auth-form.ps1 | 1 + src/Pode.psd1 | 9 +- src/Private/Authentication.ps1 | 14 +- src/Private/Cookies.ps1 | 67 ++-- src/Private/Middleware.ps1 | 2 + src/Private/Sessions.ps1 | 220 +++++------- src/Public/Cookies.ps1 | 30 +- src/Public/Middleware.ps1 | 270 --------------- src/Public/Sessions.ps1 | 362 ++++++++++++++++++++ tests/unit/Sessions.Tests.ps1 | 179 +++++----- 13 files changed, 719 insertions(+), 542 deletions(-) create mode 100644 src/Public/Sessions.ps1 diff --git a/docs/Tutorials/Cookies.md b/docs/Tutorials/Cookies.md index a55dd9fe4..c3ca4f16d 100644 --- a/docs/Tutorials/Cookies.md +++ b/docs/Tutorials/Cookies.md @@ -1,10 +1,10 @@ # Cookies -In Pode you can add/retrieve cookies for the Request/Response of the current web event. Using the cookie functions has to be done within the context of a web event, such as in Routes/Middleware/Authentication/Logging/Endware. +In Pode you can add/retrieve cookies for the Request/Response of the current web request. Using the cookie functions has to be done within the context of a web event, such as within Routes; Middleware; Authentication; Logging; and Endware. ## Adding Cookies -You can add a cookie to the response by using [`Set-PodeCookie`](../../Functions/Cookies/Set-PodeCookie), passing the Name and Value of cookie: +You can add a cookie to the response by using [`Set-PodeCookie`](../../Functions/Cookies/Set-PodeCookie), and passing the Name and Value of cookie: ```powershell Add-PodeRoute -Method Get -Path '/' -ScriptBlock { diff --git a/docs/Tutorials/Middleware/Types/Sessions.md b/docs/Tutorials/Middleware/Types/Sessions.md index a33639ac1..958a7c87c 100644 --- a/docs/Tutorials/Middleware/Types/Sessions.md +++ b/docs/Tutorials/Middleware/Types/Sessions.md @@ -1,21 +1,24 @@ # Sessions -Session Middleware is supported on web requests and responses in the form of signed-cookies/headers and server-side data storage. When configured, the middleware will check for a session-cookie/header on the request; if a cookie/header is not found on the request, or the session is not in storage, then a new session is created and attached to the response. If there is a session, then the appropriate data for that session is loaded from storage. +Session Middleware is supported on web requests and responses in the form of signed a cookie/header and server-side data storage. When configured the middleware will check for a session cookie/header (usually called `pode.sid`) on the request; if a cookie/header is not found on the request, or the session is not in storage, then a new session is created and attached to the response. If there is a session, then the appropriate data for that session is loaded from storage. -The duration of the session-cookie/header can be specified, as well as whether to extend the duration each time on each request. A secret-key to sign sessions can be supplied (default is a random GUID), as well as the ability to specify custom data stores - the default is in-memory, but custom storage could be anything like Redis/MongoDB/etc. +The duration of the session cookie/header can be specified, as well as whether to extend the duration each time on each request. A secret-key to sign sessions can be supplied (default is a random GUID), as well as the ability to specify custom data stores - the default is in-memory, but custom storage could be anything like Redis/MongoDB/etc. !!! note Using sessions via headers is best used with REST APIs and the CLI. It's not advised to use them for normal websites, as browsers don't send back response headers in new requests - unlike cookies. +!!! tip + Sessions are typically used in conjunction with Authentication, but can you use them standalone as well! + ## Usage -To initialise sessions in Pode use [`Enable-PodeSessionMiddleware`](../../../../Functions/Middleware/Enable-PodeSessionMiddleware). This function will configure and automatically create the Middleware needed to enable sessions. By default sessions are set to use cookies, but support is also available for headers. +To initialise sessions in Pode you'll need to call [`Enable-PodeSessionMiddleware`](../../../../Functions/Middleware/Enable-PodeSessionMiddleware). This function will configure and automatically create the Middleware needed to enable sessions. By default sessions are set to use cookies, but support is also available for headers. Sessions are automatically signed using a random GUID. For Pode running on a single server using the default in-memory storage this is OK, however if you're running Pode on multiple servers, or if you're defining a custom storage then a `-Secret` is required - this is so that sessions from different servers, or after a server restart, don't become corrupt and unusable. ### Cookies -The following is an example of how to setup session middleware using cookies. Each session created will expire after 2mins, but the expiry time will be extended each time the session is used: +The following is an example of how to setup session middleware using cookies. The duration of each session is defined as a total number of seconds via the `-Duration` parameter; here we set the duration to 120, so each session created will expire after 2mins, but the expiry time will be extended each time the session is used: ```powershell Start-PodeServer { @@ -27,7 +30,7 @@ The default name of the session cookie is `pode.sid`, but this can be customised ### Headers -Sessions are also supported using headers - useful for CLI requests. The following example will enable sessions use headers instead of cookies: +Sessions are also supported using headers - useful for CLI requests. The following example will enable sessions use headers instead of cookies, and will also set each session created to have a `-Duration` of 120 seconds: ```powershell Start-PodeServer { @@ -43,13 +46,13 @@ The inbuilt SessionId generator used for sessions is a GUID, but you can supply If supplied, the `-Generator` is a scriptblock that must return a valid string. The string itself should be a random unique value, that can be used as a unique session identifier. -Within a route, or middleware, you can get the current authenticated sessionId using [`Get-PodeSessionId`](../../../../Functions/Middleware/Get-PodeSessionId). If there is no session, or the session is not authenticated, then `$null` is returned. This function can also returned the fully signed sessionId as well. If you want the sessionId even if it's not authenticated, then you can supply `-Force` to get the sessionId back. +Within a route, or middleware, you can get the currently authenticated session'd ID using [`Get-PodeSessionId`](../../../../Functions/Middleware/Get-PodeSessionId). If there is no session, or the session is not authenticated, then `$null` is returned. This function can also returned the fully signed sessionId as well. If you want the sessionId even if it's not authenticated, then you can supply `-Force` to get the current SessionId back. ### Strict You can flag sessions as being strict using the `-Strict` switch. Strict sessions will extend the signing process by also using the client's UserAgent and RemoteIPAddress, to help prevent session sharing on different browsers/consoles. -Pode will automatically extend the Secret for signing for you, whether you're using the default GUID, or supplying a specific `-Secret` value. +Pode will automatically extend the Secret used for signing for you, whether you're using the default GUID, or supplying a specific `-Secret` value. ## Storage @@ -72,13 +75,14 @@ $store = New-Object -TypeName psobject # add a Get property for retreiving a session's data by SessionId $store | Add-Member -MemberType NoteProperty -Name Get -Value { param($sessionId) - return (Get-RedisKey -Key $sessionId) + $data = Get-RedisKey -Key $sessionId + return ($data | ConvertFrom-Json -AsHashtable) } # add a Set property to save a session's data $store | Add-Member -MemberType NoteProperty -Name Set -Value { param($sessionId, $data, $expiry) - Set-RedisKey -Key $sessionId -Value $data -TimeToLive $expiry + Set-RedisKey -Key $sessionId -Value ($data | ConvertTo-Json -Compress) -TimeToLive $expiry } # add a Delete property to delete a session's data by SessionId @@ -93,7 +97,7 @@ Enable-PodeSessionMiddleware -Duration 120 -Storage $store -Secret 'schwifty' ## Session Data -To add data to a session you can utilise the `.Session.Data` property within the [web event](../../../WebEvent) object accessible in a Route - or other Middleware. The data will be saved at the end of the route automatically using Endware. When a request is made using the same sessionId, the data is loaded from the store. For example, incrementing some view counter: +To add data to a session you can utilise the `.Session.Data` property within the [web event](../../../WebEvent) object accessible in a Route - or other Middleware. The data will be saved to some storage at the end of the route automatically using Endware. When a request is made using the same SessionId, the data is loaded from the store. For example, incrementing some view counter: ```powershell Add-PodeRoute -Method Get -Path '/' -ScriptBlock { @@ -101,7 +105,7 @@ Add-PodeRoute -Method Get -Path '/' -ScriptBlock { } ``` -You can also use the `$session:` variable scope, which will get/set data on the current session for the name supplied. You can use `$session:` anywhere a `$WebEvent` is available - such as routes, middleware, authentication and endware. The same view counter example above: +You can also use the `$session:` variable scope, which will get/set data on the current session for the name supplied. You can use `$session:` anywhere a `$WebEvent` is available - such as Routes, Middleware, Authentication and Endware. The same view counter example above would now be as follows: ```powershell Add-PodeRoute -Method Get -Path '/' -ScriptBlock { @@ -109,9 +113,75 @@ Add-PodeRoute -Method Get -Path '/' -ScriptBlock { } ``` -`$session:` can only be used in the main scriptblocks of routes, etc. If you attempt to use it in a function of a custom module, it will fail; even if you're using the function in a route. Pode remaps `$session:` on server start, and can only do this to the main scriptblocks supplied to functions such as `Add-PodeRoute`. +A session's data will be automatically saved by Pode at the end of each request, but you can force the data of the current session to be saved by using [`Save-PodeSession`](../../../../Functions/Sessions/Save-PodeSession). + +!!! important + `$session:` can only be used in the main scriptblocks of Routes, etc. If you attempt to use it in a function of a custom module, it will fail; even if you're using the function in a route. Pode remaps `$session:` on server start, and can only do this to the main scriptblocks supplied to functions such as `Add-PodeRoute`. In these scenarios you will have to use `$WebEvent.Session.Data`. + +## Expiry + +When you enable Sessions using [`Enable-PodeSessionMiddleware`](../../../../Functions/Sessions/Enable-PodeSessionMiddleware) you can define the duration of each session created, in seconds, using the `-Duration` parameter. When a session is created its expiry is set to `DateTime.UtcNow + Duration`, and by default a session will automatically expire when the calculated DateTime is reached: + +```powershell +Start-PodeServer { + Enable-PodeSessionMiddleware -Duration 120 +} +``` + +You can tell Pode to reset/extend each session's expiry on each request sent, that uses that SessionId, by passing the `-Extend` switch. When a session's expiry is reset/extended, the DateTime/Duration calculation is re-calculated: + +```powershell +Start-PodeServer { + Enable-PodeSessionMiddleware -Duration 120 -Extend +} +``` + +### Retrieve + +You can retrieve the expiry for the current session by using [`Get-PodeSessionExpiry`](../../../../Functions/Sessions/Get-PodeSessionExpiry). If you use this function without `-Extend` specified originally then this will return the explicit DateTime the current session will expire. However, if you did setup sessions to extend the this function will return the recalculated expiry for the current session on each call: + +```powershell +Start-PodeServer { + Enable-PodeSessionMiddleware -Duration 120 -Extend + + Add-PodeRoute -Method Get -Path '/' -ScriptBlock { + # this will return a DateTime that will always be 2mins in the future + $expiry = Get-PodeSessionExpiry + } +} +``` + +### Terminate + +To terminate the current session you can call [`Remove-PodeSession`](../../../../Functions/Sessions/Remove-PodeSession). Calling this will immediately set the session to expire now - as if somebody had clicked "Log Out". The session's data will be removed, the cookie will be discarded, and any authentication information will be dropped. + +```powershell +Start-PodeServer { + Enable-PodeSessionMiddleware -Duration 120 -Extend + + Add-PodeRoute -Method Get -Path '/logout' -ScriptBlock { + # this will terminate the current session + Remove-PodeSession + } +} +``` + +### Reset + +For any session created when `-Extend` wasn't supplied to [`Enable-PodeSessionMiddleware`](../../../../Functions/Sessions/Enable-PodeSessionMiddleware) will always have a explicit DateTime set for expiring. However, you can reset this expiry date using [`Reset-PodeSessionExpiry`](../../../../Functions/Sessions/Reset-PodeSessionExpiry), and the current session's expiry will be recalculated from now plus the specifed `-Duration`: + +```powershell +Start-PodeServer { + Enable-PodeSessionMiddleware -Duration 120 -Extend + + Add-PodeRoute -Method Get -Path '/' -ScriptBlock { + # this will reset the current session's expiry to be DateTime.Now + 2mins + Reset-PodeSessionExpiry + } +} +``` -### Example +## Example An example of using sessions in a Route to increment a views counter could be done as follows (the counter will continue to increment on each call to the route until the session expires after 2mins): @@ -121,8 +191,8 @@ Start-PodeServer { Enable-PodeSessionMiddleware -Duration 120 Add-PodeRoute -Method Get -Path '/' -ScriptBlock { - $WebEvent.Session.Data.Views++ - Write-PodeJsonResponse -Value @{ 'Views' = $WebEvent.Session.Data.Views } + $session:Views++ + Write-PodeJsonResponse -Value @{ 'Views' = $session:Views } } } ``` diff --git a/examples/views/auth-home.pode b/examples/views/auth-home.pode index 50dca99b4..34c3c01e1 100644 --- a/examples/views/auth-home.pode +++ b/examples/views/auth-home.pode @@ -6,6 +6,9 @@ Hello, $($data.Username)! You have view this page $($data.Views) times! +

+ Your session will expire on $($data.Expiry) +
diff --git a/examples/web-auth-form.ps1 b/examples/web-auth-form.ps1 index 5f01df817..6dda283a2 100644 --- a/examples/web-auth-form.ps1 +++ b/examples/web-auth-form.ps1 @@ -56,6 +56,7 @@ Start-PodeServer -Threads 2 { Write-PodeViewResponse -Path 'auth-home' -Data @{ Username = $WebEvent.Auth.User.Name Views = $session:Views + Expiry = Get-PodeSessionExpiry } } diff --git a/src/Pode.psd1 b/src/Pode.psd1 index 25f117a4b..f90ea2e43 100644 --- a/src/Pode.psd1 +++ b/src/Pode.psd1 @@ -185,15 +185,20 @@ 'Clear-PodeMiddleware', 'Add-PodeAccessRule', 'Add-PodeLimitRule', - 'Enable-PodeSessionMiddleware', 'New-PodeCsrfToken', 'Get-PodeCsrfMiddleware', 'Initialize-PodeCsrf', 'Enable-PodeCsrfMiddleware', + 'Use-PodeMiddleware', + + # sessions + 'Enable-PodeSessionMiddleware', 'Remove-PodeSession', 'Save-PodeSession', 'Get-PodeSessionId', - 'Use-PodeMiddleware', + 'Reset-PodeSessionExpiry', + 'Get-PodeSessionDuration', + 'Get-PodeSessionExpiry', # auth 'New-PodeAuthScheme', diff --git a/src/Private/Authentication.ps1 b/src/Private/Authentication.ps1 index da4a2ff67..957b21e96 100644 --- a/src/Private/Authentication.ps1 +++ b/src/Private/Authentication.ps1 @@ -1070,7 +1070,7 @@ function Get-PodeAuthMiddlewareScript # route options for using sessions $sessionless = $auth.Sessionless $usingSessions = (Test-PodeSessionsInUse) - $useHeaders = [bool]($WebEvent.Session.Properties.UseHeaders) + $useHeaders = $PodeContext.Server.Sessions.Info.UseHeaders $loginRoute = $opts.Login # check for logout command @@ -1097,7 +1097,7 @@ function Get-PodeAuthMiddlewareScript # if we're allowing anon access, and using sessions, then stop here - as a session will be created from a login route for auth'ing users if ($opts.Anon) { if (!(Test-PodeIsEmpty $WebEvent.Session.Data.Auth)) { - Revoke-PodeSession -Session $WebEvent.Session + Revoke-PodeSession } return $true @@ -1107,7 +1107,7 @@ function Get-PodeAuthMiddlewareScript # check if the login flag is set, in which case just return and load a login get-page (allowing anon access) if ($loginRoute -and !$useHeaders -and ($WebEvent.Method -ieq 'get')) { if (!(Test-PodeIsEmpty $WebEvent.Session.Data.Auth)) { - Revoke-PodeSession -Session $WebEvent.Session + Revoke-PodeSession } return $true @@ -1267,8 +1267,8 @@ function Remove-PodeAuthSession $WebEvent.Session.Data.Remove('Auth') } - # Delete the session (remove from store, blank it, and remove from Response) - Revoke-PodeSession -Session $WebEvent.Session + # Delete the current session (remove from store, blank it, and remove from Response) + Revoke-PodeSession } function Set-PodeAuthStatus @@ -1327,7 +1327,7 @@ function Set-PodeAuthStatus # check if we have a failure url redirect if (!$NoFailureRedirect -and ![string]::IsNullOrWhiteSpace($Failure.Url)) { if ($Success.UseOrigin -and ($WebEvent.Method -ieq 'get')) { - Set-PodeCookie -Name 'pode.redirecturl' -Value $WebEvent.Request.Url.PathAndQuery + $null = Set-PodeCookie -Name 'pode.redirecturl' -Value $WebEvent.Request.Url.PathAndQuery } Move-PodeResponseUrl -Url $Failure.Url @@ -1342,7 +1342,7 @@ function Set-PodeAuthStatus # if no statuscode, success, so check if we have a success url redirect (but only for auto-login routes) if ((!$NoSuccessRedirect -or $LoginRoute) -and ![string]::IsNullOrWhiteSpace($Success.Url)) { $url = $Success.Url - if ($Success.UseOrigin -and ($WebEvent.Method -ieq 'get')) { + if ($Success.UseOrigin) { $tmpUrl = Get-PodeCookieValue -Name 'pode.redirecturl' Remove-PodeCookie -Name 'pode.redirecturl' diff --git a/src/Private/Cookies.ps1 b/src/Private/Cookies.ps1 index 14a184114..7917cefae 100644 --- a/src/Private/Cookies.ps1 +++ b/src/Private/Cookies.ps1 @@ -1,6 +1,6 @@ function ConvertTo-PodeCookie { - param ( + param( [Parameter()] [System.Net.Cookie] $Cookie @@ -26,45 +26,56 @@ function ConvertTo-PodeCookie function ConvertTo-PodeCookieString { - param ( + param( [Parameter(Mandatory=$true)] $Cookie ) - $str = "$($Cookie.Name)=$($Cookie.Value)" + try { + $builder = [System.Text.StringBuilder]::new() + $null = $builder.Append($Cookie.Name) + $null = $builder.Append('=') + $null = $builder.Append($Cookie.Value) - if ($Cookie.Discard) { - $str += '; Discard' - } + if ($Cookie.Discard) { + $null = $builder.Append('; Discard') + } - if ($Cookie.HttpOnly) { - $str += '; HttpOnly' - } + if ($Cookie.HttpOnly) { + $null = $builder.Append('; HttpOnly') + } - if ($Cookie.Secure) { - $str += '; Secure' - } + if ($Cookie.Secure) { + $null = $builder.Append('; Secure') + } - if (![string]::IsNullOrWhiteSpace($Cookie.Domain)) { - $str += "; Domain=$($Cookie.Domain)" - } + if (![string]::IsNullOrEmpty($Cookie.Domain)) { + $null = $builder.Append('; Domain=') + $null = $builder.Append($Cookie.Domain) + } - if (![string]::IsNullOrWhiteSpace($Cookie.Path)) { - $str += "; Path=$($Cookie.Path)" - } + if (![string]::IsNullOrEmpty($Cookie.Path)) { + $null = $builder.Append('; Path=') + $null = $builder.Append($Cookie.Path) + } - if ($null -ne $Cookie.Expires -and $Cookie.Expires -ne [datetime]::MinValue) { - $secs = ($Cookie.Expires.ToLocalTime() - [datetime]::Now).TotalSeconds - if ($secs -lt 0) { - $secs = 0 + if (($null -ne $Cookie.Expires) -and ($Cookie.Expires.Ticks -ne 0)) { + $secs = ($Cookie.Expires.Subtract([datetime]::UtcNow)).TotalSeconds + if ($secs -lt 0) { + $secs = 0 + } + + $null = $builder.Append('; Max-Age=') + $null = $builder.Append($secs) } - $str += "; Max-Age=$($secs)" - } + if ($builder.Length -le 1) { + return $null + } - if ($str -eq '=') { - return $null + return $builder.ToString() + } + finally { + $builder = $null } - - return $str } \ No newline at end of file diff --git a/src/Private/Middleware.ps1 b/src/Private/Middleware.ps1 index e76dbc663..64a0f7466 100644 --- a/src/Private/Middleware.ps1 +++ b/src/Private/Middleware.ps1 @@ -325,6 +325,8 @@ function Get-PodeCookieMiddleware return $true } + $h_cookie | Out-Default + # parse the cookies from the header $cookies = @($h_cookie -split '; ') $WebEvent.Cookies = @{} diff --git a/src/Private/Sessions.ps1 b/src/Private/Sessions.ps1 index 42a65d981..61ca339f4 100644 --- a/src/Private/Sessions.ps1 +++ b/src/Private/Sessions.ps1 @@ -1,16 +1,12 @@ function New-PodeSession { - $sid = @{ + return @{ Name = $PodeContext.Server.Sessions.Name Id = (Invoke-PodeScriptBlock -ScriptBlock $PodeContext.Server.Sessions.GenerateId -Return) - Properties = $PodeContext.Server.Sessions.Info + Extend = $PodeContext.Server.Sessions.Info.Extend + TimeStamp = [datetime]::UtcNow Data = @{} } - - Set-PodeSessionDataHash -Session $sid - - $sid.Properties.TimeStamp = [DateTime]::UtcNow - return $sid } function ConvertTo-PodeSessionStrictSecret @@ -26,70 +22,54 @@ function ConvertTo-PodeSessionStrictSecret function Set-PodeSession { - param ( - [Parameter(Mandatory=$true)] - [ValidateNotNull()] - [hashtable] - $Session - ) + if ($null -eq $WebEvent.Session) { + throw 'there is no session available to set on the response' + } - $secure = [bool]($Session.Properties.Secure) - $strict = [bool]($Session.Properties.Strict) - $discard = [bool]($Session.Properties.Discard) - $httpOnly = [bool]($Session.Properties.HttpOnly) - $useHeaders = [bool]($Session.Properties.UseHeaders) $secret = $PodeContext.Server.Sessions.Secret # covert secret to strict mode - if ($strict) { + if ($PodeContext.Server.Sessions.Info.Strict) { $secret = ConvertTo-PodeSessionStrictSecret -Secret $secret } # set session on header - if ($useHeaders) { - Set-PodeHeader -Name $Session.Name -Value $Session.Id -Secret $secret + if ($PodeContext.Server.Sessions.Info.UseHeaders) { + Set-PodeHeader -Name $WebEvent.Session.Name -Value $WebEvent.Session.Id -Secret $secret } # set session as cookie else { $null = Set-PodeCookie ` - -Name $Session.Name ` - -Value $Session.Id ` + -Name $WebEvent.Session.Name ` + -Value $WebEvent.Session.Id ` -Secret $secret ` - -ExpiryDate (Get-PodeSessionExpiry -Session $Session) ` - -HttpOnly:$httpOnly ` - -Discard:$discard ` - -Secure:$secure + -ExpiryDate (Get-PodeSessionExpiry) ` + -HttpOnly:$PodeContext.Server.Sessions.Info.HttpOnly ` + -Secure:$PodeContext.Server.Sessions.Info.Secure } } function Get-PodeSession { - param ( - [Parameter(Mandatory=$true)] - [hashtable] - $Session - ) - - $secret = $Session.Secret - $timestamp = [datetime]::UtcNow + $secret = $PodeContext.Server.Sessions.Secret $value = $null - $name = $Session.Name + $name = $PodeContext.Server.Sessions.Name # covert secret to strict mode - if ($Session.Info.Strict) { + if ($PodeContext.Server.Sessions.Info.Strict) { $secret = ConvertTo-PodeSessionStrictSecret -Secret $secret } # session from header - if ($Session.Info.UseHeaders) { + if ($PodeContext.Server.Sessions.Info.UseHeaders) { # check that the header is validly signed - if (!(Test-PodeHeaderSigned -Name $Session.Name -Secret $secret)) { + if (!(Test-PodeHeaderSigned -Name $PodeContext.Server.Sessions.Name -Secret $secret)) { return $null } # get the header from the request - $value = Get-PodeHeader -Name $Session.Name -Secret $secret + $value = Get-PodeHeader -Name $PodeContext.Server.Sessions.Name -Secret $secret if ([string]::IsNullOrWhiteSpace($value)) { return $null } @@ -98,12 +78,12 @@ function Get-PodeSession # session from cookie else { # check that the cookie is validly signed - if (!(Test-PodeCookieSigned -Name $Session.Name -Secret $secret)) { + if (!(Test-PodeCookieSigned -Name $PodeContext.Server.Sessions.Name -Secret $secret)) { return $null } # get the cookie from the request - $cookie = Get-PodeCookie -Name $Session.Name -Secret $secret + $cookie = Get-PodeCookie -Name $PodeContext.Server.Sessions.Name -Secret $secret if ([string]::IsNullOrWhiteSpace($cookie)) { return $null } @@ -111,144 +91,115 @@ function Get-PodeSession # get details from cookie $name = $cookie.Name $value = $cookie.Value - - $timestamp = $cookie.TimeStamp - if ($null -ne $timestamp) { - $timestamp = $timestamp.ToUniversalTime() - } } # generate the session data $data = @{ Name = $name Id = $value - Properties = $Session.Info + Extend = $PodeContext.Server.Sessions.Info.Extend + TimeStamp = $null Data = @{} } - $data.Properties.TimeStamp = $timeStamp return $data } function Revoke-PodeSession { - param ( - [Parameter(Mandatory=$true)] - [ValidateNotNull()] - [hashtable] - $Session - ) + # do nothing if no current session + if ($null -eq $WebEvent.Session) { + return + } # remove from cookie - if (!$Session.Properties.UseHeaders) { - Remove-PodeCookie -Name $Session.Name + if (!$PodeContext.Server.Sessions.Info.UseHeaders) { + Remove-PodeCookie -Name $WebEvent.Session.Name } # remove session from store - Invoke-PodeScriptBlock -ScriptBlock $Session.Delete -Arguments @($Session) -Splat + Invoke-PodeScriptBlock -ScriptBlock $WebEvent.Session.Delete # blank the session - $Session.Clear() + $WebEvent.Session.Clear() } function Set-PodeSessionDataHash { - param ( - [Parameter(Mandatory=$true)] - [ValidateNotNull()] - [hashtable] - $Session - ) + if ($null -eq $WebEvent.Session) { + throw 'No session available to calculate data hash' + } - if (($null -eq $Session.Data) -or ($Session.Data.Count -eq 0)) { - $Session.Data = @{} + if (($null -eq $WebEvent.Session.Data) -or ($WebEvent.Session.Data.Count -eq 0)) { + $WebEvent.Session.Data = @{} } - $Session.DataHash = (Invoke-PodeSHA256Hash -Value (ConvertTo-Json -InputObject $Session.Data.Clone() -Depth 10 -Compress)) + $WebEvent.Session.DataHash = (Invoke-PodeSHA256Hash -Value (ConvertTo-Json -InputObject $WebEvent.Session.Data.Clone() -Depth 10 -Compress)) } function Test-PodeSessionDataHash { - param ( - [Parameter(Mandatory=$true)] - [ValidateNotNull()] - [hashtable] - $Session - ) - - if ([string]::IsNullOrWhiteSpace($Session.DataHash)) { + if ($null -eq $WebEvent.Session) { return $false } - if (($null -eq $Session.Data) -or ($Session.Data.Count -eq 0)) { - $Session.Data = @{} - } - - $hash = (Invoke-PodeSHA256Hash -Value (ConvertTo-Json -InputObject $Session.Data -Depth 10 -Compress)) - return ($Session.DataHash -eq $hash) -} - -function Get-PodeSessionExpiry -{ - param ( - [Parameter(Mandatory=$true)] - [ValidateNotNull()] - [hashtable] - $Session - ) - - if ($null -eq $Session.Properties) { - return [DateTime]::MinValue + if ([string]::IsNullOrWhiteSpace($WebEvent.Session.DataHash)) { + return $false } - $expiry = [DateTime]::UtcNow - if (!([bool]$Session.Properties.Extend)) { - $expiry = $Session.Properties.TimeStamp - if ($null -ne $expiry) { - $expiry = $expiry.ToUniversalTime() - } + if (($null -eq $WebEvent.Session.Data) -or ($WebEvent.Session.Data.Count -eq 0)) { + $WebEvent.Session.Data = @{} } - $expiry = $expiry.AddSeconds($Session.Properties.Duration) - return $expiry + $hash = (Invoke-PodeSHA256Hash -Value (ConvertTo-Json -InputObject $WebEvent.Session.Data -Depth 10 -Compress)) + return ($WebEvent.Session.DataHash -eq $hash) } function Set-PodeSessionHelpers { - param ( - [Parameter(Mandatory=$true)] - [ValidateNotNull()] - [hashtable] - $Session - ) + if ($null -eq $WebEvent.Session) { + throw 'No session available to set helpers' + } # force save a session's data to the store - $Session | Add-Member -MemberType NoteProperty -Name Save -Value { - param($session, $check) + $WebEvent.Session | Add-Member -MemberType NoteProperty -Name Save -Value { + param($check) + + # the current session + $session = $WebEvent.Session # do nothing if session has no ID if ([string]::IsNullOrWhiteSpace($session.Id)) { return } - # only save if check and hashes different, but not if extending expiry - if (!$session.Properties.Extend -and $check -and (Test-PodeSessionDataHash -Session $session)) { + # only save if check and hashes different, but not if extending expiry or updated + if (!$session.Extend -and $check -and (Test-PodeSessionDataHash)) { return } # generate the expiry - $expiry = (Get-PodeSessionExpiry -Session $session) + $expiry = Get-PodeSessionExpiry + + # the data to save - which will be the data, some extra metadata + $data = @{ + Metadata = @{ + TimeStamp = $session.TimeStamp + } + Data = $session.Data + } # save session data to store - Invoke-PodeScriptBlock -ScriptBlock $PodeContext.Server.Sessions.Store.Set -Arguments @($session.Id, $session.Data, $expiry) -Splat + Invoke-PodeScriptBlock -ScriptBlock $PodeContext.Server.Sessions.Store.Set -Arguments @($session.Id, $data, $expiry) -Splat # update session's data hash - Set-PodeSessionDataHash -Session $session + Set-PodeSessionDataHash } # delete the current session - $Session | Add-Member -MemberType NoteProperty -Name Delete -Value { - param($session) + $WebEvent.Session | Add-Member -MemberType NoteProperty -Name Delete -Value { + # the current session + $session = $WebEvent.Session # remove data from store Invoke-PodeScriptBlock -ScriptBlock $PodeContext.Server.Sessions.Store.Delete -Arguments $session.Id @@ -317,7 +268,7 @@ function Set-PodeSessionInMemClearDown $now = [DateTime]::UtcNow foreach ($key in $store.Memory.Keys) { if ($store.Memory[$key].Expiry -lt $now) { - $store.Memory.Remove($key) + $null = $store.Memory.Remove($key) } } } @@ -354,33 +305,42 @@ function Get-PodeSessionMiddleware try { - # get the session from cookie/header - $WebEvent.Session = Get-PodeSession -Session $PodeContext.Server.Sessions + # retrieve the current session from cookie/header + $WebEvent.Session = Get-PodeSession # if no session found, create a new one on the current web event if (!$WebEvent.Session) { - $WebEvent.Session = (New-PodeSession) + $WebEvent.Session = New-PodeSession $new = $true } - # get the session's data + # get the session's data from store elseif ($null -ne ($data = (Get-PodeSessionData -SessionId $WebEvent.Session.Id))) { - $WebEvent.Session.Data = $data - Set-PodeSessionDataHash -Session $WebEvent.Session + if ($null -eq $data.Metadata) { + $WebEvent.Session.Data = $data + $WebEvent.Session.TimeStamp = [datetime]::UtcNow + } + else { + $WebEvent.Session.Data = $data.Data + $WebEvent.Session.TimeStamp = $data.Metadata.TimeStamp + } } # session not in store, create a new one else { - $WebEvent.Session = (New-PodeSession) + $WebEvent.Session = New-PodeSession $new = $true } - # add helper methods to session - Set-PodeSessionHelpers -Session $WebEvent.Session + # set data hash + Set-PodeSessionDataHash + + # add helper methods to current session + Set-PodeSessionHelpers # add session to response if it's new or extendible - if ($new -or $WebEvent.Session.Properties.Extend) { - Set-PodeSession -Session $WebEvent.Session + if ($new -or $WebEvent.Session.Extend) { + Set-PodeSession } # assign endware for session to set cookie/header diff --git a/src/Public/Cookies.ps1 b/src/Public/Cookies.ps1 index cf1a1f0c0..edb51c37d 100644 --- a/src/Public/Cookies.ps1 +++ b/src/Public/Cookies.ps1 @@ -42,7 +42,7 @@ function Set-PodeCookie { [CmdletBinding(DefaultParameterSetName='Duration')] [OutputType([hashtable])] - param ( + param( [Parameter(Mandatory=$true)] [string] $Name, @@ -85,7 +85,11 @@ function Set-PodeCookie $cookie.HttpOnly = $HttpOnly $cookie.Path = '/' - if (!(Test-PodeIsEmpty $ExpiryDate)) { + if ($null -ne $ExpiryDate) { + if ($ExpiryDate.Kind -eq [System.DateTimeKind]::Local) { + $ExpiryDate = $ExpiryDate.ToUniversalTime() + } + $cookie.Expires = $ExpiryDate } elseif ($Duration -gt 0) { @@ -124,7 +128,7 @@ function Get-PodeCookie { [CmdletBinding()] [OutputType([hashtable])] - param ( + param( [Parameter(Mandatory=$true)] [string] $Name, @@ -180,7 +184,7 @@ Get-PodeCookieValue -Name 'Views' -Secret 'hunter2' function Get-PodeCookieValue { [CmdletBinding()] - param ( + param( [Parameter(Mandatory=$true)] [string] $Name, @@ -215,7 +219,7 @@ function Test-PodeCookie { [CmdletBinding()] [OutputType([bool])] - param ( + param( [Parameter(Mandatory=$true)] [string] $Name @@ -241,7 +245,7 @@ Remove-PodeCookie -Name 'Views' function Remove-PodeCookie { [CmdletBinding()] - param ( + param( [Parameter(Mandatory=$true)] [string] $Name @@ -283,7 +287,7 @@ function Test-PodeCookieSigned { [CmdletBinding()] [OutputType([bool])] - param ( + param( [Parameter(Mandatory=$true)] [string] $Name, @@ -328,7 +332,7 @@ function Update-PodeCookieExpiry { [CmdletBinding(DefaultParameterSetName='Duration')] [OutputType([hashtable])] - param ( + param( [Parameter(Mandatory=$true)] [string] $Name, @@ -349,7 +353,11 @@ function Update-PodeCookieExpiry } # extends the expiry on the cookie - if (!(Test-PodeIsEmpty $ExpiryDate)) { + if ($null -ne $ExpiryDate) { + if ($ExpiryDate.Kind -eq [System.DateTimeKind]::Local) { + $ExpiryDate = $ExpiryDate.ToUniversalTime() + } + $cookie.Expires = $ExpiryDate } elseif ($Duration -gt 0) { @@ -389,7 +397,7 @@ Set-PodeCookieSecret -Value 'hunter2' -Global function Set-PodeCookieSecret { [CmdletBinding()] - param ( + param( [Parameter(Mandatory=$true, ParameterSetName='General')] [string] $Name, @@ -433,7 +441,7 @@ function Get-PodeCookieSecret { [CmdletBinding()] [OutputType([string])] - param ( + param( [Parameter(Mandatory=$true, ParameterSetName='General')] [string] $Name, diff --git a/src/Public/Middleware.ps1 b/src/Public/Middleware.ps1 index 963aa0a20..747e0945f 100644 --- a/src/Public/Middleware.ps1 +++ b/src/Public/Middleware.ps1 @@ -130,276 +130,6 @@ function Add-PodeLimitRule } } -<# -.SYNOPSIS -Enables Middleware for creating, retrieving and using Sessions within Pode. - -.DESCRIPTION -Enables Middleware for creating, retrieving and using Sessions within Pode; with support for defining Session duration, and custom Storage. -If you're storing sessions outside of Pode, you must supply a Secret value so sessions aren't corrupted. - -.PARAMETER Secret -An optional Secret to use when signing Sessions (Default: random GUID). - -.PARAMETER Name -The name of the cookie/header used for the Session. - -.PARAMETER Duration -The duration a Session should last for, before being expired. - -.PARAMETER Generator -A custom ScriptBlock to generate a random unique SessionId. The value returned must be a String. - -.PARAMETER Storage -A custom PSObject that defines methods for Delete, Get, and Set. This allow you to store Sessions in custom Storage such as Redis. A Secret is required. - -.PARAMETER Extend -If supplied, the Sessions will have their durations extended on each successful Request. - -.PARAMETER HttpOnly -If supplied, the Session cookie will only be accessible to browsers. - -.PARAMETER Secure -If supplied, the Session cookie will only be accessible over HTTPS Requests. - -.PARAMETER Strict -If supplied, the Secret will be extended using the client request's UserAgent and RemoteIPAddress. - -.PARAMETER UseHeaders -If supplied, Sessions will be sent back in a header on the Response with the Name supplied. - -.EXAMPLE -Enable-PodeSessionMiddleware -Duration 120 - -.EXAMPLE -Enable-PodeSessionMiddleware -Duration 120 -Extend -Generator { return [System.IO.Path]::GetRandomFileName() } - -.EXAMPLE -Enable-PodeSessionMiddleware -Secret 'schwifty' -Duration 120 -UseHeaders -Strict -#> -function Enable-PodeSessionMiddleware -{ - [CmdletBinding(DefaultParameterSetName='Cookies')] - param ( - [Parameter()] - [string] - $Secret, - - [Parameter()] - [ValidateNotNullOrEmpty()] - [string] - $Name = 'pode.sid', - - [Parameter()] - [ValidateScript({ - if ($_ -lt 0) { - throw "Duration must be 0 or greater, but got: $($_)s" - } - - return $true - })] - [int] - $Duration = 0, - - [Parameter()] - [scriptblock] - $Generator, - - [Parameter()] - [psobject] - $Storage, - - [switch] - $Extend, - - [Parameter(ParameterSetName='Cookies')] - [switch] - $HttpOnly, - - [Parameter(ParameterSetName='Cookies')] - [switch] - $Secure, - - [switch] - $Strict, - - [Parameter(ParameterSetName='Headers')] - [switch] - $UseHeaders - ) - - # check that session logic hasn't already been initialised - if (Test-PodeSessionsConfigured) { - throw 'Session Middleware has already been intialised' - } - - # ensure the override store has the required methods - if (!(Test-PodeIsEmpty $Storage)) { - $members = @($Storage | Get-Member | Select-Object -ExpandProperty Name) - @('delete', 'get', 'set') | ForEach-Object { - if ($members -inotcontains $_) { - throw "Custom session storage does not implement the required '$($_)()' method" - } - } - } - - # verify the secret, set to guid if not supplied, or error if none and we have a storage - if ([string]::IsNullOrEmpty($Secret)) { - if (!(Test-PodeIsEmpty $Storage)) { - throw "A Secret is required when using custom session storage" - } - - $Secret = New-PodeGuid -Secure - } - - # if no custom storage, use the inmem one - if (Test-PodeIsEmpty $Storage) { - $Storage = (Get-PodeSessionInMemStore) - Set-PodeSessionInMemClearDown - } - - # set options against server context - $PodeContext.Server.Sessions = @{ - Name = $Name - Secret = $Secret - GenerateId = (Protect-PodeValue -Value $Generator -Default { return (New-PodeGuid) }) - Store = $Storage - Info = @{ - Duration = $Duration - Extend = $Extend - Secure = $Secure - Strict = $Strict - HttpOnly = $HttpOnly - UseHeaders = $UseHeaders - } - } - - # return scriptblock for the session middleware - $script = Get-PodeSessionMiddleware - (New-PodeMiddleware -ScriptBlock $script) | Add-PodeMiddleware -Name '__pode_mw_sessions__' -} - -<# -.SYNOPSIS -Remove the current Session, logging it out. - -.DESCRIPTION -Remove the current Session, logging it out. This will remove the session from Storage, and Cookies. - -.EXAMPLE -Remove-PodeSession -#> -function Remove-PodeSession -{ - [CmdletBinding()] - param() - - # if sessions haven't been setup, error - if (!(Test-PodeSessionsConfigured)) { - throw 'Sessions have not been configured' - } - - # error if session is null - if ($null -eq $WebEvent.Session) { - return - } - - # remove the session, and from auth and cookies - Remove-PodeAuthSession -} - -<# -.SYNOPSIS -Saves the current Session's data. - -.DESCRIPTION -Saves the current Session's data. - -.PARAMETER Force -If supplied, the data will be saved even if nothing has changed. - -.EXAMPLE -Save-PodeSession -Force -#> -function Save-PodeSession -{ - [CmdletBinding()] - param( - [switch] - $Force - ) - - # if sessions haven't been setup, error - if (!(Test-PodeSessionsConfigured)) { - throw 'Sessions have not been configured' - } - - # error if session is null - if ($null -eq $WebEvent.Session) { - throw 'There is no session available to save' - } - - # if auth is in use, then assign to session store - if (!(Test-PodeIsEmpty $WebEvent.Auth) -and $WebEvent.Auth.Store) { - $WebEvent.Session.Data.Auth = $WebEvent.Auth - } - - # save the session - Invoke-PodeScriptBlock -ScriptBlock $WebEvent.Session.Save -Arguments @($WebEvent.Session, $Force) -Splat -} - -<# -.SYNOPSIS -Returns the currently authenticated SessionId. - -.DESCRIPTION -Returns the currently authenticated SessionId. If there's no session, or it's not authenticated, then null is returned instead. -You can also have the SessionId returned as signed as well. - -.PARAMETER Signed -If supplied, the returned SessionId will also be signed. - -.PARAMETER Force -If supplied, the sessionId will be returned regardless of authentication. - -.EXAMPLE -$sessionId = Get-PodeSessionId -#> -function Get-PodeSessionId -{ - [CmdletBinding()] - param( - [switch] - $Signed, - - [switch] - $Force - ) - - $sessionId = $null - - # only return session if authenticated, or force passed - if ($Force -or (!(Test-PodeIsEmpty $WebEvent.Session.Data.Auth.User) -and $WebEvent.Session.Data.Auth.IsAuthenticated)) { - $sessionId = $WebEvent.Session.Id - - # do they want the session signed? - if ($Signed) { - $strict = $PodeContext.Server.Sessions.Info.Strict - $secret = $PodeContext.Server.Sessions.Secret - - # covert secret to strict mode - if ($strict) { - $secret = ConvertTo-PodeSessionStrictSecret -Secret $secret - } - - # sign the value if we have a secret - $sessionId = (Invoke-PodeValueSign -Value $sessionId -Secret $secret) - } - } - - return $sessionId -} - <# .SYNOPSIS Creates and returns a new secure token for use with CSRF. diff --git a/src/Public/Sessions.ps1 b/src/Public/Sessions.ps1 new file mode 100644 index 000000000..5190b889b --- /dev/null +++ b/src/Public/Sessions.ps1 @@ -0,0 +1,362 @@ +<# +.SYNOPSIS +Enables Middleware for creating, retrieving and using Sessions within Pode. + +.DESCRIPTION +Enables Middleware for creating, retrieving and using Sessions within Pode; with support for defining Session duration, and custom Storage. +If you're storing sessions outside of Pode, you must supply a Secret value so sessions aren't corrupted. + +.PARAMETER Secret +An optional Secret to use when signing Sessions (Default: random GUID). + +.PARAMETER Name +The name of the cookie/header used for the Session. + +.PARAMETER Duration +The duration a Session should last for, before being expired. + +.PARAMETER Generator +A custom ScriptBlock to generate a random unique SessionId. The value returned must be a String. + +.PARAMETER Storage +A custom PSObject that defines methods for Delete, Get, and Set. This allow you to store Sessions in custom Storage such as Redis. A Secret is required. + +.PARAMETER Extend +If supplied, the Sessions will have their durations extended on each successful Request. + +.PARAMETER HttpOnly +If supplied, the Session cookie will only be accessible to browsers. + +.PARAMETER Secure +If supplied, the Session cookie will only be accessible over HTTPS Requests. + +.PARAMETER Strict +If supplied, the Secret will be extended using the client request's UserAgent and RemoteIPAddress. + +.PARAMETER UseHeaders +If supplied, Sessions will be sent back in a header on the Response with the Name supplied. + +.EXAMPLE +Enable-PodeSessionMiddleware -Duration 120 + +.EXAMPLE +Enable-PodeSessionMiddleware -Duration 120 -Extend -Generator { return [System.IO.Path]::GetRandomFileName() } + +.EXAMPLE +Enable-PodeSessionMiddleware -Secret 'schwifty' -Duration 120 -UseHeaders -Strict +#> +function Enable-PodeSessionMiddleware +{ + [CmdletBinding(DefaultParameterSetName='Cookies')] + param ( + [Parameter()] + [string] + $Secret, + + [Parameter()] + [ValidateNotNullOrEmpty()] + [string] + $Name = 'pode.sid', + + [Parameter()] + [ValidateScript({ + if ($_ -lt 0) { + throw "Duration must be 0 or greater, but got: $($_)s" + } + + return $true + })] + [int] + $Duration = 0, + + [Parameter()] + [scriptblock] + $Generator, + + [Parameter()] + [psobject] + $Storage, + + [switch] + $Extend, + + [Parameter(ParameterSetName='Cookies')] + [switch] + $HttpOnly, + + [Parameter(ParameterSetName='Cookies')] + [switch] + $Secure, + + [switch] + $Strict, + + [Parameter(ParameterSetName='Headers')] + [switch] + $UseHeaders + ) + + # check that session logic hasn't already been initialised + if (Test-PodeSessionsConfigured) { + throw 'Session Middleware has already been intialised' + } + + # ensure the override store has the required methods + if (!(Test-PodeIsEmpty $Storage)) { + $members = @($Storage | Get-Member | Select-Object -ExpandProperty Name) + @('delete', 'get', 'set') | ForEach-Object { + if ($members -inotcontains $_) { + throw "Custom session storage does not implement the required '$($_)()' method" + } + } + } + + # verify the secret, set to guid if not supplied, or error if none and we have a storage + if ([string]::IsNullOrEmpty($Secret)) { + if (!(Test-PodeIsEmpty $Storage)) { + throw "A Secret is required when using custom session storage" + } + + $Secret = New-PodeGuid -Secure + } + + # if no custom storage, use the inmem one + if (Test-PodeIsEmpty $Storage) { + $Storage = (Get-PodeSessionInMemStore) + Set-PodeSessionInMemClearDown + } + + # set options against server context + $PodeContext.Server.Sessions = @{ + Name = $Name + Secret = $Secret + GenerateId = (Protect-PodeValue -Value $Generator -Default { return (New-PodeGuid) }) + Store = $Storage + Info = @{ + Duration = $Duration + Extend = $Extend.IsPresent + Secure = $Secure.IsPresent + Strict = $Strict.IsPresent + HttpOnly = $HttpOnly.IsPresent + UseHeaders = $UseHeaders.IsPresent + } + } + + # return scriptblock for the session middleware + $script = Get-PodeSessionMiddleware + (New-PodeMiddleware -ScriptBlock $script) | Add-PodeMiddleware -Name '__pode_mw_sessions__' +} + +<# +.SYNOPSIS +Remove the current Session, logging it out. + +.DESCRIPTION +Remove the current Session, logging it out. This will remove the session from Storage, and Cookies. + +.EXAMPLE +Remove-PodeSession +#> +function Remove-PodeSession +{ + [CmdletBinding()] + param() + + # if sessions haven't been setup, error + if (!(Test-PodeSessionsConfigured)) { + throw 'Sessions have not been configured' + } + + # do nothin if session is null + if ($null -eq $WebEvent.Session) { + return + } + + # remove the session, and from auth and cookies + Remove-PodeAuthSession +} + +<# +.SYNOPSIS +Saves the current Session's data. + +.DESCRIPTION +Saves the current Session's data. + +.PARAMETER Force +If supplied, the data will be saved even if nothing has changed. + +.EXAMPLE +Save-PodeSession -Force +#> +function Save-PodeSession +{ + [CmdletBinding()] + param( + [switch] + $Force + ) + + # if sessions haven't been setup, error + if (!(Test-PodeSessionsConfigured)) { + throw 'Sessions have not been configured' + } + + # error if session is null + if ($null -eq $WebEvent.Session) { + throw 'There is no session available to save' + } + + # if auth is in use, then assign to session store + if (!(Test-PodeIsEmpty $WebEvent.Auth) -and $WebEvent.Auth.Store) { + $WebEvent.Session.Data.Auth = $WebEvent.Auth + } + + # save the session + Invoke-PodeScriptBlock -ScriptBlock $WebEvent.Session.Save -Arguments @($Force.IsPresent) -Splat +} + +<# +.SYNOPSIS +Returns the currently authenticated SessionId. + +.DESCRIPTION +Returns the currently authenticated SessionId. If there's no session, or it's not authenticated, then null is returned instead. +You can also have the SessionId returned as signed as well. + +.PARAMETER Signed +If supplied, the returned SessionId will also be signed. + +.PARAMETER Force +If supplied, the sessionId will be returned regardless of authentication. + +.EXAMPLE +$sessionId = Get-PodeSessionId +#> +function Get-PodeSessionId +{ + [CmdletBinding()] + param( + [switch] + $Signed, + + [switch] + $Force + ) + + $sessionId = $null + + # do nothing if not authenticated, or force passed + if (!$Force -and ((Test-PodeIsEmpty $WebEvent.Session.Data.Auth.User) -or !$WebEvent.Session.Data.Auth.IsAuthenticated)) { + return $sessionId + } + + # get the sessionId + $sessionId = $WebEvent.Session.Id + + # do they want the session signed? + if ($Signed) { + $strict = $PodeContext.Server.Sessions.Info.Strict + $secret = $PodeContext.Server.Sessions.Secret + + # covert secret to strict mode + if ($strict) { + $secret = ConvertTo-PodeSessionStrictSecret -Secret $secret + } + + # sign the value if we have a secret + $sessionId = (Invoke-PodeValueSign -Value $sessionId -Secret $secret) + } + + # return the ID + return $sessionId +} + +<# +.SYNOPSIS +Resets the current Session's expiry date. + +.DESCRIPTION +Resets the current Session's expiry date, to be from the current time plus the defined Session duration. + +.EXAMPLE +Reset-PodeSessionExpiry +#> +function Reset-PodeSessionExpiry +{ + [CmdletBinding()] + param() + + # if sessions haven't been setup, error + if (!(Test-PodeSessionsConfigured)) { + throw 'Sessions have not been configured' + } + + # error if session is null + if ($null -eq $WebEvent.Session) { + throw 'There is no session available to save' + } + + # temporarily set this session to auto-extend + $WebEvent.Session.Extend = $true + + # reset on response + Set-PodeSession +} + +<# +.SYNOPSIS +Returns the defined Session duration. + +.DESCRIPTION +Returns the defined Session duration that all Session are created using. + +.EXAMPLE +$duration = Get-PodeSessionDuration +#> +function Get-PodeSessionDuration +{ + [CmdletBinding()] + param() + + return $PodeContext.Server.Sessions.Info.Duration +} + +<# +.SYNOPSIS +Returns the datetime on which the current Session's will expire. + +.DESCRIPTION +Returns the datetime on which the current Session's will expire. + +.EXAMPLE +$expiry = Get-PodeSessionExpiry +#> +function Get-PodeSessionExpiry +{ + [CmdletBinding()] + param() + + # error if session is null + if ($null -eq $WebEvent.Session) { + throw 'There is no session available to save' + } + + # default min date + if ($null -eq $WebEvent.Session.TimeStamp) { + return [datetime]::MinValue + } + + # use datetime.now or existing timestamp? + $expiry = [DateTime]::UtcNow + + if (!$WebEvent.Session.Extend -and ($null -ne $WebEvent.Session.TimeStamp)) { + $expiry = $WebEvent.Session.TimeStamp + } + + # add session duration on + $expiry = $expiry.AddSeconds($PodeContext.Server.Sessions.Info.Duration) + + # return expiry + return $expiry +} \ No newline at end of file diff --git a/tests/unit/Sessions.Tests.ps1 b/tests/unit/Sessions.Tests.ps1 index 40dea4a0a..7e685a89d 100644 --- a/tests/unit/Sessions.Tests.ps1 +++ b/tests/unit/Sessions.Tests.ps1 @@ -7,78 +7,93 @@ $now = [datetime]::UtcNow Describe 'Get-PodeSession' { Context 'Invalid parameters supplied' { It 'Throws null value error' { - { Get-PodeSession -Session @{ Name = $null } } | Should Throw 'because it is an empty string' + $PodeContext = @{ + Server = @{ + Sessions = @{ + Name = $null + } + } + } + + { Get-PodeSession } | Should Throw 'because it is an empty string' } It 'Throws an empry string value error' { - { Get-PodeSession -Session @{ Name = [string]::Empty } } | Should Throw 'because it is an empty string' + $PodeContext = @{ + Server = @{ + Sessions = @{ + Name = [string]::Empty + } + } + } + + { Get-PodeSession } | Should Throw 'because it is an empty string' } } Context 'Valid parameters' { It 'Returns no session details for invalid sessionId' { - $WebEvent = @{ 'Cookies' = @{} } + $WebEvent = @{ Cookies = @{} } $PodeContext = @{ - 'Server' = @{ - 'Cookies' = @{} - 'Sessions' = @{ - 'Name' = 'pode.sid'; - 'Secret' = 'key'; - 'Info' = @{ 'Duration' = 60; }; + Server = @{ + Cookies = @{} + Sessions = @{ + Name = 'pode.sid' + Secret = 'key' + Info = @{ 'Duration' = 60 } } } } - $data = Get-PodeSession -Session @{ Name = 'pode.sid' } + $data = Get-PodeSession $data | Should Be $null } It 'Returns no session details for invalid signed sessionId' { $cookie = [System.Net.Cookie]::new('pode.sid', 's:value.kPv88V5o2uJ29sqh2a7P/f3dxcg+JdZJZT3GTIE=') - $WebEvent = @{ 'Cookies' = @{ - 'pode.sid' = $cookie; + $WebEvent = @{ Cookies = @{ + 'pode.sid' = $cookie } } $PodeContext = @{ - 'Server' = @{ - 'Cookies' = @{} - 'Sessions' = @{ - 'Name' = 'pode.sid'; - 'Secret' = 'key'; - 'Info' = @{ 'Duration' = 60; }; + Server = @{ + Cookies = @{} + Sessions = @{ + Name = 'pode.sid' + Secret = 'key' + Info = @{ 'Duration' = 60 } } } } - $data = Get-PodeSession -Session @{ Name = 'pode.sid'; Secret = 'key'; Info = @{ Duration = 60; }; } + $data = Get-PodeSession $data | Should Be $null } It 'Returns session details' { $cookie = [System.Net.Cookie]::new('pode.sid', 's:value.kPv88V50o2uJ29sqch2a7P/f3dxcg+J/dZJZT3GTJIE=') - $WebEvent = @{ 'Cookies' = @{ - 'pode.sid' = $cookie; + $WebEvent = @{ Cookies = @{ + 'pode.sid' = $cookie } } $PodeContext = @{ - 'Server' = @{ - 'Cookies' = @{} - 'Sessions' = @{ - 'Name' = 'pode.sid'; - 'Secret' = 'key'; - 'Info' = @{ 'Duration' = 60; }; + Server = @{ + Cookies = @{} + Sessions = @{ + Name = 'pode.sid' + Secret = 'key' + Info = @{ 'Duration' = 60 } } } } - $data = Get-PodeSession -Session @{ Name = 'pode.sid'; Secret = 'key'; Info = @{ Duration = 60; }; } + $data = Get-PodeSession $data | Should Not Be $null $data.Id | Should Be 'value' $data.Name | Should Be 'pode.sid' - $data.Properties.Duration | Should Be 60 } } } @@ -86,33 +101,37 @@ Describe 'Get-PodeSession' { Describe 'Set-PodeSessionDataHash' { Context 'Invalid parameters supplied' { It 'Throws null value error' { - { Set-PodeSessionDataHash -Session $null } | Should Throw 'argument is null' + { Set-PodeSessionDataHash } | Should Throw 'No session available' } } Context 'Valid parameters' { It 'Sets a hash for no data' { - $Session = @{} - Set-PodeSessionDataHash -Session $Session - $Session.Data | Should Not Be $null + $WebEvent = @{ + Session = @{} + } + Set-PodeSessionDataHash + $WebEvent.Session.Data | Should Not Be $null $crypto = [System.Security.Cryptography.SHA256]::Create() - $hash = $crypto.ComputeHash([System.Text.Encoding]::UTF8.GetBytes(($Session.Data | ConvertTo-Json -Depth 10 -Compress))) + $hash = $crypto.ComputeHash([System.Text.Encoding]::UTF8.GetBytes(($WebEvent.Session.Data | ConvertTo-Json -Depth 10 -Compress))) $hash = [System.Convert]::ToBase64String($hash) - $Session.DataHash | Should Be $hash + $WebEvent.Session.DataHash | Should Be $hash } It 'Sets a hash for data' { - $Session = @{ 'Data' = @{ 'Counter' = 2; } } - Set-PodeSessionDataHash -Session $Session - $Session.Data | Should Not Be $null + $WebEvent = @{ + Session = @{ 'Data' = @{ 'Counter' = 2; } } + } + Set-PodeSessionDataHash + $WebEvent.Session.Data | Should Not Be $null $crypto = [System.Security.Cryptography.SHA256]::Create() - $hash = $crypto.ComputeHash([System.Text.Encoding]::UTF8.GetBytes(($Session.Data | ConvertTo-Json -Depth 10 -Compress))) + $hash = $crypto.ComputeHash([System.Text.Encoding]::UTF8.GetBytes(($WebEvent.Session.Data | ConvertTo-Json -Depth 10 -Compress))) $hash = [System.Convert]::ToBase64String($hash) - $Session.DataHash | Should Be $hash + $WebEvent.Session.DataHash | Should Be $hash } } } @@ -121,63 +140,67 @@ Describe 'New-PodeSession' { Mock 'Invoke-PodeScriptBlock' { return 'value' } It 'Creates a new session object' { + $WebEvent = @{ + Session = @{} + } + $PodeContext = @{ - 'Server' = @{ - 'Cookies' = @{} - 'Sessions' = @{ - 'Name' = 'pode.sid'; - 'Secret' = 'key'; - 'Info' = @{ 'Duration' = 60; }; - 'GenerateId' = {} + Server = @{ + Cookies = @{} + Sessions = @{ + Name = 'pode.sid' + Secret = 'key' + Info = @{ 'Duration' = 60 } + GenerateId = {} } } } - $session = New-PodeSession + $WebEvent.Session = New-PodeSession + Set-PodeSessionDataHash - $session | Should Not Be $null - $session.Id | Should Be 'value' - $session.Name | Should Be 'pode.sid' - $session.Data.Count | Should Be 0 - $session.Properties.Duration | Should Be 60 + $WebEvent.Session | Should Not Be $null + $WebEvent.Session.Id | Should Be 'value' + $WebEvent.Session.Name | Should Be 'pode.sid' + $WebEvent.Session.Data.Count | Should Be 0 $crypto = [System.Security.Cryptography.SHA256]::Create() - $hash = $crypto.ComputeHash([System.Text.Encoding]::UTF8.GetBytes(($session.Data| ConvertTo-Json -Depth 10 -Compress))) + $hash = $crypto.ComputeHash([System.Text.Encoding]::UTF8.GetBytes(($WebEvent.Session.Data| ConvertTo-Json -Depth 10 -Compress))) $hash = [System.Convert]::ToBase64String($hash) - $session.DataHash | Should Be $hash + $WebEvent.Session.DataHash | Should Be $hash } } Describe 'Test-PodeSessionDataHash' { - Context 'Invalid parameters supplied' { - It 'Throws null value error' { - { Test-PodeSessionDataHash -Session $null } | Should Throw 'argument is null' - } - } - Context 'Valid parameters' { It 'Returns false for no hash set' { - $Session = @{} - Test-PodeSessionDataHash -Session $Session | Should Be $false + $WebEvent = @{ + Session = @{} + } + Test-PodeSessionDataHash | Should Be $false } It 'Returns false for invalid hash' { - $Session = @{ 'DataHash' = 'fake' } - Test-PodeSessionDataHash -Session $Session | Should Be $false + $WebEvent = @{ + Session = @{ 'DataHash' = 'fake' } + } + Test-PodeSessionDataHash | Should Be $false } It 'Returns true for a valid hash' { - $Session = @{ - 'Data' = @{ 'Counter' = 2; }; + $WebEvent = @{ + Session = @{ + 'Data' = @{ 'Counter' = 2; }; + } } $crypto = [System.Security.Cryptography.SHA256]::Create() - $hash = $crypto.ComputeHash([System.Text.Encoding]::UTF8.GetBytes(($Session.Data| ConvertTo-Json -Depth 10 -Compress))) + $hash = $crypto.ComputeHash([System.Text.Encoding]::UTF8.GetBytes(($WebEvent.Session.Data| ConvertTo-Json -Depth 10 -Compress))) $hash = [System.Convert]::ToBase64String($hash) - $Session.DataHash = $hash + $WebEvent.Session.DataHash = $hash - Test-PodeSessionDataHash -Session $Session | Should Be $true + Test-PodeSessionDataHash | Should Be $true } } } @@ -209,13 +232,15 @@ Describe 'Set-PodeSession' { Mock Set-PodeCookie { } Mock Get-PodeSessionExpiry { return ([datetime]::UtcNow) } - $session = @{ - 'Name' = 'name'; - 'Id' = 'sessionId'; - 'Cookie' = @{}; + $WebEvent = @{ + Session = @{ + 'Name' = 'name'; + 'Id' = 'sessionId'; + 'Cookie' = @{}; + } } - Set-PodeSession -Session $session + Set-PodeSession Assert-MockCalled Set-PodeCookie -Times 1 -Scope It Assert-MockCalled Get-PodeSessionExpiry -Times 1 -Scope It @@ -225,7 +250,7 @@ Describe 'Set-PodeSession' { Describe 'Remove-PodeSession' { It 'Throws an error if sessions are not configured' { Mock Test-PodeSessionsConfigured { return $false } - { Remove-PodeSession } | Should Throw 'sessions have not been configured' + { Remove-PodeSession } | Should Throw 'Sessions have not been configured' } It 'Does nothing if there is no session' { @@ -252,7 +277,7 @@ Describe 'Remove-PodeSession' { Describe 'Save-PodeSession' { It 'Throws an error if sessions are not configured' { Mock Test-PodeSessionsConfigured { return $false } - { Save-PodeSession } | Should Throw 'sessions have not been configured' + { Save-PodeSession } | Should Throw 'Sessions have not been configured' } It 'Throws error if there is no session' { From 043eb9833d622e37ffeb772e8758b93b327d00c9 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Fri, 16 Dec 2022 12:17:00 +0000 Subject: [PATCH 11/52] #1036: debug clean-up --- src/Private/Middleware.ps1 | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/Private/Middleware.ps1 b/src/Private/Middleware.ps1 index 64a0f7466..e76dbc663 100644 --- a/src/Private/Middleware.ps1 +++ b/src/Private/Middleware.ps1 @@ -325,8 +325,6 @@ function Get-PodeCookieMiddleware return $true } - $h_cookie | Out-Default - # parse the cookies from the header $cookies = @($h_cookie -split '; ') $WebEvent.Cookies = @{} From 82160dd687f6fc449d651eb66cbfe48269bdb1da Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Fri, 16 Dec 2022 20:01:11 +0000 Subject: [PATCH 12/52] #1036: ensure duration returned is an int --- src/Public/Sessions.ps1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/Public/Sessions.ps1 b/src/Public/Sessions.ps1 index 5190b889b..a96496a02 100644 --- a/src/Public/Sessions.ps1 +++ b/src/Public/Sessions.ps1 @@ -319,7 +319,7 @@ function Get-PodeSessionDuration [CmdletBinding()] param() - return $PodeContext.Server.Sessions.Info.Duration + return [int]$PodeContext.Server.Sessions.Info.Duration } <# From c062ea944845c09dd204552d3e38639315d944b9 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sat, 17 Dec 2022 12:03:51 +0000 Subject: [PATCH 13/52] #1044: fix supplying mutliple files when the multiple property is used on input --- docs/Tutorials/Misc/UploadFiles.md | 12 ++- examples/views/web-upload-multi.html | 23 ++++++ examples/web-upload.ps1 | 14 ++++ src/Listener/PodeForm.cs | 27 ++++++- src/Listener/PodeFormData.cs | 19 ++++- src/Pode.psd1 | 1 + src/Private/Helpers.ps1 | 7 +- src/Public/Responses.ps1 | 109 +++++++++++++++++++++++---- 8 files changed, 186 insertions(+), 26 deletions(-) create mode 100644 examples/views/web-upload-multi.html diff --git a/docs/Tutorials/Misc/UploadFiles.md b/docs/Tutorials/Misc/UploadFiles.md index 878dfde6c..c01a4658f 100644 --- a/docs/Tutorials/Misc/UploadFiles.md +++ b/docs/Tutorials/Misc/UploadFiles.md @@ -3,7 +3,7 @@ Pode's inbuilt middleware supports parsing a request's body/payload and query string, and this also extends to uploading files via a ``. Like how POST data can be accessed in a Route via the [web event](../../WebEvent) as `$WebEvent.Data[]`, uploaded files can be accessed via `$WebEvent.Files[]`. !!! important - In order for uploaded files to work, your `` must contain `enctype="multipart/form-data"` + In order for uploaded files to work, your `` must contain the `enctype="multipart/form-data"` property. ## Web Form @@ -36,8 +36,6 @@ The following HTML is an example of a `` for a simple sign-up flow. Here t ``` -> You can upload multiple files from one `` - The inputs will be POSTed to the server, and accessible via the [web event](../../WebEvent)'s `.Data` and `.Files`. For the `.Data`: @@ -52,7 +50,13 @@ For the `.Files`: $WebEvent.Files['image.png'] # the bytes of the uploaded file ``` -## Script +## Multiple Files + +You can upload multiple files from one `` by either supplying multiple file `` fields, or by using the `multiple` property in a file ``. + +If you use the `multiple` property then all the file names will be available under the same `$WebEvent.Data` key. When you use [`Save-PodeRequestFile`](../../../Functions/Responses/Save-PodeRequestFile) on this key, all of the files will be saved at once. + +## Examples ### Inbuilt Save diff --git a/examples/views/web-upload-multi.html b/examples/views/web-upload-multi.html new file mode 100644 index 000000000..967bace12 --- /dev/null +++ b/examples/views/web-upload-multi.html @@ -0,0 +1,23 @@ + + + Upload Multiple Files + + + + + +
+ + +
+
+ + +
+
+ +
+ + + + \ No newline at end of file diff --git a/examples/web-upload.ps1 b/examples/web-upload.ps1 index 71417dcfe..7f88e5a6f 100644 --- a/examples/web-upload.ps1 +++ b/examples/web-upload.ps1 @@ -16,6 +16,7 @@ Start-PodeServer -Threads 2 { Add-PodeEndpoint -Address * -Port $port -Protocol Http Set-PodeViewEngine -Type HTML + New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging # GET request for web page on "localhost:8085/" Add-PodeRoute -Method Get -Path '/' -ScriptBlock { @@ -28,4 +29,17 @@ Start-PodeServer -Threads 2 { Move-PodeResponseUrl -Url '/' } + # GET request for web page on "localhost:8085/multi" + Add-PodeRoute -Method Get -Path '/multi' -ScriptBlock { + Write-PodeViewResponse -Path 'web-upload-multi' + } + + # POST request to upload multiple files + Add-PodeRoute -Method Post -Path '/upload-multi' -ScriptBlock { + # $WebEvent.Data | Out-Default + # $WebEvent.Files | Out-Default + Save-PodeRequestFile -Key 'avatar' -Path 'C:/temp' -FileName 'Ruler.png' + Move-PodeResponseUrl -Url '/multi' + } + } \ No newline at end of file diff --git a/src/Listener/PodeForm.cs b/src/Listener/PodeForm.cs index 783f9729e..72cd796ba 100644 --- a/src/Listener/PodeForm.cs +++ b/src/Listener/PodeForm.cs @@ -2,6 +2,7 @@ using System.Collections.Generic; using System.IO; using System.Text; +using System.Linq; namespace Pode { @@ -18,6 +19,7 @@ public PodeForm() public void Dispose() { + // dispose all file streams foreach (var file in Files) { file.Dispose(); @@ -28,13 +30,16 @@ public static PodeForm Parse(byte[] bytes, string contentType, Encoding contentE { var form = new PodeForm(); + // do nothing if there are no bytes to parse if (bytes == default(byte[]) || bytes.Length == 0) { return form; } + // convert to bytes to lines of bytes var lines = PodeHelpers.ConvertToByteLines(bytes); + // get the boundary start/end var parts = contentType.Split(';'); var boundaryStart = $"--{parts[1].Split('=')[1].Trim()}"; var boundaryEnd = $"{boundaryStart}--"; @@ -48,6 +53,7 @@ public static PodeForm Parse(byte[] bytes, string contentType, Encoding contentE } } + // now parse the lines for data/files return ParseHttp(form, lines, boundaryLineIndexes, contentEncoding); } @@ -57,10 +63,12 @@ private static PodeForm ParseHttp(PodeForm form, List lines, List b var disposition = string.Empty; var fields = new Dictionary(); + // loop through all boundary sections and parse them for (var i = 0; i < (boundaryLineIndexes.Count - 1); i++) { fields.Clear(); + // get the content disposition and data headers boundaryLineIndex = boundaryLineIndexes[i]; disposition = contentEncoding.GetString(lines[boundaryLineIndex + 1]).Trim(PodeHelpers.NEW_LINE_ARRAY); @@ -73,17 +81,31 @@ private static PodeForm ParseHttp(PodeForm form, List lines, List b } } + // is this just a regular data field? if (!fields.ContainsKey("filename")) { + // add the data item as name=value form.Data.Add(new PodeFormData(fields["name"], contentEncoding.GetString(lines[boundaryLineIndex + 3]).Trim(PodeHelpers.NEW_LINE_ARRAY))); } - if (fields.ContainsKey("filename")) + // otherwise it's a file field + else { - form.Data.Add(new PodeFormData(fields["name"], fields["filename"])); + // add a data item for mapping name=filename + var currentData = form.Data.FirstOrDefault(x => x.Key == fields["name"]); + if (currentData == default(PodeFormData)) + { + form.Data.Add(new PodeFormData(fields["name"], fields["filename"])); + } + else + { + currentData.AddValue(fields["filename"]); + } + // do we actually have a filename? if (!string.IsNullOrWhiteSpace(fields["filename"])) { + // parse the file contents, and create a stream for the payload var fileContentType = contentEncoding.GetString(lines[boundaryLineIndex + 2]).Trim(PodeHelpers.NEW_LINE_ARRAY); var fileBytesLength = 0; var stream = new MemoryStream(); @@ -107,6 +129,7 @@ private static PodeForm ParseHttp(PodeForm form, List lines, List b stream.Write(lines[j], 0, fileBytesLength); } + // add a file item for filename=stream [+name/content-type] form.Files.Add(new PodeFormFile(fields["filename"], stream, fields["name"], fileContentType.Split(':')[1].Trim())); } } diff --git a/src/Listener/PodeFormData.cs b/src/Listener/PodeFormData.cs index 33a7c6e11..09e1dcf76 100644 --- a/src/Listener/PodeFormData.cs +++ b/src/Listener/PodeFormData.cs @@ -1,4 +1,4 @@ -using System; +using System.Linq; using System.Collections.Generic; namespace Pode @@ -6,12 +6,25 @@ namespace Pode public class PodeFormData { public string Key { get; private set; } - public string Value { get; private set; } + + private IList _values; + public string[] Values => _values.ToArray(); + + public int Count => _values.Count; + public bool IsSingular => (_values.Count == 1); + public bool IsEmpty => (_values.Count == 0); public PodeFormData(string key, string value) { Key = key; - Value = value; + + _values = new List(); + _values.Add(value); + } + + public void AddValue(string value) + { + _values.Add(value); } } } \ No newline at end of file diff --git a/src/Pode.psd1 b/src/Pode.psd1 index d8d02aaaa..5c44f1ede 100644 --- a/src/Pode.psd1 +++ b/src/Pode.psd1 @@ -83,6 +83,7 @@ 'Read-PodeTcpClient', 'Close-PodeTcpClient', 'Save-PodeRequestFile', + 'Test-PodeRequestFile', 'Set-PodeViewEngine', 'Use-PodePartialView', 'Send-PodeSignal', diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index daeb5687e..93080669c 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -1576,7 +1576,12 @@ function ConvertFrom-PodeRequestContent } foreach ($item in $form.Data) { - $Result.Data.Add($item.Key, $item.Value) + if ($item.IsSingular) { + $Result.Data.Add($item.Key, $item.Values[0]) + } + else { + $Result.Data.Add($item.Key, $item.Values) + } } $form = $null diff --git a/src/Public/Responses.ps1 b/src/Public/Responses.ps1 index 291d75b7c..0faca117e 100644 --- a/src/Public/Responses.ps1 +++ b/src/Public/Responses.ps1 @@ -1192,57 +1192,134 @@ function Close-PodeTcpClient <# .SYNOPSIS -Saves an uploaded file on the Request to the File System. +Saves any uploaded files on the Request to the File System. .DESCRIPTION -Saves an uploaded file on the Request to the File System. +Saves any uploaded files on the Request to the File System. .PARAMETER Key -The name of the key within the web event's Data HashTable that stores the file's name. +The name of the key within the $WebEvent's Data HashTable that stores the file names. .PARAMETER Path -The path to save files. +The path to save files. If this is a directory then the file name of the uploaded file will be used, but if this is a file path then that name is used instead. +If the Request has multiple files in, and you specify a file path, then all files will be saved to that one file path - overwriting each other. + +.PARAMETER FileName +An optional FileName to save a specific files if multiple files were supplied in the Request. By default, every file is saved. .EXAMPLE Save-PodeRequestFile -Key 'avatar' .EXAMPLE Save-PodeRequestFile -Key 'avatar' -Path 'F:/Images' + +.EXAMPLE +Save-PodeRequestFile -Key 'avatar' -Path 'F:/Images' -FileName 'icon.png' #> function Save-PodeRequestFile { [CmdletBinding()] - param ( + param( [Parameter(Mandatory=$true)] [string] $Key, [Parameter()] [string] - $Path = '.' + $Path = '.', + + [Parameter()] + [string[]] + $FileName ) # if path is '.', replace with server root $Path = Get-PodeRelativePath -Path $Path -JoinRoot # ensure the parameter name exists in data - $fileName = $WebEvent.Data[$Key] - if ([string]::IsNullOrWhiteSpace($fileName)) { - throw "A parameter called '$($Key)' was not supplied in the request" + if (!(Test-PodeRequestFile -Key $Key)) { + throw "A parameter called '$($Key)' was not supplied in the request, or has no data available" + } + + # get the file names + $files = @($WebEvent.Data[$Key]) + if (($null -ne $FileName) -and ($FileName.Length -gt 0)) { + $files = @(foreach ($file in $files) { + if ($FileName -icontains $file) { + $file + } + }) } # ensure the file data exists - if (!$WebEvent.Files.ContainsKey($fileName)) { - throw "No data for file '$($fileName)' was uploaded in the request" + foreach ($file in $files) { + if (!$WebEvent.Files.ContainsKey($file)) { + throw "No data for file '$($file)' was uploaded in the request" + } + } + + # save the files + foreach ($file in $files) { + # if the path is a directory, add the filename + $filePath = $Path + if (Test-PodePathIsDirectory -Path $filePath) { + $filePath = [System.IO.Path]::Combine($filePath, $file) + } + + # save the file + $WebEvent.Files[$file].Save($filePath) + } +} + +<# +.SYNOPSIS +Test to see if the Request contains the key for any uploaded files. + +.DESCRIPTION +Test to see if the Request contains the key for any uploaded files. + +.PARAMETER Key +The name of the key within the $WebEvent's Data HashTable that stores the file names. + +.PARAMETER FileName +An optional FileName to test for a specific file within the list of uploaded files. + +.EXAMPLE +Test-PodeRequestFile -Key 'avatar' + +.EXAMPLE +Test-PodeRequestFile -Key 'avatar' -FileName 'icon.png' +#> +function Test-PodeRequestFile +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Key, + + [Parameter()] + [string] + $FileName + ) + + # ensure the parameter name exists in data + if (!$WebEvent.Data.ContainsKey($Key)) { + return $false + } + + # ensure it has filenames + if ([string]::IsNullOrEmpty($WebEvent.Data[$Key])) { + return $false } - # if the path is a directory, add the filename - if (Test-PodePathIsDirectory -Path $Path) { - $Path = [System.IO.Path]::Combine($Path, $fileName) + # do we have any specific files? + if (![string]::IsNullOrEmpty($FileName)) { + return (@($WebEvent.Data[$Key]) -icontains $FileName) } - # save the file - $WebEvent.Files[$fileName].Save($Path) + # we have files + return $true } <# From 04551eb8507b86b700419dbe99b2633b3e8c5f55 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sat, 17 Dec 2022 12:07:48 +0000 Subject: [PATCH 14/52] #1044: debug clean-up --- examples/web-upload.ps1 | 2 -- 1 file changed, 2 deletions(-) diff --git a/examples/web-upload.ps1 b/examples/web-upload.ps1 index 7f88e5a6f..8b0fc7e7a 100644 --- a/examples/web-upload.ps1 +++ b/examples/web-upload.ps1 @@ -36,8 +36,6 @@ Start-PodeServer -Threads 2 { # POST request to upload multiple files Add-PodeRoute -Method Post -Path '/upload-multi' -ScriptBlock { - # $WebEvent.Data | Out-Default - # $WebEvent.Files | Out-Default Save-PodeRequestFile -Key 'avatar' -Path 'C:/temp' -FileName 'Ruler.png' Move-PodeResponseUrl -Url '/multi' } From 45294493aef6a936c88fb25bc8ff9c60782c9f41 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sat, 17 Dec 2022 20:58:13 +0000 Subject: [PATCH 15/52] #1041: fix to uploading files from pwsh, and update to UploadFile docs --- docs/Tutorials/Misc/UploadFiles.md | 27 ++++++++ src/Listener/PodeForm.cs | 107 +++++++++++++++++++++-------- 2 files changed, 104 insertions(+), 30 deletions(-) diff --git a/docs/Tutorials/Misc/UploadFiles.md b/docs/Tutorials/Misc/UploadFiles.md index c01a4658f..af830dcc0 100644 --- a/docs/Tutorials/Misc/UploadFiles.md +++ b/docs/Tutorials/Misc/UploadFiles.md @@ -56,6 +56,33 @@ You can upload multiple files from one `
` by either supplying multiple fil If you use the `multiple` property then all the file names will be available under the same `$WebEvent.Data` key. When you use [`Save-PodeRequestFile`](../../../Functions/Responses/Save-PodeRequestFile) on this key, all of the files will be saved at once. +## CLI + +You can upload files from the CLI by using `Invoke-WebRequest` (or `Invoke-RestMethod`), and to do so you'll need to pass the `-Form` parameter. Assuming you have the following Route to save some "avatar" file: + + +```powershell +Start-PodeServer { + Add-PodeEndpoint -Address * -Port 8085 -Protocol Http + + Add-PodeRoute -Method Post -Path '/upload' -ScriptBlock { + Save-PodeRequestFile -Key 'avatar' + } +} +``` + +You can call this Route by using the following `Invoke-WebRequest` command to save the uploaded file: + +```powershell +Invoke-WebRequest -Uri "http://localhost:8085/upload" -Method Post -Form @{ avatar = (Get-Item .\path\to\file.png) } +``` + +You can also achieve the same results with `curl`: + +```bash +curl http://localhost:8085/upload -F avatar=@"C:\path\to\file.png" +``` + ## Examples ### Inbuilt Save diff --git a/src/Listener/PodeForm.cs b/src/Listener/PodeForm.cs index 72cd796ba..5bfe7e170 100644 --- a/src/Listener/PodeForm.cs +++ b/src/Listener/PodeForm.cs @@ -3,6 +3,8 @@ using System.IO; using System.Text; using System.Linq; +using System.Text.RegularExpressions; +using System.Net.Http; namespace Pode { @@ -10,6 +12,10 @@ public class PodeForm : IDisposable { public IList Files { get; private set; } public IList Data { get; private set; } + public string Boundary { get; private set; } + + private static readonly Regex BoundaryRegex = new Regex("boundary=\"?(?.+?)\"?$"); + private static readonly Regex HeaderRegex = new Regex("^(?.*?)\\:\\s+(?.*?)$"); public PodeForm() { @@ -39,9 +45,19 @@ public static PodeForm Parse(byte[] bytes, string contentType, Encoding contentE // convert to bytes to lines of bytes var lines = PodeHelpers.ConvertToByteLines(bytes); + // get the boundary + var match = BoundaryRegex.Match(contentType); + if (match.Success) + { + form.Boundary = match.Groups["boundary"].Value; + } + else + { + throw new HttpRequestException("No multipart/form-data boundary found"); + } + // get the boundary start/end - var parts = contentType.Split(';'); - var boundaryStart = $"--{parts[1].Split('=')[1].Trim()}"; + var boundaryStart = $"--{form.Boundary}"; var boundaryEnd = $"{boundaryStart}--"; var boundaryLineIndexes = new List(); @@ -59,20 +75,45 @@ public static PodeForm Parse(byte[] bytes, string contentType, Encoding contentE private static PodeForm ParseHttp(PodeForm form, List lines, List boundaryLineIndexes, Encoding contentEncoding) { - var boundaryLineIndex = 0; - var disposition = string.Empty; - var fields = new Dictionary(); + var currentLineIndex = 0; + var currentLine = string.Empty; + var fields = new Dictionary(StringComparer.InvariantCultureIgnoreCase); + var headers = new Dictionary(StringComparer.InvariantCultureIgnoreCase); // loop through all boundary sections and parse them for (var i = 0; i < (boundaryLineIndexes.Count - 1); i++) { + // reset fields and headers fields.Clear(); + headers.Clear(); + + // what's the starting line index for the current boundary? + currentLineIndex = boundaryLineIndexes[i] + 1; + + // parse headers until we see a blank line + while (!string.IsNullOrWhiteSpace((currentLine = GetLineString(lines[currentLineIndex], contentEncoding)))) + { + currentLineIndex++; + + // parse the header name=value pair + var match = HeaderRegex.Match(currentLine); + if (match.Success) + { + headers.Add(match.Groups["name"].Value, match.Groups["value"].Value); + } + } - // get the content disposition and data headers - boundaryLineIndex = boundaryLineIndexes[i]; - disposition = contentEncoding.GetString(lines[boundaryLineIndex + 1]).Trim(PodeHelpers.NEW_LINE_ARRAY); + // bump to next line, past the blank line + currentLineIndex++; - foreach (var line in disposition.Split(';')) + // get the content disposition fields + if (!headers.ContainsKey("Content-Disposition")) + { + throw new HttpRequestException("No Content-Disposition found in multipart/form-data"); + } + + // foreach (var line in disposition.Split(';')) + foreach (var line in headers["Content-Disposition"].Split(';')) { var atoms = line.Split('='); if (atoms.Length == 2) @@ -85,7 +126,7 @@ private static PodeForm ParseHttp(PodeForm form, List lines, List b if (!fields.ContainsKey("filename")) { // add the data item as name=value - form.Data.Add(new PodeFormData(fields["name"], contentEncoding.GetString(lines[boundaryLineIndex + 3]).Trim(PodeHelpers.NEW_LINE_ARRAY))); + form.Data.Add(new PodeFormData(fields["name"], GetLineString(lines[currentLineIndex], contentEncoding))); } // otherwise it's a file field @@ -103,41 +144,47 @@ private static PodeForm ParseHttp(PodeForm form, List lines, List b } // do we actually have a filename? - if (!string.IsNullOrWhiteSpace(fields["filename"])) + if (string.IsNullOrWhiteSpace(fields["filename"])) { - // parse the file contents, and create a stream for the payload - var fileContentType = contentEncoding.GetString(lines[boundaryLineIndex + 2]).Trim(PodeHelpers.NEW_LINE_ARRAY); - var fileBytesLength = 0; - var stream = new MemoryStream(); + continue; + } + + // parse the file contents, and create a stream for the payload + var fileBytesLength = 0; + var stream = new MemoryStream(); - for (var j = (boundaryLineIndex + 4); j <= (boundaryLineIndexes[i + 1] - 1); j++) + for (var j = currentLineIndex; j <= (boundaryLineIndexes[i + 1] - 1); j++) + { + fileBytesLength = lines[j].Length; + if (j == (boundaryLineIndexes[i + 1] - 1)) { - fileBytesLength = lines[j].Length; - if (j == (boundaryLineIndexes[i + 1] - 1)) + if (lines[j][fileBytesLength - 1] == PodeHelpers.NEW_LINE_BYTE) { - if (lines[j][fileBytesLength - 1] == PodeHelpers.NEW_LINE_BYTE) - { - fileBytesLength--; - } - - if (lines[j][fileBytesLength - 1] == PodeHelpers.CARRIAGE_RETURN_BYTE) - { - fileBytesLength--; - } + fileBytesLength--; } - stream.Write(lines[j], 0, fileBytesLength); + if (lines[j][fileBytesLength - 1] == PodeHelpers.CARRIAGE_RETURN_BYTE) + { + fileBytesLength--; + } } - // add a file item for filename=stream [+name/content-type] - form.Files.Add(new PodeFormFile(fields["filename"], stream, fields["name"], fileContentType.Split(':')[1].Trim())); + stream.Write(lines[j], 0, fileBytesLength); } + + // add a file item for filename=stream [+name/content-type] + form.Files.Add(new PodeFormFile(fields["filename"], stream, fields["name"], headers["Content-Type"].Trim())); } } return form; } + private static string GetLineString(byte[] bytes, Encoding contentEncoding) + { + return contentEncoding.GetString(bytes).Trim(PodeHelpers.NEW_LINE_ARRAY); + } + private static bool IsLineBoundary(byte[] bytes, string boundary, Encoding contentEncoding) { if (bytes.Length == 0) From 758e8ae1a1c39c5ae7ad6f043d9db74b891960b5 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Thu, 29 Dec 2022 23:28:11 +0000 Subject: [PATCH 16/52] #980: added support for secret management (module and custom), secret var scope, and general scope clean-up --- README.md | 1 + docs/index.md | 1 + examples/web-secrets.ps1 | 97 ++++ packers/choco/pode.nuspec | 3 +- src/Pode.psd1 | 20 +- src/Private/Authentication.ps1 | 2 +- src/Private/AutoImport.ps1 | 257 ++++++++++ src/Private/Context.ps1 | 207 ++------ src/Private/Helpers.ps1 | 102 +++- src/Private/Middleware.ps1 | 8 +- src/Private/Routes.ps1 | 6 +- src/Private/Secrets.ps1 | 437 ++++++++++++++++ src/Private/Server.ps1 | 12 +- src/Public/Authentication.ps1 | 40 +- src/Public/AutoImport.ps1 | 33 ++ src/Public/Core.ps1 | 7 +- src/Public/Events.ps1 | 8 +- src/Public/Handlers.ps1 | 8 +- src/Public/Logging.ps1 | 12 +- src/Public/Middleware.ps1 | 8 +- src/Public/Responses.ps1 | 4 +- src/Public/Routes.ps1 | 40 +- src/Public/Schedules.ps1 | 37 +- src/Public/Secrets.ps1 | 902 +++++++++++++++++++++++++++++++++ src/Public/Tasks.ps1 | 12 +- src/Public/Timers.ps1 | 37 +- src/Public/Utilities.ps1 | 8 +- src/Public/Verbs.ps1 | 8 +- src/Public/WebSockets.ps1 | 8 +- tests/unit/Server.Tests.ps1 | 13 +- 30 files changed, 2021 insertions(+), 317 deletions(-) create mode 100644 examples/web-secrets.ps1 create mode 100644 src/Private/AutoImport.ps1 create mode 100644 src/Private/Secrets.ps1 create mode 100644 src/Public/Secrets.ps1 diff --git a/README.md b/README.md index 48017d4e0..341e399e2 100644 --- a/README.md +++ b/README.md @@ -69,6 +69,7 @@ Then navigate to `http://127.0.0.1:8000` in your browser. * Authentication on requests, such as Basic, Windows and Azure AD * Support for dynamically building Routes from Functions and Modules * Generate/bind self-signed certificates +* Secret management support to load secrets from vaults * (Windows) Open the hosted server as a desktop application ## 📦 Install diff --git a/docs/index.md b/docs/index.md index 7bd899e9c..a469dd8c2 100644 --- a/docs/index.md +++ b/docs/index.md @@ -41,6 +41,7 @@ Pode is a Cross-Platform framework to create web servers that host REST APIs, We * Authentication on requests, such as Basic, Windows and Azure AD * Support for dynamically building Routes from Functions and Modules * Generate/bind self-signed certificates +* Secret management support to load secrets from vaults * (Windows) Open the hosted server as a desktop application ## 🏢 Companies using Pode diff --git a/examples/web-secrets.ps1 b/examples/web-secrets.ps1 new file mode 100644 index 000000000..efb36a890 --- /dev/null +++ b/examples/web-secrets.ps1 @@ -0,0 +1,97 @@ +param( + [Parameter(Mandatory=$true)] + [string] + $AzureSubscriptionId +) + +$path = Split-Path -Parent -Path (Split-Path -Parent -Path $MyInvocation.MyCommand.Path) +Import-Module "$($path)/src/Pode.psm1" -Force -ErrorAction Stop + +# or just: +# Import-Module Pode + +Start-PodeServer -Threads 2 { + # listen + Add-PodeEndpoint -Address * -Port 8085 -Protocol Http + + # logging + New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging + + + # secret manage azure keyvault - need to run "Connect-AzAccount" first! + Register-PodeSecretVault -Name 'PodeTest_SMAZVault' -ModuleName 'Az.KeyVault' -VaultParameters @{ + AZKVaultName = 'pode-test-kv' + SubscriptionId = $AzureSubscriptionId + } + + # custom vault cli + Register-PodeSecretVault -Name 'PodeTest_CustomVault' -CacheTtl 1 ` + -VaultParameters @{ + Address = 'http://127.0.0.1:8200' + } ` + -ScriptBlock { + param($config, $key) + return (vault kv get -format json -address $config.Address -mount secret $key | ConvertFrom-Json -AsHashtable).data.data + } ` + -SetScriptBlock { + param($config, $key, $value) + vault kv put -address $config.Address -mount secret $key "$($value.Keys[0])=$($value.Values[0])" + } ` + -RemoveScriptBlock { + param($config, $key) + vault kv destroy -address $config.Address -versions 1 -mount secret $key + } + + + # mount a secret from vault cli + Mount-PodeSecret -Name 'CustomCLI' -Vault 'PodeTest_CustomVault' -Key 'hello' -ExpandProperty 'foo' + + # mount a secret from az keyvault + Mount-PodeSecret -Name 'ModuleAz' -Vault 'PodeTest_SMAZVault' -Key 'hello' + + + # routes to get/update secret in vault cli + Add-PodeRoute -Method Get -Path '/custom' -ScriptBlock { + # $value = Get-PodeSecret -Name 'CustomCLI' + # Write-PodeJsonResponse @{ Value = $value } + Write-PodeJsonResponse @{ Value = $secret:CustomCLI } + } + + Add-PodeRoute -Method Post -Path '/custom' -ScriptBlock { + # $WebEvent.Data.Value | Update-PodeSecret -Name 'CustomCLI' + $secret:CustomCLI = $WebEvent.Data.Value + } + + + # routes to get/update secret in az keyvault + Add-PodeRoute -Method Get -Path '/module' -ScriptBlock { + # $value = Get-PodeSecret -Name 'ModuleAz' + # Write-PodeJsonResponse @{ Value = $value } + Write-PodeJsonResponse @{ Value = $secret:ModuleAz } + } + + Add-PodeRoute -Method Post -Path '/module' -ScriptBlock { + # $WebEvent.Data.Value | Update-PodeSecret -Name 'ModuleAz' + $secret:ModuleAz = $WebEvent.Data.Value + } + + + Add-PodeRoute -Method Get -Path '/adhoc/:key' -ScriptBlock { + $value = Read-PodeSecret -Key $WebEvent.Parameters['key'] -Vault 'PodeTest_CustomVault' + Write-PodeJsonResponse @{ Value = $value } + } + + Add-PodeRoute -Method Get -Path '/custom/:name' -ScriptBlock { + Write-PodeJsonResponse @{ Value = (Get-PodeSecret -Name $WebEvent.Parameters['name']) } + } + + Add-PodeRoute -Method Post -Path '/adhoc/:key' -ScriptBlock { + Set-PodeSecret -Key $WebEvent.Parameters['key'] -Vault 'PodeTest_CustomVault' -InputObject $WebEvent.Data['value'] + Mount-PodeSecret -Name $WebEvent.Data['name'] -Vault 'PodeTest_CustomVault' -Key $WebEvent.Parameters['key'] + } + + Add-PodeRoute -Method Delete -Path '/adhoc/:key' -ScriptBlock { + Remove-PodeSecret -Key $WebEvent.Parameters['key'] -Vault 'PodeTest_CustomVault' + Dismount-PodeSecret -Name $WebEvent.Parameters['key'] + } +} diff --git a/packers/choco/pode.nuspec b/packers/choco/pode.nuspec index 9e0064729..6941a6617 100644 --- a/packers/choco/pode.nuspec +++ b/packers/choco/pode.nuspec @@ -36,6 +36,7 @@ Pode is a Cross-Platform framework for creating web servers to host REST APIs an * Authentication on requests, such as Basic, Windows and Azure AD * Support for dynamically building Routes from Functions and Modules * Generate/bind self-signed certificates +* Secret management support to load secrets from vaults * (Windows) Open the hosted server as a desktop application @@ -43,7 +44,7 @@ Pode is a Cross-Platform framework for creating web servers to host REST APIs an https://github.com/Badgerati/Pode/tree/master/packers https://badgerati.github.io/Pode https://github.com/Badgerati/Pode/issues - powershell web server rest api http tcp smtp listener unix cross-platform file-monitoring multithreaded schedule middleware session authentication active-directory csrf lambda aws azure functions websockets openapi swagger redoc + powershell web server rest api smtp unix cross-platform file-monitoring multithreaded schedule middleware authentication aws azure websockets openapi Copyright 2017-2022 https://github.com/Badgerati/Pode/blob/master/LICENSE.txt false diff --git a/src/Pode.psd1 b/src/Pode.psd1 index 5c44f1ede..70af1df63 100644 --- a/src/Pode.psd1 +++ b/src/Pode.psd1 @@ -162,6 +162,7 @@ 'Get-PodeSchedule', 'Get-PodeScheduleNextTrigger', 'Use-PodeSchedules', + 'Test-PodeSchedule', # timers 'Add-PodeTimer', @@ -171,6 +172,7 @@ 'Edit-PodeTimer', 'Get-PodeTimer', 'Use-PodeTimers', + 'Test-PodeTimer', # tasks 'Add-PodeTask', @@ -282,6 +284,7 @@ 'Export-PodeModule', 'Export-PodeSnapin', 'Export-PodeFunction', + 'Export-PodeSecretVault', # Events 'Register-PodeEvent', @@ -330,6 +333,21 @@ 'Send-PodeWebSocket', 'Reset-PodeWebSocket', 'Test-PodeWebSocket' + + # Secrets + 'Register-PodeSecretVault', + 'Unregister-PodeSecretVault', + 'Unlock-PodeSecretVault', + 'Get-PodeSecretVault', + 'Test-PodeSecretVault', + 'Mount-PodeSecret', + 'Dismount-PodeSecret', + 'Get-PodeSecret', + 'Test-PodeSecret', + 'Update-PodeSecret', + 'Remove-PodeSecret', + 'Read-PodeSecret', + 'Set-PodeSecret' ) # Private data to pass to the module specified in RootModule/ModuleToProcess. This may also contain a PSData hashtable with additional module metadata used by PowerShell. @@ -341,7 +359,7 @@ 'powershell-core', 'windows', 'unix', 'linux', 'pode', 'PSEdition_Core', 'cross-platform', 'access-control', 'file-monitoring', 'multithreaded', 'rate-limiting', 'cron', 'schedule', 'middleware', 'session', 'authentication', 'active-directory', 'caching', 'csrf', 'arm', 'raspberry-pi', 'aws-lambda', - 'azure-functions', 'websockets', 'swagger', 'openapi', 'redoc') + 'azure-functions', 'websockets', 'swagger', 'openapi', 'redoc', 'secrets', 'vault') # A URL to the license for this module. LicenseUri = 'https://raw.githubusercontent.com/Badgerati/Pode/master/LICENSE.txt' diff --git a/src/Private/Authentication.ps1 b/src/Private/Authentication.ps1 index 957b21e96..fbd3f7514 100644 --- a/src/Private/Authentication.ps1 +++ b/src/Private/Authentication.ps1 @@ -1877,7 +1877,7 @@ function Import-PodeAuthADModule throw 'Active Directory module only available on Windows' } - if ($null -eq (Get-Module -Name ActiveDirectory -ListAvailable -ErrorAction Ignore)) { + if (!(Test-PodeModuleInstalled -Name ActiveDirectory)) { throw 'Active Directory module is not installed' } diff --git a/src/Private/AutoImport.ps1 b/src/Private/AutoImport.ps1 new file mode 100644 index 000000000..8d476043f --- /dev/null +++ b/src/Private/AutoImport.ps1 @@ -0,0 +1,257 @@ +function Import-PodeFunctionsIntoRunspaceState +{ + param( + [Parameter(Mandatory=$true, ParameterSetName='Script')] + [scriptblock] + $ScriptBlock, + + [Parameter(Mandatory=$true, ParameterSetName='File')] + [string] + $FilePath + ) + + # do nothing if disabled + if (!$PodeContext.Server.AutoImport.Functions.Enabled) { + return + } + + # if export only, and there are none, do nothing + if ($PodeContext.Server.AutoImport.Functions.ExportOnly -and ($PodeContext.Server.AutoImport.Functions.ExportList.Length -eq 0)) { + return + } + + # script or file functions? + switch ($PSCmdlet.ParameterSetName.ToLowerInvariant()) { + 'script' { + $funcs = (Get-PodeFunctionsFromScriptBlock -ScriptBlock $ScriptBlock) + } + + 'file' { + $funcs = (Get-PodeFunctionsFromFile -FilePath $FilePath) + } + } + + # looks like we have nothing! + if (($null -eq $funcs) -or ($funcs.Length -eq 0)) { + return + } + + # groups funcs in case there or multiple definitions + $funcs = ($funcs | Group-Object -Property { $_.Name }) + + # import them, but also check if they're exported + foreach ($func in $funcs) { + # only exported funcs? is the func exported? + if ($PodeContext.Server.AutoImport.Functions.ExportOnly -and ($PodeContext.Server.AutoImport.Functions.ExportList -inotcontains $func.Name)) { + continue + } + + # load the function + $funcDef = [System.Management.Automation.Runspaces.SessionStateFunctionEntry]::new($func.Name, $func.Group[-1].Definition) + $PodeContext.RunspaceState.Commands.Add($funcDef) + } +} + +function Import-PodeModulesIntoRunspaceState +{ + # do nothing if disabled + if (!$PodeContext.Server.AutoImport.Modules.Enabled) { + return + } + + # if export only, and there are none, do nothing + if ($PodeContext.Server.AutoImport.Modules.ExportOnly -and ($PodeContext.Server.AutoImport.Modules.ExportList.Length -eq 0)) { + return + } + + # load modules into runspaces, if allowed + $modules = Get-Module | + Where-Object { + ($_.Name -ine 'pode') -and ($_.Name -inotlike 'microsoft.powershell.*') + } | Sort-Object -Unique + + foreach ($module in $modules) { + # only exported modules? is the module exported? + if ($PodeContext.Server.AutoImport.Modules.ExportOnly -and ($PodeContext.Server.AutoImport.Modules.ExportList -inotcontains $module.Name)) { + continue + } + + # import the module + $path = Find-PodeModuleFile -Name $module.Name + + if (($module.ModuleType -ieq 'Manifest') -or ($path.EndsWith('.ps1'))) { + $PodeContext.RunspaceState.ImportPSModule($path) + } + else { + $PodeContext.Server.Modules[$module] = $path + } + } +} + +function Import-PodeSnapinsIntoRunspaceState +{ + # if non-windows or core, do nothing + if ((Test-PodeIsPSCore) -or (Test-PodeIsUnix)) { + return + } + + # do nothing if disabled + if (!$PodeContext.Server.AutoImport.Snapins.Enabled) { + return + } + + # if export only, and there are none, do nothing + if ($PodeContext.Server.AutoImport.Snapins.ExportOnly -and ($PodeContext.Server.AutoImport.Snapins.ExportList.Length -eq 0)) { + return + } + + # load snapins into runspaces, if allowed + $snapins = (Get-PSSnapin | Where-Object { !$_.IsDefault }).Name | Sort-Object -Unique + + foreach ($snapin in $snapins) { + # only exported snapins? is the snapin exported? + if ($PodeContext.Server.AutoImport.Snapins.ExportOnly -and ($PodeContext.Server.AutoImport.Snapins.ExportList -inotcontains $snapin)) { + continue + } + + $PodeContext.RunspaceState.ImportPSSnapIn($snapin, [ref]$null) + } +} + +function Initialize-PodeAutoImportConfiguration +{ + return @{ + Modules = @{ + Enabled = $true + ExportList = @() + ExportOnly = $false + } + Snapins = @{ + Enabled = $true + ExportList = @() + ExportOnly = $false + } + Functions = @{ + Enabled = $true + ExportList = @() + ExportOnly = $false + } + SecretVaults = @{ + Enabled = $true + SecretManagement = @{ + Enabled = $false + ExportList = @() + ExportOnly = $false + } + } + } +} + +function Import-PodeSecretVaultsIntoRegistry +{ + # do nothing if disabled + if (!$PodeContext.Server.AutoImport.SecretVaults.Enabled) { + return + } + + Import-PodeSecretManagementVaultsIntoRegistry +} + +function Import-PodeSecretManagementVaultsIntoRegistry +{ + # do nothing if disabled + if (!$PodeContext.Server.AutoImport.SecretVaults.SecretManagement.Enabled) { + return + } + + # if export only, and there are none, do nothing + if ($PodeContext.Server.AutoImport.SecretVaults.SecretManagement.ExportOnly -and ($PodeContext.Server.AutoImport.SecretVaults.SecretManagement.ExportList.Length -eq 0)) { + return + } + + # error if SecretManagement module not installed + if (!(Test-PodeModuleInstalled -Name Microsoft.PowerShell.SecretManagement)) { + throw 'Microsoft.PowerShell.SecretManagement module not installed' + } + + # import the module + $null = Import-Module -Name Microsoft.PowerShell.SecretManagement -Force -DisableNameChecking -Scope Global -ErrorAction Stop -Verbose:$false + + # get the current secret vaults + $vaults = @(Get-SecretVault -ErrorAction Stop) + + # register the vaults + foreach ($vault in $vaults) { + # only exported vaults? is the vault exported? + if ($PodeContext.Server.AutoImport.SecretVaults.SecretManagement.ExportOnly -and ($PodeContext.Server.AutoImport.SecretVaults.SecretManagement.ExportList -inotcontains $vault.Name)) { + continue + } + + # is a vault with this name already registered? + if (Test-PodeSecretVault -Name $vault.Name) { + throw "A Secret Vault with the name '$($vault.Name)' has already been registered while auto-importing Secret Vaults" + } + + # register the vault + $PodeContext.Server.Secrets.Vaults[$vault.Name] = @{ + Name = $vault.Name + Type = 'secretmanagement' + Parameters = $vault.VaultParameters + AutoImported = $true + Unlock = $null + Cache = $null + SecretManagement = @{ + VaultName = $vault.Name + ModuleName = $vault.ModulePath + } + } + } +} + +function Read-PodeAutoImportConfiguration +{ + param( + [Parameter()] + [hashtable] + $Configuration + ) + + $impModules = $Configuration.AutoImport.Modules + $impSnapins = $Configuration.AutoImport.Snapins + $impFuncs = $Configuration.AutoImport.Functions + $impSecretVaults = $Configuration.AutoImport.SecretVaults + + return @{ + Modules = @{ + Enabled = (($null -eq $impModules.Enable) -or [bool]$impModules.Enable) + ExportList = @() + ExportOnly = ([bool]$impModules.ExportOnly) + } + Snapins = @{ + Enabled = (($null -eq $impSnapins.Enable) -or [bool]$impSnapins.Enable) + ExportList = @() + ExportOnly = ([bool]$impSnapins.ExportOnly) + } + Functions = @{ + Enabled = (($null -eq $impFuncs.Enable) -or [bool]$impFuncs.Enable) + ExportList = @() + ExportOnly = ([bool]$impFuncs.ExportOnly) + } + SecretVaults = @{ + Enabled = (($null -eq $impSecretVaults.Enable) -or [bool]$impSecretVaults.Enable) + SecretManagement = @{ + Enabled = ((($null -eq $impSecretVaults.Enable) -and (Test-PodeModuleInstalled -Name Microsoft.PowerShell.SecretManagement)) -or [bool]$impSecretVaults.Enable) + ExportList = @() + ExportOnly = ([bool]$impSecretVaults.SecretManagement.ExportOnly) + } + } + } +} + +function Reset-PodeAutoImportConfiguration +{ + $PodeContext.Server.AutoImport.Modules.ExportList = @() + $PodeContext.Server.AutoImport.Snapins.ExportList = @() + $PodeContext.Server.AutoImport.Functions.ExportList = @() + $PodeContext.Server.AutoImport.SecretVaults.SecretManagement.ExportList = @() +} \ No newline at end of file diff --git a/src/Private/Context.ps1 b/src/Private/Context.ps1 index 88eedf7fa..8ee1d4175 100644 --- a/src/Private/Context.ps1 +++ b/src/Private/Context.ps1 @@ -114,23 +114,7 @@ function New-PodeContext } # auto importing (modules, funcs, snap-ins) - $ctx.Server.AutoImport = @{ - Modules = @{ - Enabled = $true - ExportList = @() - ExportOnly = $false - } - Snapins = @{ - Enabled = $true - ExportList = @() - ExportOnly = $false - } - Functions = @{ - Enabled = $true - ExportList = @() - ExportOnly = $false - } - } + $ctx.Server.AutoImport = Initialize-PodeAutoImportConfiguration # basic logging setup $ctx.Server.Logging = @{ @@ -302,6 +286,48 @@ function New-PodeContext # verbs for tcp $ctx.Server.Verbs = @{} + # secrets + $ctx.Server.Secrets = @{ + Vaults = @{ + # NAME = @{ + # TYPE = [CUSTOM|SECRETMANAGEMENT] + # PARAMETERS = @{} + # UNLOCK = @{ + # PASSWORD = SECURESTRING + # EXPIRY = DATE + # ENABLED = [TRUE|FALSE] + # } + # CACHE = @{ + # TTL = 0 + # ENABLED = [TRUE|FALSE] + # } + # CUSTOM = @{ + # READ = SCRIPTBLOCK + # UNLOCK = SCRIPTBLOCK + # REMOVE = SCRIPTBLOCK + # NEW = SCRIPTBLOCK + # UPDATE = SCRIPTBLOCK + # } + # SECRETMANAGEMENT = @{ + # VAULTNAME = NAME + # MODULENAME = [NAME|PATH] + # } + # } + } + Keys = @{ + # NAME = @{ + # VAULT = NAME + # ARGS = [VALUES] + # CACHE = @{ + # EXPIRY = DATE + # VALUE = SECUTESTRING + # TTL = 0 + # ENABLED = [TRUE|FALSE] + # } + # } + } + } + # custom view paths $ctx.Server.Views = @{} @@ -444,126 +470,6 @@ function New-PodeRunspaceState $PodeContext.RunspaceState = $state } -function Import-PodeFunctionsIntoRunspaceState -{ - param( - [Parameter(Mandatory=$true, ParameterSetName='Script')] - [scriptblock] - $ScriptBlock, - - [Parameter(Mandatory=$true, ParameterSetName='File')] - [string] - $FilePath - ) - - # do nothing if disabled - if (!$PodeContext.Server.AutoImport.Functions.Enabled) { - return - } - - # if export only, and there are none, do nothing - if ($PodeContext.Server.AutoImport.Functions.ExportOnly -and ($PodeContext.Server.AutoImport.Functions.ExportList.Length -eq 0)) { - return - } - - # script or file functions? - switch ($PSCmdlet.ParameterSetName.ToLowerInvariant()) { - 'script' { - $funcs = (Get-PodeFunctionsFromScriptBlock -ScriptBlock $ScriptBlock) - } - - 'file' { - $funcs = (Get-PodeFunctionsFromFile -FilePath $FilePath) - } - } - - # looks like we have nothing! - if (($null -eq $funcs) -or ($funcs.Length -eq 0)) { - return - } - - # groups funcs in case there or multiple definitions - $funcs = ($funcs | Group-Object -Property { $_.Name }) - - # import them, but also check if they're exported - foreach ($func in $funcs) { - # only exported funcs? is the func exported? - if ($PodeContext.Server.AutoImport.Functions.ExportOnly -and ($PodeContext.Server.AutoImport.Functions.ExportList -inotcontains $func.Name)) { - continue - } - - # load the function - $funcDef = [System.Management.Automation.Runspaces.SessionStateFunctionEntry]::new($func.Name, $func.Group[-1].Definition) - $PodeContext.RunspaceState.Commands.Add($funcDef) - } -} - -function Import-PodeModulesIntoRunspaceState -{ - # do nothing if disabled - if (!$PodeContext.Server.AutoImport.Modules.Enabled) { - return - } - - # if export only, and there are none, do nothing - if ($PodeContext.Server.AutoImport.Modules.ExportOnly -and ($PodeContext.Server.AutoImport.Modules.ExportList.Length -eq 0)) { - return - } - - # load modules into runspaces, if allowed - $modules = Get-Module | - Where-Object { - ($_.Name -ine 'pode') -and ($_.Name -inotlike 'microsoft.powershell.*') - } | Sort-Object -Unique - - foreach ($module in $modules) { - # only exported modules? is the module exported? - if ($PodeContext.Server.AutoImport.Modules.ExportOnly -and ($PodeContext.Server.AutoImport.Modules.ExportList -inotcontains $module.Name)) { - continue - } - - # import the module - $path = Find-PodeModuleFile -Name $module.Name - - if (($module.ModuleType -ieq 'Manifest') -or ($path.EndsWith('.ps1'))) { - $PodeContext.RunspaceState.ImportPSModule($path) - } - else { - $PodeContext.Server.Modules[$module] = $path - } - } -} - -function Import-PodeSnapinsIntoRunspaceState -{ - # if non-windows or core, do nothing - if ((Test-PodeIsPSCore) -or (Test-PodeIsUnix)) { - return - } - - # do nothing if disabled - if (!$PodeContext.Server.AutoImport.Snapins.Enabled) { - return - } - - # if export only, and there are none, do nothing - if ($PodeContext.Server.AutoImport.Snapins.ExportOnly -and ($PodeContext.Server.AutoImport.Snapins.ExportList.Length -eq 0)) { - return - } - - # load snapins into runspaces, if allowed - $snapins = (Get-PSSnapin | Where-Object { !$_.IsDefault }).Name | Sort-Object -Unique - - foreach ($snapin in $snapins) { - # only exported snapins? is the snapin exported? - if ($PodeContext.Server.AutoImport.Snapins.ExportOnly -and ($PodeContext.Server.AutoImport.Snapins.ExportList -inotcontains $snapin)) { - continue - } - - $PodeContext.RunspaceState.ImportPSSnapIn($snapin, [ref]$null) - } -} - function New-PodeRunspacePools { if ($PodeContext.Server.IsServerless) { @@ -850,7 +756,7 @@ function Open-PodeConfiguration function Set-PodeServerConfiguration { - param ( + param( [Parameter()] [hashtable] $Configuration, @@ -888,23 +794,7 @@ function Set-PodeServerConfiguration } # auto-import - $Context.Server.AutoImport = @{ - Modules = @{ - Enabled = (($null -eq $Configuration.AutoImport.Modules.Enable) -or [bool]$Configuration.AutoImport.Modules.Enable) - ExportList = @() - ExportOnly = ([bool]$Configuration.AutoImport.Modules.ExportOnly) - } - Snapins = @{ - Enabled = (($null -eq $Configuration.AutoImport.Snapins.Enable) -or [bool]$Configuration.AutoImport.Snapins.Enable) - ExportList = @() - ExportOnly = ([bool]$Configuration.AutoImport.Snapins.ExportOnly) - } - Functions = @{ - Enabled = (($null -eq $Configuration.AutoImport.Functions.Enable) -or [bool]$Configuration.AutoImport.Functions.Enable) - ExportList = @() - ExportOnly = ([bool]$Configuration.AutoImport.Functions.ExportOnly) - } - } + $Context.Server.AutoImport = Read-PodeAutoImportConfiguration # request if ([int]$Configuration.Request.Timeout -gt 0) { @@ -1028,6 +918,11 @@ function Set-PodeOutputVariables } foreach ($key in $PodeContext.Server.Output.Variables.Keys) { - Set-Variable -Name $key -Value $PodeContext.Server.Output.Variables[$key] -Force -Scope Global + try { + Set-Variable -Name $key -Value $PodeContext.Server.Output.Variables[$key] -Force -Scope Global + } + catch { + $_ | Write-PodeErrorLog + } } } \ No newline at end of file diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 93080669c..5d0c21ea9 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -117,8 +117,8 @@ function Get-PodeType $type = $Value.GetType() return @{ - 'Name' = $type.Name.ToLowerInvariant(); - 'BaseName' = $type.BaseType.Name.ToLowerInvariant(); + Name = $type.Name.ToLowerInvariant() + BaseName = $type.BaseType.Name.ToLowerInvariant() } } @@ -2450,6 +2450,59 @@ function Convert-PodeQueryStringToHashTable return (ConvertFrom-PodeNameValueToHashTable -Collection $tmpQuery) } +function Convert-PodeScopedVariables +{ + param( + [Parameter()] + [scriptblock] + $ScriptBlock, + + [Parameter()] + [System.Management.Automation.SessionState] + $PSSession, + + [Parameter()] + [ValidateSet('State', 'Session', 'Secret', 'Using')] + $Skip + ) + + # do nothing if no script + if ($null -eq $ScriptBlock) { + if (($null -ne $Skip) -and ($Skip -icontains 'Using')) { + return $ScriptBlock + } + else { + return @($ScriptBlock, $null) + } + } + + # conversions + $usingVars = $null + if (($null -eq $Skip) -or ($Skip -inotcontains 'Using')) { + $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSSession + } + + if (($null -eq $Skip) -or ($Skip -inotcontains 'State')) { + $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock + } + + if (($null -eq $Skip) -or ($Skip -inotcontains 'Session')) { + $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + } + + if (($null -eq $Skip) -or ($Skip -inotcontains 'Secret')) { + $ScriptBlock = Invoke-PodeSecretScriptConversion -ScriptBlock $ScriptBlock + } + + # return + if (($null -ne $Skip) -and ($Skip -icontains 'Using')) { + return $ScriptBlock + } + else { + return @($ScriptBlock, $usingVars) + } +} + function Invoke-PodeStateScriptConversion { param( @@ -2484,6 +2537,40 @@ function Invoke-PodeStateScriptConversion return $ScriptBlock } +function Invoke-PodeSecretScriptConversion +{ + param( + [Parameter()] + [scriptblock] + $ScriptBlock + ) + + # do nothing if no script + if ($null -eq $ScriptBlock) { + return $ScriptBlock + } + + # rename any $secret: vars + $scriptStr = "$($ScriptBlock)" + $found = $false + + while ($scriptStr -imatch '(?\$secret\:(?[a-z0-9_\?]+)\s*=)') { + $found = $true + $scriptStr = $scriptStr.Replace($Matches['full'], "Update-PodeSecret -Name '$($Matches['name'])' -InputObject ") + } + + while ($scriptStr -imatch '(?\$secret\:(?[a-z0-9_\?]+))') { + $found = $true + $scriptStr = $scriptStr.Replace($Matches['full'], "(Get-PodeSecret -Name '$($Matches['name'])')") + } + + if ($found) { + $ScriptBlock = [scriptblock]::Create($scriptStr) + } + + return $ScriptBlock +} + function Invoke-PodeSessionScriptConversion { param( @@ -2982,4 +3069,15 @@ function Set-PodeCronInterval $Cron[$Type] += "/$($Interval)" return ($Value.Length -eq 1) +} + +function Test-PodeModuleInstalled +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return ($null -ne (Get-Module -Name $Name -ListAvailable -ErrorAction Ignore)) } \ No newline at end of file diff --git a/src/Private/Middleware.ps1 b/src/Private/Middleware.ps1 index e76dbc663..6c36e0b9d 100644 --- a/src/Private/Middleware.ps1 +++ b/src/Private/Middleware.ps1 @@ -89,12 +89,8 @@ function New-PodeMiddlewareInternal # if route is empty, set it to root $Route = ConvertTo-PodeRouteRegex -Path $Route - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSSession - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSSession # create the middleware hashtable from a scriptblock $HashTable = @{ diff --git a/src/Private/Routes.ps1 b/src/Private/Routes.ps1 index 1136cc5dc..01d6c337a 100644 --- a/src/Private/Routes.ps1 +++ b/src/Private/Routes.ps1 @@ -589,11 +589,7 @@ function ConvertTo-PodeMiddleware } if ($Middleware[$i] -is [scriptblock]) { - $_script, $_usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $Middleware[$i] -PSSession $PSSession - - # check for state/session vars - $_script = Invoke-PodeStateScriptConversion -ScriptBlock $_script - $_script = Invoke-PodeSessionScriptConversion -ScriptBlock $_script + $_script, $_usingVars = Convert-PodeScopedVariables -ScriptBlock $Middleware[$i] -PSSession $PSSession $Middleware[$i] = @{ Logic = $_script diff --git a/src/Private/Secrets.ps1 b/src/Private/Secrets.ps1 new file mode 100644 index 000000000..574818231 --- /dev/null +++ b/src/Private/Secrets.ps1 @@ -0,0 +1,437 @@ +function Initialize-PodeSecretVault +{ + param( + [Parameter(Mandatory=$true, ValueFromPipeline=$true)] + [hashtable] + $VaultConfig, + + [Parameter(Mandatory=$true)] + [scriptblock] + $ScriptBlock + ) + + Invoke-PodeScriptBlock -ScriptBlock $ScriptBlock -Splat -Arguments @($VaultConfig.Parameters) +} + +function Register-PodeSecretManagementVault +{ + param( + [Parameter(Mandatory=$true, ValueFromPipeline=$true)] + [hashtable] + $VaultConfig, + + [Parameter()] + [string] + $VaultName, + + [Parameter(Mandatory=$true)] + [string] + $ModuleName + ) + + # use the Name for VaultName if not passed + if ([string]::IsNullOrWhiteSpace($VaultName)) { + $VaultName = $VaultConfig.Name + } + + # import the modules + $null = Import-Module -Name Microsoft.PowerShell.SecretManagement -Force -DisableNameChecking -Scope Global -ErrorAction Stop -Verbose:$false + $null = Import-Module -Name $ModuleName -Force -DisableNameChecking -Scope Global -ErrorAction Stop -Verbose:$false + + # attempt to register the vault + $null = Register-SecretVault -Name $VaultName -ModuleName $ModuleName -VaultParameters $VaultConfig.Parameters -Confirm:$false -AllowClobber -ErrorAction Stop + + # all is good, so set the config + $VaultConfig['SecretManagement'] = @{ + VaultName = $VaultName + ModuleName = $ModuleName + } +} + +function Register-PodeSecretCustomVault +{ + param( + [Parameter(Mandatory=$true, ValueFromPipeline=$true)] + [hashtable] + $VaultConfig, + + [Parameter(Mandatory=$true)] + [scriptblock] + $ScriptBlock, + + [Parameter()] + [scriptblock] + $UnlockScriptBlock, + + [Parameter()] + [scriptblock] + $RemoveScriptBlock, + + [Parameter()] + [scriptblock] + $SetScriptBlock, + + [Parameter()] + [scriptblock] + $UnregisterScriptBlock + ) + + # all is good, so set the config + $VaultConfig['Custom'] = @{ + Read = $ScriptBlock + Unlock = $UnlockScriptBlock + Remove = $RemoveScriptBlock + Set = $SetScriptBlock + Unregister = $UnregisterScriptBlock + } +} + +function Unlock-PodeSecretManagementVault +{ + param( + [Parameter(Mandatory=$true, ValueFromPipeline=$true)] + [hashtable] + $VaultConfig + ) + + # do we need to unlock the vault? + if (!$VaultConfig.Unlock.Enabled) { + return $null + } + + # unlock the vault, and return null for the expiry + $null = Unlock-SecretVault -Name $VaultConfig.SecretManagement.VaultName -Password $VaultConfig.Unlock.Secret -ErrorAction Stop + return $null +} + +function Unlock-PodeSecretCustomVault +{ + param( + [Parameter(Mandatory=$true, ValueFromPipeline=$true)] + [hashtable] + $VaultConfig + ) + + # do we need to unlock the vault? + if (!$VaultConfig.Unlock.Enabled) { + return + } + + # do we have an unlock scriptblock + if ($null -eq $VaultConfig.Custom.Unlock) { + throw "No Unlock ScriptBlock supplied for unlocking the vault '$($VaultConfig.Name)'" + } + + # unlock the vault, and get back an expiry + return (Invoke-PodeScriptBlock -ScriptBlock $VaultConfig.Custom.Unlock -Splat -Return -Arguments @( + $VaultConfig.Parameters, + (ConvertFrom-SecureString -SecureString $VaultConfig.Unlock.Secret -AsPlainText) + )) +} + +function Unregister-PodeSecretManagementVault +{ + param( + [Parameter(Mandatory=$true, ValueFromPipeline=$true)] + [hashtable] + $VaultConfig + ) + + # do we need to unregister the vault? + if ($VaultConfig.AutoImported) { + return + } + + # unregister the vault + $null = Unregister-SecretVault -Name $VaultConfig.SecretManagement.VaultName -Confirm:$false -ErrorAction Stop +} + +function Unregister-PodeSecretCustomVault +{ + param( + [Parameter(Mandatory=$true, ValueFromPipeline=$true)] + [hashtable] + $VaultConfig + ) + + # do we need to unregister the vault? + if ($VaultConfig.AutoImported) { + return + } + + # do we have an unregister scriptblock? if not, just do nothing + if ($null -eq $VaultConfig.Custom.Unregister) { + return + } + + # unregister the vault + Invoke-PodeScriptBlock -ScriptBlock $VaultConfig.Custom.Unregister -Splat -Arguments @( + $VaultConfig.Parameters + ) +} + +function Get-PodeSecretManagementKey +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Vault, + + [Parameter(Mandatory=$true)] + [string] + $Key + ) + + # get the vault + $_vault = $PodeContext.Server.Secrets.Vaults[$Vault] + + # fetch the secret + return (Get-Secret -Name $Key -Vault $_vault.SecretManagement.VaultName -AsPlainText -ErrorAction Stop) +} + +function Get-PodeSecretCustomKey +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Vault, + + [Parameter(Mandatory=$true)] + [string] + $Key, + + [Parameter()] + [object[]] + $ArgumentList + ) + + # get the vault + $_vault = $PodeContext.Server.Secrets.Vaults[$Vault] + + # fetch the secret + return (Invoke-PodeScriptBlock -ScriptBlock $_vault.Custom.Read -Splat -Return -Arguments (@( + $_vault.Parameters, + $Key + ) + $ArgumentList)) +} + +function Set-PodeSecretManagementKey +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Vault, + + [Parameter(Mandatory=$true)] + [string] + $Key, + + [Parameter(Mandatory=$true)] + [object] + $Value, + + [Parameter()] + [hashtable] + $Metadata + ) + + # get the vault + $_vault = $PodeContext.Server.Secrets.Vaults[$Vault] + + # set the secret + $null = Set-Secret -Name $Key -Secret $Value -Vault $_vault.SecretManagement.VaultName -Metadata $Metadata -Confirm:$false -ErrorAction Stop +} + +function Set-PodeSecretCustomKey +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Vault, + + [Parameter(Mandatory=$true)] + [string] + $Key, + + [Parameter(Mandatory=$true)] + [object] + $Value, + + [Parameter()] + [hashtable] + $Metadata, + + [Parameter()] + [object[]] + $ArgumentList + ) + + # get the vault + $_vault = $PodeContext.Server.Secrets.Vaults[$Vault] + + # do we have a set scriptblock? + if ($null -eq $_vault.Custom.Set) { + throw "No Set ScriptBlock supplied for updating/creating secrets in the vault '$($_vault.Name)'" + } + + # set the secret + Invoke-PodeScriptBlock -ScriptBlock $_vault.Custom.Set -Splat -Arguments (@( + $_vault.Parameters, + $Key, + $Value, + $Metadata + ) + $ArgumentList) +} + +function Remove-PodeSecretManagementKey +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Vault, + + [Parameter(Mandatory=$true)] + [string] + $Key + ) + + # get the vault + $_vault = $PodeContext.Server.Secrets.Vaults[$Vault] + + # remove the secret + $null = Remove-Secret -Name $Key -Vault $_vault.SecretManagement.VaultName -Confirm:$false -ErrorAction Stop +} + +function Remove-PodeSecretCustomKey +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Vault, + + [Parameter(Mandatory=$true)] + [string] + $Key, + + [Parameter()] + [object[]] + $ArgumentList + ) + + # get the vault + $_vault = $PodeContext.Server.Secrets.Vaults[$Vault] + + # do we have a remove scriptblock? + if ($null -eq $_vault.Custom.Remove) { + throw "No Remove ScriptBlock supplied for removing secrets from the vault '$($_vault.Name)'" + } + + # remove the secret + Invoke-PodeScriptBlock -ScriptBlock $_vault.Custom.Remove -Splat -Arguments (@( + $_vault.Parameters, + $Key + ) + $ArgumentList) +} + +function Start-PodeSecretCacheHousekeeper +{ + if (Test-PodeTimer -Name '__pode_secrets_cache_expiry__') { + return + } + + Add-PodeTimer -Name '__pode_secrets_cache_expiry__' -Interval 60 -ScriptBlock { + $now = [datetime]::UtcNow + + foreach ($key in $PodeContext.Server.Secrets.Keys.Values) { + if (!$key.Cache.Enabled -or ($null -eq $key.Cache.Expiry) -or ($key.Cache.Expiry -gt $now)) { + continue + } + + $key.Cache.Expiry = $null + $key.Cache.Value = $null + } + } +} + +function Start-PodeSecretVaultUnlocker +{ + if (Test-PodeTimer -Name '__pode_secrets_vault_unlock__') { + return + } + + Add-PodeTimer -Name '__pode_secrets_vault_unlock__' -Interval 60 -ScriptBlock { + $now = [datetime]::UtcNow + + foreach ($vault in $PodeContext.Server.Secrets.Vaults.Values) { + if (!$vault.Unlock.Enabled -or ($null -eq $vault.Unlock.Expiry) -or ($vault.Unlock.Expiry -gt $now)) { + continue + } + + Unlock-PodeSecretVault -Name $vault.Name + } + } +} + +function Unregister-PodeSecretVaults +{ + param( + [switch] + $ThrowError + ) + + if (Test-PodeIsEmpty $PodeContext.Server.Secrets.Vaults) { + return + } + + foreach ($vault in $PodeContext.Server.Secrets.Vaults.Values.Name) { + if ([string]::IsNullOrEmpty($vault)) { + continue + } + + try { + Unregister-PodeSecretVault -Name $vault + } + catch { + if ($ThrowError) { + throw + } + else { + $_ | Write-PodeErrorLog + } + } + } +} + +function Protect-PodeSecretValueType +{ + param( + [Parameter(Mandatory=$true)] + [object] + $Value + ) + + if ($Value -is [System.ValueType]) { + $Value = $Value.ToString() + } + + if ([string]::IsNullOrEmpty($Value)) { + $Value = [string]::Empty + } + + if ($Value -is [ordered]) { + $Value = [hashtable]$Value + } + + if (!( + ($Value -is [string]) -or + ($Value -is [securestring]) -or + ($Value -is [hashtable]) -or + ($Value -is [byte[]]) -or + ($Value -is [pscredential]) -or + ($Value -is [System.Management.Automation.OrderedHashtable]) + )) { + throw "Value to set secret to is of an invalid type. Expected either String, SecureString, HashTable, Byte[], or PSCredential. But got: $($Value.GetType().Name)" + } + + return $Value +} \ No newline at end of file diff --git a/src/Private/Server.ps1 b/src/Private/Server.ps1 index b1b9ded52..bae333e93 100644 --- a/src/Private/Server.ps1 +++ b/src/Private/Server.ps1 @@ -19,6 +19,9 @@ function Start-PodeInternalServer # if iis, setup global middleware to validate token Initialize-PodeIISMiddleware + # load any secret vaults + Import-PodeSecretVaultsIntoRegistry + # get the server's script and invoke it - to set up routes, timers, middleware, etc $_script = $PodeContext.Server.Logic if (Test-PodePath -Path $PodeContext.Server.LogicPath -NoStatus) { @@ -183,9 +186,7 @@ function Restart-PodeInternalServer $PodeContext.Tasks.Results.Clear() # auto-importers - $PodeContext.Server.AutoImport.Modules.ExportList = @() - $PodeContext.Server.AutoImport.Snapins.ExportList = @() - $PodeContext.Server.AutoImport.Functions.ExportList = @() + Reset-PodeAutoImportConfiguration # clear middle/endware $PodeContext.Server.Middleware = @() @@ -234,6 +235,11 @@ function Restart-PodeInternalServer # clear up shared state $PodeContext.Server.State.Clear() + # clear up secret vaults/cache + Unregister-PodeSecretVaults -ThrowError + $PodeContext.Server.Secrets.Vaults.Clear() + $PodeContext.Server.Secrets.Keys.Clear() + # clear up output $PodeContext.Server.Output.Variables.Clear() diff --git a/src/Public/Authentication.ps1 b/src/Public/Authentication.ps1 index c8a6ead85..81f0e1259 100644 --- a/src/Public/Authentication.ps1 +++ b/src/Public/Authentication.ps1 @@ -487,14 +487,10 @@ function New-PodeAuthScheme } 'custom' { - $ScriptBlock, $usingScriptVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + $ScriptBlock, $usingScriptVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState if (!(Test-PodeIsEmpty $PostValidator)) { - $PostValidator, $usingPostVars = Invoke-PodeUsingScriptConversion -ScriptBlock $PostValidator -PSSession $PSCmdlet.SessionState - $PostValidator = Invoke-PodeStateScriptConversion -ScriptBlock $PostValidator - $PostValidator = Invoke-PodeSessionScriptConversion -ScriptBlock $PostValidator + $PostValidator, $usingPostVars = Convert-PodeScopedVariables -ScriptBlock $PostValidator -PSSession $PSCmdlet.SessionState } return @{ @@ -761,12 +757,8 @@ function Add-PodeAuth throw 'Sessions are required to use session persistent authentication' } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # add auth method to server $PodeContext.Server.Authentications[$Name] = @{ @@ -1006,11 +998,7 @@ function Add-PodeAuthWindowsAd # if we have a scriptblock, deal with using vars if ($null -ne $ScriptBlock) { - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState } # add Windows AD auth method to server @@ -1270,11 +1258,7 @@ function Add-PodeAuthIIS # if we have a scriptblock, deal with using vars if ($null -ne $ScriptBlock) { - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState } # create the auth scheme for getting the token header @@ -1451,11 +1435,7 @@ function Add-PodeAuthUserFile # if we have a scriptblock, deal with using vars if ($null -ne $ScriptBlock) { - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState } # add Windows AD auth method to server @@ -1602,11 +1582,7 @@ function Add-PodeAuthWindowsLocal # if we have a scriptblock, deal with using vars if ($null -ne $ScriptBlock) { - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState } # add Windows Local auth method to server diff --git a/src/Public/AutoImport.ps1 b/src/Public/AutoImport.ps1 index 0d9baefdd..6fef4b004 100644 --- a/src/Public/AutoImport.ps1 +++ b/src/Public/AutoImport.ps1 @@ -76,4 +76,37 @@ function Export-PodeFunction ) $PodeContext.Server.AutoImport.Functions.ExportList += @($Name) +} + +<# +.SYNOPSIS +Exports Secret Vaults that can be auto-imported by Pode, and into its runspaces. + +.DESCRIPTION +Exports Secret Vaults that can be auto-imported by Pode, and into its runspaces. + +.PARAMETER Name +The Name(s) of a Secret Vault to export. + +.PARAMETER Type +The Type of the Secret Vault to import - only option currently is SecretManagement (default: SecretManagement) + +.EXAMPLE +Export-PodeSecretVault -Name Vault1, Vault2 +#> +function Export-PodeSecretVault +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string[]] + $Name, + + [Parameter()] + [ValidateSet('SecretManagement')] + [string] + $Type = 'SecretManagement' + ) + + $PodeContext.Server.AutoImport.SecretVaults[$Type].ExportList += @($Name) } \ No newline at end of file diff --git a/src/Public/Core.ps1 b/src/Public/Core.ps1 index 63cfb7a44..ec39a867a 100644 --- a/src/Public/Core.ps1 +++ b/src/Public/Core.ps1 @@ -151,8 +151,8 @@ function Start-PodeServer $RootPath = Get-PodeRelativePath -Path $RootPath -RootPath $MyInvocation.PSScriptRoot -JoinRoot -Resolve -TestPath } - # check for state vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -Skip Session, Using # create main context object $PodeContext = New-PodeContext ` @@ -222,6 +222,9 @@ function Start-PodeServer # set output values Set-PodeOutputVariables + # unregister secret vaults + Unregister-PodeSecretVaults + # clean the runspaces and tokens Close-PodeServerInternal -ShowDoneMessage:$ShowDoneMessage diff --git a/src/Public/Events.ps1 b/src/Public/Events.ps1 index 8c9a104d7..07bc7de90 100644 --- a/src/Public/Events.ps1 +++ b/src/Public/Events.ps1 @@ -47,12 +47,8 @@ function Register-PodeEvent throw "$($Type) event already registered: $($Name)" } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # add event $PodeContext.Server.Events[$Type][$Name] = @{ diff --git a/src/Public/Handlers.ps1 b/src/Public/Handlers.ps1 index 69efdf50c..f1a228ab4 100644 --- a/src/Public/Handlers.ps1 +++ b/src/Public/Handlers.ps1 @@ -68,12 +68,8 @@ function Add-PodeHandler $ScriptBlock = Convert-PodeFileToScriptBlock -FilePath $FilePath } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # add the handler Write-Verbose "Adding Handler: [$($Type)] $($Name)" diff --git a/src/Public/Logging.ps1 b/src/Public/Logging.ps1 index 86e1da7f9..820780033 100644 --- a/src/Public/Logging.ps1 +++ b/src/Public/Logging.ps1 @@ -208,9 +208,7 @@ function New-PodeLoggingMethod } 'custom' { - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState return @{ ScriptBlock = $ScriptBlock @@ -436,12 +434,8 @@ function Add-PodeLogger throw "The supplied output Method for the '$($Name)' Logging method requires a valid ScriptBlock" } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # add logging method to server $PodeContext.Server.Logging.Types[$Name] = @{ diff --git a/src/Public/Middleware.ps1 b/src/Public/Middleware.ps1 index 747e0945f..f3b2bddb8 100644 --- a/src/Public/Middleware.ps1 +++ b/src/Public/Middleware.ps1 @@ -371,12 +371,8 @@ function Add-PodeBodyParser throw "There is already a body parser defined for the $($ContentType) content-type" } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState $PodeContext.Server.BodyParsers[$ContentType] = @{ ScriptBlock = $ScriptBlock diff --git a/src/Public/Responses.ps1 b/src/Public/Responses.ps1 index 0faca117e..7592826d0 100644 --- a/src/Public/Responses.ps1 +++ b/src/Public/Responses.ps1 @@ -1376,9 +1376,7 @@ function Set-PodeViewEngine # check if the scriptblock has any using vars if ($null -ne $ScriptBlock) { - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState } # setup view engine config diff --git a/src/Public/Routes.ps1 b/src/Public/Routes.ps1 index c004d3287..4a4b0bc73 100644 --- a/src/Public/Routes.ps1 +++ b/src/Public/Routes.ps1 @@ -231,12 +231,8 @@ function Add-PodeRoute $ScriptBlock = Convert-PodeFileToScriptBlock -FilePath $FilePath } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # convert any middleware into valid hashtables $Middleware = @(ConvertTo-PodeMiddleware -Middleware $Middleware -PSSession $PSCmdlet.SessionState) @@ -731,12 +727,8 @@ function Add-PodeSignalRoute $ScriptBlock = Convert-PodeFileToScriptBlock -FilePath $FilePath } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # add the route(s) Write-Verbose "Adding Route: [$($Method)] $($Path)" @@ -858,12 +850,8 @@ function Add-PodeRouteGroup $Path = $null } - # check if the scriptblock has any using vars - $Routes, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $Routes -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $Routes = Invoke-PodeStateScriptConversion -ScriptBlock $Routes - $Routes = Invoke-PodeSessionScriptConversion -ScriptBlock $Routes + # check for scoped vars + $Routes, $usingVars = Convert-PodeScopedVariables -ScriptBlock $Routes -PSSession $PSCmdlet.SessionState # group details if ($null -ne $RouteGroup) { @@ -1035,12 +1023,8 @@ function Add-PodeStaticRouteGroup $Path = $null } - # check if the scriptblock has any using vars - $Routes, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $Routes -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $Routes = Invoke-PodeStateScriptConversion -ScriptBlock $Routes - $Routes = Invoke-PodeSessionScriptConversion -ScriptBlock $Routes + # check for scoped vars + $Routes, $usingVars = Convert-PodeScopedVariables -ScriptBlock $Routes -PSSession $PSCmdlet.SessionState # group details if ($null -ne $RouteGroup) { @@ -1165,12 +1149,8 @@ function Add-PodeSignalRouteGroup $Path = $null } - # check if the scriptblock has any using vars - $Routes, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $Routes -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $Routes = Invoke-PodeStateScriptConversion -ScriptBlock $Routes - $Routes = Invoke-PodeSessionScriptConversion -ScriptBlock $Routes + # check for scoped vars + $Routes, $usingVars = Convert-PodeScopedVariables -ScriptBlock $Routes -PSSession $PSCmdlet.SessionState # group details if ($null -ne $RouteGroup) { diff --git a/src/Public/Schedules.ps1 b/src/Public/Schedules.ps1 index 58073501c..3c7cfa1cc 100644 --- a/src/Public/Schedules.ps1 +++ b/src/Public/Schedules.ps1 @@ -111,12 +111,8 @@ function Add-PodeSchedule $ScriptBlock = Convert-PodeFileToScriptBlock -FilePath $FilePath } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # add the schedule $parsedCrons = ConvertFrom-PodeCronExpressions -Expressions @($Cron) @@ -328,9 +324,7 @@ function Edit-PodeSchedule # edit scriptblock if supplied if (!(Test-PodeIsEmpty $ScriptBlock)) { - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState $_schedule.Script = $ScriptBlock $_schedule.UsingVariables = $usingVars } @@ -444,6 +438,31 @@ function Get-PodeSchedule return $schedules } +<# +.SYNOPSIS +Tests whether the passed Schedule exists. + +.DESCRIPTION +Tests whether the passed Schedule exists by its name. + +.PARAMETER Name +The Name of the Schedule. + +.EXAMPLE +if (Test-PodeSchedule -Name ScheduleName) { } +#> +function Test-PodeSchedule +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return (($null -ne $PodeContext.Schedules.Items) -and $PodeContext.Schedules.Items.ContainsKey($Name)) +} + <# .SYNOPSIS Get the next trigger time for a Schedule. diff --git a/src/Public/Secrets.ps1 b/src/Public/Secrets.ps1 new file mode 100644 index 000000000..240969873 --- /dev/null +++ b/src/Public/Secrets.ps1 @@ -0,0 +1,902 @@ +<# +TODO: +- DOCS! + - auto-import +#> + +<# +.SYNOPSIS +Register a Secret Vault. + +.DESCRIPTION +Register a Secret Vault, which is defined by either custom logic or using the SecretManagement module. + +.PARAMETER Name +The unique friendly Name of the Secret Vault within Pode. + +.PARAMETER VaultParameters +A hashtable of extra parameters that should be supplied to either the SecretManagement module, or custom scriptblocks. + +.PARAMETER UnlockSecret +An optional Secret to be used to unlock the Secret Vault if need. + +.PARAMETER UnlockSecureSecret +An optional Secret, as a SecureString, to be used to unlock the Secret Vault if need. + +.PARAMETER Unlock +If supplied, the Secret Vault will be unlocked now, after being registered with Pode. + +.PARAMETER CacheTtl +An optional number of minutes that Secrets should be cached for. (Default: 0) + +.PARAMETER InitScriptBlock +An optional scriptblock to run before the Secret Vault is registered, letting you initialise any connection, contexts, etc. + +.PARAMETER VaultName +For SecretManagement module Secret Vaults, you can use thie parameter to specify the actual Vault name, and use the above Name parameter as a more friendly name if required. + +.PARAMETER ModuleName +For SecretManagement module Secret Vaults, this is the name/path of the extension module to be used. + +.PARAMETER ScriptBlock +For custom Secret Vaults, this is a scriptblock used to read the Secret from the Vault. + +.PARAMETER UnlockScriptBlock +For custom Secret Vaults, this is an optional scriptblock used to unlock the Secret Vault. + +.PARAMETER RemoveScriptBlock +For custom Secret Vaults, this is an optional scriptblock used to remove a Secret from the Vault. + +.PARAMETER SetScriptBlock +For custom Secret Vaults, this is an optional scriptblock used to create/update a Secret in the Vault. + +.PARAMETER UnregisterScriptBlock +For custom Secret Vaults, this is an optional scriptblock used unregister the Secret Vault with any custom clean-up logic. + +.EXAMPLE +Register-PodeSecretVault -Name 'VaultName' -ModuleName 'Az.KeyVault' -VaultParameters @{ AZKVaultName = $name; SubscriptionId = $subId } + +.EXAMPLE +Register-PodeSecretVault -Name 'VaultName' -VaultParameters @{ Address = 'http://127.0.0.1:8200' } -ScriptBlock { ... } +#> +function Register-PodeSecretVault +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter()] + [hashtable] + $VaultParameters, + + [Parameter()] + [string] + $UnlockSecret, + + [Parameter()] + [securestring] + $UnlockSecureSecret, + + [switch] + $Unlock, + + [Parameter()] + [int] + $CacheTtl = 0, # in minutes + + [Parameter()] + [scriptblock] + $InitScriptBlock, + + [Parameter(ParameterSetName='SecretManagement')] + [string] + $VaultName, + + [Parameter(Mandatory=$true, ParameterSetName='SecretManagement')] + [string] + $ModuleName, + + [Parameter(Mandatory=$true, ParameterSetName='Custom')] + [scriptblock] + $ScriptBlock, # Read a secret + + [Parameter(ParameterSetName='Custom')] + [scriptblock] + $UnlockScriptBlock, + + [Parameter(ParameterSetName='Custom')] + [scriptblock] + $RemoveScriptBlock, + + [Parameter(ParameterSetName='Custom')] + [scriptblock] + $SetScriptBlock, + + [Parameter(ParameterSetName='Custom')] + [scriptblock] + $UnregisterScriptBlock + ) + + # has the vault already been registered? + if (Test-PodeSecretVault -Name $Name) { + $autoImported = [string]::Empty + if ($PodeContext.Server.Secrets.Vaults[$Name].AutoImported) { + $autoImported = ' from auto-importing' + } + + throw "A Secret Vault with the name '$($Name)' has already been registered$($autoImported)" + } + + # base vault config + if (![string]::IsNullOrEmpty($UnlockSecret)) { + $UnlockSecureSecret = $UnlockSecret | ConvertTo-SecureString -AsPlainText -Force + } + + $vault = @{ + Name = $Name + Type = $PSCmdlet.ParameterSetName.ToLowerInvariant() + Parameters = $VaultParameters + AutoImported = $false + Unlock = @{ + Secret = $UnlockSecureSecret + Expiry = $null + Enabled = $Unlock.IsPresent + } + Cache = @{ + Ttl = $CacheTtl + Enabled = ($CacheTtl -gt 0) + } + } + + # initialise the secret vault + if ($null -ne $InitScriptBlock) { + $vault | Initialize-PodeSecretVault -ScriptBlock $InitScriptBlock + } + + # set vault config depending on vault type + switch ($vault.Type) { + 'custom' { + $vault | Register-PodeSecretCustomVault ` + -ScriptBlock $ScriptBlock ` + -UnlockScriptBlock $UnlockScriptBlock ` + -RemoveScriptBlock $RemoveScriptBlock ` + -SetScriptBlock $SetScriptBlock ` + -UnregisterScriptBlock $UnregisterScriptBlock + } + + 'secretmanagement' { + $vault | Register-PodeSecretManagementVault ` + -VaultName $VaultName ` + -ModuleName $ModuleName + } + } + + # create timer to clear cached secrets every minute + Start-PodeSecretCacheHousekeeper + + # add vault config to context + $PodeContext.Server.Secrets.Vaults[$Name] = $vault + + # unlock the vault? + if ($Unlock) { + Unlock-PodeSecretVault -Name $Name + } +} + +<# +.SYNOPSIS +Unregister a Secret Vault. + +.DESCRIPTION +Unregister a Secret Vault. If the Vault was via the SecretManagement module it will also be unregistered there as well. + +.PARAMETER Name +The Name of the Secret Vault in Pode to unregister. + +.EXAMPLE +Unregister-PodeSecretVault -Name 'VaultName' +#> +function Unregister-PodeSecretVault +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + # has the vault been registered? + if (!(Test-PodeSecretVault -Name $Name)) { + return + } + + # get vault + $vault = $PodeContext.Server.Secrets.Vaults[$Name] + + # unlock depending on vault type, and set expiry + switch ($vault.Type) { + 'custom' { + $vault | Unregister-PodeSecretCustomVault + } + + 'secretmanagement' { + $vault | Unregister-PodeSecretManagementVault + } + } + + # unregister from Pode + $null = $PodeContext.Server.Secrets.Vaults.Remove($Name) +} + +<# +.SYNOPSIS +Unlock the Secret Vault. + +.DESCRIPTION +Unlock the Secret Vault. + +.PARAMETER Name +The Name of the Secret Vault in Pode to be unlocked. + +.EXAMPLE +Unlock-PodeSecretVault -Name 'VaultName' +#> +function Unlock-PodeSecretVault +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + # has the vault been registered? + if (!(Test-PodeSecretVault -Name $Name)) { + throw "No Secret Vault with the name '$($Name)' has been registered" + } + + # get vault + $vault = $PodeContext.Server.Secrets.Vaults[$Name] + $expiry = $null + + # is unlocking even enabled? + if (!$vault.Unlock.Enabled) { + return + } + + # unlock depending on vault type, and set expiry + switch ($vault.Type) { + 'custom' { + $expiry = $vault | Unlock-PodeSecretCustomVault + } + + 'secretmanagement' { + $expiry = $vault | Unlock-PodeSecretManagementVault + } + } + + # if we have an expiry returned, set to UTC and configure unlock schedule + if ($null -ne $expiry) { + $expiry = ([datetime]$expiry).ToUniversalTime() + if ($expiry -le [datetime]::UtcNow) { + throw "Secret Vault unlock expiry date is in the past (UTC): $($expiry)" + } + + $vault.Unlock.Expiry = $expiry + Start-PodeSecretVaultUnlocker + } +} + +<# +.SYNOPSIS +Fetches and returns information of a Secret Vault. + +.DESCRIPTION +Fetches and returns information of a Secret Vault. + +.PARAMETER Name +The Name(s) of a Secret Vault to retrieve. + +.EXAMPLE +$vault = Get-PodeSecretVault -Name 'VaultName' + +.EXAMPLE +$vaults = Get-PodeSecretVault -Name 'VaultName1', 'VaultName2' +#> +function Get-PodeSecretVault +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string[]] + $Name + ) + + $vaults = $PodeContext.Server.Secrets.Vaults.Values + + # further filter by vault names + if (($null -ne $Name) -and ($Name.Length -gt 0)) { + $vaults = @(foreach ($_name in $Name) { + foreach ($vault in $vaults) { + if ($vault.Name -ine $_name) { + continue + } + + $vault + } + }) + } + + # return + return $vaults +} + +<# +.SYNOPSIS +Tests if a Secret Vault has been registered. + +.DESCRIPTION +Tests if a Secret Vault has been registered. + +.PARAMETER Name +The Name of the Secret Vault to test. + +.EXAMPLE +if (Test-PodeSecretVault -Name 'VaultName') { ... } +#> +function Test-PodeSecretVault +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return (($null -ne $PodeContext.Server.Secrets.Vaults) -and $PodeContext.Server.Secrets.Vaults.ContainsKey($Name)) +} + +<# +.SYNOPSIS +Mount a Secret from a Secret Vault. + +.DESCRIPTION +Mount a Secret from a Secret Vault, so it can be more easily referenced and support caching. + +.PARAMETER Name +A unique friendly Name for the Secret. + +.PARAMETER Vault +The friendly name of the Secret Vault this Secret can be found in. + +.PARAMETER Property +An optional array of Properties to be returned if the Secret contains multiple properties. + +.PARAMETER ExpandProperty +An optional Property to be expanded from the Secret and return if it contains multiple properties. + +.PARAMETER Key +The Key/Path of the Secret within the Secret Vault. + +.PARAMETER ArgumentList +An optional array of Arguments to be supplied to a custom Secret Vault's scriptblocks. + +.PARAMETER CacheTtl +An optional number of minutes to Cache the Secret's value for. You can use this parameter to override the Secret Vault's value. (Default: -1) +If the value is -1 it uses the Secret Vault's CacheTtl. A value of 0 is to disable caching for this Secret. A value >0 overrides the Secret Vault. + +.EXAMPLE +Mount-PodeSecret -Name 'SecretName' -Vault 'VaultName' -Key 'path/to/secret' -ExpandProperty 'foo' + +.EXAMPLE +Mount-PodeSecret -Name 'SecretName' -Vault 'VaultName' -Key 'key_of_secret' -CacheTtl 5 +#> +function Mount-PodeSecret +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter(Mandatory=$true)] + [string] + $Vault, + + [Parameter()] + [string[]] + $Property, + + [Parameter()] + [string] + $ExpandProperty, + + [Parameter(Mandatory=$true)] + [string] + $Key, + + [Parameter()] + [object[]] + $ArgumentList, + + # in minutes (-1 means use the vault default, 0 is off, anything higher than 0 is an override) + [Parameter()] + [int] + $CacheTtl = -1 + ) + + # has the secret been mounted already? + if (Test-PodeSecret -Name $Name) { + throw "A Secret with the name '$($Name)' has already been mounted" + } + + # does the vault exist? + if (!(Test-PodeSecretVault -Name $Vault)) { + throw "No Secret Vault with the name '$($Vault)' has been registered" + } + + # check properties + if (!(Test-PodeIsEmpty $Property) -and !(Test-PodeIsEmpty $ExpandProperty)) { + throw 'You can only provide one of either Property or ExpandPropery, but not both' + } + + # which cache value? + if ($CacheTtl -lt 0) { + $CacheTtl = [int]$PodeContext.Server.Secrets.Vaults[$Vault].Cache.Ttl + } + + # mount secret reference + $props = $Property + if (![string]::IsNullOrWhiteSpace($ExpandProperty)) { + $props = $ExpandProperty + } + + $PodeContext.Server.Secrets.Keys[$Name] = @{ + Key = $Key + Properties = @{ + Fields = $props + Expand = (![string]::IsNullOrWhiteSpace($ExpandProperty)) + Enabled = (!(Test-PodeIsEmpty $props)) + } + Vault = $Vault + Arguments = $ArgumentList + Cache = @{ + Ttl = $CacheTtl + Enabled = ($CacheTtl -gt 0) + } + } +} + +<# +.SYNOPSIS +Dismount a previously mounted Secret. + +.DESCRIPTION +Dismount a previously mounted Secret. + +.PARAMETER Name +The friendly Name of the Secret. + +.PARAMETER Remove +If supplied, the Secret will also be removed from the Secret Vault as well. + +.EXAMPLE +Dismount-PodeSecret -Name 'SecretName' + +.EXAMPLE +Dismount-PodeSecret -Name 'SecretName' -Remove +#> +function Dismount-PodeSecret +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [switch] + $Remove + ) + + # do nothing if the secret hasn't been mounted, unless Remove is specified + if (!(Test-PodeSecret -Name $Name)) { + if ($Remove) { + throw "No Secret with the name '$($Name)' has been mounted to be removed from a Secret Vault" + } + + return + } + + # if "remove" switch passed, remove the secret from the vault as well + if ($Remove) { + $secret = $PodeContext.Server.Secrets.Keys[$Name] + Remove-PodeSecret -Key $secret.Key -Vault $secret.Vault -ArgumentList $secret.Arguments + } + + # remove reference + $null = $PodeContext.Server.Secrets.Keys.Remove($Name) +} + +<# +.SYNOPSIS +Retrieve the value of a mounted Secret. + +.DESCRIPTION +Retrieve the value of a mounted Secret from a Secret Vault. You can also use "$value = $secret:" syntax in certain places. + +.PARAMETER Name +The friendly Name of a Secret. + +.EXAMPLE +$value = Get-PodeSecret -Name 'SecretName' + +.EXAMPLE +$value = $secret:SecretName +#> +function Get-PodeSecret +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + # has the secret been mounted? + if (!(Test-PodeSecret -Name $Name)) { + throw "No Secret with the name '$($Name)' has been mounted" + } + + # get the secret and vault + $secret = $PodeContext.Server.Secrets.Keys[$Name] + + # is the value cached? + if ($secret.Cache.Enabled -and ($null -ne $secret.Cache.Expiry) -and ($secret.Cache.Expiry -gt [datetime]::UtcNow)) { + return $secret.Cache.Value + } + + # fetch the secret depending on vault type + switch ($PodeContext.Server.Secrets.Vaults[$secret.Vault].Type) { + 'custom' { + $value = Get-PodeSecretCustomKey -Vault $secret.Vault -Key $secret.Key -ArgumentList $secret.Arguments + } + + 'secretmanagement' { + $value = Get-PodeSecretManagementKey -Vault $secret.Vault -Key $secret.Key + } + } + + # filter the value by any properties + if ($secret.Properties.Enabled) { + if ($secret.Properties.Expand) { + $value = Select-Object -InputObject $value -ExpandProperty $secret.Properties.Fields + } + else { + $value = Select-Object -InputObject $value -Property $secret.Properties.Fields + } + } + + # cache the value if needed + if ($secret.Cache.Enabled) { + $secret.Cache.Value = $value + $secret.Cache.Expiry = [datetime]::UtcNow.AddMinutes($secret.Cache.Ttl) + } + + # return value + return $value +} + +<# +.SYNOPSIS +Test if a Secret has been mounted. + +.DESCRIPTION +Test if a Secret has been mounted. + +.PARAMETER Name +The friendly Name of a Secret. + +.EXAMPLE +if (Test-PodeSecret -Name 'SecretName') { ... } +#> +function Test-PodeSecret +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return (($null -ne $PodeContext.Server.Secrets.Keys) -and $PodeContext.Server.Secrets.Keys.ContainsKey($Name)) +} + +<# +.SYNOPSIS +Update the value of a mounted Secret. + +.DESCRIPTION +Update the value of a mounted Secret in a Secret Vault. You can also use "$secret: = $value" syntax in certain places. + +.PARAMETER Name +The friendly Name of a Secret. + +.PARAMETER InputObject +The value to use when updating the Secret. +Only the following object types are supported: byte[], string, securestring, pscredential, hashtable. + +.PARAMETER Metadata +An optional Metadata hashtable. + +.EXAMPLE +Update-PodeSecret -Name 'SecretName' -InputObject @{ key = value } + +.EXAMPLE +Update-PodeSecret -Name 'SecretName' -InputObject 'value' + +.EXAMPLE +$secret:SecretName = 'value' +#> +function Update-PodeSecret +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + #> byte[], string, securestring, pscredential, hashtable + [Parameter(Mandatory=$true, ValueFromPipeline=$true)] + [object] + $InputObject, + + [Parameter()] + [hashtable] + $Metadata + ) + + # has the secret been mounted? + if (!(Test-PodeSecret -Name $Name)) { + throw "No Secret with the name '$($Name)' has been mounted" + } + + # make sure the value type is correct + $InputObject = Protect-PodeSecretValueType -Value $InputObject + + # get the secret and vault + $secret = $PodeContext.Server.Secrets.Keys[$Name] + + # reset the cache if enabled + if ($secret.Cache.Enabled) { + $secret.Cache.Value = $InputObject + $secret.Cache.Expiry = [datetime]::UtcNow.AddMinutes($secret.Cache.Ttl) + } + + # if we're expanding a property, convert this to a hashtable + if ($secret.Properties.Enabled -and $secret.Properties.Expand) { + $InputObject = @{ + "$($secret.Properties.Fields)" = $InputObject + } + } + + # set the secret depending on vault type + switch ($PodeContext.Server.Secrets.Vaults[$secret.Vault].Type) { + 'custom' { + Set-PodeSecretCustomKey -Vault $secret.Vault -Key $secret.Key -Value $InputObject -Metadata $Metadata -ArgumentList $secret.Arguments + } + + 'secretmanagement' { + Set-PodeSecretManagementKey -Vault $secret.Vault -Key $secret.Key -Value $InputObject -Metadata $Metadata + } + } +} + +<# +.SYNOPSIS +Remove a Secret from a Secret Vault. + +.DESCRIPTION +Remove a Secret from a Secret Vault. To remove a mounted Secret, you can pass the Remove switch to Dismount-PodeSecret. + +.PARAMETER Key +The Key/Path of the Secret within the Secret Vault. + +.PARAMETER Vault +The friendly name of the Secret Vault this Secret can be found in. + +.PARAMETER ArgumentList +An optional array of Arguments to be supplied to a custom Secret Vault's scriptblocks. + +.EXAMPLE +Remove-PodeSecret -Key 'path/to/secret' -Vault 'VaultName' +#> +function Remove-PodeSecret +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Key, + + [Parameter(Mandatory=$true)] + [string] + $Vault, + + [Parameter()] + [object[]] + $ArgumentList + ) + + # has the vault been registered? + if (!(Test-PodeSecretVault -Name $Vault)) { + throw "No Secret Vault with the name '$($Vault)' has been registered" + } + + # remove the secret depending on vault type + switch ($PodeContext.Server.Secrets.Vaults[$Vault].Type) { + 'custom' { + Remove-PodeSecretCustomKey -Vault $Vault -Key $Key -ArgumentList $ArgumentList + } + + 'secretmanagement' { + Remove-PodeSecretManagementKey -Vault $Vault -Key $Key + } + } +} + +<# +.SYNOPSIS +Read a Secret from a Secret Vault. + +.DESCRIPTION +Read a Secret from a Secret Vault. + +.PARAMETER Key +The Key/Path of the Secret within the Secret Vault. + +.PARAMETER Vault +The friendly name of the Secret Vault this Secret can be found in. + +.PARAMETER Property +An optional array of Properties to be returned if the Secret contains multiple properties. + +.PARAMETER ExpandProperty +An optional Property to be expanded from the Secret and return if it contains multiple properties. + +.PARAMETER ArgumentList +An optional array of Arguments to be supplied to a custom Secret Vault's scriptblocks. + +.EXAMPLE +$value = Read-PodeSecret -Key 'path/to/secret' -Vault 'VaultName' + +.EXAMPLE +$value = Read-PodeSecret -Key 'key_of_secret' -Vault 'VaultName' -Property prop1, prop2 +#> +function Read-PodeSecret +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Key, + + [Parameter(Mandatory=$true)] + [string] + $Vault, + + [Parameter()] + [string[]] + $Property, + + [Parameter()] + [string] + $ExpandProperty, + + [Parameter()] + [object[]] + $ArgumentList + ) + + # has the vault been registered? + if (!(Test-PodeSecretVault -Name $Vault)) { + throw "No Secret Vault with the name '$($Vault)' has been registered" + } + + # fetch the secret depending on vault type + switch ($PodeContext.Server.Secrets.Vaults[$Vault].Type) { + 'custom' { + $value = Get-PodeSecretCustomKey -Vault $Vault -Key $Key -ArgumentList $ArgumentList + } + + 'secretmanagement' { + $value = Get-PodeSecretManagementKey -Vault $Vault -Key $Key + } + } + + # filter the value by any properties + if (![string]::IsNullOrWhiteSpace($ExpandProperty)) { + $value = Select-Object -InputObject $value -ExpandProperty $ExpandProperty + } + elseif (![string]::IsNullOrEmpty($Property)) { + $value = Select-Object -InputObject $value -Property $Property + } + + # return value + return $value +} + +<# +.SYNOPSIS +Create/update a Secret in a Secret Vault. + +.DESCRIPTION +Create/update a Secret in a Secret Vault. + +.PARAMETER Key +The Key/Path of the Secret within the Secret Vault. + +.PARAMETER Vault +The friendly name of the Secret Vault this Secret should be created in. + +.PARAMETER InputObject +The value to use when updating the Secret. +Only the following object types are supported: byte[], string, securestring, pscredential, hashtable. + +.PARAMETER Metadata +An optional Metadata hashtable. + +.PARAMETER ArgumentList +An optional array of Arguments to be supplied to a custom Secret Vault's scriptblocks. + +.EXAMPLE +Set-PodeSecret -Key 'path/to/secret' -Vault 'VaultName' -InputObject 'value' + +.EXAMPLE +Set-PodeSecret -Key 'key_of_secret' -Vault 'VaultName' -InputObject @{ key = value } +#> +function Set-PodeSecret +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Key, + + [Parameter(Mandatory=$true)] + [string] + $Vault, + + #> byte[], string, securestring, pscredential, hashtable + [Parameter(Mandatory=$true, ValueFromPipeline=$true)] + [object] + $InputObject, + + [Parameter()] + [hashtable] + $Metadata, + + [Parameter()] + [object[]] + $ArgumentList + ) + + # has the vault been registered? + if (!(Test-PodeSecretVault -Name $Vault)) { + throw "No Secret Vault with the name '$($Vault)' has been registered" + } + + # make sure the value type is correct + $InputObject = Protect-PodeSecretValueType -Value $InputObject + + # set the secret depending on vault type + switch ($PodeContext.Server.Secrets.Vaults[$Vault].Type) { + 'custom' { + Set-PodeSecretCustomKey -Vault $Vault -Key $Key -Value $InputObject -Metadata $Metadata -ArgumentList $ArgumentList + } + + 'secretmanagement' { + Set-PodeSecretManagementKey -Vault $Vault -Key $Key -Value $InputObject -Metadata $Metadata + } + } +} \ No newline at end of file diff --git a/src/Public/Tasks.ps1 b/src/Public/Tasks.ps1 index dd1c346dc..ad0a6d331 100644 --- a/src/Public/Tasks.ps1 +++ b/src/Public/Tasks.ps1 @@ -53,12 +53,8 @@ function Add-PodeTask $ScriptBlock = Convert-PodeFileToScriptBlock -FilePath $FilePath } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # add the task $PodeContext.Tasks.Enabled = $true @@ -267,9 +263,7 @@ function Edit-PodeTask # edit scriptblock if supplied if (!(Test-PodeIsEmpty $ScriptBlock)) { - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState $_task.Script = $ScriptBlock $_task.UsingVariables = $usingVars } diff --git a/src/Public/Timers.ps1 b/src/Public/Timers.ps1 index dd417917a..cda89acb1 100644 --- a/src/Public/Timers.ps1 +++ b/src/Public/Timers.ps1 @@ -105,12 +105,8 @@ function Add-PodeTimer $ScriptBlock = Convert-PodeFileToScriptBlock -FilePath $FilePath } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # calculate the next tick time (based on Skip) $NextTriggerTime = [DateTime]::Now.AddSeconds($Interval) @@ -275,9 +271,7 @@ function Edit-PodeTimer # edit scriptblock if supplied if (!(Test-PodeIsEmpty $ScriptBlock)) { - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState $_timer.Script = $ScriptBlock $_timer.UsingVariables = $usingVars } @@ -332,6 +326,31 @@ function Get-PodeTimer return $timers } +<# +.SYNOPSIS +Tests whether the passed Timer exists. + +.DESCRIPTION +Tests whether the passed Timer exists by its name. + +.PARAMETER Name +The Name of the Timer. + +.EXAMPLE +if (Test-PodeTimer -Name TimerName) { } +#> +function Test-PodeTimer +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return (($null -ne $PodeContext.Timers.Items) -and $PodeContext.Timers.Items.ContainsKey($Name)) +} + <# .SYNOPSIS Automatically loads timer ps1 files diff --git a/src/Public/Utilities.ps1 b/src/Public/Utilities.ps1 index abd04204c..1b2d46e7b 100644 --- a/src/Public/Utilities.ps1 +++ b/src/Public/Utilities.ps1 @@ -342,12 +342,8 @@ function Add-PodeEndware $ArgumentList ) - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # add the scriptblock to array of endware that needs to be run $PodeContext.Server.Endware += @{ diff --git a/src/Public/Verbs.ps1 b/src/Public/Verbs.ps1 index f1a352de7..f3911f0f0 100644 --- a/src/Public/Verbs.ps1 +++ b/src/Public/Verbs.ps1 @@ -95,12 +95,8 @@ function Add-PodeVerb $ScriptBlock = Convert-PodeFileToScriptBlock -FilePath $FilePath } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # add the verb(s) Write-Verbose "Adding Verb: $($Verb)" diff --git a/src/Public/WebSockets.ps1 b/src/Public/WebSockets.ps1 index 754b48145..c7f7078e6 100644 --- a/src/Public/WebSockets.ps1 +++ b/src/Public/WebSockets.ps1 @@ -126,12 +126,8 @@ function Connect-PodeWebSocket $ScriptBlock = Convert-PodeFileToScriptBlock -FilePath $FilePath } - # check if the scriptblock has any using vars - $ScriptBlock, $usingVars = Invoke-PodeUsingScriptConversion -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState - - # check for state/session vars - $ScriptBlock = Invoke-PodeStateScriptConversion -ScriptBlock $ScriptBlock - $ScriptBlock = Invoke-PodeSessionScriptConversion -ScriptBlock $ScriptBlock + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState # connect try { diff --git a/tests/unit/Server.Tests.ps1 b/tests/unit/Server.Tests.ps1 index 493324e63..02f5f6969 100644 --- a/tests/unit/Server.Tests.ps1 +++ b/tests/unit/Server.Tests.ps1 @@ -99,9 +99,9 @@ Describe 'Restart-PodeInternalServer' { It 'Resetting the server values' { $PodeContext = @{ Tokens = @{ - Cancellation = New-Object System.Threading.CancellationTokenSource; - Restart = New-Object System.Threading.CancellationTokenSource; - }; + Cancellation = New-Object System.Threading.CancellationTokenSource + Restart = New-Object System.Threading.CancellationTokenSource + } Server = @{ Routes = @{ GET = @{ 'key' = 'value' } @@ -153,6 +153,9 @@ Describe 'Restart-PodeInternalServer' { Modules = @{ Exported = @() } Snapins = @{ Exported = @() } Functions = @{ Exported = @() } + SecretVaults = @{ + SecretManagement = @{ Exported = @() } + } } Views = @{ 'key' = 'value' } Events = @{ @@ -166,6 +169,10 @@ Describe 'Restart-PodeInternalServer' { PermissionsPolicy = @{} } } + Secrets = @{ + Vaults = @{} + Keys = @{} + } } Metrics = @{ Server = @{ From b4fd354b2fe82f59a0f6bb4905f8eec81d7241bc Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Fri, 30 Dec 2022 20:54:47 +0000 Subject: [PATCH 17/52] #980: update docs for secret support --- docs/Tutorials/Scoping.md | 101 ++++++++ docs/Tutorials/Secrets/Overview.md | 231 ++++++++++++++++++ docs/Tutorials/Secrets/Types/Custom.md | 220 +++++++++++++++++ .../Secrets/Types/SecretManagement.md | 87 +++++++ src/Pode.psd1 | 2 +- src/Private/Context.ps1 | 40 +-- src/Private/Secrets.ps1 | 26 +- src/Public/Secrets.ps1 | 29 ++- 8 files changed, 684 insertions(+), 52 deletions(-) create mode 100644 docs/Tutorials/Secrets/Overview.md create mode 100644 docs/Tutorials/Secrets/Types/Custom.md create mode 100644 docs/Tutorials/Secrets/Types/SecretManagement.md diff --git a/docs/Tutorials/Scoping.md b/docs/Tutorials/Scoping.md index 9d0ec0b91..c11b7e44b 100644 --- a/docs/Tutorials/Scoping.md +++ b/docs/Tutorials/Scoping.md @@ -258,3 +258,104 @@ Start-PodeServer -ScriptBlock { } } ``` + +## Secret Vaults + +This mostly only applies to vaults registered via the SecretManagement module, and its `Register-SecretVault` function. You can register vaults as the user that will run your Pode server, before you start the server using the normal `Register-SecretVault` function. On start, Pode will detect these vaults and will automatically register then within Pode for you - ready to be used for mounting secrets with [`Mount-PodeSecret`](../../Functions/Secrets/Mount-PodeSecret). + +Below, `AzVault1` will be automatically registered within Pode: + +```powershell +Register-SecretVault -Name 'AzVault1' -ModuleName 'Az.KeyVault' -VaultParameters @{ + AZKVaultName = 'VaultNameInAzure' + SubscriptionId = $SubscriptionId +} + +Start-PodeServer -ScriptBlock { + Add-PodeEndpoint -Address localhost -Port 9000 -Protocol Http + + Mount-PodeSecret -Name 'SecretName' -Vault 'AzVault1' -Key 'SecretKeyNameInVault' + + Add-PodeRoute -Method Get -Path '/' -ScriptBlock { + Write-PodeJsonResponse @{ Value = $secret:SecretName } + } +} +``` + +### Disable + +If Pode detects that the SecretManagement module isn't installed, then this functionality is automatically disabled. However, if it is, and you don't need any secret vaults, or want to stop the auto-importing from occurring, you can use disable it via the `server.psd1` [configuration file](../Configuration). + +The following will disable it for every registration type: + +```powershell +@{ + Server = @{ + AutoImport = @{ + SecretVaults = @{ + Enable = $false + } + } + } +} +``` + +Whereas the following will disable it solely for the SecretManagement type: + +```powershell +@{ + Server = @{ + AutoImport = @{ + SecretVaults = @{ + SecretManagement = @{ + Enable = $false + } + } + } + } +} +``` + +### Export + +If you want finer control over which vaults are auto-imported, then you can set the auto-import to use an export list. To do so, you set Pode to only import exported secret vaults: + +```powershell +@{ + Server = @{ + AutoImport = @{ + SecretVaults = @{ + SecretManagement = @{ + Enable = $true + ExportOnly = $true + } + } + } + } +} +``` + +Then you can "export" vaults that Pode should import by using [`Export-PodeSecretVault`](../../Functions/AutoImport/Export-PodeSecretVault). Below only `AzVault2` will be auto-imported into Pode: + +```powershell +Register-SecretVault -Name 'AzVault1' -ModuleName 'Az.KeyVault' -VaultParameters @{ + AZKVaultName = 'VaultNameInAzure1' + SubscriptionId = $SubscriptionId +} + +Register-SecretVault -Name 'AzVault2' -ModuleName 'Az.KeyVault' -VaultParameters @{ + AZKVaultName = 'VaultNameInAzure2' + SubscriptionId = $SubscriptionId +} + +Start-PodeServer -ScriptBlock { + Add-PodeEndpoint -Address localhost -Port 9000 -Protocol Http + + Export-PodeSecretVault -Name AzVault2 + Mount-PodeSecret -Name 'SecretName' -Vault 'AzVault2' -Key 'SecretKeyNameInVault' + + Add-PodeRoute -Method Get -Path '/' -ScriptBlock { + Write-PodeJsonResponse @{ Value = $secret:SecretName } + } +} +``` diff --git a/docs/Tutorials/Secrets/Overview.md b/docs/Tutorials/Secrets/Overview.md new file mode 100644 index 000000000..e652d9e85 --- /dev/null +++ b/docs/Tutorials/Secrets/Overview.md @@ -0,0 +1,231 @@ +# Overview + +You can register and mount secret values from secret vaults, like Azure KeyVault or HashiCorp Vault, into Pode for use in Routes, Middleware, etc. + +Secrets can also be referenced from a vault in an adhoc manor, without needing to mount them first. You can also create, update and remove secrets in vaults. + +The values of mounted secrets may also be cached for a period of time, to reduce load on the vault. + +## Registering + +In order to reference Secrets from a vault you first need to register that vault using [`Register-PodeSecretVault`](../../../Functions/Secrets/Register-PodeSecretVault). Registering a vault registers the vault within Pode, but will also call any logic needed by the registration type being used. For example, if using Secret Management then Pode will call `Register-SecretVault` for you. + +A further example, as follows, will use the Secret Management PowerShell module to register an Azure KeyVault within Pode: + +```powershell +Register-PodeSecretVault -Name 'FriendlyVaultName' -ModuleName 'Az.KeyVault' -VaultParameters @{ + AZKVaultName = 'VaultNameInAzure' + SubscriptionId = $SubscriptionId +} +``` + +You can find more information on the [Secret Management](../Types/SecretManagement) page. The general parameters for all types are: + +| Parameter | Description | +| --------- | ----------- | +| Name | This is a friendly name for the vault within Pode that you can reference | +| VaultParameters | This is a hashtable of options, for the vault, that is supplied to vault scripts | + +!!! note + You can unregister a vault via [`Unregister-PodeSecretVault`](../../../Functions/Secrets/Unregister-PodeSecretVault). All vaults are automatically unregistered at when the server stops - unless it was auto-imported. + +### Types + +At present there are just two registration types implemented for registering secret vaults: + +* [Secret Management](../Types/SecretManagement) (powershell module) +* [Custom](../Types/Custom) + +### Initialise + +If there is any logic that needs to be invoked before a vault is registered, such as connecting to a cloud provider first (ie: `Connect-AzAccount`), this can be achieved via the `-InitScriptBlock` parameter on [`Register-PodeSecretVault`](../../../Functions/Secrets/Register-PodeSecretVault). + +This scriptblock is run just before Pode invokes any registration logic, and applies to all registration types. The scriptblock is supplied the `-VaultParameters` hashtable as a parameter: + +```powershell +Register-PodeSecretVault -Name 'VaultName' -ModuleName 'Az.KeyVault' ` + -VaultParameters @{ + AZKVaultName = 'VaultNameInAzure' + SubscriptionId = $SubscriptionId + } ` + -InitScriptBlock { + param($config) + Connect-AzAccount -Subscription $config.SubscriptionId + } +``` + +### Auto-Import + +Similar to modules and functions, Pode will auto-import any secret vaults registered outside of vault. You can find more [information here](../../Scoping#secret-vaults) + +### Unlock + +Some vaults require unlocking first, or an authorization token to be be acquired to access the vault. Unlocking applies to all registration types, and to configure unlocking for use you'll first need to supply either an `-UnlockSecret` or an `-UnlockSecureSecret` to [`Register-PodeSecretVault`](../../../Functions/Secrets/Register-PodeSecretVault). + +!!! important + If you're using a custom registration type, you'll also need to supply an `-UnlockScriptBlock`. + +```powershell +Register-PodeSecretVault -Name 'VaultName' -ModuleName 'Az.KeyVault' -UnlockSecret 'some-vault-password' ` + -VaultParameters @{ + AZKVaultName = 'VaultNameInAzure' + SubscriptionId = $SubscriptionId + } ` + -UnlockScriptBlock { + param($config, $secret) + Unlock-SomeVault -Secret $secret + } +``` + +Pode will automatically call the unlock logic after registration, but you can stop this from occurring by passing `-NoUnlock`. If you do, you'll need to call [`Unlock-PodeSecretVault`](../../../Functions/Secrets/Unlock-PodeSecretVault) to unlock the vault: + +```powershell +Unlock-PodeSecretVault -Name 'VaultName' +``` + +If you need to periodically check/unlock your vault, then Pode can do this automatically for you. To achieve this you can supply a number of minutes for the `-UnlockInterval` parameter on [`Register-PodeSecretVault`](../../../Functions/Secrets/Register-PodeSecretVault), this will tell Pode to automatically check/unlock the vault after the first unlock as occurred. + +## Mounting + +After registering a secret vault, you can now mount secrets from that vault for use within Pode Routes, Middleware, etc. The logic for mounting a secret is the same regardless of vault type or registration type. A secret can be mounted via [`Mount-PodeSecret`](../../../Functions/Secrets/Mount-PodeSecret): + +```powershell +Mount-PodeSecret -Name 'SecretName' -Vault 'VaultName' -Key 'SecretKeyNameInVault' +``` + +The `-Name` is the name of the secret you'll be using to reference the secret throughout Pode. The `-Vault` parameter is the name of the vault from [`Register-PodeSecretVault`](../../../Functions/Secrets/Register-PodeSecretVault), and the `-Key` is the path/name of the secret within the vault itself. + +Some secrets will be returned as hashtables - such as from HashiCorp Vault. In some cases you might only want certain properties to be returned from this secret, and you can limit the properties returned by using the `-Property` parameter. For example, if a secret has 5 keys named key1 to key5, you can limit this to just key2 and key4: + +```powershell +Mount-PodeSecret -Name 'SecretName' -Vault 'VaultName' -Key 'SecretKeyNameInVault' -Property key2, key4 +``` + +Or, you can limit the result to a single property, and expand on it - so now you get a string returned and not a hashtable: + +```powershell +Mount-PodeSecret -Name 'SecretName' -Vault 'VaultName' -Key 'SecretKeyNameInVault' -ExpandProperty key3 +``` + +!!! note + When using `-ExpandProperty` and you want to update the value, just pass a raw string (or whatever the inner type is). Pode will automatically wrap the original property key back for you. + +!!! note + You can dismount a mounted secret via [`Dismount-PodeSecret`](../../../Functions/Secrets/Dismount-PodeSecret). If you also supply the `-Remove` switch the secret will be deleted within the vault as well. + +### Secret Scope + +To retrieve the values of mounted secrets you can use [`Get-PodeSecret`](../../../Functions/Secrets/Get-PodeSecret), and then to update the value of the secret you can use [`Update-PodeSecret`](../../../Functions/Secrets/Update-PodeSecret): + +```powershell +# get secret +Add-PodeRoute -Method Get -Path '/secret' -ScriptBlock { + Write-PodeJsonResponse @{ Value = (Get-PodeSecret -Name 'SecretName') } +} + +# update secret +Add-PodeRoute -Method Post -Path '/secret' -ScriptBlock { + $WebEvent.Data.Value | Update-PodeSecret -Name 'SecretName' +} +``` + +Or, you can do the same but using the `$secret:` scope: + +```powershell +# get secret +Add-PodeRoute -Method Get -Path '/secret' -ScriptBlock { + Write-PodeJsonResponse @{ Value = $secret:SecretName } +} + +# update secret +Add-PodeRoute -Method Post -Path '/secret' -ScriptBlock { + $secret:SecretName = $WebEvent.Data.Value +} +``` + +### Caching + +!!! important + The cache is an in-memory in-application cache, and is unencrypted. It is never stored to disk, and cached values are wiped once their expiry is up; the cache is also wiped when the server is stopped. + +To reduce round-trip time by constantly going to a vault, as well as to reduce stress on a vault, you can optionally enable caching on secrets - the cache by default is disabled. + +You can either supply a `-CacheTtl` as a number of minutes to [`Register-PodeSecretVault`](../../../Functions/Secrets/Register-PodeSecretVault), and all secrets mounted will be cached. Or you can supply a `-CacheTtl` only to specific mounted secrets - you can also use this option to disable caching for a secret, by supplying a value of 0, even if the vault itself is registered with a cache TTL. + +To enable caching for all mounted secrets from a vault - but with one disabled, and one overriding: + +```powershell +# register a vault with a secret cache TTL of 5 minutes +Register-PodeSecretVault -Name 'FriendlyVaultName' -ModuleName 'Az.KeyVault' -CacheTtl 5 -VaultParameters @{ + AZKVaultName = 'VaultNameInAzure' + SubscriptionId = $SubscriptionId +} + +# mount a secret, that will use the vault cache of 5mins +Mount-PodeSecret -Name 'SecretName1' -Vault 'VaultName' -Key 'SecretKeyNameInVault1' + +# mount a secret, that will use a custom cache TTL of 10mins +Mount-PodeSecret -Name 'SecretName2' -Vault 'VaultName' -Key 'SecretKeyNameInVault2' -CacheTtl 10 + +# mount a secret, that uses no caching +Mount-PodeSecret -Name 'SecretName3' -Vault 'VaultName' -Key 'SecretKeyNameInVault3' -CacheTtl 0 +``` + +And then to enable caching for just specific mounted secrets: + +```powershell +# register a vault with no secret cache +Register-PodeSecretVault -Name 'FriendlyVaultName' -ModuleName 'Az.KeyVault' -VaultParameters @{ + AZKVaultName = 'VaultNameInAzure' + SubscriptionId = $SubscriptionId +} + +# mount a secret, that won't be cached +Mount-PodeSecret -Name 'SecretName1' -Vault 'VaultName' -Key 'SecretKeyNameInVault1' + +# mount a secret, that will use a custom cache TTL of 10mins +Mount-PodeSecret -Name 'SecretName2' -Vault 'VaultName' -Key 'SecretKeyNameInVault2' -CacheTtl 10 +``` + +## Adhoc + +There is also support for creating/updating, retrieving, and removing secrets in an adhoc manor from registered vaults - without having to mount them. + + +### Create + +To create a new secret, and well as update an existing ones value, you can use [`Set-PodeSecret`](../../../Functions/Secrets/Set-PodeSecret): + +```powershell +Add-PodeRoute -Method Post -Path '/adhoc/:key' -ScriptBlock { + Set-PodeSecret -Key $WebEvent.Parameters['key'] -Vault 'VaultName' -InputObject $WebEvent.Data['value'] + + # if you needed to, afterwards, you could mount the secret as well + Mount-PodeSecret -Name $WebEvent.Data['name'] -Vault 'VaultName' -Key $WebEvent.Parameters['key'] +} +``` + +### Read + +To retrieve the secret's value you can use [`Read-PodeSecret`](../../../Functions/Secrets/Read-PodeSecret): + +```powershell +Add-PodeRoute -Method Get -Path '/adhoc/:key' -ScriptBlock { + $value = Read-PodeSecret -Key $WebEvent.Parameters['key'] -Vault 'VaultName' + Write-PodeJsonResponse @{ Value = $value } +} +``` + +### Remove + +And to then remove the secret from the vault you can use [`Remove-PodeSecret`](../../../Functions/Secrets/Remove-PodeSecret): + +```powershell +Add-PodeRoute -Method Delete -Path '/adhoc/:key' -ScriptBlock { + # if the secret wasn't mounted, you can just call Remove-PodeSecret directly + Remove-PodeSecret -Key $WebEvent.Parameters['key'] -Vault 'VaultName' + + # if the secret was mounted, you can dismount and remove from the vault via + Dismount-PodeSecret -Name $WebEvent.Parameters['key'] -Remove +} +``` diff --git a/docs/Tutorials/Secrets/Types/Custom.md b/docs/Tutorials/Secrets/Types/Custom.md new file mode 100644 index 000000000..adade41e2 --- /dev/null +++ b/docs/Tutorials/Secrets/Types/Custom.md @@ -0,0 +1,220 @@ +# Custom + +You can register custom secret vault logic in Pode, which lets you use vaults that either [SecretManagement](../SecretManagement) doesn't have extensions for, or that you want to have extra logic around. + +!!! note + An overview of general features can be [found here](../../Overview). + +## Register + +When registering a secret vault via [`Register-PodeSecretVault`](../../../../Functions/Secrets/Register-PodeSecretVault) using custom logic, besides a `-Name` and `-VaultParameters` the only other mandatory parameter is `-ScriptBlock`; this is the main scriptblock that will be used for retrieving secrets from the vault. + +For example, this following registers a custom vault for HashiCorp Vault and uses the `vault` CLI to retrieve a secret. When the `-ScriptBlock` is invoked, it is passed the `-VaultParameters` and the key/path to the secret within the vault: + +```powershell +# register a custom vault for HashiCorp Vault +Register-PodeSecretVault -Name 'HcpVault' ` + -VaultParameters @{ + Address = 'http://127.0.0.1:8200' + } ` + -ScriptBlock { + param($config, $key) + return (vault kv get -format json -address $config.Address -mount secret $key | ConvertFrom-Json -AsHashtable).data.data + } + +# mount a secret using the above vault, and retrieve the "secret/data/tools/github" +Mount-PodeSecret -Name 'Github' -Vault 'HcpVault' -Key 'tools/github' + +# reference this secret in a route to "release" something +Add-PodeRoute -Method Post -Path '/release' -ScriptBlock { + Publish-Release -ApiToken $secret:Github.api_token +} +``` + +When registering a custom vault, you will also need to supply additional optional scriptblocks to enable other secret functionality: + +* `-SetScriptBlock` +* `-RemoveScriptBlock` +* `-UnlockScriptBlock` +* `-UnregisterScriptBlock` + +If you attempt to use the functionality without supplying a scriptblock for it, errors will be thrown. + +### Set/Update + +In order to be able to update and set/create secrets, you will need to supply a `-SetScriptBlock` parameter on [`Register-PodeSecretVault`](../../../../Functions/Secrets/Register-PodeSecretVault). Without this, the following will fail: + +* `$secret: = ` +* [`Update-PodeSecret`](../../../../Functions/Secrets/Update-PodeSecret) +* [`Set-PodeSecret`](../../../../Functions/Secrets/Set-PodeSecret) + +The `-SetScriptBlock` scriptblock, when invoked, will be passed the `-VaultParameters`; the key/path to the secret within in the vault; and the value to update/create the secret with. + +Using the base HashiCorp Vault example from the top of the page, this can be extended to update secrets as follows: + +```powershell +Register-PodeSecretVault -Name 'HcpVault' ` + -VaultParameters @{ + Address = 'http://127.0.0.1:8200' + } ` + -ScriptBlock { + param($config, $key) + return (vault kv get -format json -address $config.Address -mount secret $key | ConvertFrom-Json -AsHashtable).data.data + } ` + -SetScriptBlock { + param($config, $key, $value) + vault kv put -address $config.Address -mount secret $key "$($value.Keys[0])=$($value.Values[0])" + } + +# mount a secret using the above vault, and retrieve the "secret/data/tools/github" +Mount-PodeSecret -Name 'Github' -Vault 'HcpVault' -Key 'tools/github' + +# reference this secret in a route to update the api token +Add-PodeRoute -Method Put -Path '/api-token' -ScriptBlock { + $secret:Github = @{ + api_token = $WebEvent.Data.ApiToken + } +} +``` + +### Remove + +In order to be able to remove secrets, you will need to supply a `-RemoveScriptBlock` parameter on [`Register-PodeSecretVault`](../../../../Functions/Secrets/Register-PodeSecretVault). Without this, the following will fail: + +* [`Remove-PodeSecret`](../../../../Functions/Secrets/Remove-PodeSecret) + +The `-RemoveScriptBlock` scriptblock, when invoked, will be passed the `-VaultParameters` and the key/path to the secret within in the vault. + +Using the base HashiCorp Vault example from the top of the page, this can be extended to remove secrets as follows: + +```powershell +Register-PodeSecretVault -Name 'HcpVault' ` + -VaultParameters @{ + Address = 'http://127.0.0.1:8200' + } ` + -ScriptBlock { + param($config, $key) + return (vault kv get -format json -address $config.Address -mount secret $key | ConvertFrom-Json -AsHashtable).data.data + } ` + -RemoveScriptBlock { + param($config, $key) + vault kv destroy -address $config.Address -versions 1 -mount secret $key + } + +# mount a secret using the above vault, and retrieve the "secret/data/tools/github" +Mount-PodeSecret -Name 'Github' -Vault 'HcpVault' -Key 'tools/github' + +# reference this secret in a route to delete it +Add-PodeRoute -Method Delete -Path '/api-token' -ScriptBlock { + Dismount-PodeSecret -Name 'Github' -Remove +} +``` + +### Unlock + +In order to be able to unlock the vault, you will need to supply an `-UnlockScriptBlock` parameter on [`Register-PodeSecretVault`](../../../../Functions/Secrets/Register-PodeSecretVault). Without this, the following will fail: + +* Unlocking the vault with [`Register-PodeSecretVault`](../../../../Functions/Secrets/Register-PodeSecretVault) +* [`Unlock-PodeSecretVault`](../../../../Functions/Secrets/Unlock-PodeSecretVault) + +The `-UnlockScriptBlock` scriptblock, when invoked, will be passed the `-VaultParameters` and the unlock secret supplied to [`Register-PodeSecretVault`](../../../../Functions/Secrets/Register-PodeSecretVault). + +Using the base HashiCorp Vault example from the top of the page, this can be extended to unlock the vault: + +```powershell +Register-PodeSecretVault -Name 'HcpVault' ` + -VaultParameters @{ + Address = 'http://127.0.0.1:8200' + } ` + -ScriptBlock { + param($config, $key) + return (vault kv get -format json -address $config.Address -mount secret $key | ConvertFrom-Json -AsHashtable).data.data + } ` + -UnlockSecret 'some-secret-vault' -UnlockScriptBlock { + param($config, $secret) + vault operator unseal -address $config.Address $secret + } +``` + +!!! tip + You can return a DateTime object from the `-UnlockScriptBlock` to specify a custom time for Pode to re-invoke the scriptblock again - rather than using the `-UnlockInterval` parameter. If no DateTime is returned, then `-UnlockInterval` will be used by default if it has been supplied. + +### Unregister + +You can still call [`Unregister-PodeSecretVault`](../../../../Functions/Secrets/Unregister-PodeSecretVault) for a custom vault without supplying an `-UnregisterScriptBlock`, but nothing will occur - other than the vault being removed from within Pode. + +If you do supply an `-UnregisterScriptBlock`, then this will be called just before the vault is removed from within Pode. The scriptblock, when invoked, will be just supplied the `-VaultParameters`. + +Using the base HashiCorp Vault example from the top of the page, this can be extended to run some custom logic when the vault is unregistered - in this case, as an example only(!), the vault will be locked: + +```powershell +Register-PodeSecretVault -Name 'HcpVault' ` + -VaultParameters @{ + Address = 'http://127.0.0.1:8200' + } ` + -ScriptBlock { + param($config, $key) + return (vault kv get -format json -address $config.Address -mount secret $key | ConvertFrom-Json -AsHashtable).data.data + } ` + -UnregisterScriptBlock { + param($config) + vault operator seal -address $config.Address + } +``` + +## Example + +The following example is an aggregated example of everything mentioned above put together. It will register a custom vault for HashiCorp Vault, with Set, Remove, Unlock and Unregister support. + +```powershell +Start-PodeServer { + # listen on localhost:8080 + Add-PodeEndpoint -Address * -Port 8080 -Protocol Http + + # register a custom vault for HashiCorp Vault + Register-PodeSecretVault -Name 'HcpVault' ` + -VaultParameters @{ + Address = 'http://127.0.0.1:8200' + } ` + -ScriptBlock { + param($config, $key) + return (vault kv get -format json -address $config.Address -mount secret $key | ConvertFrom-Json -AsHashtable).data.data + } ` + -SetScriptBlock { + param($config, $key, $value) + vault kv put -address $config.Address -mount secret $key "$($value.Keys[0])=$($value.Values[0])" + } ` + -RemoveScriptBlock { + param($config, $key) + vault kv destroy -address $config.Address -versions 1 -mount secret $key + } ` + -UnlockSecret 'some-secret-vault' -UnlockScriptBlock { + param($config, $secret) + vault operator unseal -address $config.Address $secret + } ` + -UnregisterScriptBlock { + param($config) + vault operator seal -address $config.Address + } + + # mount a secret using the above vault, and retrieve the "secret/data/tools/github" + Mount-PodeSecret -Name 'Github' -Vault 'HcpVault' -Key 'tools/github' + + # reference this secret in a route to "release" something + Add-PodeRoute -Method Post -Path '/release' -ScriptBlock { + Publish-Release -ApiToken $secret:Github.api_token + } + + # reference this secret in a route to update the api token + Add-PodeRoute -Method Put -Path '/api-token' -ScriptBlock { + $secret:Github = @{ + api_token = $WebEvent.Data.ApiToken + } + } + + # reference this secret in a route to delete it + Add-PodeRoute -Method Delete -Path '/api-token' -ScriptBlock { + Dismount-PodeSecret -Name 'Github' -Remove + } +} +``` diff --git a/docs/Tutorials/Secrets/Types/SecretManagement.md b/docs/Tutorials/Secrets/Types/SecretManagement.md new file mode 100644 index 000000000..4e0e08045 --- /dev/null +++ b/docs/Tutorials/Secrets/Types/SecretManagement.md @@ -0,0 +1,87 @@ +# Secret Management + +Pode can register secret vaults using Microsoft's [SecretManagement](https://github.com/powershell/secretmanagement) PowerShell module, plus extensions. + +!!! note + An overview of general features can be [found here](../../Overview). + +## Register + +When registering a secret vault via [`Register-PodeSecretVault`](../../../../Functions/Secrets/Register-PodeSecretVault) using the SecretManagement module, besides a `-Name` and `-VaultParameters` the only other mandatory parameter is `-ModuleName`; this is the name of the extension module to use with the SecretManagement module. Besides calling the SecretManagement's `Register-SecretVault`, Pode will also automatically import the SecretManagement and extension modules into Pode's runspaces for you. + +For example, if we were registering an Azure KeyVault this would be `Az.KeyVault`: + +```powershell +Register-PodeSecretVault -Name 'FriendlyVaultName' -ModuleName 'Az.KeyVault' -VaultParameters @{ + AZKVaultName = 'VaultNameInAzure' + SubscriptionId = $SubscriptionId +} +``` + +The only other SecretManagement specific parameter is `-VaultName`. This parameter can be used to give the actual name of the vault, while keeping the `-Name` parameter as a better more friendlier name. If no `-VaultName` is supplied then `-Name` is used instead. Using the same example as above, but this time we specify a specific vault name to pass to `Register-SecretVault`: + +```powershell +Register-PodeSecretVault -Name 'FriendlyVaultName' -VaultName 'VaultNameInAzure' -ModuleName 'Az.KeyVault' -VaultParameters @{ + AZKVaultName = 'VaultNameInAzure' + SubscriptionId = $SubscriptionId +} +``` + +If you use [`Unregister-PodeSecretVault`](../../../../Functions/Secrets/Unregister-PodeSecretVault), then Pode will also call the SecretManagement's `Unregister-SecretVault`. + +## Auto-Import + +More information can be [found here](../../../Scoping), but if the SecretManagement module is installed then Pode will automatically import/register any secret vaults already registered. + +Any secret vaults registered this way will no be automatically unregistered when the server stops. + +## Example + +The following example registered an Azure KeyVault, mounts a secret from the vault into Pode, and then adds two Routes - one to retrieve the value, and another one to update the value: + +```powershell +param( + [Parameter(Mandatory=$true)] + [string] + $VaultName, + + [Parameter(Mandatory=$true)] + [string] + $SubscriptionId +) + +Start-PodeServer { + Add-PodeEndpoint -Address * -Port 8080 -Protocol Http + + # secret manage azure keyvault - need to run "Connect-AzAccount" first! + Register-PodeSecretVault -Name 'FriendlyVaultName' -ModuleName 'Az.KeyVault' -VaultParameters @{ + AZKVaultName = $VaultName + SubscriptionId = $SubscriptionId + } + + # mount a secret from az keyvault + Mount-PodeSecret -Name 'SecretName' -Vault 'FriendlyVaultName' -Key 'AKVSecretName' + + + # routes to get/update secret in az keyvault + Add-PodeRoute -Method Get -Path '/secret' -ScriptBlock { + Write-PodeJsonResponse @{ Value = $secret:SecretName } + } + + Add-PodeRoute -Method Post -Path '/secret' -ScriptBlock { + $secret:SecretName = $WebEvent.Data.Value + } +} +``` + +To retrieve the value: +```powershell +Invoke-RestMethod -Uri 'http://localhost:8080/secret' +``` + +And to update the value: +```powershell +Invoke-RestMethod -Uri 'http://localhost:8080/secret' -Method Post -Body @{ + Value = '' +} +``` diff --git a/src/Pode.psd1 b/src/Pode.psd1 index 70af1df63..72ad4a394 100644 --- a/src/Pode.psd1 +++ b/src/Pode.psd1 @@ -359,7 +359,7 @@ 'powershell-core', 'windows', 'unix', 'linux', 'pode', 'PSEdition_Core', 'cross-platform', 'access-control', 'file-monitoring', 'multithreaded', 'rate-limiting', 'cron', 'schedule', 'middleware', 'session', 'authentication', 'active-directory', 'caching', 'csrf', 'arm', 'raspberry-pi', 'aws-lambda', - 'azure-functions', 'websockets', 'swagger', 'openapi', 'redoc', 'secrets', 'vault') + 'azure-functions', 'websockets', 'swagger', 'openapi', 'redoc', 'webserver', 'secrets', 'vault') # A URL to the license for this module. LicenseUri = 'https://raw.githubusercontent.com/Badgerati/Pode/master/LICENSE.txt' diff --git a/src/Private/Context.ps1 b/src/Private/Context.ps1 index 8ee1d4175..2777f6f09 100644 --- a/src/Private/Context.ps1 +++ b/src/Private/Context.ps1 @@ -288,44 +288,8 @@ function New-PodeContext # secrets $ctx.Server.Secrets = @{ - Vaults = @{ - # NAME = @{ - # TYPE = [CUSTOM|SECRETMANAGEMENT] - # PARAMETERS = @{} - # UNLOCK = @{ - # PASSWORD = SECURESTRING - # EXPIRY = DATE - # ENABLED = [TRUE|FALSE] - # } - # CACHE = @{ - # TTL = 0 - # ENABLED = [TRUE|FALSE] - # } - # CUSTOM = @{ - # READ = SCRIPTBLOCK - # UNLOCK = SCRIPTBLOCK - # REMOVE = SCRIPTBLOCK - # NEW = SCRIPTBLOCK - # UPDATE = SCRIPTBLOCK - # } - # SECRETMANAGEMENT = @{ - # VAULTNAME = NAME - # MODULENAME = [NAME|PATH] - # } - # } - } - Keys = @{ - # NAME = @{ - # VAULT = NAME - # ARGS = [VALUES] - # CACHE = @{ - # EXPIRY = DATE - # VALUE = SECUTESTRING - # TTL = 0 - # ENABLED = [TRUE|FALSE] - # } - # } - } + Vaults = @{} + Keys = @{} } # custom view paths diff --git a/src/Private/Secrets.ps1 b/src/Private/Secrets.ps1 index 574818231..596f1610b 100644 --- a/src/Private/Secrets.ps1 +++ b/src/Private/Secrets.ps1 @@ -76,6 +76,11 @@ function Register-PodeSecretCustomVault $UnregisterScriptBlock ) + # unlock secret with no script? + if ($VaultConfig.Unlock.Enabled -and (Test-PodeIsEmpty $UnlockScriptBlock)) { + throw 'Unlock secret supplied for custom Secret Vault type, but not Unlock ScriptBlock supplied' + } + # all is good, so set the config $VaultConfig['Custom'] = @{ Read = $ScriptBlock @@ -99,8 +104,14 @@ function Unlock-PodeSecretManagementVault return $null } - # unlock the vault, and return null for the expiry + # unlock the vault $null = Unlock-SecretVault -Name $VaultConfig.SecretManagement.VaultName -Password $VaultConfig.Unlock.Secret -ErrorAction Stop + + # interval? + if ($VaultConfig.Unlock.Interval -gt 0) { + return ([datetime]::UtcNow.AddMinutes($VaultConfig.Unlock.Interval)) + } + return $null } @@ -123,10 +134,21 @@ function Unlock-PodeSecretCustomVault } # unlock the vault, and get back an expiry - return (Invoke-PodeScriptBlock -ScriptBlock $VaultConfig.Custom.Unlock -Splat -Return -Arguments @( + $expiry = (Invoke-PodeScriptBlock -ScriptBlock $VaultConfig.Custom.Unlock -Splat -Return -Arguments @( $VaultConfig.Parameters, (ConvertFrom-SecureString -SecureString $VaultConfig.Unlock.Secret -AsPlainText) )) + + # return expiry if given, otherwise check interval + if ($null -ne $expiry) { + return $expiry + } + + if ($VaultConfig.Unlock.Interval -gt 0) { + return ([datetime]::UtcNow.AddMinutes($VaultConfig.Unlock.Interval)) + } + + return $null } function Unregister-PodeSecretManagementVault diff --git a/src/Public/Secrets.ps1 b/src/Public/Secrets.ps1 index 240969873..bcfe71cd7 100644 --- a/src/Public/Secrets.ps1 +++ b/src/Public/Secrets.ps1 @@ -1,9 +1,3 @@ -<# -TODO: -- DOCS! - - auto-import -#> - <# .SYNOPSIS Register a Secret Vault. @@ -23,8 +17,11 @@ An optional Secret to be used to unlock the Secret Vault if need. .PARAMETER UnlockSecureSecret An optional Secret, as a SecureString, to be used to unlock the Secret Vault if need. -.PARAMETER Unlock -If supplied, the Secret Vault will be unlocked now, after being registered with Pode. +.PARAMETER UnlockInterval +An optional number of minutes that Pode will periodically check/unlock the Secret Vault. (Default: 0) + +.PARAMETER NoUnlock +If supplied, the Secret Vault will not be unlocked after registration. To unlock you'll need to call Unlock-PodeSecretVault. .PARAMETER CacheTtl An optional number of minutes that Secrets should be cached for. (Default: 0) @@ -79,8 +76,12 @@ function Register-PodeSecretVault [securestring] $UnlockSecureSecret, + [Parameter()] + [int] + $UnlockInterval = 0, + [switch] - $Unlock, + $NoUnlock, [Parameter()] [int] @@ -95,6 +96,7 @@ function Register-PodeSecretVault $VaultName, [Parameter(Mandatory=$true, ParameterSetName='SecretManagement')] + [Alias('Module')] [string] $ModuleName, @@ -103,18 +105,22 @@ function Register-PodeSecretVault $ScriptBlock, # Read a secret [Parameter(ParameterSetName='Custom')] + [Alias('Unlock')] [scriptblock] $UnlockScriptBlock, [Parameter(ParameterSetName='Custom')] + [Alias('Remove')] [scriptblock] $RemoveScriptBlock, [Parameter(ParameterSetName='Custom')] + [Alias('Set')] [scriptblock] $SetScriptBlock, [Parameter(ParameterSetName='Custom')] + [Alias('Unregister')] [scriptblock] $UnregisterScriptBlock ) @@ -142,7 +148,8 @@ function Register-PodeSecretVault Unlock = @{ Secret = $UnlockSecureSecret Expiry = $null - Enabled = $Unlock.IsPresent + Interval = $UnlockInterval + Enabled = (!(Test-PodeIsEmpty $UnlockSecureSecret)) } Cache = @{ Ttl = $CacheTtl @@ -180,7 +187,7 @@ function Register-PodeSecretVault $PodeContext.Server.Secrets.Vaults[$Name] = $vault # unlock the vault? - if ($Unlock) { + if (!$NoUnlock -and $vault.Unlock.Enabled) { Unlock-PodeSecretVault -Name $Name } } From 26ddd70ac51d9dd3441178b8fa1a2d821578b490 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sun, 1 Jan 2023 17:24:35 +0000 Subject: [PATCH 18/52] #980: small clean-up tweaks --- src/Private/AutoImport.ps1 | 2 +- src/Private/Helpers.ps1 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/Private/AutoImport.ps1 b/src/Private/AutoImport.ps1 index 8d476043f..ba311f2da 100644 --- a/src/Private/AutoImport.ps1 +++ b/src/Private/AutoImport.ps1 @@ -178,7 +178,7 @@ function Import-PodeSecretManagementVaultsIntoRegistry $null = Import-Module -Name Microsoft.PowerShell.SecretManagement -Force -DisableNameChecking -Scope Global -ErrorAction Stop -Verbose:$false # get the current secret vaults - $vaults = @(Get-SecretVault -ErrorAction Stop) + $vaults = @(Get-SecretVault -ErrorAction Stop) # register the vaults foreach ($vault in $vaults) { diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 5d0c21ea9..0239a0405 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -3079,5 +3079,5 @@ function Test-PodeModuleInstalled $Name ) - return ($null -ne (Get-Module -Name $Name -ListAvailable -ErrorAction Ignore)) + return ($null -ne (Get-Module -Name $Name -ListAvailable -ErrorAction Ignore -Verbose:$false)) } \ No newline at end of file From 5bf10576dab8fc65a1ec36c4136fea0f5d037e04 Mon Sep 17 00:00:00 2001 From: Mason Date: Sun, 1 Jan 2023 12:21:55 -0600 Subject: [PATCH 19/52] Update Errors.md Fixing typo --- docs/Tutorials/Logging/Types/Errors.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Tutorials/Logging/Types/Errors.md b/docs/Tutorials/Logging/Types/Errors.md index 3794f5de4..075a28a00 100644 --- a/docs/Tutorials/Logging/Types/Errors.md +++ b/docs/Tutorials/Logging/Types/Errors.md @@ -65,7 +65,7 @@ New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging The following example will enable Error logging, and it will log all errors levels except Debug: ```powershell -New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging -Levels Error, Warning, Information, Verbose +New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging -Levels Error, Warning, Informational, Verbose ``` ### Using Raw Item From f8cfb7f060a73409b0da6d5a67440c1313595db8 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sun, 1 Jan 2023 18:27:01 +0000 Subject: [PATCH 20/52] #1029: resolve path when creating new PS drive. also replace with .NET method for perf bump --- src/Pode.psm1 | 4 ++-- src/Private/Helpers.ps1 | 11 +++++++---- src/Public/Core.ps1 | 2 +- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/src/Pode.psm1 b/src/Pode.psm1 index 48abff47f..29af54048 100644 --- a/src/Pode.psm1 +++ b/src/Pode.psm1 @@ -18,7 +18,7 @@ if ($PODE_SCOPE_RUNSPACE) { } # load private functions -Get-ChildItem "$($root)/Private/*.ps1" | Resolve-Path | ForEach-Object { . $_ } +Get-ChildItem "$($root)/Private/*.ps1" | ForEach-Object { . ([System.IO.Path]::GetFullPath($_)) } # only import public functions if not in a runspace if (!$PODE_SCOPE_RUNSPACE) { @@ -26,7 +26,7 @@ if (!$PODE_SCOPE_RUNSPACE) { } # load public functions -Get-ChildItem "$($root)/Public/*.ps1" | Resolve-Path | ForEach-Object { . $_ } +Get-ChildItem "$($root)/Public/*.ps1" | ForEach-Object { . ([System.IO.Path]::GetFullPath($_)) } # get functions from memory and compare to existing to find new functions added $funcs = Get-ChildItem Function: | Where-Object { $sysfuncs -notcontains $_ } diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 0239a0405..32ca41cf9 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -848,6 +848,9 @@ function New-PodePSDrive throw "Path does not exist: $($Path)" } + # resolve the path + $Path = [System.IO.Path]::GetFullPath($Path) + # create the temp drive if (!(Test-PodePSDrive -Name $Name -Path $Path)) { $drive = (New-PSDrive -Name $Name -PSProvider FileSystem -Root $Path -Scope Global -ErrorAction Stop) @@ -2178,13 +2181,13 @@ function Find-PodeFileForContentType function Test-PodePathIsRelative { - param ( + param( [Parameter(Mandatory=$true)] [string] $Path ) - if (@('.', '..') -contains $Path) { + if (($Path.Length -le 2) -and (@('.', '..') -contains $Path)) { return $true } @@ -2193,7 +2196,7 @@ function Test-PodePathIsRelative function Get-PodeRelativePath { - param ( + param( [Parameter(Mandatory=$true)] [string] $Path, @@ -2224,7 +2227,7 @@ function Get-PodeRelativePath # if flagged, resolve the path if ($Resolve) { $_rawPath = $Path - $Path = (Resolve-Path -Path $Path -ErrorAction Ignore).Path + $Path = [System.IO.Path]::GetFullPath($Path) } # if flagged, test the path and throw error if it doesn't exist diff --git a/src/Public/Core.ps1 b/src/Public/Core.ps1 index ec39a867a..cba47e534 100644 --- a/src/Public/Core.ps1 +++ b/src/Public/Core.ps1 @@ -634,7 +634,7 @@ function Show-PodeGui # set the window's icon path if (![string]::IsNullOrWhiteSpace($Icon)) { - $PodeContext.Server.Gui.Icon = (Resolve-Path $Icon).Path + $PodeContext.Server.Gui.Icon = [System.IO.Path]::GetFullPath($Icon) if (!(Test-Path $PodeContext.Server.Gui.Icon)) { throw "Path to icon for GUI does not exist: $($PodeContext.Server.Gui.Icon)" } From a0b7af51c3ad61714ee42a784e68531d34e6e6d3 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sun, 1 Jan 2023 18:32:35 +0000 Subject: [PATCH 21/52] #1029: minor whitespace tweak --- src/Private/Helpers.ps1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 32ca41cf9..20a257455 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -823,7 +823,7 @@ function Close-PodeServerInternal function New-PodePSDrive { - param ( + param( [Parameter(Mandatory=$true)] [string] $Path, From 562db0e8c35622a9d251be9faaed2e74caaf7a14 Mon Sep 17 00:00:00 2001 From: Mason Date: Sun, 1 Jan 2023 13:34:58 -0600 Subject: [PATCH 22/52] Update UserFile.md Added clarification around the password formats and provided functions to convert plaintext passwords to expected hashes. --- .../Authentication/Inbuilt/UserFile.md | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/docs/Tutorials/Authentication/Inbuilt/UserFile.md b/docs/Tutorials/Authentication/Inbuilt/UserFile.md index 44bdc39e3..37cd11325 100644 --- a/docs/Tutorials/Authentication/Inbuilt/UserFile.md +++ b/docs/Tutorials/Authentication/Inbuilt/UserFile.md @@ -59,6 +59,30 @@ Start-PodeServer { } ``` +Regardless of whether the password is a standard SHA256 hash or HMAC hash, the hashed output should be a base64 string. The following functions will return the a hashed value in the expected format: + +**SHA256 HASH**: +```powershell +function ConvertTo-SHA256([string]$String) +{ + $SHA256 = New-Object System.Security.Cryptography.SHA256Managed + $SHA256Hash = $SHA256.ComputeHash([Text.Encoding]::ASCII.GetBytes($String)) + $SHA256HashString = [Convert]::ToBase64String($SHA256Hash) + return $SHA256HashString +} +``` + +**HMAC HASH:** +```powershell +function ConvertTo-HMACSHA256([string]$String, [string]$Secret) { + $HMACSHA256 = New-Object System.Security.Cryptography.HMACSHA256 + $HMACSHA256.Secret = [Text.Encoding]::ASCII.GetBytes($Secret) + $HMACSHA256Hash = $HMACSHA256.ComputeHash([Text.Encoding]::ASCII.GetBytes($String)) + $HMACSHA256HashString = [Convert]::ToBase64String($HMACSHA256Hash) + return $HMACSHA256HashString +} +``` + ### User Object The User object returned, and accessible on Routes, and other functions via the [web event](../../../WebEvent)'s `$WebEvent.Auth.User` property, will contain the following information: From d215f64f0db797a52893e746b440c645021eceef Mon Sep 17 00:00:00 2001 From: Mason Date: Sun, 1 Jan 2023 13:36:36 -0600 Subject: [PATCH 23/52] Update UserFile.md --- docs/Tutorials/Authentication/Inbuilt/UserFile.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Tutorials/Authentication/Inbuilt/UserFile.md b/docs/Tutorials/Authentication/Inbuilt/UserFile.md index 37cd11325..2a28456b7 100644 --- a/docs/Tutorials/Authentication/Inbuilt/UserFile.md +++ b/docs/Tutorials/Authentication/Inbuilt/UserFile.md @@ -59,7 +59,7 @@ Start-PodeServer { } ``` -Regardless of whether the password is a standard SHA256 hash or HMAC hash, the hashed output should be a base64 string. The following functions will return the a hashed value in the expected format: +Regardless of whether the password is a standard SHA256 hash or HMAC hash, the hashed output should be a base64 string. The following functions will return the hashed value in the expected format: **SHA256 HASH**: ```powershell From e96b251eb9e17f5f92e604d639d4d26514cb2d26 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Tue, 3 Jan 2023 23:07:33 +0000 Subject: [PATCH 24/52] #1029: fix tests, and change relative path check to all regex for perf --- src/Private/Helpers.ps1 | 26 +++------------ src/Public/Core.ps1 | 2 +- tests/unit/Helpers.Tests.ps1 | 62 ++++++------------------------------ 3 files changed, 16 insertions(+), 74 deletions(-) diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 20a257455..01d973a61 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -849,7 +849,7 @@ function New-PodePSDrive } # resolve the path - $Path = [System.IO.Path]::GetFullPath($Path) + $Path = [System.IO.Path]::GetFullPath($Path, $pwd.Path) # create the temp drive if (!(Test-PodePSDrive -Name $Name -Path $Path)) { @@ -1664,9 +1664,8 @@ function Get-PodeCount function Test-PodePathAccess { - param ( + param( [Parameter(Mandatory=$true)] - [ValidateNotNullOrEmpty()] [string] $Path ) @@ -1683,7 +1682,7 @@ function Test-PodePathAccess function Test-PodePath { - param ( + param( [Parameter()] $Path, @@ -2179,21 +2178,6 @@ function Find-PodeFileForContentType return $null } -function Test-PodePathIsRelative -{ - param( - [Parameter(Mandatory=$true)] - [string] - $Path - ) - - if (($Path.Length -le 2) -and (@('.', '..') -contains $Path)) { - return $true - } - - return ($Path -match '^\.{1,2}[\\/]') -} - function Get-PodeRelativePath { param( @@ -2216,7 +2200,7 @@ function Get-PodeRelativePath ) # if the path is relative, join to root if flagged - if ($JoinRoot -and (Test-PodePathIsRelative -Path $Path)) { + if ($JoinRoot -and ($Path -match '^\.{1,2}([\\\/]|$)')) { if ([string]::IsNullOrWhiteSpace($RootPath)) { $RootPath = $PodeContext.Server.Root } @@ -2227,7 +2211,7 @@ function Get-PodeRelativePath # if flagged, resolve the path if ($Resolve) { $_rawPath = $Path - $Path = [System.IO.Path]::GetFullPath($Path) + $Path = [System.IO.Path]::GetFullPath($Path, $pwd.Path) } # if flagged, test the path and throw error if it doesn't exist diff --git a/src/Public/Core.ps1 b/src/Public/Core.ps1 index cba47e534..2a7857175 100644 --- a/src/Public/Core.ps1 +++ b/src/Public/Core.ps1 @@ -634,7 +634,7 @@ function Show-PodeGui # set the window's icon path if (![string]::IsNullOrWhiteSpace($Icon)) { - $PodeContext.Server.Gui.Icon = [System.IO.Path]::GetFullPath($Icon) + $PodeContext.Server.Gui.Icon = [System.IO.Path]::GetFullPath($Icon, $pwd.Path) if (!(Test-Path $PodeContext.Server.Gui.Icon)) { throw "Path to icon for GUI does not exist: $($PodeContext.Server.Gui.Icon)" } diff --git a/tests/unit/Helpers.Tests.ps1 b/tests/unit/Helpers.Tests.ps1 index 9bb8715b5..052c15b70 100644 --- a/tests/unit/Helpers.Tests.ps1 +++ b/tests/unit/Helpers.Tests.ps1 @@ -1,5 +1,6 @@ $path = $MyInvocation.MyCommand.Path $src = (Split-Path -Parent -Path $path) -ireplace '[\\/]tests[\\/]unit', '/src/' +Add-Type -LiteralPath "$($src)/Libs/netstandard2.0/Pode.dll" -ErrorAction Stop Get-ChildItem "$($src)/*.ps1" -Recurse | Resolve-Path | ForEach-Object { . $_ } Describe 'Get-PodeType' { @@ -1009,87 +1010,44 @@ Describe 'ConvertFrom-PodeFile' { } } -Describe 'Test-PodePathIsRelative' { - It 'Returns true for .' { - Test-PodePathIsRelative -Path '.' | Should Be $true - } - - It 'Returns true for ..' { - Test-PodePathIsRelative -Path '..' | Should Be $true - } - - It 'Returns true for relative file' { - Test-PodePathIsRelative -Path './file.txt' | Should Be $true - } - - It 'Returns true for relative folder' { - Test-PodePathIsRelative -Path '../folder' | Should Be $true - } - - It 'Returns false for literal windows path' { - Test-PodePathIsRelative -Path 'c:/path' | Should Be $false - } - - It 'Returns false for literal nix path' { - Test-PodePathIsRelative -Path '/path' | Should Be $false - } -} - Describe 'Get-PodeRelativePath' { $PodeContext = @{ 'Server' = @{ 'Root' = 'c:/' } } It 'Returns back a literal path' { - Mock Test-PodePathIsRelative { return $false } Get-PodeRelativePath -Path 'c:/path' | Should Be 'c:/path' } - It 'Returns null for non-existent literal path when resolving' { - Mock Test-PodePathIsRelative { return $false } - Mock Resolve-Path { return $null } - Get-PodeRelativePath -Path 'c:/path' -Resolve | Should Be ([string]::Empty) - } - It 'Returns path for literal path when resolving' { - Mock Test-PodePathIsRelative { return $false } - Mock Resolve-Path { return @{ 'Path' = 'c:/path' } } - Get-PodeRelativePath -Path 'c:/path' -Resolve | Should Be 'c:/path' + Get-PodeRelativePath -Path $pwd.Path -Resolve | Should Be $pwd.Path } It 'Returns back a relative path' { - Mock Test-PodePathIsRelative { return $true } Get-PodeRelativePath -Path './path' | Should Be './path' } - It 'Returns null for a non-existent relative path when resolving' { - Mock Test-PodePathIsRelative { return $true } - Mock Resolve-Path { return $null } - Get-PodeRelativePath -Path './path' -Resolve | Should Be ([string]::Empty) - } - It 'Returns path for a relative path when resolving' { - Mock Test-PodePathIsRelative { return $true } - Mock Resolve-Path { return @{ 'Path' = 'c:/path' } } - Get-PodeRelativePath -Path './path' -Resolve | Should Be 'c:/path' + Get-PodeRelativePath -Path ".\src" -Resolve | Should Be (Join-Path $pwd.Path "src") } It 'Returns path for a relative path joined to default root' { - Mock Test-PodePathIsRelative { return $true } Get-PodeRelativePath -Path './path' -JoinRoot | Should Be 'c:/./path' } It 'Returns resolved path for a relative path joined to default root when resolving' { - Mock Test-PodePathIsRelative { return $true } - Mock Resolve-Path { return @{ 'Path' = 'c:/path' } } - Get-PodeRelativePath -Path './path' -JoinRoot -Resolve | Should Be 'c:/path' + $PodeContext = @{ + Server = @{ + Root = $pwd.Path + } + } + + Get-PodeRelativePath -Path './src' -JoinRoot -Resolve | Should Be (Join-Path $pwd.Path "src") } It 'Returns path for a relative path joined to passed root' { - Mock Test-PodePathIsRelative { return $true } Get-PodeRelativePath -Path './path' -JoinRoot -RootPath 'e:/' | Should Be 'e:/./path' } It 'Throws error for path ot existing' { - Mock Test-PodePathIsRelative { return $false } Mock Test-PodePath { return $false } { Get-PodeRelativePath -Path './path' -TestPath } | Should Throw 'The path does not exist' } From 8e0922c7e92cfc2a1644caf2dcf2be17a12bea08 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Wed, 4 Jan 2023 19:07:33 +0000 Subject: [PATCH 25/52] #1029: fix tests --- tests/unit/Helpers.Tests.ps1 | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/unit/Helpers.Tests.ps1 b/tests/unit/Helpers.Tests.ps1 index 052c15b70..f147043de 100644 --- a/tests/unit/Helpers.Tests.ps1 +++ b/tests/unit/Helpers.Tests.ps1 @@ -1,6 +1,5 @@ $path = $MyInvocation.MyCommand.Path $src = (Split-Path -Parent -Path $path) -ireplace '[\\/]tests[\\/]unit', '/src/' -Add-Type -LiteralPath "$($src)/Libs/netstandard2.0/Pode.dll" -ErrorAction Stop Get-ChildItem "$($src)/*.ps1" -Recurse | Resolve-Path | ForEach-Object { . $_ } Describe 'Get-PodeType' { From 805cefc7a07ce63a89538d0258c5bf223590be49 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Wed, 4 Jan 2023 19:35:52 +0000 Subject: [PATCH 26/52] #1029: fix tests --- src/Private/Helpers.ps1 | 4 ++-- src/Public/Core.ps1 | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 01d973a61..5d4eb9020 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -849,7 +849,7 @@ function New-PodePSDrive } # resolve the path - $Path = [System.IO.Path]::GetFullPath($Path, $pwd.Path) + $Path = [System.IO.Path]::GetFullPath($Path.Replace('\', '/') , $pwd.Path) # create the temp drive if (!(Test-PodePSDrive -Name $Name -Path $Path)) { @@ -2211,7 +2211,7 @@ function Get-PodeRelativePath # if flagged, resolve the path if ($Resolve) { $_rawPath = $Path - $Path = [System.IO.Path]::GetFullPath($Path, $pwd.Path) + $Path = [System.IO.Path]::GetFullPath($Path.Replace('\', '/'), $pwd.Path) } # if flagged, test the path and throw error if it doesn't exist diff --git a/src/Public/Core.ps1 b/src/Public/Core.ps1 index 2a7857175..cd0df9166 100644 --- a/src/Public/Core.ps1 +++ b/src/Public/Core.ps1 @@ -634,7 +634,7 @@ function Show-PodeGui # set the window's icon path if (![string]::IsNullOrWhiteSpace($Icon)) { - $PodeContext.Server.Gui.Icon = [System.IO.Path]::GetFullPath($Icon, $pwd.Path) + $PodeContext.Server.Gui.Icon = [System.IO.Path]::GetFullPath($Icon.Replace('\', '/'), $pwd.Path) if (!(Test-Path $PodeContext.Server.Gui.Icon)) { throw "Path to icon for GUI does not exist: $($PodeContext.Server.Gui.Icon)" } From 4a30c2727ce513b2899406359ff6614430450fc9 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Thu, 5 Jan 2023 19:09:46 +0000 Subject: [PATCH 27/52] #1050: bump dockerfiles to PS7.3 --- Dockerfile | 2 +- alpine.dockerfile | 2 +- arm32.dockerfile | 2 +- docs/Getting-Started/Installation.md | 2 +- docs/Hosting/Docker.md | 4 ++-- packers/docker/arm32/Dockerfile | 2 +- 6 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Dockerfile b/Dockerfile index cb3923c3a..f6149007d 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -FROM mcr.microsoft.com/powershell:7.2-ubuntu-20.04 +FROM mcr.microsoft.com/powershell:7.3-ubuntu-22.04 LABEL maintainer="Matthew Kelly (Badgerati)" RUN mkdir -p /usr/local/share/powershell/Modules/Pode COPY ./pkg/ /usr/local/share/powershell/Modules/Pode \ No newline at end of file diff --git a/alpine.dockerfile b/alpine.dockerfile index 9fa946e66..dafff22aa 100644 --- a/alpine.dockerfile +++ b/alpine.dockerfile @@ -1,4 +1,4 @@ -FROM mcr.microsoft.com/powershell:7.2-alpine-3.14 +FROM mcr.microsoft.com/powershell:7.3-alpine-3.15 LABEL maintainer="Matthew Kelly (Badgerati)" RUN mkdir -p /usr/local/share/powershell/Modules/Pode COPY ./pkg/ /usr/local/share/powershell/Modules/Pode \ No newline at end of file diff --git a/arm32.dockerfile b/arm32.dockerfile index 34811aa36..1574f6e31 100644 --- a/arm32.dockerfile +++ b/arm32.dockerfile @@ -1,4 +1,4 @@ -FROM mcr.microsoft.com/powershell:7.2-arm32v7-ubuntu-18.04 +FROM mcr.microsoft.com/powershell:preview-7.3-arm32v7-ubuntu-18.04 LABEL maintainer="Matthew Kelly (Badgerati)" RUN mkdir -p /usr/local/share/powershell/Modules/Pode COPY ./pkg/ /usr/local/share/powershell/Modules/Pode \ No newline at end of file diff --git a/docs/Getting-Started/Installation.md b/docs/Getting-Started/Installation.md index 209da30e7..87645f5c2 100644 --- a/docs/Getting-Started/Installation.md +++ b/docs/Getting-Started/Installation.md @@ -41,7 +41,7 @@ Install-Module -Name Pode [![Docker](https://img.shields.io/docker/stars/badgerati/pode.svg?label=Stars)](https://hub.docker.com/r/badgerati/pode/) [![Docker](https://img.shields.io/docker/pulls/badgerati/pode.svg?label=Pulls)](https://hub.docker.com/r/badgerati/pode/) -Pode can run on *nix environments, therefore it only makes sense for there to be Docker images for you to use! The images use PowerShell v7.2 on either an Ubuntu Focal image (default), an Alpine image, or an ARM32 image (for Raspberry Pis). +Pode can run on *nix environments, therefore it only makes sense for there to be Docker images for you to use! The images use PowerShell v7.3 on either an Ubuntu Focal image (default), an Alpine image, or an ARM32 image (for Raspberry Pis). * To pull down the latest Pode image you can do: diff --git a/docs/Hosting/Docker.md b/docs/Hosting/Docker.md index 2797e761f..944a456ba 100644 --- a/docs/Hosting/Docker.md +++ b/docs/Hosting/Docker.md @@ -2,7 +2,7 @@ Pode has a Docker image that you can use to host your server, for instructions on pulling these images you can [look here](../../Installation). -The images use PowerShell v7.2 on either an Ubuntu Focal (default), Alpine, or ARM32 image. +The images use PowerShell v7.3 on either an Ubuntu Focal (default), Alpine, or ARM32 image. ## Images @@ -11,7 +11,7 @@ The images use PowerShell v7.2 on either an Ubuntu Focal (default), Alpine, or A ### Default -The default Pode image is an Ubuntu Focal image with PowerShell v7.2 and Pode installed. An example of using this image in your Dockerfile could be as follows: +The default Pode image is an Ubuntu Focal image with PowerShell v7.3 and Pode installed. An example of using this image in your Dockerfile could be as follows: ```dockerfile # pull down the pode image diff --git a/packers/docker/arm32/Dockerfile b/packers/docker/arm32/Dockerfile index 324366850..9f842034a 100644 --- a/packers/docker/arm32/Dockerfile +++ b/packers/docker/arm32/Dockerfile @@ -1,6 +1,6 @@ FROM arm32v7/ubuntu:bionic -ENV PS_VERSION=7.2.4 +ENV PS_VERSION=7.3.1 ENV PS_PACKAGE=powershell-${PS_VERSION}-linux-arm32.tar.gz ENV PS_PACKAGE_URL=https://github.com/PowerShell/PowerShell/releases/download/v${PS_VERSION}/${PS_PACKAGE} From eee460ae71b47dbad69e06c6658249360a25bbf7 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Thu, 5 Jan 2023 20:09:28 +0000 Subject: [PATCH 28/52] #1050: minor docs update --- docs/Getting-Started/Installation.md | 2 +- docs/Hosting/Docker.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/Getting-Started/Installation.md b/docs/Getting-Started/Installation.md index 87645f5c2..5828a80e2 100644 --- a/docs/Getting-Started/Installation.md +++ b/docs/Getting-Started/Installation.md @@ -41,7 +41,7 @@ Install-Module -Name Pode [![Docker](https://img.shields.io/docker/stars/badgerati/pode.svg?label=Stars)](https://hub.docker.com/r/badgerati/pode/) [![Docker](https://img.shields.io/docker/pulls/badgerati/pode.svg?label=Pulls)](https://hub.docker.com/r/badgerati/pode/) -Pode can run on *nix environments, therefore it only makes sense for there to be Docker images for you to use! The images use PowerShell v7.3 on either an Ubuntu Focal image (default), an Alpine image, or an ARM32 image (for Raspberry Pis). +Pode can run on *nix environments, therefore it only makes sense for there to be Docker images for you to use! The images use PowerShell v7.3 on either an Ubuntu Jammy image (default), an Alpine image, or an ARM32 image (for Raspberry Pis). * To pull down the latest Pode image you can do: diff --git a/docs/Hosting/Docker.md b/docs/Hosting/Docker.md index 944a456ba..1ce829a7d 100644 --- a/docs/Hosting/Docker.md +++ b/docs/Hosting/Docker.md @@ -2,7 +2,7 @@ Pode has a Docker image that you can use to host your server, for instructions on pulling these images you can [look here](../../Installation). -The images use PowerShell v7.3 on either an Ubuntu Focal (default), Alpine, or ARM32 image. +The images use PowerShell v7.3 on either an Ubuntu Jammy (default), Alpine, or ARM32 image. ## Images @@ -11,7 +11,7 @@ The images use PowerShell v7.3 on either an Ubuntu Focal (default), Alpine, or A ### Default -The default Pode image is an Ubuntu Focal image with PowerShell v7.3 and Pode installed. An example of using this image in your Dockerfile could be as follows: +The default Pode image is an Ubuntu Jammy image with PowerShell v7.3 and Pode installed. An example of using this image in your Dockerfile could be as follows: ```dockerfile # pull down the pode image From b4629b4b588582908d8b1e8a5c8fc0b1b08a867b Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Thu, 5 Jan 2023 20:37:55 +0000 Subject: [PATCH 29/52] #1052: bump mkdocs and material theme --- mkdocs.yml | 2 ++ pode.build.ps1 | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/mkdocs.yml b/mkdocs.yml index ac2a2e9e9..907fae4e8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -10,6 +10,8 @@ theme: - navigation.tabs.sticky - navigation.tracking - navigation.top + - content.copy.code + - content.action.edit font: text: Fira Sans code: Fira Code diff --git a/pode.build.ps1 b/pode.build.ps1 index b0e9b5e0a..0854d6fa5 100644 --- a/pode.build.ps1 +++ b/pode.build.ps1 @@ -9,12 +9,12 @@ param ( $Versions = @{ Pester = '4.8.0' - MkDocs = '1.2.3' + MkDocs = '1.4.2' PSCoveralls = '1.0.0' SevenZip = '18.5.0.20180730' DotNet = '6.0.1' Checksum = '0.2.0' - MkDocsTheme = '8.1.2' + MkDocsTheme = '9.0.2' PlatyPS = '0.14.0' } From 4db3fd1584ad09109dc33b0db1fe5e37d480be53 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Thu, 5 Jan 2023 22:15:51 +0000 Subject: [PATCH 30/52] #1051: bump listener to .NET7 for PS7.3+. also fix small resolving issue on Windows --- pode.build.ps1 | 37 +++++++++++++++++------------------- src/Listener/Pode.csproj | 2 +- src/Pode.psm1 | 8 +++++++- src/Private/Helpers.ps1 | 4 ++-- src/Public/Core.ps1 | 2 +- tests/unit/Helpers.Tests.ps1 | 16 ++++++++++++++-- 6 files changed, 42 insertions(+), 27 deletions(-) diff --git a/pode.build.ps1 b/pode.build.ps1 index 0854d6fa5..fdf269070 100644 --- a/pode.build.ps1 +++ b/pode.build.ps1 @@ -12,7 +12,7 @@ $Versions = @{ MkDocs = '1.4.2' PSCoveralls = '1.0.0' SevenZip = '18.5.0.20180730' - DotNet = '6.0.1' + DotNet = '7.0.1' Checksum = '0.2.0' MkDocsTheme = '9.0.2' PlatyPS = '0.14.0' @@ -95,6 +95,19 @@ function Install-PodeBuildModule($name) Install-Module -Name "$($name)" -Scope CurrentUser -RequiredVersion "$($Versions[$name])" -Force -SkipPublisherCheck } +function Invoke-PodeBuildDotnetBuild($target) +{ + dotnet build --configuration Release --self-contained --framework $target + if (!$?) { + throw "dotnet build failed for $($target)" + } + + dotnet publish --configuration Release --self-contained --framework $target --output ../Libs/$target + if (!$?) { + throw "dotnet publish failed for $($target)" + } +} + <# # Helper Tasks @@ -192,25 +205,9 @@ task Build BuildDeps, { Push-Location ./src/Listener try { - dotnet build --configuration Release --self-contained --framework netstandard2.0 - if (!$?) { - throw "dotnet build failed for netstandard2" - } - - dotnet publish --configuration Release --self-contained --framework netstandard2.0 --output ../Libs/netstandard2.0 - if (!$?) { - throw "dotnet publish failed for netstandard2" - } - - dotnet build --configuration Release --self-contained --framework net6.0 - if (!$?) { - throw "dotnet build failed for net6" - } - - dotnet publish --configuration Release --self-contained --framework net6.0 --output ../Libs/net6.0 - if (!$?) { - throw "dotnet publish failed for net6" - } + Invoke-PodeBuildDotnetBuild -target 'netstandard2.0' + Invoke-PodeBuildDotnetBuild -target 'net6.0' + Invoke-PodeBuildDotnetBuild -target 'net7.0' } finally { Pop-Location diff --git a/src/Listener/Pode.csproj b/src/Listener/Pode.csproj index 53515d4cb..f65cedc7f 100644 --- a/src/Listener/Pode.csproj +++ b/src/Listener/Pode.csproj @@ -1,6 +1,6 @@ - netstandard2.0;net6.0 + netstandard2.0;net6.0;net7.0 $(NoWarn);SYSLIB0001 diff --git a/src/Pode.psm1 b/src/Pode.psm1 index 29af54048..702715c79 100644 --- a/src/Pode.psm1 +++ b/src/Pode.psm1 @@ -5,12 +5,18 @@ $root = Split-Path -Parent -Path $MyInvocation.MyCommand.Path Add-Type -AssemblyName System.Web Add-Type -AssemblyName System.Net.Http +# netstandard2 for <7.2 if ($PSVersionTable.PSVersion -lt [version]'7.2.0') { Add-Type -LiteralPath "$($root)/Libs/netstandard2.0/Pode.dll" -ErrorAction Stop } -else { +# net6 for =7.2 +elseif ($PSVersionTable.PSVersion -lt [version]'7.3.0') { Add-Type -LiteralPath "$($root)/Libs/net6.0/Pode.dll" -ErrorAction Stop } +# net7 for >7.2 +else { + Add-Type -LiteralPath "$($root)/Libs/net7.0/Pode.dll" -ErrorAction Stop +} # import everything if in a runspace if ($PODE_SCOPE_RUNSPACE) { diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 5d4eb9020..bdfbc66dc 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -849,7 +849,7 @@ function New-PodePSDrive } # resolve the path - $Path = [System.IO.Path]::GetFullPath($Path.Replace('\', '/') , $pwd.Path) + $Path = Get-PodeRelativePath -Path $Path -JoinRoot -Resolve # create the temp drive if (!(Test-PodePSDrive -Name $Name -Path $Path)) { @@ -2211,7 +2211,7 @@ function Get-PodeRelativePath # if flagged, resolve the path if ($Resolve) { $_rawPath = $Path - $Path = [System.IO.Path]::GetFullPath($Path.Replace('\', '/'), $pwd.Path) + $Path = [System.IO.Path]::GetFullPath($Path.Replace('\', '/')) } # if flagged, test the path and throw error if it doesn't exist diff --git a/src/Public/Core.ps1 b/src/Public/Core.ps1 index cd0df9166..39883f4d2 100644 --- a/src/Public/Core.ps1 +++ b/src/Public/Core.ps1 @@ -634,7 +634,7 @@ function Show-PodeGui # set the window's icon path if (![string]::IsNullOrWhiteSpace($Icon)) { - $PodeContext.Server.Gui.Icon = [System.IO.Path]::GetFullPath($Icon.Replace('\', '/'), $pwd.Path) + $PodeContext.Server.Gui.Icon = Get-PodeRelativePath -Path $Icon -JoinRoot -Resolve if (!(Test-Path $PodeContext.Server.Gui.Icon)) { throw "Path to icon for GUI does not exist: $($PodeContext.Server.Gui.Icon)" } diff --git a/tests/unit/Helpers.Tests.ps1 b/tests/unit/Helpers.Tests.ps1 index f147043de..99a86cde0 100644 --- a/tests/unit/Helpers.Tests.ps1 +++ b/tests/unit/Helpers.Tests.ps1 @@ -1017,7 +1017,13 @@ Describe 'Get-PodeRelativePath' { } It 'Returns path for literal path when resolving' { - Get-PodeRelativePath -Path $pwd.Path -Resolve | Should Be $pwd.Path + $PodeContext = @{ + Server = @{ + Root = $pwd.Path + } + } + + Get-PodeRelativePath -Path $pwd.Path -Resolve -JoinRoot | Should Be $pwd.Path } It 'Returns back a relative path' { @@ -1025,7 +1031,13 @@ Describe 'Get-PodeRelativePath' { } It 'Returns path for a relative path when resolving' { - Get-PodeRelativePath -Path ".\src" -Resolve | Should Be (Join-Path $pwd.Path "src") + $PodeContext = @{ + Server = @{ + Root = $pwd.Path + } + } + + Get-PodeRelativePath -Path ".\src" -Resolve -JoinRoot | Should Be (Join-Path $pwd.Path "src") } It 'Returns path for a relative path joined to default root' { From 2480542c18b945d66c107ef6476760bc309d9850 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Thu, 5 Jan 2023 22:27:35 +0000 Subject: [PATCH 31/52] #1051: bump .net to 7 in github actions --- .github/workflows/ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index cf72a2b92..ff02d022d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -25,7 +25,7 @@ jobs: - name: Setup .NET uses: actions/setup-dotnet@v1.9.0 with: - dotnet-version: 6.0.x + dotnet-version: 7.0.x - name: Install Invoke-Build shell: pwsh From d9e0ac36de02a7ea9d784793f362a38a9cad7e64 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Thu, 12 Jan 2023 20:41:18 +0000 Subject: [PATCH 32/52] #1063: initial work for FIM support --- examples/Dockerfile | 2 +- examples/file-watchers.ps1 | 40 ++ .../BufferingFileSystemWatcher.cs | 430 ++++++++++++++++++ .../EventQueueOverflowException.cs | 14 + .../FileWatcherErrorEventArgs.cs | 16 + .../RecoveringFileSystemWatcher.cs | 186 ++++++++ src/Listener/PodeConnector.cs | 4 - src/Listener/PodeFileEvent.cs | 55 +++ src/Listener/PodeFileWatcher.cs | 91 ++++ src/Listener/PodeFileWatcherChangeType.cs | 12 + src/Listener/PodeListener.cs | 2 +- src/Listener/PodeReceiver.cs | 6 +- src/Listener/PodeWatcher.cs | 77 ++++ src/Listener/PodeWebSocket.cs | 2 +- src/Listener/PodeWebSocketRequest.cs | 1 + src/Private/Context.ps1 | 23 +- src/Private/FIM.ps1 | 418 +++++++++++++++++ src/Private/Helpers.ps1 | 78 +++- src/Private/Routes.ps1 | 54 +-- src/Private/Server.ps1 | 7 + src/Public/Core.ps1 | 2 +- src/Public/FIM.ps1 | 206 +++++++++ src/Public/Routes.ps1 | 12 +- src/Public/Verbs.ps1 | 6 +- tests/unit/Routes.Tests.ps1 | 14 +- 25 files changed, 1688 insertions(+), 70 deletions(-) create mode 100644 examples/file-watchers.ps1 create mode 100644 src/Listener/FileSystemWatcher/BufferingFileSystemWatcher.cs create mode 100644 src/Listener/FileSystemWatcher/EventQueueOverflowException.cs create mode 100644 src/Listener/FileSystemWatcher/FileWatcherErrorEventArgs.cs create mode 100644 src/Listener/FileSystemWatcher/RecoveringFileSystemWatcher.cs create mode 100644 src/Listener/PodeFileEvent.cs create mode 100644 src/Listener/PodeFileWatcher.cs create mode 100644 src/Listener/PodeFileWatcherChangeType.cs create mode 100644 src/Listener/PodeWatcher.cs create mode 100644 src/Private/FIM.ps1 create mode 100644 src/Public/FIM.ps1 diff --git a/examples/Dockerfile b/examples/Dockerfile index 65ff8c543..b4424f241 100644 --- a/examples/Dockerfile +++ b/examples/Dockerfile @@ -1,4 +1,4 @@ FROM badgerati/pode:test COPY . /usr/src/app/ EXPOSE 8085 -CMD [ "pwsh", "-c", "cd /usr/src/app; ./web-pages-docker.ps1" ] +CMD [ "pwsh", "-c", "cd /usr/src/app; ./file-watchers.ps1" ] diff --git a/examples/file-watchers.ps1 b/examples/file-watchers.ps1 new file mode 100644 index 000000000..b1587b6ee --- /dev/null +++ b/examples/file-watchers.ps1 @@ -0,0 +1,40 @@ +$path = Split-Path -Parent -Path (Split-Path -Parent -Path $MyInvocation.MyCommand.Path) +Import-Module "$($path)/src/Pode.psm1" -Force -ErrorAction Stop + +# or just: +# Import-Module Pode + +# create a server, and start listening on port 9000 +Start-PodeServer -Verbose { + + # add two endpoints + Add-PodeEndpoint -Address * -Port 9000 -Protocol Http + + # enable logging + New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging + + # watchers + # Add-PodeFileWatcher -Name 'Test' -Path './public' -Include '*.txt', '*.md' -ScriptBlock { + # "[$($FileEvent.Type)]: $($FileEvent.FullPath)" | Out-Default + # } + + # Add-PodeFileWatcher -Path 'C:/Projects/:project/src' -Include '*.ps1' -ScriptBlock { + # "[$($FileEvent.Type)][$($FileEvent.Parameters['project'])]: $($FileEvent.FullPath)" | Out-Default + # } + # Add-PodeFileWatcher -Path 'C:/Projects/Pode/:folder' -Include '*.psd1', '*.txt' -ScriptBlock { + # "[$($FileEvent.Type)][$($FileEvent.Parameters['folder'])]: $($FileEvent.FullPath)" | Out-Default + # } + # Add-PodeFileWatcher -Path '/mnt/c/Projects/:project/src' -Include '*.ps1' -ScriptBlock { + # "[$($FileEvent.Type)][$($FileEvent.Parameters['project'])]: $($FileEvent.FullPath)" | Out-Default + # } + + Add-PodeTimer -Name 'Test' -Interval 10 -ScriptBlock { + $root = Get-PodeServerPath + $file = Join-Path $root 'myfile.txt' + 'hi!' | Out-File -FilePath $file -Append -Force + } + + Add-PodeFileWatcher -Path '.' -Include '*.txt' -ScriptBlock { + "[$($FileEvent.Type)]: $($FileEvent.FullPath)" | Out-Default + } +} \ No newline at end of file diff --git a/src/Listener/FileSystemWatcher/BufferingFileSystemWatcher.cs b/src/Listener/FileSystemWatcher/BufferingFileSystemWatcher.cs new file mode 100644 index 000000000..317c12e2a --- /dev/null +++ b/src/Listener/FileSystemWatcher/BufferingFileSystemWatcher.cs @@ -0,0 +1,430 @@ +using System; +using System.Collections.Concurrent; +using System.ComponentModel; +using System.IO; +using System.Linq; +using System.Threading; +using System.Threading.Tasks; + +// A better FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher +namespace Pode.FileSystemWatcher +{ + public class BufferingFileSystemWatcher : Component + { + public PodeItemQueue Contexts { get; private set; } + public Task GetContextAsync(CancellationToken cancellationToken = default(CancellationToken)) + { + return Contexts.GetAsync(cancellationToken); + } + + + private System.IO.FileSystemWatcher _containedFSW = null; + private FileSystemEventHandler _onExistedHandler = null; + private FileSystemEventHandler _onAllChangesHandler = null; + private FileSystemEventHandler _onCreatedHandler = null; + private FileSystemEventHandler _onChangedHandler = null; + private FileSystemEventHandler _onDeletedHandler = null; + private RenamedEventHandler _onRenamedHandler = null; + private ErrorEventHandler _onErrorHandler = null; + private BlockingCollection _fileSystemEventBuffer = null; + private CancellationTokenSource _cancellationTokenSource = null; + + public bool EnableRaisingEvents + { + get + { + return _containedFSW.EnableRaisingEvents; + } + set + { + if (_containedFSW.EnableRaisingEvents == value) + { + return; + } + + StopRaisingBufferedEvents(); + _cancellationTokenSource = new CancellationTokenSource(); + + // We EnableRaisingEvents, before NotifyExistingFiles + // to prevent missing any events + // accepting more duplicates (which may occur anyway). + _containedFSW.EnableRaisingEvents = value; + if (value) + { + RaiseBufferedEventsUntilCancelled(); + } + } + } + + public string Filter + { + get { return _containedFSW.Filter; } + set { _containedFSW.Filter = value; } + } + + public bool IncludeSubdirectories + { + get { return _containedFSW.IncludeSubdirectories; } + set { _containedFSW.IncludeSubdirectories = value; } + } + + public int InternalBufferSize + { + get { return _containedFSW.InternalBufferSize; } + set { _containedFSW.InternalBufferSize = value; } + } + + public NotifyFilters NotifyFilter + { + get { return _containedFSW.NotifyFilter; } + set { _containedFSW.NotifyFilter = value; } + } + + public string Path + { + get { return _containedFSW.Path; } + set { _containedFSW.Path = value; } + } + + public ISynchronizeInvoke SynchronizingObject + { + get { return _containedFSW.SynchronizingObject; } + set { _containedFSW.SynchronizingObject = value; } + } + + public override ISite Site + { + get { return _containedFSW.Site; } + set { _containedFSW.Site = value; } + } + + public bool OrderByOldestFirst { get; set; } = false; + + private int _eventQueueSize = int.MaxValue; + public int EventQueueCapacity + { + get { return _eventQueueSize; } + set { _eventQueueSize = value; } + } + + public int Count + { + get { return _fileSystemEventBuffer.Count; } + } + + + public BufferingFileSystemWatcher() + { + _containedFSW = new System.IO.FileSystemWatcher(); + Contexts = new PodeItemQueue(); + } + + public BufferingFileSystemWatcher(string path) + { + _containedFSW = new System.IO.FileSystemWatcher(path, "*.*"); + Contexts = new PodeItemQueue(); + } + + public BufferingFileSystemWatcher(string path, string filter) + { + _containedFSW = new System.IO.FileSystemWatcher(path, filter); + Contexts = new PodeItemQueue(); + } + + + public event FileSystemEventHandler Existed + { + add { _onExistedHandler += value; } + remove { _onExistedHandler -= value; } + } + + public event FileSystemEventHandler All + { + add + { + if (_onAllChangesHandler == null) + { + _containedFSW.Created += BufferEvent; + _containedFSW.Changed += BufferEvent; + _containedFSW.Renamed += BufferEvent; + _containedFSW.Deleted += BufferEvent; + } + + _onAllChangesHandler += value; + } + remove + { + _containedFSW.Created -= BufferEvent; + _containedFSW.Changed -= BufferEvent; + _containedFSW.Renamed -= BufferEvent; + _containedFSW.Deleted -= BufferEvent; + _onAllChangesHandler -= value; + } + } + + //- The _fsw events add to the buffer. + //- The public events raise from the buffer to the consumer. + public event FileSystemEventHandler Created + { + add + { + if (_onCreatedHandler == null) + { + _containedFSW.Created += BufferEvent; + } + + _onCreatedHandler += value; + } + remove + { + _containedFSW.Created -= BufferEvent; + _onCreatedHandler -= value; + } + } + + public event FileSystemEventHandler Changed + { + add + { + if (_onChangedHandler == null) + { + _containedFSW.Changed += BufferEvent; + } + + _onChangedHandler += value; + } + remove + { + _containedFSW.Changed -= BufferEvent; + _onChangedHandler -= value; + } + } + + public event FileSystemEventHandler Deleted + { + add + { + if (_onDeletedHandler == null) + { + _containedFSW.Deleted += BufferEvent; + } + + _onDeletedHandler += value; + } + remove + { + _containedFSW.Deleted -= BufferEvent; + _onDeletedHandler -= value; + } + } + + public event RenamedEventHandler Renamed + { + add + { + if (_onRenamedHandler == null) + { + _containedFSW.Renamed += BufferEvent; + } + + _onRenamedHandler += value; + } + remove + { + _containedFSW.Renamed -= BufferEvent; + _onRenamedHandler -= value; + } + } + + public event ErrorEventHandler Error + { + add + { + if (_onErrorHandler == null) + { + _containedFSW.Error += BufferingFileSystemWatcher_Error; + } + + _onErrorHandler += value; + } + remove + { + if (_onErrorHandler == null) + { + _containedFSW.Error -= BufferingFileSystemWatcher_Error; + } + + _onErrorHandler -= value; + } + } + + + private string _lastFilePath = string.Empty; + private DateTime _lastDateTime = DateTime.MinValue; + + private void BufferEvent(object _, FileSystemEventArgs e) + { + // prevent duplicate change events + if (e.ChangeType == WatcherChangeTypes.Changed) + { + lock (_lastFilePath) + { + if (e.FullPath == _lastFilePath && _lastDateTime.AddMilliseconds(500) > DateTime.UtcNow) + { + return; + } + + _lastFilePath = e.FullPath; + _lastDateTime = DateTime.UtcNow; + } + } + + // add event to buffer + if (!_fileSystemEventBuffer.TryAdd(e)) + { + var ex = new EventQueueOverflowException($"Event queue size {_fileSystemEventBuffer.BoundedCapacity} events exceeded."); + InvokeHandler(_onErrorHandler, new ErrorEventArgs(ex)); + } + } + + private void StopRaisingBufferedEvents(object _ = null, EventArgs __ = null) + { + _cancellationTokenSource?.Cancel(); + _fileSystemEventBuffer = new BlockingCollection(_eventQueueSize); + } + + + private void BufferingFileSystemWatcher_Error(object sender, ErrorEventArgs e) + { + InvokeHandler(_onErrorHandler, e); + } + + private void RaiseBufferedEventsUntilCancelled() + { + Task.Run(() => + { + try + { + if (_onExistedHandler != null || _onAllChangesHandler != null) + { + NotifyExistingFiles(); + } + + foreach (var e in _fileSystemEventBuffer.GetConsumingEnumerable(_cancellationTokenSource.Token)) + { + if (_onAllChangesHandler != null) + { + InvokeHandler(_onAllChangesHandler, e); + } + else + { + switch (e.ChangeType) + { + case WatcherChangeTypes.Created: + InvokeHandler(_onCreatedHandler, e); + break; + case WatcherChangeTypes.Changed: + InvokeHandler(_onChangedHandler, e); + break; + case WatcherChangeTypes.Deleted: + InvokeHandler(_onDeletedHandler, e); + break; + case WatcherChangeTypes.Renamed: + InvokeHandler(_onRenamedHandler, e as RenamedEventArgs); + break; + } + } + } + } + catch (OperationCanceledException) { } + catch (Exception ex) + { + BufferingFileSystemWatcher_Error(this, new ErrorEventArgs(ex)); + } + }); + } + + private void NotifyExistingFiles() + { + var searchSubDirectoriesOption = (IncludeSubdirectories ? SearchOption.AllDirectories : SearchOption.TopDirectoryOnly); + + var files = new DirectoryInfo(Path).GetFiles(Filter, searchSubDirectoriesOption); + if (OrderByOldestFirst) + { + files = files.OrderBy(x => x.LastWriteTimeUtc).ToArray(); + } + + foreach (var file in files) + { + InvokeHandler(_onExistedHandler, new FileSystemEventArgs(WatcherChangeTypes.All, file.DirectoryName, file.Name)); + InvokeHandler(_onAllChangesHandler, new FileSystemEventArgs(WatcherChangeTypes.All, file.DirectoryName, file.Name)); + } + } + + private void InvokeHandler(FileSystemEventHandler eventHandler, FileSystemEventArgs e) + { + if (eventHandler != null) + { + if (_containedFSW.SynchronizingObject != null && this._containedFSW.SynchronizingObject.InvokeRequired) + { + _containedFSW.SynchronizingObject.BeginInvoke(eventHandler, new object[] { this, e }); + } + else + { + eventHandler(this, e); + } + } + } + + private void InvokeHandler(RenamedEventHandler eventHandler, RenamedEventArgs e) + { + if (eventHandler != null) + { + if (_containedFSW.SynchronizingObject != null && this._containedFSW.SynchronizingObject.InvokeRequired) + { + _containedFSW.SynchronizingObject.BeginInvoke(eventHandler, new object[] { this, e }); + } + else + { + eventHandler(this, e); + } + } + } + + private void InvokeHandler(ErrorEventHandler eventHandler, ErrorEventArgs e) + { + if (eventHandler != null) + { + if (_containedFSW.SynchronizingObject != null && this._containedFSW.SynchronizingObject.InvokeRequired) + { + _containedFSW.SynchronizingObject.BeginInvoke(eventHandler, new object[] { this, e }); + } + else + { + eventHandler(this, e); + } + } + } + + + protected override void Dispose(bool disposing) + { + if (disposing) + { + _cancellationTokenSource?.Cancel(); + _containedFSW?.Dispose(); + _fileSystemEventBuffer?.Dispose(); + + _onExistedHandler = null; + _onAllChangesHandler = null; + _onCreatedHandler = null; + _onChangedHandler = null; + _onDeletedHandler = null; + _onRenamedHandler = null; + _onErrorHandler = null; + } + + base.Dispose(disposing); + } + } +} \ No newline at end of file diff --git a/src/Listener/FileSystemWatcher/EventQueueOverflowException.cs b/src/Listener/FileSystemWatcher/EventQueueOverflowException.cs new file mode 100644 index 000000000..9ffa3236d --- /dev/null +++ b/src/Listener/FileSystemWatcher/EventQueueOverflowException.cs @@ -0,0 +1,14 @@ +using System; + +// A better FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher +namespace Pode.FileSystemWatcher +{ + class EventQueueOverflowException : Exception + { + public EventQueueOverflowException() + : base() { } + + public EventQueueOverflowException(string message) + : base(message) { } + } +} \ No newline at end of file diff --git a/src/Listener/FileSystemWatcher/FileWatcherErrorEventArgs.cs b/src/Listener/FileSystemWatcher/FileWatcherErrorEventArgs.cs new file mode 100644 index 000000000..318046c42 --- /dev/null +++ b/src/Listener/FileSystemWatcher/FileWatcherErrorEventArgs.cs @@ -0,0 +1,16 @@ +using System; +using System.ComponentModel; + +// A better FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher +namespace Pode.FileSystemWatcher +{ + public class FileWatcherErrorEventArgs : HandledEventArgs + { + public readonly Exception Error; + + public FileWatcherErrorEventArgs(Exception exception) + { + this.Error = exception; + } + } +} \ No newline at end of file diff --git a/src/Listener/FileSystemWatcher/RecoveringFileSystemWatcher.cs b/src/Listener/FileSystemWatcher/RecoveringFileSystemWatcher.cs new file mode 100644 index 000000000..a12b184e4 --- /dev/null +++ b/src/Listener/FileSystemWatcher/RecoveringFileSystemWatcher.cs @@ -0,0 +1,186 @@ +using System; +using System.ComponentModel; +using System.IO; +using System.Threading; + +// A better FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher +namespace Pode.FileSystemWatcher +{ + public class RecoveringFileSystemWatcher : BufferingFileSystemWatcher + { + public TimeSpan DirectoryMonitorInterval = TimeSpan.FromMinutes(5); + public TimeSpan DirectoryRetryInterval = TimeSpan.FromSeconds(5); + private System.Threading.Timer _monitorTimer = null; + private bool _isRecovering = false; + + + public RecoveringFileSystemWatcher() + : base() { } + + public RecoveringFileSystemWatcher(string path) + : base(path, "*.*") { } + + public RecoveringFileSystemWatcher(string path, string filter) + : base(path, filter) { } + + + // To allow consumer to cancel default error handling + private EventHandler _onErrorHandler = null; + public new event EventHandler Error + { + add { _onErrorHandler += value; } + remove { _onErrorHandler -= value; } + } + + public new bool EnableRaisingEvents + { + get { return base.EnableRaisingEvents; } + set + { + if (value == EnableRaisingEvents) + { + return; + } + + base.EnableRaisingEvents = value; + if (EnableRaisingEvents) + { + base.Error += BufferingFileSystemWatcher_Error; + Start(); + } + else + { + base.Error -= BufferingFileSystemWatcher_Error; + } + } + } + + private void Start() + { + try + { + _monitorTimer = new System.Threading.Timer(_monitorTimer_Elapsed); + + Disposed += (_, __) => + { + _monitorTimer.Dispose(); + }; + + RestartIfNecessary(TimeSpan.Zero); + } + catch (Exception) + { + throw; + } + } + + private void _monitorTimer_Elapsed(object state) + { + try + { + if (!Directory.Exists(Path)) + { + throw new DirectoryNotFoundException($"Directory not found {Path}"); + } + else + { + if (!EnableRaisingEvents) + { + EnableRaisingEvents = true; + } + + RestartIfNecessary(DirectoryMonitorInterval); + } + } + catch (Exception ex) when (ex is FileNotFoundException || ex is DirectoryNotFoundException) + { + // Handles race condition too: Path loses accessibility between .Exists() and .EnableRaisingEvents + if (ExceptionWasHandledByCaller(ex)) + { + return; + } + + if (!_isRecovering) + { + _isRecovering = true; + } + + EnableRaisingEvents = false; + _isRecovering = true; + RestartIfNecessary(DirectoryRetryInterval); + } + catch (Exception) + { + throw; + } + } + + private void RestartIfNecessary(TimeSpan delay) + { + try + { + _monitorTimer.Change(delay, Timeout.InfiniteTimeSpan); + } + catch (ObjectDisposedException) { } // ignore timer disposed + } + + private void BufferingFileSystemWatcher_Error(object sender, ErrorEventArgs e) + { + // These exceptions have the same HResult + var NetworkNameNoLongerAvailable = -2147467259; // occurs on network outage + var AccessIsDenied = -2147467259; // occurs after directory was deleted + + var ex = e.GetException(); + if (ExceptionWasHandledByCaller(e.GetException())) + { + return; + } + + // The base FSW does set .EnableRaisingEvents=False AFTER raising OnError() + EnableRaisingEvents = false; + + if (ex is InternalBufferOverflowException || ex is EventQueueOverflowException) + { + RestartIfNecessary(DirectoryRetryInterval); + } + else if (ex is Win32Exception && (ex.HResult == NetworkNameNoLongerAvailable | ex.HResult == AccessIsDenied)) + { + RestartIfNecessary(DirectoryRetryInterval); + } + else + { + throw ex; + } + } + + private bool ExceptionWasHandledByCaller(Exception ex) + { + // Allow consumer to handle error + if (_onErrorHandler != null) + { + var e = new FileWatcherErrorEventArgs(ex); + InvokeHandler(_onErrorHandler, e); + return e.Handled; + } + else + { + return false; + } + } + + private void InvokeHandler(EventHandler eventHandler, FileWatcherErrorEventArgs e) + { + if (eventHandler != null) + { + if (SynchronizingObject != null && this.SynchronizingObject.InvokeRequired) + { + SynchronizingObject.BeginInvoke(eventHandler, new object[] { this, e }); + } + else + { + eventHandler(this, e); + } + } + } + } +} \ No newline at end of file diff --git a/src/Listener/PodeConnector.cs b/src/Listener/PodeConnector.cs index 897bcbfc6..be7a3d5a8 100644 --- a/src/Listener/PodeConnector.cs +++ b/src/Listener/PodeConnector.cs @@ -1,9 +1,5 @@ using System; -using System.Linq; -using System.Collections.Concurrent; -using System.Collections.Generic; using System.Threading; -using System.Threading.Tasks; namespace Pode { diff --git a/src/Listener/PodeFileEvent.cs b/src/Listener/PodeFileEvent.cs new file mode 100644 index 000000000..57087acd9 --- /dev/null +++ b/src/Listener/PodeFileEvent.cs @@ -0,0 +1,55 @@ +using System; +using System.IO; + +namespace Pode +{ + public class PodeFileEvent : IDisposable + { + public PodeFileWatcher FileWatcher { get; private set; } + public PodeFileWatcherChangeType ChangeType { get; private set; } + public string Name { get; private set; } + public string FullPath { get; private set; } + public string OldName { get; private set; } + public string OldFullPath { get; private set; } + + public PodeFileEvent(PodeFileWatcher watcher, FileSystemEventArgs e) + { + FileWatcher = watcher; + ChangeType = MapChangeType(e.ChangeType); + Name = e.Name; + FullPath = e.FullPath; + + if (ChangeType == PodeFileWatcherChangeType.Renamed) + { + var re = e as RenamedEventArgs; + OldName = re.OldName; + OldFullPath = re.OldFullPath; + } + } + + private PodeFileWatcherChangeType MapChangeType(WatcherChangeTypes type) + { + switch (type) + { + case WatcherChangeTypes.All: + return PodeFileWatcherChangeType.Existed; + case WatcherChangeTypes.Changed: + return PodeFileWatcherChangeType.Changed; + case WatcherChangeTypes.Created: + return PodeFileWatcherChangeType.Created; + case WatcherChangeTypes.Deleted: + return PodeFileWatcherChangeType.Deleted; + case WatcherChangeTypes.Renamed: + return PodeFileWatcherChangeType.Renamed; + default: + return PodeFileWatcherChangeType.Errored; + } + } + + public void Dispose() + { + FileWatcher.Watcher.RemoveProcessingFileEvent(this); + FileWatcher = null; + } + } +} \ No newline at end of file diff --git a/src/Listener/PodeFileWatcher.cs b/src/Listener/PodeFileWatcher.cs new file mode 100644 index 000000000..9a2d2f1e7 --- /dev/null +++ b/src/Listener/PodeFileWatcher.cs @@ -0,0 +1,91 @@ +using System.Collections.Generic; +using System.IO; +using Pode.FileSystemWatcher; + +namespace Pode +{ + public class PodeFileWatcher + { + public PodeWatcher Watcher; + private RecoveringFileSystemWatcher FileWatcher; + + public string Name { get; private set; } + public ISet EventsRegistered {get; private set; } + + public PodeFileWatcher(string name, string path, bool includeSubdirectories, int internalBufferSize, NotifyFilters notifyFilters) + { + Name = name; + + FileWatcher = new RecoveringFileSystemWatcher(path); + FileWatcher.IncludeSubdirectories = includeSubdirectories; + FileWatcher.InternalBufferSize = internalBufferSize; + FileWatcher.NotifyFilter = notifyFilters; + + EventsRegistered = new HashSet(); + RegisterEvent(PodeFileWatcherChangeType.Errored); + } + + public void BindWatcher(PodeWatcher watcher) + { + Watcher = watcher; + } + + public void RegisterEvent(PodeFileWatcherChangeType type) + { + EventsRegistered.Add(type); + } + + public void Start() + { + foreach (var evt in EventsRegistered) + { + switch (evt) + { + case PodeFileWatcherChangeType.Created: + FileWatcher.Created += FileEventHandler; + break; + + case PodeFileWatcherChangeType.Changed: + FileWatcher.Changed += FileEventHandler; + break; + + case PodeFileWatcherChangeType.Deleted: + FileWatcher.Deleted += FileEventHandler; + break; + + case PodeFileWatcherChangeType.Existed: + FileWatcher.Existed += FileEventHandler; + break; + + case PodeFileWatcherChangeType.Renamed: + FileWatcher.Renamed += FileEventHandler; + break; + + case PodeFileWatcherChangeType.Errored: + FileWatcher.Error += FileErrorEventHandler; + break; + } + } + + FileWatcher.EnableRaisingEvents = true; + } + + public void Dispose() + { + if (FileWatcher != default(RecoveringFileSystemWatcher)) + { + FileWatcher.Dispose(); + } + } + + private void FileEventHandler(object _, FileSystemEventArgs e) + { + Watcher.AddFileEvent(new PodeFileEvent(this, e)); + } + + private void FileErrorEventHandler(object _, FileWatcherErrorEventArgs e) + { + PodeHelpers.WriteException(e.Error, Watcher); + } + } +} \ No newline at end of file diff --git a/src/Listener/PodeFileWatcherChangeType.cs b/src/Listener/PodeFileWatcherChangeType.cs new file mode 100644 index 000000000..74d469b52 --- /dev/null +++ b/src/Listener/PodeFileWatcherChangeType.cs @@ -0,0 +1,12 @@ +namespace Pode +{ + public enum PodeFileWatcherChangeType + { + Created, + Changed, + Deleted, + Renamed, + Existed, + Errored + } +} \ No newline at end of file diff --git a/src/Listener/PodeListener.cs b/src/Listener/PodeListener.cs index c97d91858..c87636401 100644 --- a/src/Listener/PodeListener.cs +++ b/src/Listener/PodeListener.cs @@ -36,6 +36,7 @@ public int RequestBodySize } public PodeListener(CancellationToken cancellationToken = default(CancellationToken)) + : base(cancellationToken) { Sockets = new List(); Signals = new Dictionary(); @@ -150,7 +151,6 @@ public override void Start() base.Start(); } - // public void Dispose() protected override void Close() { // shutdown the sockets diff --git a/src/Listener/PodeReceiver.cs b/src/Listener/PodeReceiver.cs index 8b325b215..c723e2e2c 100644 --- a/src/Listener/PodeReceiver.cs +++ b/src/Listener/PodeReceiver.cs @@ -1,6 +1,5 @@ using System; using System.Linq; -using System.Collections.Concurrent; using System.Collections.Generic; using System.Threading; using System.Threading.Tasks; @@ -74,6 +73,11 @@ public void AddWebSocketRequest(PodeWebSocketRequest request) Requests.Add(request); } + public void RemoveProcessingWebSocketRequest(PodeWebSocketRequest request) + { + Requests.RemoveProcessing(request); + } + public PodeWebSocketRequest GetWebSocketRequest(CancellationToken cancellationToken = default(CancellationToken)) { return Requests.Get(cancellationToken); diff --git a/src/Listener/PodeWatcher.cs b/src/Listener/PodeWatcher.cs new file mode 100644 index 000000000..eebdec709 --- /dev/null +++ b/src/Listener/PodeWatcher.cs @@ -0,0 +1,77 @@ +using System.Collections.Generic; +using System.Threading; +using System.Linq; +using System.Threading.Tasks; + +namespace Pode +{ + public class PodeWatcher : PodeConnector + { + private IList FileWatchers; + + public PodeItemQueue FileEvents { get; private set; } + + public PodeWatcher(CancellationToken cancellationToken = default(CancellationToken)) + : base(cancellationToken) + { + FileWatchers = new List(); + FileEvents = new PodeItemQueue(); + } + + public void AddFileWatcher(PodeFileWatcher watcher) + { + watcher.BindWatcher(this); + FileWatchers.Add(watcher); + } + + public Task GetFileEventAsync(CancellationToken cancellationToken = default(CancellationToken)) + { + return FileEvents.GetAsync(cancellationToken); + } + + public void AddFileEvent(PodeFileEvent fileEvent) + { + FileEvents.Add(fileEvent); + } + + public void RemoveProcessingFileEvent(PodeFileEvent fileEvent) + { + FileEvents.RemoveProcessing(fileEvent); + } + + public override void Start() + { + foreach (var watcher in FileWatchers) + { + watcher.Start(); + } + + base.Start(); + } + + protected override void Close() + { + // dispose watchers + PodeHelpers.WriteErrorMessage($"Closing file watchers", this, PodeLoggingLevel.Verbose); + + foreach (var _watcher in FileWatchers.ToArray()) + { + _watcher.Dispose(); + } + + FileWatchers.Clear(); + PodeHelpers.WriteErrorMessage($"Closed file watchers", this, PodeLoggingLevel.Verbose); + + // dispose existing file events + PodeHelpers.WriteErrorMessage($"Closing file events", this, PodeLoggingLevel.Verbose); + + foreach (var _evt in FileEvents.ToArray()) + { + _evt.Dispose(); + } + + FileEvents.Clear(); + PodeHelpers.WriteErrorMessage($"Closed file events", this, PodeLoggingLevel.Verbose); + } + } +} \ No newline at end of file diff --git a/src/Listener/PodeWebSocket.cs b/src/Listener/PodeWebSocket.cs index 9cfb6ca67..10d6a75f9 100644 --- a/src/Listener/PodeWebSocket.cs +++ b/src/Listener/PodeWebSocket.cs @@ -11,6 +11,7 @@ namespace Pode { public class PodeWebSocket : IDisposable { + public PodeReceiver Receiver { get; private set; } public string Name { get; private set; } public Uri URL { get; private set; } public string ContentType { get; private set; } @@ -20,7 +21,6 @@ public bool IsConnected } private ClientWebSocket WebSocket; - private PodeReceiver Receiver; public PodeWebSocket(string name, string url, string contentType) { diff --git a/src/Listener/PodeWebSocketRequest.cs b/src/Listener/PodeWebSocketRequest.cs index 875a69707..7305d9666 100644 --- a/src/Listener/PodeWebSocketRequest.cs +++ b/src/Listener/PodeWebSocketRequest.cs @@ -38,6 +38,7 @@ public PodeWebSocketRequest(PodeWebSocket webSocket, MemoryStream bytes) public void Dispose() { + WebSocket.Receiver.RemoveProcessingWebSocketRequest(this); RawBody = default(byte[]); _body = string.Empty; } diff --git a/src/Private/Context.ps1 b/src/Private/Context.ps1 index 2777f6f09..bd200ab59 100644 --- a/src/Private/Context.ps1 +++ b/src/Private/Context.ps1 @@ -79,7 +79,9 @@ function New-PodeContext Add-Member -MemberType NoteProperty -Name Server -Value @{} -PassThru | Add-Member -MemberType NoteProperty -Name Metrics -Value @{} -PassThru | Add-Member -MemberType NoteProperty -Name Listeners -Value @() -PassThru | - Add-Member -MemberType NoteProperty -Name Receivers -Value @() -PassThru + Add-Member -MemberType NoteProperty -Name Receivers -Value @() -PassThru | + Add-Member -MemberType NoteProperty -Name Watchers -Value @() -PassThru | + Add-Member -MemberType NoteProperty -Name Fim -Value @{} -PassThru # set the server name, logic and root, and other basic properties $ctx.Server.Name = $Name @@ -94,8 +96,9 @@ function New-PodeContext # list of created listeners/receivers $ctx.Listeners = @() $ctx.Receivers = @() + $ctx.Watchers = @() - # list of timers/schedules/tasks + # list of timers/schedules/tasks/fim $ctx.Timers = @{ Enabled = ($EnablePool -icontains 'timers') Items = @{} @@ -113,6 +116,11 @@ function New-PodeContext Results = @{} } + $ctx.Fim = @{ + Enabled = ($EnablePool -icontains 'files') + Items = @{} + } + # auto importing (modules, funcs, snap-ins) $ctx.Server.AutoImport = Initialize-PodeAutoImportConfiguration @@ -126,6 +134,7 @@ function New-PodeContext $ctx.Threads = @{ General = $Threads Schedules = 10 + Files = 1 Tasks = 2 WebSockets = 2 } @@ -375,6 +384,7 @@ function New-PodeContext Schedules = $null Gui = $null Tasks = $null + Files = $null } # session state @@ -526,6 +536,14 @@ function New-PodeRunspacePools } } + # setup files runspace pool -if we have any file watchers + if (Test-PodeFileWatchersExist) { + $PodeContext.RunspacePools.Files = @{ + Pool = [runspacefactory]::CreateRunspacePool(1, $PodeContext.Threads.Files + 1, $PodeContext.RunspaceState, $Host) + State = 'Waiting' + } + } + # setup gui runspace pool (only for non-ps-core) - if gui enabled if (Test-PodeGuiEnabled) { $PodeContext.RunspacePools.Gui = @{ @@ -676,6 +694,7 @@ function New-PodeStateContext Add-Member -MemberType NoteProperty -Name Timers -Value $Context.Timers -PassThru | Add-Member -MemberType NoteProperty -Name Schedules -Value $Context.Schedules -PassThru | Add-Member -MemberType NoteProperty -Name Tasks -Value $Context.Tasks -PassThru | + Add-Member -MemberType NoteProperty -Name Fim -Value $Context.Fim -PassThru | Add-Member -MemberType NoteProperty -Name RunspacePools -Value $Context.RunspacePools -PassThru | Add-Member -MemberType NoteProperty -Name Tokens -Value $Context.Tokens -PassThru | Add-Member -MemberType NoteProperty -Name Metrics -Value $Context.Metrics -PassThru | diff --git a/src/Private/FIM.ps1 b/src/Private/FIM.ps1 new file mode 100644 index 000000000..ac3ebf3db --- /dev/null +++ b/src/Private/FIM.ps1 @@ -0,0 +1,418 @@ +using namespace Pode + +function Test-PodeFileWatchersExist +{ + return (($null -ne $PodeContext.Fim) -and (($PodeContext.Fim.Enabled) -or ($PodeContext.Fim.Items.Count -gt 0))) +} + +function New-PodeFileWatcher +{ + $watcher = [PodeWatcher]::new($PodeContext.Tokens.Cancellation.Token) + $watcher.ErrorLoggingEnabled = (Test-PodeErrorLoggingEnabled) + $watcher.ErrorLoggingLevels = @(Get-PodeErrorLoggingLevels) + return $watcher +} + +function Start-PodeFileWatcherRunspace +{ + if (!(Test-PodeFileWatchersExist)) { + return + } + + try { + # create the watcher + $watcher = New-PodeFileWatcher + + # register file watchers and events + foreach ($item in $PodeContext.Fim.Items.Values) { + for ($i = 0; $i -lt $item.Watchers.Length; $i++) { + Write-Verbose "Creating FileWatcher for '$($item.Watchers[$i].Path)'" + $fileWatcher = [PodeFileWatcher]::new($item.Name, $item.Watchers[$i].Path, $item.IncludeSubdirectories, $item.InternalBufferSize, $item.NotifyFilters) + + foreach ($evt in $item.Events) { + Write-Verbose "-> Registering event: $($evt)" + $fileWatcher.RegisterEvent($evt) + } + + $watcher.AddFileWatcher($fileWatcher) + } + } + + $watcher.Start() + $PodeContext.Watchers += $watcher + } + catch { + $_ | Write-PodeErrorLog + $_.Exception | Write-PodeErrorLog -CheckInnerException + Close-PodeDisposable -Disposable $watcher + throw $_.Exception + } + + $watchScript = { + param( + [Parameter(Mandatory=$true)] + $Watcher, + + [Parameter(Mandatory=$true)] + [int] + $ThreadId + ) + + try + { + while ($Watcher.IsConnected -and !$PodeContext.Tokens.Cancellation.IsCancellationRequested) + { + $evt = (Wait-PodeTask -Task $Watcher.GetFileEventAsync($PodeContext.Tokens.Cancellation.Token)) + + try + { + try + { + # get file watcher + $fileWatcher = $PodeContext.Fim.Items[$evt.FileWatcher.Name] + if ($null -eq $fileWatcher) { + continue + } + + # if there are exclusions, and one matches, return + if (($null -ne $fileWatcher.Exclude) -and ($evt.Name -imatch $fileWatcher.Exclude)) { + continue + } + + # if there are inclusions, and none match, return + if (($null -ne $fileWatcher.Include) -and ($evt.Name -inotmatch $fileWatcher.Include)) { + continue + } + + # set file event object + $FileEvent = @{ + Type = $evt.ChangeType + FullPath = $evt.FullPath + Name = $evt.Name + Old = @{ + FullPath = $evt.OldFullPath + Name = $evt.OldName + } + Parameters = @{} + Lockable = $PodeContext.Lockables.Global + Timestamp = [datetime]::UtcNow + } + + # do we have any parameters? + if ($fileWatcher.Placeholders.Exist -and ($FileEvent.FullPath -imatch $fileWatcher.Placeholders.Path)) { + $FileEvent.Parameters = $Matches + } + + # invoke main script + $_args = @(Get-PodeScriptblockArguments -ArgumentList $fileWatcher.Arguments -UsingVariables $fileWatcher.UsingVariables) + Invoke-PodeScriptBlock -ScriptBlock $fileWatcher.Script -Arguments $_args -Scoped -Splat + } + catch [System.OperationCanceledException] {} + catch { + $_ | Write-PodeErrorLog + $_.Exception | Write-PodeErrorLog -CheckInnerException + } + } + finally { + $FileEvent = $null + Close-PodeDisposable -Disposable $evt + } + } + } + catch [System.OperationCanceledException] {} + catch { + $_ | Write-PodeErrorLog + $_.Exception | Write-PodeErrorLog -CheckInnerException + throw $_.Exception + } + } + + 1..$PodeContext.Threads.Files | ForEach-Object { + Add-PodeRunspace -Type Files -ScriptBlock $watchScript -Parameters @{ 'Watcher' = $watcher; 'ThreadId' = $_ } + } + + # script to keep file watcher server alive until cancelled + $waitScript = { + param( + [Parameter(Mandatory=$true)] + $Watcher + ) + + try { + while ($Watcher.IsConnected -and !$PodeContext.Tokens.Cancellation.IsCancellationRequested) { + Start-Sleep -Seconds 1 + } + } + catch [System.OperationCanceledException] {} + catch { + $_ | Write-PodeErrorLog + $_.Exception | Write-PodeErrorLog -CheckInnerException + throw $_.Exception + } + finally { + Close-PodeDisposable -Disposable $Watcher + } + } + + Add-PodeRunspace -Type Files -ScriptBlock $waitScript -Parameters @{ 'Watcher' = $watcher } -NoProfile + + + + + # $script = { + # # wrapper action handler + # $action = Get-PodeFileWatcherAction + + # # error action hanlder + # $errAction = Get-PodeFileWatcherErrorAction + + # try { + # # register file watchers for each one setup + # foreach ($item in $PodeContext.Fim.Items.Values) { + # for ($i = 0; $i -lt $item.Watchers.Length; $i++) { + # # create .net file watcher for path + # $item.Watchers[$i].Watcher = New-Object Pode.FileWatcher.RecoveringFileSystemWatcher $item.Watchers[$i].Path -Property @{ + # IncludeSubdirectories = $item.IncludeSubdirectories + # InternalBufferSize = $item.InternalBufferSize + # NotifyFilter = $item.NotifyFilters + # EnableRaisingEvents = $true + # } + + # # setup message data ith script/args + # $msgData = @{ + # ScriptBlock = $item.Script + # ArgumentList = @(Get-PodeScriptblockArguments -ArgumentList $item.Arguments -UsingVariables $item.UsingVariables) + # Exclude = $item.Exclude + # Include = $item.Include + # Placeholders = $item.Placeholders + # } + + # # register defined events + # foreach ($evt in $item.Events) { + # Register-PodeFileWatcherEvent ` + # -Name $item.Name ` + # -Index $i ` + # -EventName $evt ` + # -Watcher $item.Watchers[$i].Watcher ` + # -ScriptBlock $action ` + # -MessageData $msgData + # } + + # # register "Error" event type - log the exception + # Register-PodeFileWatcherEvent ` + # -Name $item.Name ` + # -Index $i ` + # -EventName 'Error' ` + # -Watcher $item.Watchers[$i].Watcher ` + # -ScriptBlock $errAction + # } + # } + + # # keep the runspace alive + # while (!$PodeContext.Tokens.Cancellation.IsCancellationRequested) { + # $PodeContext.Fim.Items.Values[0].Watchers[0].Watcher | Out-Default + # $PodeContext.Tokens.Cancellation.Token | Out-Default + # $null = (Wait-PodeTask -Task $PodeContext.Fim.Items.Values[0].Watchers[0].Watcher.GetContextAsync($PodeContext.Tokens.Cancellation.Token)) + # # $processing = $false + + # # foreach ($item in $PodeContext.Fim.Items.Values) { + # # foreach ($watcher in $item.Watchers) { + # # if ($watcher.Watcher.Count -gt 0) { + # # $processing = $true + # # break + # # } + # # } + # # } + + # # if ($processing) { + # # Start-Sleep -Milliseconds 10 + # # } + # # else { + # # Start-Sleep -Seconds 1 + # # } + # } + # } + # catch [System.OperationCanceledException] {} + # catch { + # $_ | Write-PodeErrorLog + # $_.Exception | Write-PodeErrorLog -CheckInnerException + # throw + # } + # finally { + # # dispose/unregister all watchers + # foreach ($item in $PodeContext.Fim.Items.Values) { + # for ($i = 0; $i -lt $item.Watchers.Length; $i++) { + # # unregister events + # foreach ($evt in $item.Events) { + # Unregister-PodeFileWatcherEvent -Name $item.Name -Index $i -EventName $evt + # } + + # # unregister error event + # Unregister-PodeFileWatcherEvent -Name $item.Name -Index $i -EventName 'Error' + + # # dispose watcher + # if ($null -ne $item.Watchers[$i].Watcher) { + # $item.Watchers[$i].Watcher.Dispose() + # } + # } + # } + # } + # } + + # Add-PodeRunspace -Type Files -ScriptBlock $script -NoProfile +} + +function Get-PodeFileWatcherIdenifierName +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter(Mandatory=$true)] + [int] + $Index, + + [Parameter(Mandatory=$true)] + [string] + $EventName + ) + + return "Pode.Fim.$($Name -replace '\s+', '_').$($Index).$($EventName)" +} + +function Test-PodeFileWatcherEventRegistered +{ + param( + [Parameter(Mandatory=$true)] + [string] + $SourceIdentifier + ) + + return (($null -ne (Get-Event -SourceIdentifier $SourceIdentifier -ErrorAction Ignore))) +} + +function Get-PodeFileWatcherAction +{ + return { + try { + # if there are exclusions, and one matches, return + if (($null -ne $Event.MessageData.Exclude) -and ($Event.SourceEventArgs.Name -imatch $Event.MessageData.Exclude)) { + return + } + + # if there are inclusions, and none match, return + if (($null -ne $Event.MessageData.Include) -and ($Event.SourceEventArgs.Name -inotmatch $Event.MessageData.Include)) { + return + } + + # set file event object + $global:FileEvent = @{ + Type = $Event.SourceEventArgs.ChangeType + FullPath = $Event.SourceEventArgs.FullPath + Name = $Event.SourceEventArgs.Name + Old = @{ + FullPath = $Event.SourceEventArgs.OldFullPath + Name = $Event.SourceEventArgs.OldName + } + Parameters = @{} + Lockable = $PodeContext.Lockables.Global + Timestamp = [datetime]::UtcNow + } + + # do we have any parameters? + if ($Event.MessageData.Placeholders.Exist -and ($FileEvent.FullPath -imatch $Event.MessageData.Placeholders.Path)) { + $FileEvent.Parameters = $Matches + } + + # invoke main script + Invoke-PodeScriptBlock ` + -ScriptBlock $Event.MessageData.ScriptBlock ` + -Arguments $Event.MessageData.ArgumentList ` + -Scoped ` + -Splat + } + catch [System.OperationCanceledException] {} + catch { + $_ | Write-PodeErrorLog + } + } +} + +function Get-PodeFileWatcherErrorAction +{ + return { + $Event.SourceEventArgs.GetException() | Write-PodeErrorLog + } +} + +function Register-PodeFileWatcherEvent +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter(Mandatory=$true)] + [int] + $Index, + + [Parameter(Mandatory=$true)] + [string] + $EventName, + + [Parameter(Mandatory=$true)] + [Pode.FileWatcher.RecoveringFileSystemWatcher] + $Watcher, + + [Parameter(Mandatory=$true)] + [scriptblock] + $ScriptBlock, + + [Parameter()] + [hashtable] + $MessageData = @{} + ) + + $id = Get-PodeFileWatcherIdenifierName -Name $Name -Index $Index -EventName $EventName + + if (Test-PodeFileWatcherEventRegistered -SourceIdentifier $id) { + throw "An event handler has already been registered with the identifier '$($id)'" + } + + if ($null -eq $MessageData) { + $MessageData = @{} + } + + Register-ObjectEvent ` + -InputObject $Watcher ` + -EventName $EventName ` + -SourceIdentifier $id ` + -Action $ScriptBlock ` + -MessageData $MessageData ` + -SupportEvent ` + -ErrorAction Stop +} + +function Unregister-PodeFileWatcherEvent +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter(Mandatory=$true)] + [int] + $Index, + + [Parameter(Mandatory=$true)] + [string] + $EventName + ) + + $id = Get-PodeFileWatcherIdenifierName -Name $Name -Index $Index -EventName $EventName + + if (Test-PodeFileWatcherEventRegistered -SourceIdentifier $id) { + Unregister-Event -SourceIdentifier $id -Force -ErrorAction SilentlyContinue + } +} \ No newline at end of file diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index bdfbc66dc..cb62fe655 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -507,7 +507,7 @@ function Add-PodeRunspace { param ( [Parameter(Mandatory=$true)] - [ValidateSet('Main', 'Signals', 'Schedules', 'Gui', 'Web', 'Smtp', 'Tcp', 'Tasks', 'WebSockets')] + [ValidateSet('Main', 'Signals', 'Schedules', 'Gui', 'Web', 'Smtp', 'Tcp', 'Tasks', 'WebSockets', 'Files')] [string] $Type, @@ -651,6 +651,13 @@ function Close-PodeRunspaces } } + foreach ($watcher in $PodeContext.Watchers) { + if (!$watcher.IsDisposed) { + $continue = $true + break + } + } + if ($continue) { continue } @@ -1828,7 +1835,7 @@ function Convert-PodePathPatternToRegex function Convert-PodePathPatternsToRegex { - param ( + param( [Parameter()] [string[]] $Paths, @@ -3067,4 +3074,71 @@ function Test-PodeModuleInstalled ) return ($null -ne (Get-Module -Name $Name -ListAvailable -ErrorAction Ignore -Verbose:$false)) +} + +function Get-PodePlaceholderRegex +{ + return '\:(?[\w]+)' +} + +function Resolve-PodePlaceholders +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Path, + + [Parameter()] + [string] + $Pattern, + + [Parameter()] + [string] + $Prepend = '(?<', + + [Parameter()] + [string] + $Append = '>[^\/]+?)', + + [switch] + $Slashes + ) + + if ([string]::IsNullOrWhiteSpace($Pattern)) { + $Pattern = Get-PodePlaceholderRegex + } + + if ($Path -imatch $Pattern) { + $Path = [regex]::Escape($Path) + } + + if ($Slashes) { + $Path = ($Path.TrimEnd('\/') -replace '(\\\\|\/)', '[\\\/]') + $Path = "$($Path)[\\\/]" + } + + while ($Path -imatch $Pattern) { + $Path = ($Path -ireplace $Matches[0], "$($Prepend)$($Matches['tag'])$($Append)") + } + + return $Path +} + +function Test-PodePlaceholders +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Path, + + [Parameter()] + [string] + $Placeholder + ) + + if ([string]::IsNullOrWhiteSpace($Placeholder)) { + $Placeholder = Get-PodePlaceholderRegex + } + + return ($Path -imatch $Placeholder) } \ No newline at end of file diff --git a/src/Private/Routes.ps1 b/src/Private/Routes.ps1 index 01d6c337a..1b9229ead 100644 --- a/src/Private/Routes.ps1 +++ b/src/Private/Routes.ps1 @@ -1,6 +1,6 @@ function Test-PodeRouteFromRequest { - param ( + param( [Parameter(Mandatory=$true)] [ValidateSet('DELETE', 'GET', 'HEAD', 'MERGE', 'OPTIONS', 'PATCH', 'POST', 'PUT', 'TRACE', 'STATIC', 'SIGNAL', '*')] [string] @@ -280,61 +280,33 @@ function Get-PodeRoutesByUrl return $null } -function Update-PodeRoutePlaceholders -{ - param( - [Parameter(Mandatory=$true)] - [string] - $Path - ) - - # replace placeholder parameters with regex - $placeholder = '\:(?[\w]+)' - if ($Path -imatch $placeholder) { - $Path = [regex]::Escape($Path) - } - - while ($Path -imatch $placeholder) { - $Path = ($Path -ireplace $Matches[0], "(?<$($Matches['tag'])>[^\/]+?)") - } - - return $Path -} - function ConvertTo-PodeOpenApiRoutePath { - param ( + param( [Parameter(Mandatory=$true)] [string] $Path ) - # replace placeholder parameters with regex - $placeholder = '\:(?[\w]+)' - if ($Path -imatch $placeholder) { - $Path = [regex]::Escape($Path) - } - - while ($Path -imatch $placeholder) { - $Path = ($Path -ireplace $Matches[0], "{$($Matches['tag'])}") - } - - return $Path + return (Resolve-PodePlaceholders -Path $Path -Pattern '\:(?[\w]+)' -Prepend '{' -Append '}') } function Update-PodeRouteSlashes { - param ( + param( [Parameter(Mandatory=$true)] [string] $Path, [switch] - $Static + $Static, + + [switch] + $NoLeadingSlash ) # ensure route starts with a '/' - if (!$Path.StartsWith('/')) { + if (!$NoLeadingSlash -and !$Path.StartsWith('/')) { $Path = "/$($Path)" } @@ -351,7 +323,7 @@ function Update-PodeRouteSlashes function Split-PodeRouteQuery { - param ( + param( [Parameter(Mandatory=$true)] [string] $Path @@ -362,7 +334,7 @@ function Split-PodeRouteQuery function ConvertTo-PodeRouteRegex { - param ( + param( [Parameter()] [string] $Path @@ -376,7 +348,7 @@ function ConvertTo-PodeRouteRegex $Path = Split-PodeRouteQuery -Path $Path $Path = Protect-PodeValue -Value $Path -Default '/' $Path = Update-PodeRouteSlashes -Path $Path - $Path = Update-PodeRoutePlaceholders -Path $Path + $Path = Resolve-PodePlaceholders -Path $Path return $Path } @@ -457,7 +429,7 @@ function Test-PodeRouteInternal function Convert-PodeFunctionVerbToHttpMethod { - param ( + param( [Parameter()] [string] $Verb diff --git a/src/Private/Server.ps1 b/src/Private/Server.ps1 index bae333e93..8a4ccf3f1 100644 --- a/src/Private/Server.ps1 +++ b/src/Private/Server.ps1 @@ -66,6 +66,9 @@ function Start-PodeInternalServer # start runspace for websockets Start-PodeWebSocketRunspace + + # start runspace for file watchers + Start-PodeFileWatcherRunspace } # start the appropriate server @@ -185,6 +188,9 @@ function Restart-PodeInternalServer $PodeContext.Tasks.Items.Clear() $PodeContext.Tasks.Results.Clear() + # clear file watchers + $PodeContext.Fim.Items.Clear() + # auto-importers Reset-PodeAutoImportConfiguration @@ -216,6 +222,7 @@ function Restart-PodeInternalServer $PodeContext.Server.Signals.Listener = $null $PodeContext.Listeners = @() $PodeContext.Receivers = @() + $PodeContext.Watchers = @{} # set view engine back to default $PodeContext.Server.ViewEngine = @{ diff --git a/src/Public/Core.ps1 b/src/Public/Core.ps1 index 39883f4d2..88849bee9 100644 --- a/src/Public/Core.ps1 +++ b/src/Public/Core.ps1 @@ -108,7 +108,7 @@ function Start-PodeServer $ListenerType = [string]::Empty, [Parameter()] - [ValidateSet('Timers', 'Schedules', 'Tasks', 'WebSockets')] + [ValidateSet('Timers', 'Schedules', 'Tasks', 'WebSockets', 'Files')] [string[]] $EnablePool, diff --git a/src/Public/FIM.ps1 b/src/Public/FIM.ps1 new file mode 100644 index 000000000..1b6b31782 --- /dev/null +++ b/src/Public/FIM.ps1 @@ -0,0 +1,206 @@ +function Add-PodeFileWatcher +{ + [CmdletBinding(DefaultParameterSetName='Script')] + param( + [Parameter()] + [string] + $Name = $null, + + [Parameter()] + [ValidateSet('Changed', 'Created', 'Deleted', 'Renamed', 'Existed', '*')] + [string[]] + $EventName = '*', + + [Parameter(Mandatory=$true)] + [string] + $Path, + + [Parameter(Mandatory=$true, ParameterSetName='Script')] + [scriptblock] + $ScriptBlock, + + [Parameter(Mandatory=$true, ParameterSetName='File')] + [string] + $FilePath, + + [Parameter()] + [object[]] + $ArgumentList, + + [Parameter()] + [System.IO.NotifyFilters[]] + $NotifyFilter = @('FileName', 'DirectoryName', 'LastWrite', 'CreationTime'), + + [Parameter()] + [string[]] + $Exclude, + + [Parameter()] + [ValidateNotNullOrEmpty()] + [string[]] + $Include = '*.*', + + [Parameter()] + [ValidateRange(4kb, 64kb)] + [int] + $InternalBufferSize = 8kb, + + [switch] + $NoSubdirectories, + + [switch] + $PassThru + ) + + # set random name + if ([string]::IsNullOrEmpty($Name)) { + $Name = New-PodeGuid -Secure + } + + # set all for * event + if ('*' -iin $EventName) { + $EventName = @('Changed', 'Created', 'Deleted', 'Renamed') + } + + # resolve path if relative + $Path = Get-PodeRelativePath -Path $Path -JoinRoot -Resolve + + # resolve path, and test it + $hasPlaceholders = Test-PodePlaceholders -Path $Path + if ($hasPlaceholders) { + $rgxPath = Update-PodeRouteSlashes -Path $Path -NoLeadingSlash + $rgxPath = Resolve-PodePlaceholders -Path $rgxPath -Slashes + $Path = $Path -ireplace (Get-PodePlaceholderRegex), '*' + } + + # test path to make sure it exists + if (!(Test-PodePath $Path -NoStatus)) { + throw "The path does not exist: $($Path)" + } + + # test if we have the file watcher already + if (Test-PodeFileWatcher -Name $Name) { + throw "A File Watcher with the name '$($Name)' has already been defined" + } + + # if we have a file path supplied, load that path as a scriptblock + if ($PSCmdlet.ParameterSetName -ieq 'file') { + $ScriptBlock = Convert-PodeFileToScriptBlock -FilePath $FilePath + } + + # check for scoped vars + $ScriptBlock, $usingVars = Convert-PodeScopedVariables -ScriptBlock $ScriptBlock -PSSession $PSCmdlet.SessionState + + # enable the file watcher threads + $PodeContext.Fim.Enabled = $true + + # resolve the path's widacards if any + $paths = @($Path) + if ($Path.Contains('*')) { + $paths = @(Get-ChildItem -Path $Path -Directory -Force | Select-Object -ExpandProperty FullName) + } + + $watchers = @(foreach ($p in $paths) { + @{ + Path = $p + Watcher = $null + } + }) + + # add the file watcher + $PodeContext.Fim.Items[$Name] = @{ + Name = $Name + Events = @($EventName) + Path = $Path + Placeholders = @{ + Path = $rgxPath + Exist = $hasPlaceholders + } + Script = $ScriptBlock + UsingVariables = $usingVars + Arguments = $ArgumentList + NotifyFilters = @($NotifyFilter) + IncludeSubdirectories = !$NoSubdirectories.IsPresent + InternalBufferSize = $InternalBufferSize + Exclude = (Convert-PodePathPatternsToRegex -Paths @($Exclude)) + Include = (Convert-PodePathPatternsToRegex -Paths @($Include)) + Watchers = $watchers + } + + # return? + if ($PassThru) { + return $PodeContext.Fim.Items[$Name] + } +} + +function Test-PodeFileWatcher +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return (($null -ne $PodeContext.Fim.Items) -and $PodeContext.Fim.Items.ContainsKey($Name)) +} + +function Get-PodeFileWatcher +{ + [CmdletBinding()] + param( + [Parameter()] + [string[]] + $Name + ) + + $watchers = $PodeContext.Fim.Items.Values + + # further filter by file watcher names + if (($null -ne $Name) -and ($Name.Length -gt 0)) { + $watchers = @(foreach ($_name in $Name) { + foreach ($watcher in $watchers) { + if ($watcher.Name -ine $_name) { + continue + } + + $watcher + } + }) + } + + # return + return $watchers +} + +function Remove-PodeFileWatcher +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + $null = $PodeContext.Fim.Items.Remove($Name) +} + +function Clear-PodeFileWatchers +{ + [CmdletBinding()] + param() + + $PodeContext.Fim.Items.Clear() +} + +function Use-PodeFileWatchers +{ + [CmdletBinding()] + param( + [Parameter()] + [string] + $Path + ) + + Use-PodeFolder -Path $Path -DefaultPath 'filewatchers' +} diff --git a/src/Public/Routes.ps1 b/src/Public/Routes.ps1 index 4a4b0bc73..1ec651f49 100644 --- a/src/Public/Routes.ps1 +++ b/src/Public/Routes.ps1 @@ -186,7 +186,7 @@ function Add-PodeRoute # ensure the route has appropriate slashes $Path = Update-PodeRouteSlashes -Path $Path $OpenApiPath = ConvertTo-PodeOpenApiRoutePath -Path $Path - $Path = Update-PodeRoutePlaceholders -Path $Path + $Path = Resolve-PodePlaceholders -Path $Path # get endpoints from name if (!$PodeContext.Server.FindEndpoints.Route) { @@ -483,7 +483,7 @@ function Add-PodeStaticRoute # ensure the route has appropriate slashes $Path = Update-PodeRouteSlashes -Path $Path -Static $OpenApiPath = ConvertTo-PodeOpenApiRoutePath -Path $Path - $Path = Update-PodeRoutePlaceholders -Path $Path + $Path = Resolve-PodePlaceholders -Path $Path # get endpoints from name if (!$PodeContext.Server.FindEndpoints.Route) { @@ -1226,7 +1226,7 @@ function Remove-PodeRoute # ensure the route has appropriate slashes and replace parameters $Path = Update-PodeRouteSlashes -Path $Path - $Path = Update-PodeRoutePlaceholders -Path $Path + $Path = Resolve-PodePlaceholders -Path $Path # ensure route does exist if (!$PodeContext.Server.Routes[$Method].Contains($Path)) { @@ -1838,7 +1838,7 @@ function Get-PodeRoute if (![string]::IsNullOrWhiteSpace($Path)) { $Path = Split-PodeRouteQuery -Path $Path $Path = Update-PodeRouteSlashes -Path $Path - $Path = Update-PodeRoutePlaceholders -Path $Path + $Path = Resolve-PodePlaceholders -Path $Path $routes = @(foreach ($route in $routes) { if ($route.Path -ine $Path) { @@ -2119,7 +2119,7 @@ function Test-PodeRoute # ensure the route has appropriate slashes $Path = Update-PodeRouteSlashes -Path $Path - $Path = Update-PodeRoutePlaceholders -Path $Path + $Path = Resolve-PodePlaceholders -Path $Path # get endpoint from name $endpoint = @(Find-PodeEndpoints -EndpointName $EndpointName)[0] @@ -2173,7 +2173,7 @@ function Test-PodeStaticRoute # ensure the route has appropriate slashes $Path = Update-PodeRouteSlashes -Path $Path -Static - $Path = Update-PodeRoutePlaceholders -Path $Path + $Path = Resolve-PodePlaceholders -Path $Path # get endpoint from name $endpoint = @(Find-PodeEndpoints -EndpointName $EndpointName)[0] diff --git a/src/Public/Verbs.ps1 b/src/Public/Verbs.ps1 index f3911f0f0..750740f8a 100644 --- a/src/Public/Verbs.ps1 +++ b/src/Public/Verbs.ps1 @@ -71,7 +71,7 @@ function Add-PodeVerb ) # find placeholder parameters in verb (ie: COMMAND :parameter) - $Verb = Update-PodeRoutePlaceholders -Path $Verb + $Verb = Resolve-PodePlaceholders -Path $Verb # get endpoints from name if (!$PodeContext.Server.FindEndpoints.Tcp) { @@ -152,7 +152,7 @@ function Remove-PodeVerb ) # ensure the verb placeholders are replaced - $Verb = Update-PodeRoutePlaceholders -Path $Verb + $Verb = Resolve-PodePlaceholders -Path $Verb # ensure verb does exist if (!$PodeContext.Server.Verbs.Contains($Verb)) { @@ -225,7 +225,7 @@ function Get-PodeVerb # if we have a verb, filter if (![string]::IsNullOrWhiteSpace($Verb)) { - $Verb = Update-PodeRoutePlaceholders -Path $Verb + $Verb = Resolve-PodePlaceholders -Path $Verb $verbs = $PodeContext.Server.Verbs[$Verb] } else { diff --git a/tests/unit/Routes.Tests.ps1 b/tests/unit/Routes.Tests.ps1 index 3c8e39b0c..5ae80a3d2 100644 --- a/tests/unit/Routes.Tests.ps1 +++ b/tests/unit/Routes.Tests.ps1 @@ -710,35 +710,35 @@ Describe 'Update-PodeRouteSlashes' { } } -Describe 'Update-PodeRoutePlaceholders' { +Describe 'Resolve-PodePlaceholders' { It 'Update route placeholders, basic' { $input = 'route' - Update-PodeRoutePlaceholders -Path $input | Should Be 'route' + Resolve-PodePlaceholders -Path $input | Should Be 'route' } It 'Update route placeholders' { $input = ':route' - Update-PodeRoutePlaceholders -Path $input | Should Be '(?[^\/]+?)' + Resolve-PodePlaceholders -Path $input | Should Be '(?[^\/]+?)' } It 'Update route placeholders, double with no spacing' { $input = ':route:placeholder' - Update-PodeRoutePlaceholders -Path $input | Should Be '(?[^\/]+?)(?[^\/]+?)' + Resolve-PodePlaceholders -Path $input | Should Be '(?[^\/]+?)(?[^\/]+?)' } It 'Update route placeholders, double with double ::' { $input = '::route:placeholder' - Update-PodeRoutePlaceholders -Path $input | Should Be ':(?[^\/]+?)(?[^\/]+?)' + Resolve-PodePlaceholders -Path $input | Should Be ':(?[^\/]+?)(?[^\/]+?)' } It 'Update route placeholders, double with slash' { $input = ':route/:placeholder' - Update-PodeRoutePlaceholders -Path $input | Should Be '(?[^\/]+?)/(?[^\/]+?)' + Resolve-PodePlaceholders -Path $input | Should Be '(?[^\/]+?)/(?[^\/]+?)' } It 'Update route placeholders, no update' { $input = ': route' - Update-PodeRoutePlaceholders -Path $input | Should Be ': route' + Resolve-PodePlaceholders -Path $input | Should Be ': route' } } From e996da3fa5268c969a211e203efa8a864682241b Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Thu, 12 Jan 2023 22:44:30 +0000 Subject: [PATCH 33/52] #1063: fix paths when on windows powershell, fix tests, and improve support for running watchers on their own - without a web server --- examples/file-watchers.ps1 | 30 ++--- src/Pode.psd1 | 18 ++- src/Private/{FIM.ps1 => FileWatchers.ps1} | 0 src/Private/Helpers.ps1 | 27 +++++ src/Private/Server.ps1 | 18 ++- src/Public/Core.ps1 | 2 +- src/Public/{FIM.ps1 => FileWatchers.ps1} | 131 ++++++++++++++++++++++ tests/unit/Server.Tests.ps1 | 10 +- 8 files changed, 209 insertions(+), 27 deletions(-) rename src/Private/{FIM.ps1 => FileWatchers.ps1} (100%) rename src/Public/{FIM.ps1 => FileWatchers.ps1} (59%) diff --git a/examples/file-watchers.ps1 b/examples/file-watchers.ps1 index b1587b6ee..06d655f79 100644 --- a/examples/file-watchers.ps1 +++ b/examples/file-watchers.ps1 @@ -8,7 +8,7 @@ Import-Module "$($path)/src/Pode.psm1" -Force -ErrorAction Stop Start-PodeServer -Verbose { # add two endpoints - Add-PodeEndpoint -Address * -Port 9000 -Protocol Http + # Add-PodeEndpoint -Address * -Port 9000 -Protocol Http # enable logging New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging @@ -18,23 +18,17 @@ Start-PodeServer -Verbose { # "[$($FileEvent.Type)]: $($FileEvent.FullPath)" | Out-Default # } - # Add-PodeFileWatcher -Path 'C:/Projects/:project/src' -Include '*.ps1' -ScriptBlock { - # "[$($FileEvent.Type)][$($FileEvent.Parameters['project'])]: $($FileEvent.FullPath)" | Out-Default - # } - # Add-PodeFileWatcher -Path 'C:/Projects/Pode/:folder' -Include '*.psd1', '*.txt' -ScriptBlock { - # "[$($FileEvent.Type)][$($FileEvent.Parameters['folder'])]: $($FileEvent.FullPath)" | Out-Default - # } - # Add-PodeFileWatcher -Path '/mnt/c/Projects/:project/src' -Include '*.ps1' -ScriptBlock { - # "[$($FileEvent.Type)][$($FileEvent.Parameters['project'])]: $($FileEvent.FullPath)" | Out-Default - # } - - Add-PodeTimer -Name 'Test' -Interval 10 -ScriptBlock { - $root = Get-PodeServerPath - $file = Join-Path $root 'myfile.txt' - 'hi!' | Out-File -FilePath $file -Append -Force + Add-PodeFileWatcher -Path 'C:/Projects/:project/src' -Include '*.ps1' -ScriptBlock { + "[$($FileEvent.Type)][$($FileEvent.Parameters['project'])]: $($FileEvent.FullPath)" | Out-Default } - Add-PodeFileWatcher -Path '.' -Include '*.txt' -ScriptBlock { - "[$($FileEvent.Type)]: $($FileEvent.FullPath)" | Out-Default - } + # Add-PodeTimer -Name 'Test' -Interval 10 -ScriptBlock { + # $root = Get-PodeServerPath + # $file = Join-Path $root 'myfile.txt' + # 'hi!' | Out-File -FilePath $file -Append -Force + # } + + # Add-PodeFileWatcher -Path '.' -Include '*.txt' -ScriptBlock { + # "[$($FileEvent.Type)]: $($FileEvent.FullPath)" | Out-Default + # } } \ No newline at end of file diff --git a/src/Pode.psd1 b/src/Pode.psd1 index 72ad4a394..12dbf3b30 100644 --- a/src/Pode.psd1 +++ b/src/Pode.psd1 @@ -347,7 +347,15 @@ 'Update-PodeSecret', 'Remove-PodeSecret', 'Read-PodeSecret', - 'Set-PodeSecret' + 'Set-PodeSecret', + + # File Watchers + 'Add-PodeFileWatcher', + 'Test-PodeFileWatcher', + 'Get-PodeFileWatcher', + 'Remove-PodeFileWatcher', + 'Clear-PodeFileWatchers', + 'Use--PodeFileWatchers' ) # Private data to pass to the module specified in RootModule/ModuleToProcess. This may also contain a PSData hashtable with additional module metadata used by PowerShell. @@ -356,10 +364,10 @@ # Tags applied to this module. These help with module discovery in online galleries. Tags = @('powershell', 'web', 'server', 'http', 'listener', 'rest', 'api', 'tcp', 'smtp', 'websites', - 'powershell-core', 'windows', 'unix', 'linux', 'pode', 'PSEdition_Core', 'cross-platform', 'access-control', - 'file-monitoring', 'multithreaded', 'rate-limiting', 'cron', 'schedule', 'middleware', 'session', - 'authentication', 'active-directory', 'caching', 'csrf', 'arm', 'raspberry-pi', 'aws-lambda', - 'azure-functions', 'websockets', 'swagger', 'openapi', 'redoc', 'webserver', 'secrets', 'vault') + 'powershell-core', 'windows', 'unix', 'linux', 'pode', 'PSEdition_Core', 'cross-platform', + 'file-monitoring', 'multithreaded', 'schedule', 'middleware', 'session', + 'authentication', 'arm', 'raspberry-pi', 'aws-lambda', + 'azure-functions', 'websockets', 'swagger', 'openapi', 'webserver', 'secrets', 'fim') # A URL to the license for this module. LicenseUri = 'https://raw.githubusercontent.com/Badgerati/Pode/master/LICENSE.txt' diff --git a/src/Private/FIM.ps1 b/src/Private/FileWatchers.ps1 similarity index 100% rename from src/Private/FIM.ps1 rename to src/Private/FileWatchers.ps1 diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index cb62fe655..6ce875d3b 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -3117,6 +3117,33 @@ function Resolve-PodePlaceholders $Path = "$($Path)[\\\/]" } + return (Convert-PodePlaceholders -Path $Path -Pattern $Pattern -Prepend $Prepend -Append $Append) +} + +function Convert-PodePlaceholders +{ + param( + [Parameter(Mandatory=$true)] + [string] + $Path, + + [Parameter()] + [string] + $Pattern, + + [Parameter()] + [string] + $Prepend = '(?<', + + [Parameter()] + [string] + $Append = '>[^\/]+?)' + ) + + if ([string]::IsNullOrWhiteSpace($Pattern)) { + $Pattern = Get-PodePlaceholderRegex + } + while ($Path -imatch $Pattern) { $Path = ($Path -ireplace $Matches[0], "$($Prepend)$($Matches['tag'])$($Append)") } diff --git a/src/Private/Server.ps1 b/src/Private/Server.ps1 index 8a4ccf3f1..218762bf3 100644 --- a/src/Private/Server.ps1 +++ b/src/Private/Server.ps1 @@ -50,7 +50,7 @@ function Start-PodeInternalServer New-PodeRunspacePools Open-PodeRunspacePools - if (!$PodeContext.Server.IsServerless -and ($PodeContext.Server.Types.Length -gt 0)) + if (!$PodeContext.Server.IsServerless) { # start runspace for loggers Start-PodeLoggingRunspace @@ -273,4 +273,20 @@ function Restart-PodeInternalServer $_ | Write-PodeErrorLog throw $_.Exception } +} + +function Test-PodeServerKeepOpen +{ + # if we have any timers/schedules/fim - keep open + if ((Test-PodeTimersExist) -or (Test-PodeSchedulesExist) -or (Test-PodeFileWatchersExist)) { + return $true + } + + # if not a service, and not any type/serverless - close server + if (!$PodeContext.Server.IsService -and (($PodeContext.Server.Types.Length -eq 0) -or $PodeContext.Server.IsServerless)) { + return $false + } + + # keep server open + return $true } \ No newline at end of file diff --git a/src/Public/Core.ps1 b/src/Public/Core.ps1 index 88849bee9..cf8871625 100644 --- a/src/Public/Core.ps1 +++ b/src/Public/Core.ps1 @@ -180,7 +180,7 @@ function Start-PodeServer Start-PodeInternalServer -Request $Request -Browse:$Browse # at this point, if it's just a one-one off script, return - if (!$PodeContext.Server.IsService -and (($PodeContext.Server.Types.Length -eq 0) -or $PodeContext.Server.IsServerless)) { + if (!(Test-PodeServerKeepOpen)) { return } diff --git a/src/Public/FIM.ps1 b/src/Public/FileWatchers.ps1 similarity index 59% rename from src/Public/FIM.ps1 rename to src/Public/FileWatchers.ps1 index 1b6b31782..012b197e8 100644 --- a/src/Public/FIM.ps1 +++ b/src/Public/FileWatchers.ps1 @@ -1,3 +1,58 @@ +<# +.SYNOPSIS +Adds a new File Watcher to monitor file changes in a directory. + +.DESCRIPTION +Adds a new File Watcher to monitor file changes in a directory. + +.PARAMETER Name +An optional Name for the File Watcher. (Default: GUID) + +.PARAMETER EventName +An optional EventName to be monitored. Note: '*' refers to Created, Deleted, Changed, and Renamed. (Default: *) + +.PARAMETER Path +The Path to a directory which contains the files to be monitored. + +.PARAMETER ScriptBlock +The ScriptBlock defining logic to be run when events are triggered. + +.PARAMETER FilePath +A literal, or relative, path to a file containing a ScriptBlock for the File Watcher's logic. + +.PARAMETER ArgumentList +A hashtable of arguments to supply to the File Watcher's ScriptBlock. + +.PARAMETER NotifyFilter +The attributes on files to monitor and notify about. (Default: FileName, DirectoryName, LastWrite, CreationTime) + +.PARAMETER Exclude +An optional array of file patterns to be excluded. + +.PARAMETER Include +An optional array of file patterns to be included. (Default: *.*) + +.PARAMETER InternalBufferSize +The InternalBufferSize of the file monitor, used when temporarily storing events. (Default: 8kb) + +.PARAMETER NoSubdirectories +If supplied, the File Watcher will only monitor files in the specified directory path, and not in all sub-directories as well. + +.PARAMETER PassThru +If supplied, the File Watcher object registered will be returned. + +.EXAMPLE +Add-PodeFileWatcher -Path 'C:/Projects/:project/src' -Include '*.ps1' -ScriptBlock {} + +.EXAMPLE +Add-PodeFileWatcher -Path 'C:/Websites/:site' -Include '*.config' -EventName Changed -ScriptBlock {} + +.EXAMPLE +Add-PodeFileWatcher -Path '/temp/logs' -EventName Created -NotifyFilter CreationTime -ScriptBlock {} + +.EXAMPLE +$watcher = Add-PodeFileWatcher -Path '/temp/logs' -Exclude *.txt -ScriptBlock {} -PassThru +#> function Add-PodeFileWatcher { [CmdletBinding(DefaultParameterSetName='Script')] @@ -63,8 +118,16 @@ function Add-PodeFileWatcher } # resolve path if relative + if (!(Test-PodeIsPSCore)) { + $Path = Convert-PodePlaceholders -Path $Path -Prepend '%' -Append '%' + } + $Path = Get-PodeRelativePath -Path $Path -JoinRoot -Resolve + if (!(Test-PodeIsPSCore)) { + $Path = Convert-PodePlaceholders -Path $Path -Pattern '\%(?[\w]+)\%' -Prepend ':' -Append ([string]::Empty) + } + # resolve path, and test it $hasPlaceholders = Test-PodePlaceholders -Path $Path if ($hasPlaceholders) { @@ -133,6 +196,19 @@ function Add-PodeFileWatcher } } +<# +.SYNOPSIS +Tests whether the passed File Watcher exists. + +.DESCRIPTION +Tests whether the passed File Watcher exists by its name. + +.PARAMETER Name +The Name of the File Watcher. + +.EXAMPLE +if (Test-PodeFileWatcher -Name WatcherName) { } +#> function Test-PodeFileWatcher { [CmdletBinding()] @@ -145,6 +221,22 @@ function Test-PodeFileWatcher return (($null -ne $PodeContext.Fim.Items) -and $PodeContext.Fim.Items.ContainsKey($Name)) } +<# +.SYNOPSIS +Returns any defined File Watchers. + +.DESCRIPTION +Returns any defined File Watchers. + +.PARAMETER Name +An optional File Watcher Name(s) to be returned. + +.EXAMPLE +Get-PodeFileWatcher + +.EXAMPLE +Get-PodeFileWatcher -Name Name1, Name2 +#> function Get-PodeFileWatcher { [CmdletBinding()] @@ -173,6 +265,19 @@ function Get-PodeFileWatcher return $watchers } +<# +.SYNOPSIS +Removes a specific File Watchers. + +.DESCRIPTION +Removes a specific File Watchers. + +.PARAMETER Name +The Name of the File Watcher to be removed. + +.EXAMPLE +Remove-PodeFileWatcher -Name 'Logs' +#> function Remove-PodeFileWatcher { [CmdletBinding()] @@ -185,6 +290,16 @@ function Remove-PodeFileWatcher $null = $PodeContext.Fim.Items.Remove($Name) } +<# +.SYNOPSIS +Removes all File Watchers. + +.DESCRIPTION +Removes all File Watchers. + +.EXAMPLE +Clear-PodeFileWatchers +#> function Clear-PodeFileWatchers { [CmdletBinding()] @@ -193,6 +308,22 @@ function Clear-PodeFileWatchers $PodeContext.Fim.Items.Clear() } +<# +.SYNOPSIS +Automatically loads File Watchers ps1 files + +.DESCRIPTION +Automatically loads File Watchers ps1 files from either a /filewatcher folder, or a custom folder. Saves space dot-sourcing them all one-by-one. + +.PARAMETER Path +Optional Path to a folder containing ps1 files, can be relative or literal. + +.EXAMPLE +Use-PodeFileWatchers + +.EXAMPLE +Use-PodeFileWatchers -Path './my-watchers' +#> function Use-PodeFileWatchers { [CmdletBinding()] diff --git a/tests/unit/Server.Tests.ps1 b/tests/unit/Server.Tests.ps1 index 02f5f6969..2b5e1d8ca 100644 --- a/tests/unit/Server.Tests.ps1 +++ b/tests/unit/Server.Tests.ps1 @@ -36,8 +36,8 @@ Describe 'Start-PodeInternalServer' { Assert-MockCalled Invoke-PodeScriptBlock -Times 1 -Scope It Assert-MockCalled New-PodeRunspacePools -Times 1 -Scope It Assert-MockCalled New-PodeRunspaceState -Times 1 -Scope It - Assert-MockCalled Start-PodeTimerRunspace -Times 0 -Scope It - Assert-MockCalled Start-PodeScheduleRunspace -Times 0 -Scope It + Assert-MockCalled Start-PodeTimerRunspace -Times 1 -Scope It + Assert-MockCalled Start-PodeScheduleRunspace -Times 1 -Scope It Assert-MockCalled Start-PodeSmtpServer -Times 0 -Scope It Assert-MockCalled Start-PodeTcpServer -Times 0 -Scope It Assert-MockCalled Start-PodeWebServer -Times 0 -Scope It @@ -199,6 +199,12 @@ Describe 'Restart-PodeInternalServer' { } Results = @{} } + Fim = @{ + Enabled = $true + Items = @{ + key = 'value' + } + } } Restart-PodeInternalServer | Out-Null From b89a430895023a561ed1c504eea6e9d384f5228a Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sun, 15 Jan 2023 13:11:48 +0000 Subject: [PATCH 34/52] #1063: add docs for file watchers, and small tweaks for include/exclude --- LICENSE.txt | 2 +- docs/Tutorials/FileWatchers.md | 252 ++++++++++++++++++ mkdocs.yml | 1 + packers/choco/pode.nuspec | 2 +- .../BufferingFileSystemWatcher.cs | 2 +- .../EventQueueOverflowException.cs | 2 +- .../FileWatcherErrorEventArgs.cs | 2 +- .../RecoveringFileSystemWatcher.cs | 2 +- src/Pode.psd1 | 2 +- src/Private/FileWatchers.ps1 | 117 +------- src/Private/Helpers.ps1 | 104 ++++---- src/Public/FileWatchers.ps1 | 19 +- 12 files changed, 322 insertions(+), 185 deletions(-) create mode 100644 docs/Tutorials/FileWatchers.md diff --git a/LICENSE.txt b/LICENSE.txt index 3c190e4e5..e6d70bc0b 100644 --- a/LICENSE.txt +++ b/LICENSE.txt @@ -1,6 +1,6 @@ The MIT License (MIT) -Copyright (c) [2017-2022] [Matthew Kelly (Badgerati)] +Copyright (c) [2017-2023] [Matthew Kelly (Badgerati)] Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal diff --git a/docs/Tutorials/FileWatchers.md b/docs/Tutorials/FileWatchers.md new file mode 100644 index 000000000..2306afc0c --- /dev/null +++ b/docs/Tutorials/FileWatchers.md @@ -0,0 +1,252 @@ +# File Watchers + +Creating a File Watcher lets you monitor for file changes at a given path. You can specify the [event](#events) to monitor (Created, Changed, etc.), and the attributes to use to trigger events (LastWrite, Size, etc.). + +File Watcher paths can be literal or relative, and they can also support [parameters](#parameters) similar to Routes (ie: `C:/Websites/:site`) + +??? note + Under the hood the File Watcher uses the `System.IO.FileSystemWatcher` class, however the version used is an improved version from [GitHub](https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher) with further additional tweaks. + +!!! info + The File Watchers have been tested on Windows and Unix systems, as well as UNC paths from Windows and Azure Storage Account file shares. The File Watcher will not work on WSL. + +## Create a File Watcher + +You can create a File Watcher using [`Add-PodeFileWatcher`](../../Functions/FileWatchers/Add-PodeFileWatcher), this will let you specify a path to a directory to monitor and a script to invoke whenever an event is observed: + +```powershell +Add-PodeFileWatcher -Path 'C:/websites' -ScriptBlock { + "[$($FileEvent.Type)]: $($FileEvent.FullPath)" | Out-Default +} +``` + +By default the above will monitor every file, and in every subdirectory, at `C:/websites`. It will also monitor files for the following `-EventName`: Created, Changed, Deleted, and Renamed. These events will be triggered observing the following `-NotifyFilter`: FileName, DirectoryName, LastWrite, CreationTime. Whenever a file event is observed the `-ScriptBlock` is triggered, and in the case of the above it will simply output the event to the console. + +The `$FileEvent` variable works in a similar fashion to `$WebEvent`, and is detailed [below](#file-event). + +!!! note + When a File Watcher is created, it will be created with a random name. You can use a specific name by supplying the `-Name` parameter during creation. If you don't need a specific name, but you want to know the random name used, you can use the `-PassThru` switch to return the File Watcher object created. + +### File Event + +When an event is triggered, and the File Watcher's `-ScriptBlock` is invoked, a `$FileEvent` variable will be created and accessible from within the scriptblock. + +```powershell +Add-PodeFileWatcher -Path 'C:/websites' -ScriptBlock { + $FileEvent | Out-Default +} +``` + +The `$FileEvent` variable contains the following properties: + +| Name | Type | Description | +| ---- | ---- | ----------- | +| FullPath | string | The full path to the file that triggered that event | +| Lockable | hashtable | A synchronized hashtable that can be used with `Lock-PodeObject` | +| Name | string | The name of the file that triggered the event | +| Old | hashtable | When the event Type is "Renamed", this will contain the Name and FullPath for the previous name of the renamed file | +| Parameters | hashtable | Contains the parsed parameter values from the File Watcher#s path | +| Timestamp | datetime | The current date and time of the event | +| Type | PodeFileWatcherChangeType | The type of event that has been triggered | + +### Events + +The File Watcher can monitor a number of events, which can be specified by using the `-EventName` parameter. When you create a File Watcher without passing `-EventName`, a File Watcher will be created using events Changed, Created, Deleted and Renamed by default. + +!!! tip + There is a special event name of `*` which will create a File Watcher using every event type. + +#### Changed + +A File Watcher that triggers on the Changed `-EventName` will call the defined `-ScriptBlock` whenever a file is changed. When then scriptblock in invoked, the `$FileEvent` base Name/FullPath properties will be the Name/FullPath of the changed file. + +```powershell +Add-PodeFileWatcher -EventName Changed -Path 'C:/websites' -ScriptBlock { + # the Type will be set to "Changed" + $FileEvent.Type | Out-Default + + # file name and path + $FileEvent.Name | Out-Default + $FileEvent.FullPath | Out-Default +} +``` + +#### Created + +A File Watcher that triggers on the Created `-EventName` will call the defined `-ScriptBlock` whenever a file is created. When then scriptblock in invoked, the `$FileEvent` base Name/FullPath properties will be the Name/FullPath of the created file. + +```powershell +Add-PodeFileWatcher -EventName Created -Path 'C:/websites' -ScriptBlock { + # the Type will be set to "Created" + $FileEvent.Type | Out-Default + + # file name and path + $FileEvent.Name | Out-Default + $FileEvent.FullPath | Out-Default +} +``` + +#### Deleted + +A File Watcher that triggers on the Deleted `-EventName` will call the defined `-ScriptBlock` whenever a file is deleted. When then scriptblock in invoked, the `$FileEvent` base Name/FullPath properties will be the Name/FullPath of the deleted file. + +```powershell +Add-PodeFileWatcher -EventName Deleted -Path 'C:/websites' -ScriptBlock { + # the Type will be set to "Deleted" + $FileEvent.Type | Out-Default + + # file name and path + $FileEvent.Name | Out-Default + $FileEvent.FullPath | Out-Default +} +``` + +#### Existed + +A File Watcher that triggers on the Existed `-EventName` will call the defined `-ScriptBlock` on server start for every file that currently exists in the defined `-Path` location. When then scriptblock in invoked, the `$FileEvent` base Name/FullPath properties will be the Name/FullPath of the existing file. + +```powershell +Add-PodeFileWatcher -EventName Existed -Path 'C:/websites' -ScriptBlock { + # the Type will be set to "Existed" + $FileEvent.Type | Out-Default + + # file name and path + $FileEvent.Name | Out-Default + $FileEvent.FullPath | Out-Default +} +``` + +#### Renamed + +A File Watcher that triggers on the Renamed `-EventName` will call the defined `-ScriptBlock` whenever a file is renamed. When then scriptblock in invoked, the `$FileEvent` `Old` property will be initialised, and will contain the original Name/FullPath of the file before the rename. The base Name/FullPath properties will be the new Name/FullPath of the renamed file. + +```powershell +Add-PodeFileWatcher -EventName Renamed -Path 'C:/websites' -ScriptBlock { + # the Type will be set to "Renamed" + $FileEvent.Type | Out-Default + + # new file name and path + $FileEvent.Name | Out-Default + $FileEvent.FullPath | Out-Default + + # original file name and path + $FileEvent.Old.Name | Out-Default + $FileEvent.Old.FullPath | Out-Default +} +``` + +### Parameters + +The `-Path` parameter also supports URL parameter style syntax similar to Routes. For example, say you want to monitor every file for each website found at `C:/websites`, and each site is also its on directory at this path (ie: `C:/websites/example.com`). If you have multiple sites in this directory, you can use `C:/websites/:site` to watch every file in those subdirectories, but also be able to reference the website that had the file change via `$FileEvent.Parameters['site']`: + +```powershell +Add-PodeFileWatcher -Path 'C:/websites/:site' -ScriptBlock { + "[$($FileEvent.Type)][$($FileEvent.Parameters['site'])]: $($FileEvent.FullPath)" | Out-Default +} +``` + +You can have multiple parameters in a path, and reference them all via `$FileEvent.Parameters`. + +### Include / Exclude + +By default every file is monitored (ie: `*.*` on `-Include`). However you can customise the monitored files by using the `-Include` and `-Exclude` parameters. + +For example, to monitor only config file you might use: + +```powershell +Add-PodeFileWatcher -Path 'C:/websites' -Include '*.config' -ScriptBlock { + "[$($FileEvent.Type)]: $($FileEvent.FullPath)" | Out-Default +} +``` + +Or to monitor every file except for log files you might use: + +```powershell +Add-PodeFileWatcher -Path 'C:/websites' -Exclude '*.log' -ScriptBlock { + "[$($FileEvent.Type)]: $($FileEvent.FullPath)" | Out-Default +} +``` + +The `-Include` and `-Exclude` parameters allow for an array of values to be supplied; so you might want to monitor all config files, but then also one specific "web.sitemap" file as well: + +```powershell +Add-PodeFileWatcher -Path 'C:/websites' -Include '*.config', 'web.sitemap' -ScriptBlock { + "[$($FileEvent.Type)]: $($FileEvent.FullPath)" | Out-Default +} +``` + +### Subdirectories + +When you create a File Watcher is will automatically monitor every file in every subdirectory. However you can change this behaviour to only monitor files in the specified directory by supplying `-NoSubdirectories`: + +```powershell +Add-PodeFileWatcher -Path 'C:/websites' -NoSubdirectories -ScriptBlock { + "[$($FileEvent.Type)]: $($FileEvent.FullPath)" | Out-Default +} +``` + +## Arguments + +You can supply custom arguments to be passed to your File Watchers by using the `-ArgumentList` parameter. This parameter takes an array of objects, which will be splatted onto the File Watcher's scriptblock: + +```powershell +Add-PodeFileWatcher -Path 'C:/websites' -ArgumentList 'Item1', 'Item2' -ScriptBlock { + param($i1, $i2) + + # $i1 will be 'Item1' +} +``` + +## Script from File + +You normally define a File Watcher's script using the `-ScriptBlock` parameter however, you can also reference a file with the required scriptblock using `-FilePath`. Using the `-FilePath` parameter will dot-source a scriptblock from the file, and set it as the File Watcher's script. + +For example, to create a File Watcher from a file that will output the file events to the console: + +* File.ps1 +```powershell +{ + "[$($FileEvent.Type)]: $($FileEvent.FullPath)" | Out-Default +} +``` + +* File Watcher +```powershell +Add-PodeFileWatcher -Path 'C:/websites' -FilePath './FileWatchers/File.ps1' +``` + +## Getting File Watchers + +The [`Get-PodeFileWatcher`](../../Functions/FileWatchers/Get-PodeFileWatcher) helper function will allow you to retrieve a list of File Watchers configured within Pode. You can use it to retrieve all of the File Watchers, or supply filters to retrieve specific ones. + +To retrieve all of the File Watchers, you can call the function will no parameters. To filter, here are some examples: + +```powershell +# one File Watchers by name +Get-PodeFileWatcher -Name Name1 + +# multiple File Watchers by name +Get-PodeFileWatcher -Name Name1, Name2 +``` + +## File Watcher Object + +!!! warning + Be careful if you choose to edit these objects, as they will affect the server. + +The following is the structure of the File Watcher object internally, as well as the object that is returned from [`Get-PodeFileWatcher`](../../Functions/FileWatchers/Get-PodeFileWatcher): + +| Name | Type | Description | +| ---- | ---- | ----------- | +| Name | string | The name of the File Watcher | +| Events | string[] | The events that the File Watcher will trigger on | +| Path | string | The given main path the File Watcher is monitoring | +| Placeholders | hashtable | Specifies whether the given path contained parameters, and the regex path to retrieve them | +| Script | scriptblock | The scriptblock of the File Watcher | +| Arguments | object[] | The arguments supplied from ArgumentList | +| NotifyFilters | NotifyFilter[] | The attributes that will be used to trigger the defined events | +| IncludeSubdirectories | bool | Whether the File Watcher will monitor files in subdirectories or not | +| InternalBufferSize | int | The size of the internal buffer cache that stored triggered events | +| Exclude | string[] | The list of file types that should be excluded from triggering events | +| Include | string[] | The list of file types that should be included when triggering events | +| Paths | string[] | The list of all paths that will be monitored. For example, when the original path was a wildcard this will contain all resolved directory paths that the wildcard references | diff --git a/mkdocs.yml b/mkdocs.yml index 907fae4e8..1025062f1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -34,6 +34,7 @@ markdown_extensions: - pymdownx.superfences - pymdownx.tasklist: custom_checkbox: true + - pymdownx.details - admonition - meta diff --git a/packers/choco/pode.nuspec b/packers/choco/pode.nuspec index 6941a6617..079c7fb38 100644 --- a/packers/choco/pode.nuspec +++ b/packers/choco/pode.nuspec @@ -45,7 +45,7 @@ Pode is a Cross-Platform framework for creating web servers to host REST APIs an https://badgerati.github.io/Pode https://github.com/Badgerati/Pode/issues powershell web server rest api smtp unix cross-platform file-monitoring multithreaded schedule middleware authentication aws azure websockets openapi - Copyright 2017-2022 + Copyright 2017-2023 https://github.com/Badgerati/Pode/blob/master/LICENSE.txt false https://raw.githubusercontent.com/Badgerati/Pode/master/images/icon.png diff --git a/src/Listener/FileSystemWatcher/BufferingFileSystemWatcher.cs b/src/Listener/FileSystemWatcher/BufferingFileSystemWatcher.cs index 317c12e2a..e85e1eeda 100644 --- a/src/Listener/FileSystemWatcher/BufferingFileSystemWatcher.cs +++ b/src/Listener/FileSystemWatcher/BufferingFileSystemWatcher.cs @@ -6,7 +6,7 @@ using System.Threading; using System.Threading.Tasks; -// A better FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher +// A tweaked version of the FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher namespace Pode.FileSystemWatcher { public class BufferingFileSystemWatcher : Component diff --git a/src/Listener/FileSystemWatcher/EventQueueOverflowException.cs b/src/Listener/FileSystemWatcher/EventQueueOverflowException.cs index 9ffa3236d..7075a3da8 100644 --- a/src/Listener/FileSystemWatcher/EventQueueOverflowException.cs +++ b/src/Listener/FileSystemWatcher/EventQueueOverflowException.cs @@ -1,6 +1,6 @@ using System; -// A better FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher +// A tweaked version of the FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher namespace Pode.FileSystemWatcher { class EventQueueOverflowException : Exception diff --git a/src/Listener/FileSystemWatcher/FileWatcherErrorEventArgs.cs b/src/Listener/FileSystemWatcher/FileWatcherErrorEventArgs.cs index 318046c42..7b7e90c34 100644 --- a/src/Listener/FileSystemWatcher/FileWatcherErrorEventArgs.cs +++ b/src/Listener/FileSystemWatcher/FileWatcherErrorEventArgs.cs @@ -1,7 +1,7 @@ using System; using System.ComponentModel; -// A better FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher +// A tweaked version of the FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher namespace Pode.FileSystemWatcher { public class FileWatcherErrorEventArgs : HandledEventArgs diff --git a/src/Listener/FileSystemWatcher/RecoveringFileSystemWatcher.cs b/src/Listener/FileSystemWatcher/RecoveringFileSystemWatcher.cs index a12b184e4..4de222115 100644 --- a/src/Listener/FileSystemWatcher/RecoveringFileSystemWatcher.cs +++ b/src/Listener/FileSystemWatcher/RecoveringFileSystemWatcher.cs @@ -3,7 +3,7 @@ using System.IO; using System.Threading; -// A better FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher +// A tweaked version of the FileSystemWatcher from https://github.com/petermeinl/LeanWork.IO.FileSystem.Watcher namespace Pode.FileSystemWatcher { public class RecoveringFileSystemWatcher : BufferingFileSystemWatcher diff --git a/src/Pode.psd1 b/src/Pode.psd1 index 12dbf3b30..8d0a2acc2 100644 --- a/src/Pode.psd1 +++ b/src/Pode.psd1 @@ -20,7 +20,7 @@ Author = 'Matthew Kelly (Badgerati)' # Copyright statement for this module - Copyright = 'Copyright (c) 2017-2022 Matthew Kelly (Badgerati), licensed under the MIT License.' + Copyright = 'Copyright (c) 2017-2023 Matthew Kelly (Badgerati), licensed under the MIT License.' # Description of the functionality provided by this module Description = 'A Cross-Platform PowerShell framework for creating web servers to host REST APIs and Websites. Pode also has support for being used in Azure Functions and AWS Lambda.' diff --git a/src/Private/FileWatchers.ps1 b/src/Private/FileWatchers.ps1 index ac3ebf3db..98bd1ab3f 100644 --- a/src/Private/FileWatchers.ps1 +++ b/src/Private/FileWatchers.ps1 @@ -25,9 +25,9 @@ function Start-PodeFileWatcherRunspace # register file watchers and events foreach ($item in $PodeContext.Fim.Items.Values) { - for ($i = 0; $i -lt $item.Watchers.Length; $i++) { - Write-Verbose "Creating FileWatcher for '$($item.Watchers[$i].Path)'" - $fileWatcher = [PodeFileWatcher]::new($item.Name, $item.Watchers[$i].Path, $item.IncludeSubdirectories, $item.InternalBufferSize, $item.NotifyFilters) + foreach ($path in $item.Paths) { + Write-Verbose "Creating FileWatcher for '$($path)'" + $fileWatcher = [PodeFileWatcher]::new($item.Name, $path, $item.IncludeSubdirectories, $item.InternalBufferSize, $item.NotifyFilters) foreach ($evt in $item.Events) { Write-Verbose "-> Registering event: $($evt)" @@ -75,12 +75,14 @@ function Start-PodeFileWatcherRunspace } # if there are exclusions, and one matches, return - if (($null -ne $fileWatcher.Exclude) -and ($evt.Name -imatch $fileWatcher.Exclude)) { + $exc = (Convert-PodePathPatternsToRegex -Paths $fileWatcher.Exclude) + if (($null -ne $exc) -and ($evt.Name -imatch $exc)) { continue } # if there are inclusions, and none match, return - if (($null -ne $fileWatcher.Include) -and ($evt.Name -inotmatch $fileWatcher.Include)) { + $inc = (Convert-PodePathPatternsToRegex -Paths $fileWatcher.Include) + if (($null -ne $inc) -and ($evt.Name -inotmatch $inc)) { continue } @@ -155,111 +157,6 @@ function Start-PodeFileWatcherRunspace } Add-PodeRunspace -Type Files -ScriptBlock $waitScript -Parameters @{ 'Watcher' = $watcher } -NoProfile - - - - - # $script = { - # # wrapper action handler - # $action = Get-PodeFileWatcherAction - - # # error action hanlder - # $errAction = Get-PodeFileWatcherErrorAction - - # try { - # # register file watchers for each one setup - # foreach ($item in $PodeContext.Fim.Items.Values) { - # for ($i = 0; $i -lt $item.Watchers.Length; $i++) { - # # create .net file watcher for path - # $item.Watchers[$i].Watcher = New-Object Pode.FileWatcher.RecoveringFileSystemWatcher $item.Watchers[$i].Path -Property @{ - # IncludeSubdirectories = $item.IncludeSubdirectories - # InternalBufferSize = $item.InternalBufferSize - # NotifyFilter = $item.NotifyFilters - # EnableRaisingEvents = $true - # } - - # # setup message data ith script/args - # $msgData = @{ - # ScriptBlock = $item.Script - # ArgumentList = @(Get-PodeScriptblockArguments -ArgumentList $item.Arguments -UsingVariables $item.UsingVariables) - # Exclude = $item.Exclude - # Include = $item.Include - # Placeholders = $item.Placeholders - # } - - # # register defined events - # foreach ($evt in $item.Events) { - # Register-PodeFileWatcherEvent ` - # -Name $item.Name ` - # -Index $i ` - # -EventName $evt ` - # -Watcher $item.Watchers[$i].Watcher ` - # -ScriptBlock $action ` - # -MessageData $msgData - # } - - # # register "Error" event type - log the exception - # Register-PodeFileWatcherEvent ` - # -Name $item.Name ` - # -Index $i ` - # -EventName 'Error' ` - # -Watcher $item.Watchers[$i].Watcher ` - # -ScriptBlock $errAction - # } - # } - - # # keep the runspace alive - # while (!$PodeContext.Tokens.Cancellation.IsCancellationRequested) { - # $PodeContext.Fim.Items.Values[0].Watchers[0].Watcher | Out-Default - # $PodeContext.Tokens.Cancellation.Token | Out-Default - # $null = (Wait-PodeTask -Task $PodeContext.Fim.Items.Values[0].Watchers[0].Watcher.GetContextAsync($PodeContext.Tokens.Cancellation.Token)) - # # $processing = $false - - # # foreach ($item in $PodeContext.Fim.Items.Values) { - # # foreach ($watcher in $item.Watchers) { - # # if ($watcher.Watcher.Count -gt 0) { - # # $processing = $true - # # break - # # } - # # } - # # } - - # # if ($processing) { - # # Start-Sleep -Milliseconds 10 - # # } - # # else { - # # Start-Sleep -Seconds 1 - # # } - # } - # } - # catch [System.OperationCanceledException] {} - # catch { - # $_ | Write-PodeErrorLog - # $_.Exception | Write-PodeErrorLog -CheckInnerException - # throw - # } - # finally { - # # dispose/unregister all watchers - # foreach ($item in $PodeContext.Fim.Items.Values) { - # for ($i = 0; $i -lt $item.Watchers.Length; $i++) { - # # unregister events - # foreach ($evt in $item.Events) { - # Unregister-PodeFileWatcherEvent -Name $item.Name -Index $i -EventName $evt - # } - - # # unregister error event - # Unregister-PodeFileWatcherEvent -Name $item.Name -Index $i -EventName 'Error' - - # # dispose watcher - # if ($null -ne $item.Watchers[$i].Watcher) { - # $item.Watchers[$i].Watcher.Dispose() - # } - # } - # } - # } - # } - - # Add-PodeRunspace -Type Files -ScriptBlock $script -NoProfile } function Get-PodeFileWatcherIdenifierName diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 6ce875d3b..a61b0ecf4 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -3,7 +3,7 @@ using namespace Pode # read in the content from a dynamic pode file and invoke its content function ConvertFrom-PodeFile { - param ( + param( [Parameter(Mandatory=$true)] [ValidateNotNull()] $Content, @@ -26,7 +26,7 @@ function ConvertFrom-PodeFile function Get-PodeViewEngineType { - param ( + param( [Parameter(Mandatory=$true)] [string] $Path @@ -45,7 +45,7 @@ function Get-PodeViewEngineType function Get-PodeFileContentUsingViewEngine { - param ( + param( [Parameter(Mandatory=$true)] [string] $Path, @@ -95,7 +95,7 @@ function Get-PodeFileContentUsingViewEngine function Get-PodeFileContent { - param ( + param( [Parameter(Mandatory=$true)] [string] $Path @@ -106,7 +106,7 @@ function Get-PodeFileContent function Get-PodeType { - param ( + param( [Parameter()] $Value ) @@ -151,7 +151,7 @@ function Test-PodeIsAdminUser function Get-PodeHostIPRegex { - param ( + param( [Parameter(Mandatory=$true)] [ValidateSet('Both', 'Hostname', 'IP')] [string] @@ -184,7 +184,7 @@ function Get-PortRegex function Get-PodeEndpointInfo { - param ( + param( [Parameter()] [string] $Address, @@ -237,7 +237,7 @@ function Get-PodeEndpointInfo function Test-PodeIPAddress { - param ( + param( [Parameter()] [string] $IP, @@ -265,7 +265,7 @@ function Test-PodeIPAddress function Test-PodeHostname { - param ( + param( [Parameter()] [string] $Hostname @@ -276,7 +276,7 @@ function Test-PodeHostname function ConvertTo-PodeIPAddress { - param ( + param( [Parameter(Mandatory=$true)] [ValidateNotNull()] $Address @@ -287,7 +287,7 @@ function ConvertTo-PodeIPAddress function Get-PodeIPAddressesForHostname { - param ( + param( [Parameter(Mandatory=$true)] [string] $Hostname, @@ -335,7 +335,7 @@ function Get-PodeIPAddressesForHostname function Test-PodeIPAddressLocal { - param ( + param( [Parameter(Mandatory=$true)] [string] $IP @@ -346,7 +346,7 @@ function Test-PodeIPAddressLocal function Test-PodeIPAddressAny { - param ( + param( [Parameter(Mandatory=$true)] [string] $IP @@ -357,7 +357,7 @@ function Test-PodeIPAddressAny function Test-PodeIPAddressLocalOrAny { - param ( + param( [Parameter(Mandatory=$true)] [string] $IP @@ -368,7 +368,7 @@ function Test-PodeIPAddressLocalOrAny function Get-PodeIPAddress { - param ( + param( [Parameter()] [string] $IP @@ -400,7 +400,7 @@ function Get-PodeIPAddress function Test-PodeIPAddressInRange { - param ( + param( [Parameter(Mandatory=$true)] $IP, @@ -429,7 +429,7 @@ function Test-PodeIPAddressInRange function Test-PodeIPAddressIsSubnetMask { - param ( + param( [Parameter(Mandatory=$true)] [ValidateNotNullOrEmpty()] [string] @@ -441,7 +441,7 @@ function Test-PodeIPAddressIsSubnetMask function Get-PodeSubnetRange { - param ( + param( [Parameter(Mandatory=$true)] [ValidateNotNullOrEmpty()] [string] @@ -505,7 +505,7 @@ function Get-PodeSubnetRange function Add-PodeRunspace { - param ( + param( [Parameter(Mandatory=$true)] [ValidateSet('Main', 'Signals', 'Schedules', 'Gui', 'Web', 'Smtp', 'Tcp', 'Tasks', 'WebSockets', 'Files')] [string] @@ -617,7 +617,7 @@ function Open-PodeRunspace function Close-PodeRunspaces { - param ( + param( [switch] $ClosePool ) @@ -795,7 +795,7 @@ function Test-PodeKeyPressed function Close-PodeServerInternal { - param ( + param( [switch] $ShowDoneMessage ) @@ -951,7 +951,7 @@ function Remove-PodePSDrives function Join-PodeServerRoot { - param ( + param( [Parameter(Mandatory=$true)] [ValidateNotNullOrEmpty()] [string] @@ -977,7 +977,7 @@ function Join-PodeServerRoot function Remove-PodeEmptyItemsFromArray { - param ( + param( [Parameter(ValueFromPipeline=$true)] $Array ) @@ -1032,7 +1032,7 @@ function Remove-PodeNullKeysFromHashtable function Get-PodeFileExtension { - param ( + param( [Parameter()] [string] $Path, @@ -1051,7 +1051,7 @@ function Get-PodeFileExtension function Get-PodeFileName { - param ( + param( [Parameter()] [string] $Path, @@ -1069,7 +1069,7 @@ function Get-PodeFileName function Test-PodeValidNetworkFailure { - param ( + param( [Parameter()] $Exception ) @@ -1623,7 +1623,7 @@ function Split-PodeContentType function ConvertFrom-PodeNameValueToHashTable { - param ( + param( [Parameter()] [System.Collections.Specialized.NameValueCollection] $Collection @@ -1643,7 +1643,7 @@ function ConvertFrom-PodeNameValueToHashTable function Get-PodeCount { - param ( + param( [Parameter()] $Object ) @@ -1732,7 +1732,7 @@ function Test-PodePath function Test-PodePathIsFile { - param ( + param( [Parameter()] [string] $Path, @@ -1754,7 +1754,7 @@ function Test-PodePathIsFile function Test-PodePathIsWildcard { - param ( + param( [Parameter()] [string] $Path @@ -1769,7 +1769,7 @@ function Test-PodePathIsWildcard function Test-PodePathIsDirectory { - param ( + param( [Parameter(Mandatory=$true)] [ValidateNotNullOrEmpty()] [string] @@ -1788,7 +1788,7 @@ function Test-PodePathIsDirectory function Convert-PodePathSeparators { - param ( + param( [Parameter()] $Paths ) @@ -1802,7 +1802,7 @@ function Convert-PodePathSeparators function Convert-PodePathPatternToRegex { - param ( + param( [Parameter()] [string] $Path, @@ -1823,7 +1823,6 @@ function Convert-PodePathPatternToRegex } $Path = $Path -ireplace '\.', '\.' - $Path = $Path -ireplace '\*', '.*?' if ($NotStrict) { @@ -1847,28 +1846,23 @@ function Convert-PodePathPatternsToRegex $NotStrict ) - # remove any empty entries - $Paths = @($Paths | Where-Object { - !(Test-PodeIsEmpty $_) + # replace certain chars + $Paths = @(foreach ($path in $Paths) { + if (![string]::IsNullOrEmpty($path)) { + Convert-PodePathPatternToRegex -Path $path -NotStrict -NotSlashes:$NotSlashes + } }) # if no paths, return null - if (Test-PodeIsEmpty $Paths) { + if (($null -eq $Paths) -or ($Paths.Length -eq 0)) { return $null } - # replace certain chars - $Paths = @($Paths | ForEach-Object { - if (!(Test-PodeIsEmpty $_)) { - Convert-PodePathPatternToRegex -Path $_ -NotStrict -NotSlashes:$NotSlashes - } - }) - # join them all together $joined = "($($Paths -join '|'))" if ($NotStrict) { - return "$($joined)" + return $joined } return "^$($joined)$" @@ -1943,7 +1937,7 @@ function Get-PodeUrl function Find-PodeErrorPage { - param ( + param( [Parameter()] [int] $Code, @@ -2017,7 +2011,7 @@ function Find-PodeErrorPage function Get-PodeErrorPage { - param ( + param( [Parameter()] [int] $Code, @@ -2052,7 +2046,7 @@ function Get-PodeErrorPage function Find-PodeCustomErrorPage { - param ( + param( [Parameter()] [int] $Code, @@ -2088,7 +2082,7 @@ function Find-PodeCustomErrorPage function Find-PodeFileForContentType { - param ( + param( [Parameter()] [string] $Path, @@ -2231,7 +2225,7 @@ function Get-PodeRelativePath function Get-PodeWildcardFiles { - param ( + param( [Parameter(Mandatory=$true)] [string] $Path, @@ -2261,7 +2255,7 @@ function Get-PodeWildcardFiles function Test-PodeIsServerless { - param ( + param( [Parameter()] [string] $FunctionName, @@ -2281,7 +2275,7 @@ function Test-PodeIsServerless function Get-PodeEndpointUrl { - param ( + param( [Parameter()] $Endpoint ) @@ -2358,7 +2352,7 @@ function Get-PodeDefaultPort function Set-PodeServerHeader { - param ( + param( [Parameter()] [string] $Type, @@ -2377,7 +2371,7 @@ function Set-PodeServerHeader function Get-PodeHandler { - param ( + param( [Parameter(Mandatory=$true)] [ValidateSet('Service', 'Smtp')] [string] diff --git a/src/Public/FileWatchers.ps1 b/src/Public/FileWatchers.ps1 index 012b197e8..6448849db 100644 --- a/src/Public/FileWatchers.ps1 +++ b/src/Public/FileWatchers.ps1 @@ -9,7 +9,7 @@ Adds a new File Watcher to monitor file changes in a directory. An optional Name for the File Watcher. (Default: GUID) .PARAMETER EventName -An optional EventName to be monitored. Note: '*' refers to Created, Deleted, Changed, and Renamed. (Default: *) +An optional EventName to be monitored. Note: '*' refers to all event names. (Default: Changed, Created, Deleted, Renamed) .PARAMETER Path The Path to a directory which contains the files to be monitored. @@ -64,7 +64,7 @@ function Add-PodeFileWatcher [Parameter()] [ValidateSet('Changed', 'Created', 'Deleted', 'Renamed', 'Existed', '*')] [string[]] - $EventName = '*', + $EventName = @('Changed', 'Created', 'Deleted', 'Renamed'), [Parameter(Mandatory=$true)] [string] @@ -114,7 +114,7 @@ function Add-PodeFileWatcher # set all for * event if ('*' -iin $EventName) { - $EventName = @('Changed', 'Created', 'Deleted', 'Renamed') + $EventName = @('Changed', 'Created', 'Deleted', 'Renamed', 'Existed') } # resolve path if relative @@ -163,13 +163,6 @@ function Add-PodeFileWatcher $paths = @(Get-ChildItem -Path $Path -Directory -Force | Select-Object -ExpandProperty FullName) } - $watchers = @(foreach ($p in $paths) { - @{ - Path = $p - Watcher = $null - } - }) - # add the file watcher $PodeContext.Fim.Items[$Name] = @{ Name = $Name @@ -185,9 +178,9 @@ function Add-PodeFileWatcher NotifyFilters = @($NotifyFilter) IncludeSubdirectories = !$NoSubdirectories.IsPresent InternalBufferSize = $InternalBufferSize - Exclude = (Convert-PodePathPatternsToRegex -Paths @($Exclude)) - Include = (Convert-PodePathPatternsToRegex -Paths @($Include)) - Watchers = $watchers + Exclude = $Exclude + Include = $Include + Paths = $paths } # return? From 609854bb8bb33b45db2ba138bd6c5f98636aa9d0 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sun, 15 Jan 2023 14:02:23 +0000 Subject: [PATCH 35/52] #1063: clean-up after reviewing PR --- examples/Dockerfile | 2 +- src/Private/FileWatchers.ps1 | 155 ----------------------------------- 2 files changed, 1 insertion(+), 156 deletions(-) diff --git a/examples/Dockerfile b/examples/Dockerfile index b4424f241..65ff8c543 100644 --- a/examples/Dockerfile +++ b/examples/Dockerfile @@ -1,4 +1,4 @@ FROM badgerati/pode:test COPY . /usr/src/app/ EXPOSE 8085 -CMD [ "pwsh", "-c", "cd /usr/src/app; ./file-watchers.ps1" ] +CMD [ "pwsh", "-c", "cd /usr/src/app; ./web-pages-docker.ps1" ] diff --git a/src/Private/FileWatchers.ps1 b/src/Private/FileWatchers.ps1 index 98bd1ab3f..dbc68b116 100644 --- a/src/Private/FileWatchers.ps1 +++ b/src/Private/FileWatchers.ps1 @@ -158,158 +158,3 @@ function Start-PodeFileWatcherRunspace Add-PodeRunspace -Type Files -ScriptBlock $waitScript -Parameters @{ 'Watcher' = $watcher } -NoProfile } - -function Get-PodeFileWatcherIdenifierName -{ - param( - [Parameter(Mandatory=$true)] - [string] - $Name, - - [Parameter(Mandatory=$true)] - [int] - $Index, - - [Parameter(Mandatory=$true)] - [string] - $EventName - ) - - return "Pode.Fim.$($Name -replace '\s+', '_').$($Index).$($EventName)" -} - -function Test-PodeFileWatcherEventRegistered -{ - param( - [Parameter(Mandatory=$true)] - [string] - $SourceIdentifier - ) - - return (($null -ne (Get-Event -SourceIdentifier $SourceIdentifier -ErrorAction Ignore))) -} - -function Get-PodeFileWatcherAction -{ - return { - try { - # if there are exclusions, and one matches, return - if (($null -ne $Event.MessageData.Exclude) -and ($Event.SourceEventArgs.Name -imatch $Event.MessageData.Exclude)) { - return - } - - # if there are inclusions, and none match, return - if (($null -ne $Event.MessageData.Include) -and ($Event.SourceEventArgs.Name -inotmatch $Event.MessageData.Include)) { - return - } - - # set file event object - $global:FileEvent = @{ - Type = $Event.SourceEventArgs.ChangeType - FullPath = $Event.SourceEventArgs.FullPath - Name = $Event.SourceEventArgs.Name - Old = @{ - FullPath = $Event.SourceEventArgs.OldFullPath - Name = $Event.SourceEventArgs.OldName - } - Parameters = @{} - Lockable = $PodeContext.Lockables.Global - Timestamp = [datetime]::UtcNow - } - - # do we have any parameters? - if ($Event.MessageData.Placeholders.Exist -and ($FileEvent.FullPath -imatch $Event.MessageData.Placeholders.Path)) { - $FileEvent.Parameters = $Matches - } - - # invoke main script - Invoke-PodeScriptBlock ` - -ScriptBlock $Event.MessageData.ScriptBlock ` - -Arguments $Event.MessageData.ArgumentList ` - -Scoped ` - -Splat - } - catch [System.OperationCanceledException] {} - catch { - $_ | Write-PodeErrorLog - } - } -} - -function Get-PodeFileWatcherErrorAction -{ - return { - $Event.SourceEventArgs.GetException() | Write-PodeErrorLog - } -} - -function Register-PodeFileWatcherEvent -{ - param( - [Parameter(Mandatory=$true)] - [string] - $Name, - - [Parameter(Mandatory=$true)] - [int] - $Index, - - [Parameter(Mandatory=$true)] - [string] - $EventName, - - [Parameter(Mandatory=$true)] - [Pode.FileWatcher.RecoveringFileSystemWatcher] - $Watcher, - - [Parameter(Mandatory=$true)] - [scriptblock] - $ScriptBlock, - - [Parameter()] - [hashtable] - $MessageData = @{} - ) - - $id = Get-PodeFileWatcherIdenifierName -Name $Name -Index $Index -EventName $EventName - - if (Test-PodeFileWatcherEventRegistered -SourceIdentifier $id) { - throw "An event handler has already been registered with the identifier '$($id)'" - } - - if ($null -eq $MessageData) { - $MessageData = @{} - } - - Register-ObjectEvent ` - -InputObject $Watcher ` - -EventName $EventName ` - -SourceIdentifier $id ` - -Action $ScriptBlock ` - -MessageData $MessageData ` - -SupportEvent ` - -ErrorAction Stop -} - -function Unregister-PodeFileWatcherEvent -{ - param( - [Parameter(Mandatory=$true)] - [string] - $Name, - - [Parameter(Mandatory=$true)] - [int] - $Index, - - [Parameter(Mandatory=$true)] - [string] - $EventName - ) - - $id = Get-PodeFileWatcherIdenifierName -Name $Name -Index $Index -EventName $EventName - - if (Test-PodeFileWatcherEventRegistered -SourceIdentifier $id) { - Unregister-Event -SourceIdentifier $id -Force -ErrorAction SilentlyContinue - } -} \ No newline at end of file From 7994ed5b229484de81b8b38f9b06ff1f2f6fa028 Mon Sep 17 00:00:00 2001 From: ili101 Date: Mon, 16 Jan 2023 23:54:05 +0200 Subject: [PATCH 36/52] Fix Query with no key parse --- src/Private/Helpers.ps1 | 9 +++++++-- tests/unit/Helpers.Tests.ps1 | 11 ++++++++++- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index a61b0ecf4..7de2a5dd5 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -1387,7 +1387,7 @@ function ConvertTo-PodeResponseContent if ([string]::IsNullOrWhiteSpace($ContentType)) { return ([string]$InputObject) } - + # run action for the content type switch ($ContentType) { { $_ -ilike '*/json' } { @@ -1634,8 +1634,13 @@ function ConvertFrom-PodeNameValueToHashTable } $ht = @{} + $i = 0 foreach ($key in $Collection.Keys) { - $ht[$key] = $Collection[$key] + if (!$key) { + $key = '' + } + $ht[$key] = $Collection[$i] + $i++ } return $ht diff --git a/tests/unit/Helpers.Tests.ps1 b/tests/unit/Helpers.Tests.ps1 index 99a86cde0..b49181862 100644 --- a/tests/unit/Helpers.Tests.ps1 +++ b/tests/unit/Helpers.Tests.ps1 @@ -66,7 +66,7 @@ Describe 'Test-PodeIsEmpty' { It 'Return true for no value' { Test-PodeIsEmpty | Should be $true } - + It 'Return true for null value' { Test-PodeIsEmpty -Value $null | Should be $true } @@ -857,6 +857,15 @@ Describe 'ConvertFrom-PodeNameValueToHashTable' { $r.GetType().Name | Should Be 'Hashtable' $r.colour | Should Be 'blue' } + + It 'Returns a hashtable from a value without key collection' { + $c = [System.Collections.Specialized.NameValueCollection]::new() + $c.Add($null, 'blue') + + $r = ConvertFrom-PodeNameValueToHashTable -Collection $c + $r.GetType().Name | Should Be 'Hashtable' + $r[''] | Should Be 'blue' + } } Describe 'Get-PodeUrl' { From 520a72992f33ed8a85173875b0f12c4633294276 Mon Sep 17 00:00:00 2001 From: ili101 Date: Tue, 17 Jan 2023 12:06:38 +0200 Subject: [PATCH 37/52] Fix test and remove Get-PodeCount restriction --- src/Private/Helpers.ps1 | 10 ++-------- tests/unit/Helpers.Tests.ps1 | 5 ++--- 2 files changed, 4 insertions(+), 11 deletions(-) diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 7de2a5dd5..77aa1422c 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -1661,14 +1661,8 @@ function Get-PodeCount return $Object.Length } - if ($Object -is [System.Collections.Specialized.NameValueCollection]) { - if ($Object.Count -eq 0) { - return 0 - } - - if (($Object.Count -eq 1) -and ($null -eq $Object.Keys[0])) { - return 0 - } + if ($Object -is [System.Collections.Specialized.NameValueCollection] -and $Object.Count -eq 0) { + return 0 } return $Object.Count diff --git a/tests/unit/Helpers.Tests.ps1 b/tests/unit/Helpers.Tests.ps1 index b49181862..d99e0083a 100644 --- a/tests/unit/Helpers.Tests.ps1 +++ b/tests/unit/Helpers.Tests.ps1 @@ -859,12 +859,11 @@ Describe 'ConvertFrom-PodeNameValueToHashTable' { } It 'Returns a hashtable from a value without key collection' { - $c = [System.Collections.Specialized.NameValueCollection]::new() - $c.Add($null, 'blue') + $c = [System.Web.HttpUtility]::ParseQueryString('?blue') $r = ConvertFrom-PodeNameValueToHashTable -Collection $c $r.GetType().Name | Should Be 'Hashtable' - $r[''] | Should Be 'blue' + $r.'' | Should Be 'blue' } } From df4cdfeb4a9d8a4010a78e273094cbeff31f766f Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Tue, 17 Jan 2023 21:57:31 +0000 Subject: [PATCH 38/52] #1067: add support for mutexes and semaphores --- examples/lockables.ps1 | 41 -- examples/threading.ps1 | 151 ++++++ src/Pode.psd1 | 33 +- src/Private/Context.ps1 | 11 +- src/Private/FileWatchers.ps1 | 2 +- src/Private/Helpers.ps1 | 12 +- src/Private/PodeServer.ps1 | 4 +- src/Private/Schedules.ps1 | 2 +- src/Private/Server.ps1 | 7 +- src/Private/Serverless.ps1 | 4 +- src/Private/ServiceServer.ps1 | 2 +- src/Private/SmtpServer.ps1 | 2 +- src/Private/Tasks.ps1 | 2 +- src/Private/TcpServer.ps1 | 2 +- src/Private/Timers.ps1 | 2 +- src/Private/WebSockets.ps1 | 2 +- src/Public/Threading.ps1 | 968 ++++++++++++++++++++++++++++++++++ src/Public/Utilities.ps1 | 213 +------- tests/unit/Server.Tests.ps1 | 5 + 19 files changed, 1197 insertions(+), 268 deletions(-) delete mode 100644 examples/lockables.ps1 create mode 100644 examples/threading.ps1 create mode 100644 src/Public/Threading.ps1 diff --git a/examples/lockables.ps1 b/examples/lockables.ps1 deleted file mode 100644 index eb30437be..000000000 --- a/examples/lockables.ps1 +++ /dev/null @@ -1,41 +0,0 @@ -$path = Split-Path -Parent -Path (Split-Path -Parent -Path $MyInvocation.MyCommand.Path) -Import-Module "$($path)/src/Pode.psm1" -Force -ErrorAction Stop - -# or just: -# Import-Module Pode - -# create a basic server -Start-PodeServer -Threads 2 { - - Add-PodeEndpoint -Address * -Port 8090 -Protocol Http - New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging - - New-PodeLockable -Name 'TestLock' - - Add-PodeRoute -Method Get -Path '/custom-route1' -ScriptBlock { - Get-PodeLockable -Name 'TestLock' | Lock-PodeObject -ScriptBlock { - Start-Sleep -Seconds 10 - } - - Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } - } - - Add-PodeRoute -Method Get -Path '/custom-route2' -ScriptBlock { - Get-PodeLockable -Name 'TestLock' | Lock-PodeObject -ScriptBlock {} - Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } - } - - Add-PodeRoute -Method Get -Path '/global-route1' -ScriptBlock { - Lock-PodeObject -ScriptBlock { - Start-Sleep -Seconds 10 - } - - Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } - } - - Add-PodeRoute -Method Get -Path '/global-route2' -ScriptBlock { - Get-PodeLockable -Name 'TestLock' | Lock-PodeObject -CheckGlobal -ScriptBlock {} - Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } - } - -} \ No newline at end of file diff --git a/examples/threading.ps1 b/examples/threading.ps1 new file mode 100644 index 000000000..6d7f700ae --- /dev/null +++ b/examples/threading.ps1 @@ -0,0 +1,151 @@ +param( + [Parameter()] + [int] + $Port = 8090 +) + +$path = Split-Path -Parent -Path (Split-Path -Parent -Path $MyInvocation.MyCommand.Path) +Import-Module "$($path)/src/Pode.psm1" -Force -ErrorAction Stop + +# or just: +# Import-Module Pode + +<# +# Demostrates Lockables, Mutexes, and Semaphores +#> + +Start-PodeServer -Threads 2 { + + Add-PodeEndpoint -Address * -Port $Port -Protocol Http + New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging + + + # custom locks + New-PodeLockable -Name 'TestLock' + + Add-PodeRoute -Method Get -Path '/lock/custom/route1' -ScriptBlock { + Lock-PodeObject -Name 'TestLock' -ScriptBlock { + Start-Sleep -Seconds 10 + } + + Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } + } + + Add-PodeRoute -Method Get -Path '/lock/custom/route2' -ScriptBlock { + Lock-PodeObject -Name 'TestLock' -ScriptBlock {} + Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } + } + + # global locks + Add-PodeRoute -Method Get -Path '/lock/global/route1' -ScriptBlock { + Lock-PodeObject -ScriptBlock { + Start-Sleep -Seconds 10 + } + + Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } + } + + Add-PodeRoute -Method Get -Path '/lock/global/route2' -ScriptBlock { + Get-PodeLockable -Name 'TestLock' | Lock-PodeObject -CheckGlobal -ScriptBlock {} + Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } + } + + + # self mutex + New-PodeMutex -Name 'SelfMutex' + + Add-PodeRoute -Method Get -Path '/mutex/self/route1' -ScriptBlock { + Use-PodeMutex -Name 'SelfMutex' -ScriptBlock { + Start-Sleep -Seconds 10 + } + + Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } + } + + Add-PodeRoute -Method Get -Path '/mutex/self/route2' -ScriptBlock { + Use-PodeMutex -Name 'SelfMutex' -ScriptBlock {} + Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } + } + + # local mutex + New-PodeMutex -Name 'LocalMutex' -Scope Local + + Add-PodeRoute -Method Get -Path '/mutex/local/route1' -ScriptBlock { + Use-PodeMutex -Name 'LocalMutex' -ScriptBlock { + Start-Sleep -Seconds 10 + } + + Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } + } + + Add-PodeRoute -Method Get -Path '/mutex/local/route2' -ScriptBlock { + Use-PodeMutex -Name 'LocalMutex' -ScriptBlock {} + Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } + } + + # global mutex + New-PodeMutex -Name 'GlobalMutex' -Scope Global + + Add-PodeRoute -Method Get -Path '/mutex/global/route1' -ScriptBlock { + Use-PodeMutex -Name 'GlobalMutex' -ScriptBlock { + Start-Sleep -Seconds 10 + } + + Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } + } + + Add-PodeRoute -Method Get -Path '/mutex/global/route2' -ScriptBlock { + Use-PodeMutex -Name 'GlobalMutex' -ScriptBlock {} + Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } + } + + + # self semaphore + New-PodeSemaphore -Name 'SelfSemaphore' + + Add-PodeRoute -Method Get -Path '/semaphore/self/route1' -ScriptBlock { + Use-PodeSemaphore -Name 'SelfSemaphore' -ScriptBlock { + Start-Sleep -Seconds 10 + } + + Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } + } + + Add-PodeRoute -Method Get -Path '/semaphore/self/route2' -ScriptBlock { + Use-PodeSemaphore -Name 'SelfSemaphore' -ScriptBlock {} + Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } + } + + # local semaphore + New-PodeSemaphore -Name 'LocalSemaphore' -Scope Local + + Add-PodeRoute -Method Get -Path '/semaphore/local/route1' -ScriptBlock { + Use-PodeSemaphore -Name 'LocalSemaphore' -ScriptBlock { + Start-Sleep -Seconds 10 + } + + Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } + } + + Add-PodeRoute -Method Get -Path '/semaphore/local/route2' -ScriptBlock { + Use-PodeSemaphore -Name 'LocalSemaphore' -ScriptBlock {} + Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } + } + + # global semaphore + New-PodeSemaphore -Name 'GlobalSemaphore' -Scope Global -Count 1 + + Add-PodeRoute -Method Get -Path '/semaphore/global/route1' -ScriptBlock { + Use-PodeSemaphore -Name 'GlobalSemaphore' -ScriptBlock { + Start-Sleep -Seconds 10 + } + + Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } + } + + Add-PodeRoute -Method Get -Path '/semaphore/global/route2' -ScriptBlock { + Use-PodeSemaphore -Name 'GlobalSemaphore' -ScriptBlock {} + Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } + } + +} \ No newline at end of file diff --git a/src/Pode.psd1 b/src/Pode.psd1 index 8d0a2acc2..10cc580da 100644 --- a/src/Pode.psd1 +++ b/src/Pode.psd1 @@ -91,7 +91,6 @@ # utility helpers 'Close-PodeDisposable', - 'Lock-PodeObject', 'Get-PodeServerPath', 'Start-PodeStopwatch', 'Use-PodeStream', @@ -114,10 +113,6 @@ 'Test-PodeIsIIS', 'Test-PodeIsHeroku', 'Get-PodeIISApplicationPath', - 'New-PodeLockable', - 'Remove-PodeLockable', - 'Get-PodeLockable', - 'Test-PodeLockable', 'Out-PodeVariable', 'Test-PodeIsHosted', 'New-PodeCron', @@ -355,7 +350,33 @@ 'Get-PodeFileWatcher', 'Remove-PodeFileWatcher', 'Clear-PodeFileWatchers', - 'Use--PodeFileWatchers' + 'Use-PodeFileWatchers', + + # Threading + 'Lock-PodeObject', + 'New-PodeLockable', + 'Remove-PodeLockable', + 'Get-PodeLockable', + 'Test-PodeLockable', + 'Enter-PodeLockable', + 'Exit-PodeLockable', + 'Clear-PodeLockables', + 'New-PodeMutex', + 'Test-PodeMutex', + 'Get-PodeMutex', + 'Remove-PodeMutex', + 'Use-PodeMutex', + 'Enter-PodeMutex', + 'Exit-PodeMutex', + 'Clear-PodeMutexes', + 'New-PodeSemaphore', + 'Test-PodeSemaphore', + 'Get-PodeSemaphore', + 'Remove-PodeSemaphore', + 'Use-PodeSemaphore', + 'Enter-PodeSemaphore', + 'Exit-PodeSemaphore', + 'Clear-PodeSemaphores' ) # Private data to pass to the module specified in RootModule/ModuleToProcess. This may also contain a PSData hashtable with additional module metadata used by PowerShell. diff --git a/src/Private/Context.ps1 b/src/Private/Context.ps1 index bd200ab59..f6cceed77 100644 --- a/src/Private/Context.ps1 +++ b/src/Private/Context.ps1 @@ -75,7 +75,7 @@ function New-PodeContext Add-Member -MemberType NoteProperty -Name RunspaceState -Value $null -PassThru | Add-Member -MemberType NoteProperty -Name Tokens -Value @{} -PassThru | Add-Member -MemberType NoteProperty -Name LogsToProcess -Value $null -PassThru | - Add-Member -MemberType NoteProperty -Name Lockables -Value $null -PassThru | + Add-Member -MemberType NoteProperty -Name Threading -Value @{} -PassThru | Add-Member -MemberType NoteProperty -Name Server -Value @{} -PassThru | Add-Member -MemberType NoteProperty -Name Metrics -Value @{} -PassThru | Add-Member -MemberType NoteProperty -Name Listeners -Value @() -PassThru | @@ -387,12 +387,15 @@ function New-PodeContext Files = $null } - # session state - $ctx.Lockables = @{ + # threading locks, etc. + $ctx.Threading.Lockables = @{ Global = [hashtable]::Synchronized(@{}) Custom = @{} } + $ctx.Threading.Mutexes = @{} + $ctx.Threading.Semaphores = @{} + # setup runspaces $ctx.Runspaces = @() @@ -699,7 +702,7 @@ function New-PodeStateContext Add-Member -MemberType NoteProperty -Name Tokens -Value $Context.Tokens -PassThru | Add-Member -MemberType NoteProperty -Name Metrics -Value $Context.Metrics -PassThru | Add-Member -MemberType NoteProperty -Name LogsToProcess -Value $Context.LogsToProcess -PassThru | - Add-Member -MemberType NoteProperty -Name Lockables -Value $Context.Lockables -PassThru | + Add-Member -MemberType NoteProperty -Name Threading -Value $Context.Threading -PassThru | Add-Member -MemberType NoteProperty -Name Server -Value $Context.Server -PassThru) } diff --git a/src/Private/FileWatchers.ps1 b/src/Private/FileWatchers.ps1 index dbc68b116..0ad5b848b 100644 --- a/src/Private/FileWatchers.ps1 +++ b/src/Private/FileWatchers.ps1 @@ -96,7 +96,7 @@ function Start-PodeFileWatcherRunspace Name = $evt.OldName } Parameters = @{} - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global Timestamp = [datetime]::UtcNow } diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index a61b0ecf4..93c4ccb26 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -802,25 +802,35 @@ function Close-PodeServerInternal # ensure the token is cancelled if ($null -ne $PodeContext.Tokens.Cancellation) { + Write-Verbose "Cancelling main cancellation token" $PodeContext.Tokens.Cancellation.Cancel() } # stop all current runspaces + Write-Verbose "Closing runspaces" Close-PodeRunspaces -ClosePool # stop the file monitor if it's running + Write-Verbose "Stopping file monitor" Stop-PodeFileMonitor try { # remove all the cancellation tokens + Write-Verbose "Disposing cancellation tokens" Close-PodeDisposable -Disposable $PodeContext.Tokens.Cancellation Close-PodeDisposable -Disposable $PodeContext.Tokens.Restart + + # dispose mutex/semaphores + Write-Verbose "Diposing mutex and semaphores" + Clear-PodeMutexes + Clear-PodeSemaphores } catch { - $_ | Write-PodeErrorLog + $_ | Out-Default } # remove all of the pode temp drives + Write-Verbose "Removing internal PSDrives" Remove-PodePSDrives if ($ShowDoneMessage -and ($PodeContext.Server.Types.Length -gt 0) -and !$PodeContext.Server.IsServerless) { diff --git a/src/Private/PodeServer.ps1 b/src/Private/PodeServer.ps1 index 21dfd12e7..0ef9f6a6a 100644 --- a/src/Private/PodeServer.ps1 +++ b/src/Private/PodeServer.ps1 @@ -126,7 +126,7 @@ function Start-PodeWebServer Auth = @{} Response = $Response Request = $Request - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global Path = [System.Web.HttpUtility]::UrlDecode($Request.Url.AbsolutePath) Method = $Request.HttpMethod.ToLowerInvariant() Query = $null @@ -352,7 +352,7 @@ function Start-PodeWebServer $SignalEvent = @{ Response = $Response Request = $Request - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global Path = [System.Web.HttpUtility]::UrlDecode($Request.Url.AbsolutePath) Data = @{ Path = [System.Web.HttpUtility]::UrlDecode($payload.path) diff --git a/src/Private/Schedules.ps1 b/src/Private/Schedules.ps1 index 53cde8b7b..1cdfe2354 100644 --- a/src/Private/Schedules.ps1 +++ b/src/Private/Schedules.ps1 @@ -175,7 +175,7 @@ function Invoke-PodeInternalScheduleLogic # setup event param $parameters = @{ Event = @{ - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global Sender = $Schedule } } diff --git a/src/Private/Server.ps1 b/src/Private/Server.ps1 index 218762bf3..c292f8e21 100644 --- a/src/Private/Server.ps1 +++ b/src/Private/Server.ps1 @@ -222,7 +222,7 @@ function Restart-PodeInternalServer $PodeContext.Server.Signals.Listener = $null $PodeContext.Listeners = @() $PodeContext.Receivers = @() - $PodeContext.Watchers = @{} + $PodeContext.Watchers = @() # set view engine back to default $PodeContext.Server.ViewEngine = @{ @@ -247,6 +247,11 @@ function Restart-PodeInternalServer $PodeContext.Server.Secrets.Vaults.Clear() $PodeContext.Server.Secrets.Keys.Clear() + # dispose mutex/semaphores + Clear-PodeLockables + Clear-PodeMutexes + Clear-PodeSemaphores + # clear up output $PodeContext.Server.Output.Variables.Clear() diff --git a/src/Private/Serverless.ps1 b/src/Private/Serverless.ps1 index f08943aef..9d0b56de3 100644 --- a/src/Private/Serverless.ps1 +++ b/src/Private/Serverless.ps1 @@ -34,7 +34,7 @@ function Start-PodeAzFuncServer Auth = @{} Response = $response Request = $request - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global Path = [string]::Empty Method = $request.Method.ToLowerInvariant() Query = $request.Query @@ -161,7 +161,7 @@ function Start-PodeAwsLambdaServer Auth = @{} Response = $response Request = $request - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global Path = [System.Web.HttpUtility]::UrlDecode($request.path) Method = $request.httpMethod.ToLowerInvariant() Query = $request.queryStringParameters diff --git a/src/Private/ServiceServer.ps1 b/src/Private/ServiceServer.ps1 index 70c076c87..540f93b7f 100644 --- a/src/Private/ServiceServer.ps1 +++ b/src/Private/ServiceServer.ps1 @@ -16,7 +16,7 @@ function Start-PodeServiceServer { # the event object $ServiceEvent = @{ - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global } # invoke the service handlers diff --git a/src/Private/SmtpServer.ps1 b/src/Private/SmtpServer.ps1 index 3b7a9101a..eec5e486d 100644 --- a/src/Private/SmtpServer.ps1 +++ b/src/Private/SmtpServer.ps1 @@ -98,7 +98,7 @@ function Start-PodeSmtpServer $SmtpEvent = @{ Response = $Response Request = $Request - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global Email = @{ From = $Request.From To = $Request.To diff --git a/src/Private/Tasks.ps1 b/src/Private/Tasks.ps1 index 0dff999e5..e12e377d8 100644 --- a/src/Private/Tasks.ps1 +++ b/src/Private/Tasks.ps1 @@ -82,7 +82,7 @@ function Invoke-PodeInternalTask # setup event param $parameters = @{ Event = @{ - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global Sender = $Task } } diff --git a/src/Private/TcpServer.ps1 b/src/Private/TcpServer.ps1 index 7b469d98b..d506184cb 100644 --- a/src/Private/TcpServer.ps1 +++ b/src/Private/TcpServer.ps1 @@ -95,7 +95,7 @@ function Start-PodeTcpServer $TcpEvent = @{ Response = $Response Request = $Request - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global Endpoint = @{ Protocol = $Request.Scheme Address = $Request.Address diff --git a/src/Private/Timers.ps1 b/src/Private/Timers.ps1 index 33a2a8c53..c7181ae89 100644 --- a/src/Private/Timers.ps1 +++ b/src/Private/Timers.ps1 @@ -78,7 +78,7 @@ function Invoke-PodeInternalTimer try { $global:TimerEvent = @{ - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global Sender = $Timer } diff --git a/src/Private/WebSockets.ps1 b/src/Private/WebSockets.ps1 index ac693e998..fc21c187d 100644 --- a/src/Private/WebSockets.ps1 +++ b/src/Private/WebSockets.ps1 @@ -70,7 +70,7 @@ function Start-PodeWebSocketRunspace Request = $request Data = $null Files = $null - Lockable = $PodeContext.Lockables.Global + Lockable = $PodeContext.Threading.Lockables.Global Timestamp = [datetime]::UtcNow } diff --git a/src/Public/Threading.ps1 b/src/Public/Threading.ps1 new file mode 100644 index 000000000..041649e9f --- /dev/null +++ b/src/Public/Threading.ps1 @@ -0,0 +1,968 @@ +<# +.SYNOPSIS +Places a temporary lock on an object, or Lockable, while a ScriptBlock is invoked. + +.DESCRIPTION +Places a temporary lock on an object, or Lockable, while a ScriptBlock is invoked. + +.PARAMETER Object +The Object, or Lockable, to lock. If no Object is supplied then the global lockable is used by default. + +.PARAMETER Name +The Name of a Lockable object in Pode to lock, if no Name is supplied then the global lockable is used by default. + +.PARAMETER ScriptBlock +The ScriptBlock to invoke. + +.PARAMETER Timeout +If supplied, a number of milliseconds to timeout after if a lock cannot be acquired. (Default: Infinite) + +.PARAMETER Return +If supplied, any values from the ScriptBlock will be returned. + +.PARAMETER CheckGlobal +If supplied, will check the global Lockable object and wait until it's freed-up before locking the passed object. + +.EXAMPLE +Lock-PodeObject -ScriptBlock { /* logic */ } + +.EXAMPLE +Lock-PodeObject -Object $SomeArray -ScriptBlock { /* logic */ } + +.EXAMPLE +Lock-PodeObject -Name 'LockName' -Timeout 5000 -ScriptBlock { /* logic */ } + +.EXAMPLE +$result = (Lock-PodeObject -Return -Object $SomeArray -ScriptBlock { /* logic */ }) +#> +function Lock-PodeObject +{ + [CmdletBinding(DefaultParameterSetName='Object')] + [OutputType([object])] + param( + [Parameter(ValueFromPipeline=$true, ParameterSetName='Object')] + [object] + $Object, + + [Parameter(Mandatory=$true, ParameterSetName='Name')] + [string] + $Name, + + [Parameter(Mandatory=$true)] + [scriptblock] + $ScriptBlock, + + [Parameter()] + [int] + $Timeout = [System.Threading.Timeout]::Infinite, + + [switch] + $Return, + + [switch] + $CheckGlobal + ) + + try { + if ([string]::IsNullOrEmpty($Name)) { + Enter-PodeLockable -Object $Object -Timeout $Timeout -CheckGlobal:$CheckGlobal + } + else { + Enter-PodeLockable -Name $Name -Timeout $Timeout -CheckGlobal:$CheckGlobal + } + + if ($null -ne $ScriptBlock) { + Invoke-PodeScriptBlock -ScriptBlock $ScriptBlock -NoNewClosure -Return:$Return + } + } + catch { + $_ | Write-PodeErrorLog + throw $_.Exception + } + finally { + if ([string]::IsNullOrEmpty($Name)) { + Exit-PodeLockable -Object $Object + } + else { + Exit-PodeLockable -Name $Name + } + } +} + +<# +.SYNOPSIS +Creates a new custom Lockable object. + +.DESCRIPTION +Creates a new custom Lockable object for use with Lock-PodeObject, and Enter/Exit-PodeLockable. + +.PARAMETER Name +The Name of the Lockable object. + +.EXAMPLE +New-PodeLockable -Name 'Lock1' +#> +function New-PodeLockable +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + if (Test-PodeLockable -Name $Name) { + return + } + + $PodeContext.Threading.Lockables.Custom[$Name] = [hashtable]::Synchronized(@{}) +} + +<# +.SYNOPSIS +Removes a custom Lockable object. + +.DESCRIPTION +Removes a custom Lockable object. + +.PARAMETER Name +The Name of the Lockable object to remove. + +.EXAMPLE +Remove-PodeLockable -Name 'Lock1' +#> +function Remove-PodeLockable +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + if (Test-PodeLockable -Name $Name) { + $PodeContext.Threading.Lockables.Custom.Remove($Name) + } +} + +<# +.SYNOPSIS +Get a custom Lockable object. + +.DESCRIPTION +Get a custom Lockable object for use with Lock-PodeObject, and Enter/Exit-PodeLockable. + +.PARAMETER Name +The Name of the Lockable object. + +.EXAMPLE +Get-PodeLockable -Name 'Lock1' | Lock-PodeObject -ScriptBlock {} +#> +function Get-PodeLockable +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return $PodeContext.Threading.Lockables.Custom[$Name] +} + +<# +.SYNOPSIS +Test if a custom Lockable object exists. + +.DESCRIPTION +Test if a custom Lockable object exists. + +.PARAMETER Name +The Name of the Lockable object. + +.EXAMPLE +Test-PodeLockable -Name 'Lock1' +#> +function Test-PodeLockable +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return $PodeContext.Threading.Lockables.Custom.ContainsKey($Name) +} + +<# +.SYNOPSIS +Place a lock on an object or Lockable. + +.DESCRIPTION +Place a lock on an object or Lockable. This should eventually be followed by a call to Exit-PodeLockable. + +.PARAMETER Object +The Object, or Lockable, to lock. If no Object is supplied then the global lockable is used by default. + +.PARAMETER Name +The Name of a Lockable object in Pode to lock, if no Name is supplied then the global lockable is used by default. + +.PARAMETER Timeout +If supplied, a number of milliseconds to timeout after if a lock cannot be acquired. (Default: Infinite) + +.PARAMETER CheckGlobal +If supplied, will check the global Lockable object and wait until it's freed-up before locking the passed object. + +.EXAMPLE +Enter-PodeLockable -Object $SomeArray + +.EXAMPLE +Enter-PodeLockable -Name 'LockName' -Timeout 5000 +#> +function Enter-PodeLockable +{ + [CmdletBinding(DefaultParameterSetName='Object')] + param( + [Parameter(ValueFromPipeline=$true, ParameterSetName='Object')] + [object] + $Object, + + [Parameter(Mandatory=$true, ParameterSetName='Name')] + [string] + $Name, + + [Parameter()] + [int] + $Timeout = [System.Threading.Timeout]::Infinite, + + [switch] + $CheckGlobal + ) + + # get object by name if set + if (![string]::IsNullOrEmpty($Name)) { + $Object = Get-PodeLockable -Name $Name + } + + # if object is null, default to global + if ($null -eq $Object) { + $Object = $PodeContext.Threading.Lockables.Global + } + + # check if value type and throw + if ($Object -is [valuetype]) { + throw 'Cannot lock value types' + } + + # check if null and throw + if ($null -eq $Object) { + throw 'Cannot lock a null object' + } + + # check if the global lockable is locked + if ($CheckGlobal) { + Lock-PodeObject -Object $PodeContext.Threading.Lockables.Global -ScriptBlock {} -Timeout $Timeout + } + + # attempt to acquire lock + $locked = $false + [System.Threading.Monitor]::TryEnter($Object.SyncRoot, $Timeout, [ref]$locked) + if (!$locked) { + throw "Failed to acquire lock on object" + } +} + +<# +.SYNOPSIS +Remove a lock from an object or Lockable. + +.DESCRIPTION +Remove a lock from an object or Lockable, that was originally locked via Enter-PodeLockable. + +.PARAMETER Object +The Object, or Lockable, to unlock. If no Object is supplied then the global lockable is used by default. + +.PARAMETER Name +The Name of a Lockable object in Pode to unlock, if no Name is supplied then the global lockable is used by default. + +.EXAMPLE +Exit-PodeLockable -Object $SomeArray + +.EXAMPLE +Exit-PodeLockable -Name 'LockName' +#> +function Exit-PodeLockable +{ + [CmdletBinding(DefaultParameterSetName='Object')] + param( + [Parameter(ValueFromPipeline=$true, ParameterSetName='Object')] + [object] + $Object, + + [Parameter(Mandatory=$true, ParameterSetName='Name')] + [string] + $Name + ) + + # get object by name if set + if (![string]::IsNullOrEmpty($Name)) { + $Object = Get-PodeLockable -Name $Name + } + + # if object is null, default to global + if ($null -eq $Object) { + $Object = $PodeContext.Threading.Lockables.Global + } + + # check if value type and throw + if ($Object -is [valuetype]) { + throw 'Cannot unlock value types' + } + + # check if null and throw + if ($null -eq $Object) { + throw 'Cannot unlock a null object' + } + + if ([System.Threading.Monitor]::IsEntered($Object.SyncRoot)) { + [System.Threading.Monitor]::Pulse($Object.SyncRoot) + [System.Threading.Monitor]::Exit($Object.SyncRoot) + } +} + +<# +.SYNOPSIS +Remove all Lockables. + +.DESCRIPTION +Remove all Lockables. + +.EXAMPLE +Clear-PodeLockables +#> +function Clear-PodeLockables +{ + [CmdletBinding()] + param() + + if (Test-PodeIsEmpty $PodeContext.Threading.Lockables.Custom) { + return + } + + foreach ($name in $PodeContext.Threading.Lockables.Custom.Keys.Clone()) { + Remove-PodeLockable -Name $name + } +} + +<# +.SYNOPSIS +Create a new Mutex. + +.DESCRIPTION +Create a new Mutex. + +.PARAMETER Name +The Name of the Mutex. + +.PARAMETER Scope +The Scope of the Mutex, can be either Self, Local, or Global. (Default: Self) +Self: The current process, or child processes. +Local: All processes for the current login session on Windows, or the the same as Self on Unix. +Global: All processes on the system, across every session. + +.EXAMPLE +New-PodeMutex -Name 'SelfMutex' + +.EXAMPLE +New-PodeMutex -Name 'LocalMutex' -Scope Local + +.EXAMPLE +New-PodeMutex -Name 'GlobalMutex' -Scope Global +#> +function New-PodeMutex +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter()] + [ValidateSet('Self', 'Local', 'Global')] + [string] + $Scope = 'Self' + ) + + if (Test-PodeMutex -Name $Name) { + throw "A mutex with the following name already exists: $($Name)" + } + + $mutex = $null + + switch ($Scope.ToLowerInvariant()) { + 'self' { + $mutex = [System.Threading.Mutex]::new($false) + } + + 'local' { + $mutex = [System.Threading.Mutex]::new($false, "Local\$($Name)") + } + + 'global' { + $mutex = [System.Threading.Mutex]::new($false, "Global\$($Name)") + } + } + + $PodeContext.Threading.Mutexes[$Name] = $mutex +} + +<# +.SYNOPSIS +Test if a Mutex exists. + +.DESCRIPTION +Test if a Mutex exists. + +.PARAMETER Name +The Name of the Mutex. + +.EXAMPLE +Test-PodeMutex -Name 'LocalMutex' +#> +function Test-PodeMutex +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return $PodeContext.Threading.Mutexes.ContainsKey($Name) +} + +<# +.SYNOPSIS +Get a Mutex. + +.DESCRIPTION +Get a Mutex. + +.PARAMETER Name +The Name of the Mutex. + +.EXAMPLE +$mutex = Get-PodeMutex -Name 'SelfMutex' +#> +function Get-PodeMutex +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return $PodeContext.Threading.Mutexes[$Name] +} + +<# +.SYNOPSIS +Remove a Mutex. + +.DESCRIPTION +Remove a Mutex. + +.PARAMETER Name +The Name of the Mutex. + +.EXAMPLE +Remove-PodeMutex -Name 'GlobalMutex' +#> +function Remove-PodeMutex +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + if (Test-PodeMutex -Name $Name) { + $PodeContext.Threading.Mutexes[$Name].Dispose() + $PodeContext.Threading.Mutexes.Remove($Name) + } +} + +<# +.SYNOPSIS +Places a temporary hold on a Mutex, invokes a ScriptBlock, then releases the Mutex. + +.DESCRIPTION +Places a temporary hold on a Mutex, invokes a ScriptBlock, then releases the Mutex. + +.PARAMETER Name +The Name of the Mutex. + +.PARAMETER ScriptBlock +The ScriptBlock to invoke. + +.PARAMETER Timeout +If supplied, a number of milliseconds to timeout after if a hold cannot be acquired on the Mutex. (Default: Infinite) + +.PARAMETER Return +If supplied, any values from the ScriptBlock will be returned. + +.EXAMPLE +Use-PodeMutex -Name 'SelfMutex' -Timeout 5000 -ScriptBlock {} + +.EXAMPLE +$result = Use-PodeMutex -Name 'LocalMutex' -Return -ScriptBlock {} +#> +function Use-PodeMutex +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter(Mandatory=$true)] + [scriptblock] + $ScriptBlock, + + [Parameter()] + [int] + $Timeout = [System.Threading.Timeout]::Infinite, + + [switch] + $Return + ) + + try { + $acquired = $false + Enter-PodeMutex -Name $Name -Timeout $Timeout + $acquired = $true + Invoke-PodeScriptBlock -ScriptBlock $ScriptBlock -NoNewClosure -Return:$Return + } + catch { + $_ | Write-PodeErrorLog + throw $_.Exception + } + finally { + if ($acquired) { + Exit-PodeMutex -Name $Name + } + } +} + +<# +.SYNOPSIS +Acquires a hold on a Mutex. + +.DESCRIPTION +Acquires a hold on a Mutex. This should eventually by followed by a call to Exit-PodeMutex. + +.PARAMETER Name +The Name of the Mutex. + +.PARAMETER Timeout +If supplied, a number of milliseconds to timeout after if a hold cannot be acquired on the Mutex. (Default: Infinite) + +.EXAMPLE +Enter-PodeMutex -Name 'SelfMutex' -Timeout 5000 +#> +function Enter-PodeMutex +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter()] + [int] + $Timeout = [System.Threading.Timeout]::Infinite + ) + + $mutex = Get-PodeMutex -Name $Name + if ($null -eq $mutex) { + throw "No mutex found called '$($Name)'" + } + + if (!$mutex.WaitOne($Timeout)) { + throw "Failed to acquire mutex ownership. Mutex name: $($Name)" + } +} + +<# +.SYNOPSIS +Release the hold on a Mutex. + +.DESCRIPTION +Release the hold on a Mutex, that was originally acquired by Enter-PodeMutex. + +.PARAMETER Name +The Name of the Mutex. + +.EXAMPLE +Exit-PodeMutex -Name 'SelfMutex' +#> +function Exit-PodeMutex +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + $mutex = Get-PodeMutex -Name $Name + if ($null -eq $mutex) { + throw "No mutex found called '$($Name)'" + } + + $mutex.ReleaseMutex() +} + +<# +.SYNOPSIS +Removes all Mutexes. + +.DESCRIPTION +Removes all Mutexes. + +.EXAMPLE +Clear-PodeMutexes +#> +function Clear-PodeMutexes +{ + [CmdletBinding()] + param() + + if (Test-PodeIsEmpty $PodeContext.Threading.Mutexes) { + return + } + + foreach ($name in $PodeContext.Threading.Mutexes.Keys.Clone()) { + Remove-PodeMutex -Name $name + } +} + +<# +.SYNOPSIS +Create a new Semaphore. + +.DESCRIPTION +Create a new Semaphore. + +.PARAMETER Name +The Name of the Semaphore. + +.PARAMETER Count +The number of threads to allow a hold on the Semaphore. (Default: 1) + +.PARAMETER Scope +The Scope of the Semaphore, can be either Self, Local, or Global. (Default: Self) +Self: The current process, or child processes. +Local: All processes for the current login session on Windows, or the the same as Self on Unix. +Global: All processes on the system, across every session. + +.EXAMPLE +New-PodeSemaphore -Name 'SelfSemaphore' + +.EXAMPLE +New-PodeSemaphore -Name 'LocalSemaphore' -Scope Local + +.EXAMPLE +New-PodeSemaphore -Name 'GlobalSemaphore' -Count 3 -Scope Global +#> +function New-PodeSemaphore +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter()] + [int] + $Count = 1, + + [Parameter()] + [ValidateSet('Self', 'Local', 'Global')] + [string] + $Scope = 'Self' + ) + + if (Test-PodeSemaphore -Name $Name) { + throw "A semaphore with the following name already exists: $($Name)" + } + + if ($Count -le 0) { + $Count = 1 + } + + $semaphore = $null + + switch ($Scope.ToLowerInvariant()) { + 'self' { + $semaphore = [System.Threading.Semaphore]::new($Count, $Count) + } + + 'local' { + $semaphore = [System.Threading.Semaphore]::new($Count, $Count, "Local\$($Name)") + } + + 'global' { + $semaphore = [System.Threading.Semaphore]::new($Count, $Count, "Global\$($Name)") + } + } + + $PodeContext.Threading.Semaphores[$Name] = $semaphore +} + +<# +.SYNOPSIS +Test if a Semaphore exists. + +.DESCRIPTION +Test if a Semaphore exists. + +.PARAMETER Name +The Name of the Semaphore. + +.EXAMPLE +Test-PodeSemaphore -Name 'LocalSemaphore' +#> +function Test-PodeSemaphore +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return $PodeContext.Threading.Semaphores.ContainsKey($Name) +} + +<# +.SYNOPSIS +Get a Semaphore. + +.DESCRIPTION +Get a Semaphore. + +.PARAMETER Name +The Name of the Semaphore. + +.EXAMPLE +$semaphore = Get-PodeSemaphore -Name 'SelfSemaphore' +#> +function Get-PodeSemaphore +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + return $PodeContext.Threading.Semaphores[$Name] +} + +<# +.SYNOPSIS +Remove a Semaphore. + +.DESCRIPTION +Remove a Semaphore. + +.PARAMETER Name +The Name of the Semaphore. + +.EXAMPLE +Remove-PodeSemaphore -Name 'GlobalSemaphore' +#> +function Remove-PodeSemaphore +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name + ) + + if (Test-PodeSemaphore -Name $Name) { + $PodeContext.Threading.Semaphores[$Name].Dispose() + $PodeContext.Threading.Semaphores.Remove($Name) + } +} + +<# +.SYNOPSIS +Places a temporary hold on a Semaphore, invokes a ScriptBlock, then releases the Semaphore. + +.DESCRIPTION +Places a temporary hold on a Semaphore, invokes a ScriptBlock, then releases the Semaphore. + +.PARAMETER Name +The Name of the Semaphore. + +.PARAMETER ScriptBlock +The ScriptBlock to invoke. + +.PARAMETER Timeout +If supplied, a number of milliseconds to timeout after if a hold cannot be acquired on the Semaphore. (Default: Infinite) + +.PARAMETER Return +If supplied, any values from the ScriptBlock will be returned. + +.EXAMPLE +Use-PodeSemaphore -Name 'SelfSemaphore' -Timeout 5000 -ScriptBlock {} + +.EXAMPLE +$result = Use-PodeSemaphore -Name 'LocalSemaphore' -Return -ScriptBlock {} +#> +function Use-PodeSemaphore +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter(Mandatory=$true)] + [scriptblock] + $ScriptBlock, + + [Parameter()] + [int] + $Timeout = [System.Threading.Timeout]::Infinite, + + [switch] + $Return + ) + + try { + $acquired = $false + Enter-PodeSemaphore -Name $Name -Timeout $Timeout + $acquired = $true + Invoke-PodeScriptBlock -ScriptBlock $ScriptBlock -NoNewClosure -Return:$Return + } + catch { + $_ | Write-PodeErrorLog + throw $_.Exception + } + finally { + if ($acquired) { + Exit-PodeSemaphore -Name $Name + } + } +} + +<# +.SYNOPSIS +Acquires a hold on a Semaphore. + +.DESCRIPTION +Acquires a hold on a Semaphore. This should eventually by followed by a call to Exit-PodeSemaphore. + +.PARAMETER Name +The Name of the Semaphore. + +.PARAMETER Timeout +If supplied, a number of milliseconds to timeout after if a hold cannot be acquired on the Semaphore. (Default: Infinite) + +.EXAMPLE +Enter-PodeSemaphore -Name 'SelfSemaphore' -Timeout 5000 +#> +function Enter-PodeSemaphore +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter()] + [int] + $Timeout = [System.Threading.Timeout]::Infinite + ) + + $semaphore = Get-PodeSemaphore -Name $Name + if ($null -eq $semaphore) { + throw "No semaphore found called '$($Name)'" + } + + if (!$semaphore.WaitOne($Timeout)) { + throw "Failed to acquire semaphore ownership. Semaphore name: $($Name)" + } +} + +<# +.SYNOPSIS +Release the hold on a Semaphore. + +.DESCRIPTION +Release the hold on a Semaphore, that was originally acquired by Enter-PodeSemaphore. + +.PARAMETER Name +The Name of the Semaphore. + +.PARAMETER ReleaseCount +The number of releases to release in one go. (Default: 1) + +.EXAMPLE +Exit-PodeSemaphore -Name 'SelfSemaphore' +#> +function Exit-PodeSemaphore +{ + [CmdletBinding()] + param( + [Parameter(Mandatory=$true)] + [string] + $Name, + + [Parameter()] + [int] + $ReleaseCount = 1 + ) + + $semaphore = Get-PodeSemaphore -Name $Name + if ($null -eq $semaphore) { + throw "No semaphore found called '$($Name)'" + } + + if ($ReleaseCount -lt 1) { + $ReleaseCount = 1 + } + + $semaphore.Release($ReleaseCount) +} + +<# +.SYNOPSIS +Removes all Semaphores. + +.DESCRIPTION +Removes all Semaphores. + +.EXAMPLE +Clear-PodeSemaphores +#> +function Clear-PodeSemaphores +{ + [CmdletBinding()] + param() + + if (Test-PodeIsEmpty $PodeContext.Threading.Semaphores) { + return + } + + foreach ($name in $PodeContext.Threading.Semaphores.Keys.Clone()) { + Remove-PodeSemaphore -Name $name + } +} \ No newline at end of file diff --git a/src/Public/Utilities.ps1 b/src/Public/Utilities.ps1 index 1b2d46e7b..c40e52b63 100644 --- a/src/Public/Utilities.ps1 +++ b/src/Public/Utilities.ps1 @@ -20,7 +20,7 @@ Close-PodeDisposable -Disposable $stream -Close function Close-PodeDisposable { [CmdletBinding()] - param ( + param( [Parameter()] [System.IDisposable] $Disposable, @@ -54,93 +54,6 @@ function Close-PodeDisposable } } -<# -.SYNOPSIS -Places a temporary lock on a object while a ScriptBlock is invoked. - -.DESCRIPTION -Places a temporary lock on a object while a ScriptBlock is invoked. - -.PARAMETER Object -The object to lock, if no object is supplied then the global lockable is used by default. - -.PARAMETER ScriptBlock -The ScriptBlock to invoke. - -.PARAMETER Return -If supplied, any values from the ScriptBlock will be returned. - -.PARAMETER CheckGlobal -If supplied, will check the global Lockable object and wait until it's freed-up before locking the passed object. - -.EXAMPLE -Lock-PodeObject -ScriptBlock { /* logic */ } - -.EXAMPLE -Lock-PodeObject -Object $SomeArray -ScriptBlock { /* logic */ } - -.EXAMPLE -$result = (Lock-PodeObject -Return -Object $SomeArray -ScriptBlock { /* logic */ }) -#> -function Lock-PodeObject -{ - [CmdletBinding()] - [OutputType([object])] - param ( - [Parameter(ValueFromPipeline=$true)] - [object] - $Object, - - [Parameter(Mandatory=$true)] - [scriptblock] - $ScriptBlock, - - [switch] - $Return, - - [switch] - $CheckGlobal - ) - - if ($null -eq $Object) { - $Object = $PodeContext.Lockables.Global - } - - if ($Object -is [valuetype]) { - throw 'Cannot lock value types' - } - - $locked = $false - - try { - if ($CheckGlobal) { - Lock-PodeObject -Object $PodeContext.Lockables.Global -ScriptBlock {} - } - - [System.Threading.Monitor]::Enter($Object.SyncRoot) - $locked = $true - - if ($null -ne $ScriptBlock) { - if ($Return) { - return (Invoke-PodeScriptBlock -ScriptBlock $ScriptBlock -NoNewClosure -Return) - } - else { - Invoke-PodeScriptBlock -ScriptBlock $ScriptBlock -NoNewClosure - } - } - } - catch { - $_ | Write-PodeErrorLog - throw $_.Exception - } - finally { - if ($locked) { - [System.Threading.Monitor]::Pulse($Object.SyncRoot) - [System.Threading.Monitor]::Exit($Object.SyncRoot) - } - } -} - <# .SYNOPSIS Returns the literal path of the server. @@ -179,7 +92,7 @@ Start-PodeStopwatch -Name 'ReadFile' -ScriptBlock { $content = Get-Content './fi function Start-PodeStopwatch { [CmdletBinding()] - param ( + param( [Parameter(Mandatory=$true)] [string] $Name, @@ -223,7 +136,7 @@ function Use-PodeStream { [CmdletBinding()] [OutputType([object])] - param ( + param( [Parameter(Mandatory=$true)] [System.IDisposable] $Stream, @@ -261,7 +174,7 @@ Use-PodeScript -Path './scripts/tools.ps1' function Use-PodeScript { [CmdletBinding()] - param ( + param( [Parameter(Mandatory=$true)] [string] $Path @@ -332,7 +245,7 @@ Add-PodeEndware -ScriptBlock { /* logic */ } function Add-PodeEndware { [CmdletBinding()] - param ( + param( [Parameter(Mandatory=$true, ValueFromPipeline=$true)] [scriptblock] $ScriptBlock, @@ -473,7 +386,7 @@ Import-PodeSnapin -Name 'WDeploySnapin3.0' function Import-PodeSnapin { [CmdletBinding()] - param ( + param( [Parameter(Mandatory=$true)] [string] $Name @@ -508,7 +421,7 @@ function Protect-PodeValue { [CmdletBinding()] [OutputType([object])] - param ( + param( [Parameter()] $Value, @@ -542,7 +455,7 @@ function Resolve-PodeValue { [CmdletBinding()] [OutputType([object])] - param ( + param( [Parameter(Mandatory=$true)] [bool] $Check, @@ -596,7 +509,7 @@ function Invoke-PodeScriptBlock { [CmdletBinding()] [OutputType([object])] - param ( + param( [Parameter(Mandatory=$true)] [scriptblock] $ScriptBlock, @@ -664,7 +577,7 @@ function Test-PodeIsEmpty { [CmdletBinding()] [OutputType([bool])] - param ( + param( [Parameter()] $Value ) @@ -928,112 +841,6 @@ function Test-PodeIsHosted return ((Test-PodeIsIIS) -or (Test-PodeIsHeroku)) } -<# -.SYNOPSIS -Creates a new custom lockable object for use with Lock-PodeObject. - -.DESCRIPTION -Creates a new custom lockable object for use with Lock-PodeObject. - -.PARAMETER Name -The Name of the lockable object. - -.EXAMPLE -New-PodeLockable -Name 'Lock1' -#> -function New-PodeLockable -{ - [CmdletBinding()] - param( - [Parameter(Mandatory=$true)] - [string] - $Name - ) - - if (Test-PodeLockable -Name $Name) { - return - } - - $PodeContext.Lockables.Custom[$Name] = [hashtable]::Synchronized(@{}) -} - -<# -.SYNOPSIS -Removes a custom lockable object. - -.DESCRIPTION -Removes a custom lockable object. - -.PARAMETER Name -The Name of the lockable object to remove. - -.EXAMPLE -Remove-PodeLockable -Name 'Lock1' -#> -function Remove-PodeLockable -{ - [CmdletBinding()] - param( - [Parameter(Mandatory=$true)] - [string] - $Name - ) - - if (Test-PodeLockable -Name $Name) { - $PodeContext.Lockables.Custom.Remove($Name) - } -} - -<# -.SYNOPSIS -Get a custom lockable object for use with Lock-PodeObject. - -.DESCRIPTION -Get a custom lockable object for use with Lock-PodeObject. - -.PARAMETER Name -The Name of the lockable object. - -.EXAMPLE -Get-PodeLockable -Name 'Lock1' | Lock-PodeObject -ScriptBlock {} -#> -function Get-PodeLockable -{ - [CmdletBinding()] - param( - [Parameter(Mandatory=$true)] - [string] - $Name - ) - - return $PodeContext.Lockables.Custom[$Name] -} - -<# -.SYNOPSIS -Test if a custom lockable object exists. - -.DESCRIPTION -Test if a custom lockable object exists. - -.PARAMETER Name -The Name of the lockable object. - -.EXAMPLE -Test-PodeLockable -Name 'Lock1' -#> -function Test-PodeLockable -{ - [CmdletBinding()] - param( - [Parameter(Mandatory=$true)] - [string] - $Name - ) - - return $PodeContext.Lockables.Custom.ContainsKey($Name) -} - <# .SYNOPSIS Defines variables to be created when the Pode server stops. diff --git a/tests/unit/Server.Tests.ps1 b/tests/unit/Server.Tests.ps1 index 2b5e1d8ca..ef5020646 100644 --- a/tests/unit/Server.Tests.ps1 +++ b/tests/unit/Server.Tests.ps1 @@ -205,6 +205,11 @@ Describe 'Restart-PodeInternalServer' { key = 'value' } } + Threading = @{ + Lockables = @{ Custom = @{} } + Mutexes = @{} + Semaphores = @{} + } } Restart-PodeInternalServer | Out-Null From e4ac52fb2ef07ef36bf84bb771d07b18080a7753 Mon Sep 17 00:00:00 2001 From: ili101 Date: Wed, 18 Jan 2023 02:57:17 +0200 Subject: [PATCH 39/52] Use GetValues --- src/Private/Helpers.ps1 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 77aa1422c..8c233b16b 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -1634,13 +1634,13 @@ function ConvertFrom-PodeNameValueToHashTable } $ht = @{} - $i = 0 foreach ($key in $Collection.Keys) { + $htKey = $key if (!$key) { - $key = '' + $htKey = '' } - $ht[$key] = $Collection[$i] - $i++ + + $ht[$htKey] = $Collection.GetValues($key) } return $ht From cc4df1de8369d2070d88eca883fae63502c3c276 Mon Sep 17 00:00:00 2001 From: ili101 Date: Wed, 18 Jan 2023 23:49:29 +0200 Subject: [PATCH 40/52] Keep valuse values as string --- src/Private/Helpers.ps1 | 2 +- tests/unit/Helpers.Tests.ps1 | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 8c233b16b..fa0ee7a92 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -1640,7 +1640,7 @@ function ConvertFrom-PodeNameValueToHashTable $htKey = '' } - $ht[$htKey] = $Collection.GetValues($key) + $ht[$htKey] = $Collection.Get($key) } return $ht diff --git a/tests/unit/Helpers.Tests.ps1 b/tests/unit/Helpers.Tests.ps1 index d99e0083a..b39695c5b 100644 --- a/tests/unit/Helpers.Tests.ps1 +++ b/tests/unit/Helpers.Tests.ps1 @@ -855,6 +855,7 @@ Describe 'ConvertFrom-PodeNameValueToHashTable' { $r = ConvertFrom-PodeNameValueToHashTable -Collection $c $r.GetType().Name | Should Be 'Hashtable' + $r.colour.GetType().Name | Should Be 'string' $r.colour | Should Be 'blue' } From a1b872c7feb07cbe67b3fd4b2a5402754e730f2c Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Wed, 18 Jan 2023 22:12:20 +0000 Subject: [PATCH 41/52] #1067: update docs --- docs/Tutorials/Threading.md | 70 ------------ docs/Tutorials/Threading/Lockables.md | 127 ++++++++++++++++++++++ docs/Tutorials/Threading/Mutexes.md | 103 ++++++++++++++++++ docs/Tutorials/Threading/Semaphores.md | 108 ++++++++++++++++++ docs/Tutorials/Threading/ServerThreads.md | 11 ++ 5 files changed, 349 insertions(+), 70 deletions(-) delete mode 100644 docs/Tutorials/Threading.md create mode 100644 docs/Tutorials/Threading/Lockables.md create mode 100644 docs/Tutorials/Threading/Mutexes.md create mode 100644 docs/Tutorials/Threading/Semaphores.md create mode 100644 docs/Tutorials/Threading/ServerThreads.md diff --git a/docs/Tutorials/Threading.md b/docs/Tutorials/Threading.md deleted file mode 100644 index d4fa9aa3c..000000000 --- a/docs/Tutorials/Threading.md +++ /dev/null @@ -1,70 +0,0 @@ -# Threading - -By default Pode deals with incoming requests synchronously in a single thread. You can increase the number of threads/runspaces that Pode uses to handle requests by using the `-Threads` parameter on [`Start-PodeServer`](../../Functions/Core/Start-PodeServer): - -```powershell -Start-PodeServer -Threads 2 { - # logic -} -``` - -The number of threads supplied only applies to Web, SMTP, and TCP servers. If `-Threads` is not supplied, or is <=0 then the number of threads is forced to the default of 1. - -## Locking - -When using multi-threading in Pode at times you'll want to ensure certain functions run thread-safe. To do this you can use [`Lock-PodeObject`](../../Functions/Utilities/Lock-PodeObject) which will lock an object cross-thread. - -### Global - -In event objects, like `$WebEvent`, there is a global `Lockable` object that you can use - this object is synchronized across every thread, so locking it on one will lock it on all: - -```powershell -Add-PodeRoute -Method Get -Path '/save' -ScriptBlock { - Lock-PodeObject -ScriptBlock { - Save-PodeState -Path './state.json' - } -} -``` - -### Custom - -The global lockable is good, but at times you will have separate processes where they can use different lockables objects - rather than sharing a global one and locking each other out needlessly. - -To create a custom lockable object you can use [`New-PodeLockable`](../../Functions/Utilities/New-PodeLockable), and this will create a synchronized object across all threads that you can use. You cna then use this object via [`Get-PodeLockable`](../../Functions/Utilities/Get-PodeLockable) and pipe it into [`Lock-PodeObject`](../../Functions/Utilities/Lock-PodeObject): - -```powershell -New-PodeLockable -Name 'Lock1' - -Add-PodeRoute -Method Get -Path '/save' -ScriptBlock { - Get-PodeLockable -Name 'Lock1' | Lock-PodeObject -ScriptBlock { - Save-PodeState -Path './state.json' - } -} -``` - -On [`Lock-PodeObject`](../../Functions/Utilities/Lock-PodeObject) there's also a `-CheckGlobal` switch. This switch will check if the global lockable is locked, and wait for it to free up before locking the specified object and running the script. This is useful if you have a number of custom lockables, and then when saving the current state you lock the global. Every other process could lock their custom lockables, but then also check the global lockable and block until saving state is finished. - -For example, the following has two routes. The first route locks the global lockable and sleeps for 10 seconds, whereas the second route locks the custom object but checks the global for locking. Calling the first route then the second straight after, they will both return after 10 seconds: - -```powershell -Start-PodeServer -Threads 2 { - Add-PodeEndpoint -Address * -Port 8090 -Protocol Http - - New-PodeLockable -Name 'TestLock' - - # lock global, sleep for 10secs - Add-PodeRoute -Method Get -Path '/route1' -ScriptBlock { - Lock-PodeObject -ScriptBlock { - Start-Sleep -Seconds 10 - } - - Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } - } - - # lock custom, but check global - Add-PodeRoute -Method Get -Path '/route2' -ScriptBlock { - Get-PodeLockable -Name 'TestLock' | Lock-PodeObject -CheckGlobal -ScriptBlock {} - Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } - } -} -``` diff --git a/docs/Tutorials/Threading/Lockables.md b/docs/Tutorials/Threading/Lockables.md new file mode 100644 index 000000000..8455298ac --- /dev/null +++ b/docs/Tutorials/Threading/Lockables.md @@ -0,0 +1,127 @@ +# Lockables + +When using multi-threading in Pode at times you'll want to ensure certain functions run thread-safe. To do this you can use [`Lock-PodeObject`](../../../Functions/Threading/Lock-PodeObject) which will lock an object across threads. + +## Global + +In event objects, like `$WebEvent`, there is a global `Lockable` object that you can use - this object is synchronized across every thread, so locking it on one thread will lock it on all threads: + +```powershell +Add-PodeRoute -Method Get -Path '/save' -ScriptBlock { + Lock-PodeObject -ScriptBlock { + Save-PodeState -Path './state.json' + } +} +``` + +## Custom + +The global lockable is good, but at times you will have separate processes where they can use different lockable objects - rather than sharing a global one and locking each other out needlessly. + +To create a custom lockable object you can use [`New-PodeLockable`](../../../Functions/Threading/New-PodeLockable), and this will create a synchronized object across all threads that you can use. You can then use this object via [`Get-PodeLockable`](../../../Functions/Threading/Get-PodeLockable) and pipe it into [`Lock-PodeObject`](../../../Functions/Threading/Lock-PodeObject): + +```powershell +New-PodeLockable -Name 'Lock1' + +Add-PodeRoute -Method Get -Path '/save' -ScriptBlock { + Get-PodeLockable -Name 'Lock1' | Lock-PodeObject -ScriptBlock { + Save-PodeState -Path './state.json' + } +} +``` + +Similarly, you can reference the lockable directly by name: + +```powershell +New-PodeLockable -Name 'Lock1' + +Add-PodeRoute -Method Get -Path '/save' -ScriptBlock { + Lock-PodeObject -Name 'Lock1' -ScriptBlock { + Save-PodeState -Path './state.json' + } +} +``` + +On [`Lock-PodeObject`](../../../Functions/Threading/Lock-PodeObject) there's also a `-CheckGlobal` switch. This switch will check if the global lockable is locked, and wait for it to free up before locking the specified object and running the script. This is useful if you have a number of custom lockables, and then when saving the current state you lock the global. Every other process could lock their custom lockables, but then also check the global lockable and block until saving state is finished. + +For example, the following has two routes. The first route locks the global lockable and sleeps for 10 seconds, whereas the second route locks the custom object but checks the global for locking. Calling the first route then the second straight after, they will both return after 10 seconds: + +```powershell +Start-PodeServer -Threads 2 { + Add-PodeEndpoint -Address * -Port 8090 -Protocol Http + + New-PodeLockable -Name 'TestLock' + + # lock global, sleep for 10secs + Add-PodeRoute -Method Get -Path '/route1' -ScriptBlock { + Lock-PodeObject -ScriptBlock { + Start-Sleep -Seconds 10 + } + + Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } + } + + # lock custom, but check global + Add-PodeRoute -Method Get -Path '/route2' -ScriptBlock { + Lock-PodeObject -Name 'TestLock' -CheckGlobal -ScriptBlock {} + Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } + } +} +``` + +## Enter / Exit + +You can have more advanced control over the locking of lockables, and other objects, by using [`Enter-PodeLockable`](../../../Functions/Threading/Enter-PodeLockable) and [`Exit-PodeLockable`](../../../Functions/Threading/Exit-PodeLockable). Using these functions you can place a lock on a lockable, run some logic, and then remove the lock - but you don't have to do it all in the same function or scriptblock. + +[`Enter-PodeLockable`](../../../Functions/Threading/Enter-PodeLockable) can be used to initially place a lock on a lockable, and then you must call [`Exit-PodeLockable`](../../../Functions/Threading/Exit-PodeLockable) later on to remove the lock: + +```powershell +Start-PodeServer -Threads 2 { + Add-PodeEndpoint -Address * -Port 8090 -Protocol Http + + New-PodeLockable -Name 'TestLock' + + # lock object and sleep for 10secs + Add-PodeRoute -Method Get -Path '/route1' -ScriptBlock { + try { + Enter-PodeLockable -Name 'TestLock' + Start-Sleep -Seconds 10 + } + finally { + Exit-PodeLockable -Name 'TestLock' + } + + Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } + } +} +``` + +## Timeout + +When locking a lockable, or another object, by default Pode will wait indefinitely for the object to be unlocked before locking it. You can use the `-Timeout` parameter to specify a number of milliseconds to timeout after if a lock cannot be acquired. + +```powershell +Start-PodeServer -Threads 2 { + Add-PodeEndpoint -Address * -Port 8090 -Protocol Http + + New-PodeLockable -Name 'TestLock' + + # lock object and sleep for 10secs + Add-PodeRoute -Method Get -Path '/route1' -ScriptBlock { + Lock-PodeObject -Name 'TestLock' -ScriptBlock { + Start-Sleep -Seconds 10 + } + + Write-PodeJsonResponse -Value @{ Route = 1; Thread = $ThreadId } + } + + # lock object, but timeout after 2s + Add-PodeRoute -Method Get -Path '/route2' -ScriptBlock { + Lock-PodeObject -Name 'TestLock' -Timeout 2000 -ScriptBlock { + Start-Sleep -Seconds 2 + } + + Write-PodeJsonResponse -Value @{ Route = 2; Thread = $ThreadId } + } +} +``` diff --git a/docs/Tutorials/Threading/Mutexes.md b/docs/Tutorials/Threading/Mutexes.md new file mode 100644 index 000000000..fb97a7ccf --- /dev/null +++ b/docs/Tutorials/Threading/Mutexes.md @@ -0,0 +1,103 @@ +# Mutexes + +Other than [Lockables](./Lockables), Pode also lets you create Mutexes. A Mutex lets you control thread synchronisation across different processes, whether they be child-processes; processes running on the current login session; or every process running on the system. + +## Creating a Mutex + +To create a Mutex in Pode you can use [`New-PodeMutex`](../../../Functions/Threading/New-PodeMutex), this will either create a Mutex or retrieve an existing Mutex if one already exists within the selected scope. + +A Mutex will need a `-Name` but also a `-Scope`. The default scope is Self, but other options are Local or Global: + +| Scope | Description | +| ----- | ----------- | +| Self | The current process, or child processes | +| Local | All processes for the current login session on Windows, or the the same as Self on Unix | +| Global | All processes on the system, across every session | + +The following example will create a new global Mutex: + +```powershell +Start-PodeServer { + New-PodeMutex -Name 'ExampleMutex' -Scope Global +} +``` + +Once created, you can use the Mutex in [`Use-PodeMutex`](../../../Functions/Threading/Use-PodeMutex), [`Enter-PodeMutex`](../../../Functions/Threading/Enter-PodeMutex), and [`Exit-PodeMutex`](../../../Functions/Threading/Exit-PodeMutex). + +## Using a Mutex + +To use a Mutex after creating one you can use [`Use-PodeMutex`](../../../Functions/Threading/Use-PodeMutex) to enter a Mutex; invoke a scriptblock; and then exit a Mutex to free it up for another process. + +Below are 2 scripts for creating 2 Pode servers. Both create the same global Mutex, and then use te Mutex to invoke a scriptblock. In 1 server the scriptblock sleeps for 10 seconds, and in the other for 2 seconds. If you run both servers and call the first server's Route, and then the second, the second will block until the first finishes - even though they're in different servers/threads. + +* First server +```powershell +Start-PodeServer { + Add-PodeEndpoint -Address localhost -Port 8080 -Protocol Http + + New-PodeMutex -Name 'GlobalMutex' -Scope Global + + Add-PodeRoute -Method Get -Path '/sleep' -ScriptBlock { + Use-PodeMutex -Name 'GlobalMutex' -ScriptBlock { + Start-Sleep -Seconds 10 + } + } +} +``` + +* Second server +```powershell +Start-PodeServer { + Add-PodeEndpoint -Address localhost -Port 8081 -Protocol Http + + New-PodeMutex -Name 'GlobalMutex' -Scope Global + + Add-PodeRoute -Method Get -Path '/sleep' -ScriptBlock { + Use-PodeMutex -Name 'GlobalMutex' -ScriptBlock { + Start-Sleep -Seconds 2 + } + } +} +``` + +## Enter / Exit + +You can have more advanced control over the using of Mutexes via [`Enter-PodeMutex`](../../../Functions/Threading/Enter-PodeMutex) and [`Exit-PodeMutex`](../../../Functions/Threading/Exit-PodeMutex). Using these functions you can enter a Mutex, run some logic, and then exit the Mutex - but you don't have to do it all in the same function or scriptblock. + +[`Enter-PodeMutex`](../../../Functions/Threading/Enter-PodeMutex) can be used to initially place enter a Mutex, and then you must call [`Exit-PodeMutex`](../../../Functions/Threading/Exit-PodeMutex) later on to exit the Mutex: + +```powershell +Start-PodeServer -Threads 2 { + Add-PodeEndpoint -Address * -Port 8090 -Protocol Http + + New-PodeMutex -Name 'GlobalMutex' -Scope Global + + Add-PodeRoute -Method Get -Path '/sleep' -ScriptBlock { + try { + Enter-PodeMutex -Name 'GlobalMutex' + Start-Sleep -Seconds 10 + } + finally { + Exit-PodeMutex -Name 'GlobalMutex' + } + } +} +``` + +## Timeout + +When using a Mutex by default Pode will wait indefinitely for the Mutex to be released before entering it. You can use the `-Timeout` parameter to specify a number of milliseconds to timeout after if a Mutex fails be released. + +```powershell +Start-PodeServer -Threads 2 { + Add-PodeEndpoint -Address * -Port 8090 -Protocol Http + + New-PodeMutex -Name 'GlobalMutex' -Scope Global + + Add-PodeRoute -Method Get -Path '/sleep' -ScriptBlock { + Use-PodeMutex -Name 'GlobalMutex' -Timeout 2000 -ScriptBlock { + Start-Sleep -Seconds 2 + } + } +} +``` diff --git a/docs/Tutorials/Threading/Semaphores.md b/docs/Tutorials/Threading/Semaphores.md new file mode 100644 index 000000000..b1226f569 --- /dev/null +++ b/docs/Tutorials/Threading/Semaphores.md @@ -0,0 +1,108 @@ +# Semaphores + +Semaphores are similar to [Mutexes](./Mutexes), in that they let you control thread synchronisation across different processes, whether they be child-processes; processes running on the current login session; or every process running on the system. The difference being that a Semaphore allows you to control the number of processes that can enter the Semaphore at once, unlike a Mutex which is just one at a time. + +!!! note + When the `-Count` for a Semaphore is set to 1 they're basically the same as Mutexes. + +## Creating a Semaphore + +To create a Semaphore in Pode you can use [`New-PodeSemaphore`](../../../Functions/Threading/New-PodeSemaphore), this will either create a Semaphore or retrieve an existing Semaphore if one already exists within the selected scope. + +A Semaphore will need a `-Name` and a `-Scope`. You can also optionally supply a `-Count` which is the number of processes allowed to enter the Semaphore at once - the default be 1. The default scope is Self, but other options are Local or Global: + +| Scope | Description | +| ----- | ----------- | +| Self | The current process, or child processes | +| Local | All processes for the current login session on Windows, or the the same as Self on Unix | +| Global | All processes on the system, across every session | + +The following example will create a new global Semaphore, which allows 2 processes to enter at once: + +```powershell +Start-PodeServer { + New-PodeSemaphore -Name 'ExampleSemaphore' -Scope Global -Count 2 +} +``` + +Once created, you can use the Semaphore in [`Use-PodeSemaphore`](../../../Functions/Threading/Use-PodeSemaphore), [`Enter-PodeSemaphore`](../../../Functions/Threading/Enter-PodeSemaphore), and [`Exit-PodeSemaphore`](../../../Functions/Threading/Exit-PodeSemaphore). + +## Using a Semaphore + +To use a Semaphore after creating one you can use [`Use-PodeSemaphore`](../../../Functions/Threading/Use-PodeSemaphore) to enter a Semaphore; invoke a scriptblock; and then exit a Semaphore to free it up for another process - depending on how many processes can enter the Semaphore at once. + +Below are 2 scripts for creating 2 Pode servers, similar to the example in [Mutexes](./Mutexes#using-a-mutex). Both create the same global Semaphore, and then use te Semaphore to invoke a scriptblock. In 1 server the scriptblock sleeps for 10 seconds, and in the other for 2 seconds. If you run both servers and call the first server's Route, and then the second, the second won't block while the first is still running. However, if you were to call the second server's Route twice in different terminals the first call would return after 2 seconds, but the second call would return after 4 seconds - as only 2 processes can enter the Semaphore at once. + +* First server +```powershell +Start-PodeServer { + Add-PodeEndpoint -Address localhost -Port 8080 -Protocol Http + + New-PodeSemaphore -Name 'GlobalSemaphore' -Scope Global -Count 2 + + Add-PodeRoute -Method Get -Path '/sleep' -ScriptBlock { + Use-PodeSemaphore -Name 'GlobalSemaphore' -ScriptBlock { + Start-Sleep -Seconds 10 + } + } +} +``` + +* Second server +```powershell +Start-PodeServer -Threads 2 { + Add-PodeEndpoint -Address localhost -Port 8081 -Protocol Http + + New-PodeSemaphore -Name 'GlobalSemaphore' -Scope Global -Count 2 + + Add-PodeRoute -Method Get -Path '/sleep' -ScriptBlock { + Use-PodeSemaphore -Name 'GlobalSemaphore' -ScriptBlock { + Start-Sleep -Seconds 2 + } + } +} +``` + +## Enter / Exit + +You can have more advanced control over the using of Semaphores via [`Enter-PodeSemaphore`](../../../Functions/Threading/Enter-PodeSemaphore) and [`Exit-PodeSemaphore`](../../../Functions/Threading/Exit-PodeSemaphore). Using these functions you can enter a Semaphore, run some logic, and then exit the Semaphore - but you don't have to do it all in the same function or scriptblock. + +[`Enter-PodeSemaphore`](../../../Functions/Threading/Enter-PodeSemaphore) can be used to initially place enter a Semaphore, and then you must call [`Exit-PodeSemaphore`](../../../Functions/Threading/Exit-PodeSemaphore) later on to exit the Semaphore: + +```powershell +Start-PodeServer -Threads 2 { + Add-PodeEndpoint -Address * -Port 8090 -Protocol Http + + New-PodeSemaphore -Name 'GlobalSemaphore' -Scope Global -Count 2 + + Add-PodeRoute -Method Get -Path '/sleep' -ScriptBlock { + try { + Enter-PodeSemaphore -Name 'GlobalSemaphore' + Start-Sleep -Seconds 10 + } + finally { + Exit-PodeSemaphore -Name 'GlobalSemaphore' + } + } +} +``` + +[`Exit-PodeSemaphore`](../../../Functions/Threading/Exit-PodeSemaphore) also allows you to exit the Semaphore X number of times via `-ReleaseCount`. + +## Timeout + +When using a Semaphore by default Pode will wait indefinitely for the Semaphore to be released before entering it. You can use the `-Timeout` parameter to specify a number of milliseconds to timeout after if a Semaphore fails be released. + +```powershell +Start-PodeServer -Threads 2 { + Add-PodeEndpoint -Address * -Port 8090 -Protocol Http + + New-PodeSemaphore -Name 'GlobalSemaphore' -Scope Global -Count 2 + + Add-PodeRoute -Method Get -Path '/sleep' -ScriptBlock { + Use-PodeSemaphore -Name 'GlobalSemaphore' -Timeout 2000 -ScriptBlock { + Start-Sleep -Seconds 2 + } + } +} +``` diff --git a/docs/Tutorials/Threading/ServerThreads.md b/docs/Tutorials/Threading/ServerThreads.md new file mode 100644 index 000000000..dd6d0f8ce --- /dev/null +++ b/docs/Tutorials/Threading/ServerThreads.md @@ -0,0 +1,11 @@ +# Server Threads + +By default Pode deals with incoming requests synchronously in a single thread (or runspace). You can increase the number of threads/runspaces that Pode uses to handle requests by using the `-Threads` parameter on [`Start-PodeServer`](../../../Functions/Core/Start-PodeServer): + +```powershell +Start-PodeServer -Threads 2 { + # logic +} +``` + +The number of threads supplied only applies to Web, SMTP, and TCP servers. If `-Threads` is not supplied, or is <=0 then the number of threads is forced to the default of 1. From 427821d7810697205e3cd17d722272137c0ecb20 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Thu, 19 Jan 2023 19:31:32 +0000 Subject: [PATCH 42/52] #1054: PlatyPS doesnt like HTML in Examples, so replace with a plain comment --- pode.build.ps1 | 2 +- src/Public/Responses.ps1 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pode.build.ps1 b/pode.build.ps1 index fdf269070..7ecf744cd 100644 --- a/pode.build.ps1 +++ b/pode.build.ps1 @@ -15,7 +15,7 @@ $Versions = @{ DotNet = '7.0.1' Checksum = '0.2.0' MkDocsTheme = '9.0.2' - PlatyPS = '0.14.0' + PlatyPS = '0.14.2' } <# diff --git a/src/Public/Responses.ps1 b/src/Public/Responses.ps1 index 7592826d0..8f55dad3a 100644 --- a/src/Public/Responses.ps1 +++ b/src/Public/Responses.ps1 @@ -539,7 +539,7 @@ The path to a HTML file. The status code to set against the response. .EXAMPLE -Write-PodeHtmlResponse -Value 'Hello!' +Write-PodeHtmlResponse -Value "Raw HTML can be placed here" .EXAMPLE Write-PodeHtmlResponse -Value @{ Message = 'Hello, all!' } From 9339aeec2aa2599cfe3a794e96b7a7aba5383e5e Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Thu, 19 Jan 2023 20:34:42 +0000 Subject: [PATCH 43/52] #996: allow -Method on Add-PodeRoute to support handling multiple methods --- docs/Tutorials/Routes/Overview.md | 3 + examples/web-pages.ps1 | 4 +- src/Public/Routes.ps1 | 137 ++++++++++++++++-------------- 3 files changed, 78 insertions(+), 66 deletions(-) diff --git a/docs/Tutorials/Routes/Overview.md b/docs/Tutorials/Routes/Overview.md index cc31a23a2..e8584407f 100644 --- a/docs/Tutorials/Routes/Overview.md +++ b/docs/Tutorials/Routes/Overview.md @@ -32,6 +32,9 @@ Here, anyone who calls `http://localhost:8080/ping` will receive the following r } ``` +!!! tip + You can supply more than 1 `-Method` if required, such as: `-Method Get, Post` + The scriptblock for the route will have access to the `$WebEvent` variable which contains information about the current [web event](../../WebEvent). This argument will contain the `Request` and `Response` objects, `Data` (from POST), and the `Query` (from the query string of the URL), as well as any `Parameters` from the route itself (eg: `/:accountId`). You can add your routes straight into the [`Start-PodeServer`](../../../Functions/Core/Start-PodeServer) scriptblock, or separate them into different files. These files can then be dot-sourced, or you can use [`Use-PodeRoutes`](../../../Functions/Routes/Use-PodeRoutes) to automatically load all ps1 files within a `/routes` directory at the root of your server. diff --git a/examples/web-pages.ps1 b/examples/web-pages.ps1 index 9d774582a..90fbf799e 100644 --- a/examples/web-pages.ps1 +++ b/examples/web-pages.ps1 @@ -86,8 +86,8 @@ Start-PodeServer -Threads 2 -Verbose { Set-PodeResponseAttachment -Path 'Anger.jpg' } - # GET request with parameters - Add-PodeRoute -Method Get -Path '/:userId/details' -ScriptBlock { + # GET and POST request with parameters + Add-PodeRoute -Method Get, Post -Path '/:userId/details' -ScriptBlock { Write-PodeJsonResponse -Value @{ 'userId' = $WebEvent.Parameters['userId'] } } diff --git a/src/Public/Routes.ps1 b/src/Public/Routes.ps1 index 1ec651f49..775143e43 100644 --- a/src/Public/Routes.ps1 +++ b/src/Public/Routes.ps1 @@ -1,12 +1,12 @@ <# .SYNOPSIS -Adds a Route for a specific HTTP Method. +Adds a Route for a specific HTTP Method(s). .DESCRIPTION -Adds a Route for a specific HTTP Method, with path, that when called with invoke any logic and/or Middleware. +Adds a Route for a specific HTTP Method(s), with path, that when called with invoke any logic and/or Middleware. .PARAMETER Method -The HTTP Method of this Route. +The HTTP Method of this Route, multiple can be supplied. .PARAMETER Path The URI path for the Route. @@ -77,7 +77,7 @@ function Add-PodeRoute param( [Parameter(Mandatory=$true)] [ValidateSet('Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] - [string] + [string[]] $Method, [Parameter(Mandatory=$true)] @@ -174,13 +174,16 @@ function Add-PodeRoute } } + # var for new routes created + $newRoutes = @() + # store the original path $origPath = $Path # split route on '?' for query $Path = Split-PodeRouteQuery -Path $Path if ([string]::IsNullOrWhiteSpace($Path)) { - throw "[$($Method)]: No Path supplied for Route" + throw "No Path supplied for Route" } # ensure the route has appropriate slashes @@ -200,30 +203,9 @@ function Add-PodeRoute $IfExists = Get-PodeRouteIfExistsPreference } - # ensure the route doesn't already exist for each endpoint - $endpoints = @(foreach ($_endpoint in $endpoints) { - $found = Test-PodeRouteInternal -Method $Method -Path $Path -Protocol $_endpoint.Protocol -Address $_endpoint.Address -ThrowError:($IfExists -ieq 'Error') - - if ($found) { - if ($IfExists -ieq 'Overwrite') { - Remove-PodeRoute -Method $Method -Path $origPath -EndpointName $_endpoint.Name - } - - if ($IfExists -ieq 'Skip') { - continue - } - } - - $_endpoint - }) - - if (($null -eq $endpoints) -or ($endpoints.Length -eq 0)) { - return - } - # if middleware, scriptblock and file path are all null/empty, error if ((Test-PodeIsEmpty $Middleware) -and (Test-PodeIsEmpty $ScriptBlock) -and (Test-PodeIsEmpty $FilePath) -and (Test-PodeIsEmpty $Authentication)) { - throw "[$($Method)] $($Path): No logic passed" + throw "No logic passed for Route: $($Path)" } # if we have a file path supplied, load that path as a scriptblock @@ -259,51 +241,78 @@ function Add-PodeRoute # workout a default transfer encoding for the route $TransferEncoding = Find-PodeRouteTransferEncoding -Path $Path -TransferEncoding $TransferEncoding - # add the route(s) - Write-Verbose "Adding Route: [$($Method)] $($Path)" - $newRoutes = @(foreach ($_endpoint in $endpoints) { - @{ - Logic = $ScriptBlock - UsingVariables = $usingVars - Middleware = $Middleware - Authentication = $Authentication - Endpoint = @{ - Protocol = $_endpoint.Protocol - Address = $_endpoint.Address.Trim() - Name = $_endpoint.Name - } - ContentType = $ContentType - TransferEncoding = $TransferEncoding - ErrorType = $ErrorContentType - Arguments = $ArgumentList - Method = $Method - Path = $Path - OpenApi = @{ - Path = $OpenApiPath - Responses = @{ - '200' = @{ description = 'OK' } - 'default' = @{ description = 'Internal server error' } + # loop through each method + foreach ($_method in $Method) { + # ensure the route doesn't already exist for each endpoint + $endpoints = @(foreach ($_endpoint in $endpoints) { + $found = Test-PodeRouteInternal -Method $_method -Path $Path -Protocol $_endpoint.Protocol -Address $_endpoint.Address -ThrowError:($IfExists -ieq 'Error') + + if ($found) { + if ($IfExists -ieq 'Overwrite') { + Remove-PodeRoute -Method $_method -Path $origPath -EndpointName $_endpoint.Name + } + + if ($IfExists -ieq 'Skip') { + continue } - Parameters = $null - RequestBody = $null - Authentication = @() } - IsStatic = $false - Metrics = @{ - Requests = @{ - Total = 0 - StatusCodes = @{} + + $_endpoint + }) + + if (($null -eq $endpoints) -or ($endpoints.Length -eq 0)) { + continue + } + + # add the route(s) + Write-Verbose "Adding Route: [$($_method)] $($Path)" + $methodRoutes = @(foreach ($_endpoint in $endpoints) { + @{ + Logic = $ScriptBlock + UsingVariables = $usingVars + Middleware = $Middleware + Authentication = $Authentication + Endpoint = @{ + Protocol = $_endpoint.Protocol + Address = $_endpoint.Address.Trim() + Name = $_endpoint.Name + } + ContentType = $ContentType + TransferEncoding = $TransferEncoding + ErrorType = $ErrorContentType + Arguments = $ArgumentList + Method = $_method + Path = $Path + OpenApi = @{ + Path = $OpenApiPath + Responses = @{ + '200' = @{ description = 'OK' } + 'default' = @{ description = 'Internal server error' } + } + Parameters = $null + RequestBody = $null + Authentication = @() + } + IsStatic = $false + Metrics = @{ + Requests = @{ + Total = 0 + StatusCodes = @{} + } } } + }) + + if (![string]::IsNullOrWhiteSpace($Authentication)) { + Set-PodeOAAuth -Route $methodRoutes -Name $Authentication } - }) - if (![string]::IsNullOrWhiteSpace($Authentication)) { - Set-PodeOAAuth -Route $newRoutes -Name $Authentication + $PodeContext.Server.Routes[$_method][$Path] += @($methodRoutes) + if ($PassThru) { + $newRoutes += $methodRoutes + } } - $PodeContext.Server.Routes[$Method][$Path] += @($newRoutes) - # return the routes? if ($PassThru) { return $newRoutes From 917fe429ce7ce714c0bd9390df41c0ef25900ff1 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Thu, 19 Jan 2023 21:03:25 +0000 Subject: [PATCH 44/52] #1071: add support for the CONNECT http method --- docs/Tutorials/Routes/Overview.md | 2 +- src/Listener/PodeHelpers.cs | 2 +- src/Private/Context.ps1 | 1 + src/Private/Middleware.ps1 | 2 +- src/Private/Routes.ps1 | 4 ++-- src/Public/Middleware.ps1 | 4 ++-- src/Public/Routes.ps1 | 12 ++++++------ src/Public/Security.ps1 | 2 +- 8 files changed, 15 insertions(+), 14 deletions(-) diff --git a/docs/Tutorials/Routes/Overview.md b/docs/Tutorials/Routes/Overview.md index e8584407f..9fbcae868 100644 --- a/docs/Tutorials/Routes/Overview.md +++ b/docs/Tutorials/Routes/Overview.md @@ -8,7 +8,7 @@ Routes can also be bound against a specific protocol or endpoint. This allows yo !!! info The following HTTP methods are supported by routes in Pode: - DELETE, GET, HEAD, MERGE, OPTIONS, PATCH, POST, PUT, and TRACE. + CONNECT, DELETE, GET, HEAD, MERGE, OPTIONS, PATCH, POST, PUT, and TRACE. ## Usage diff --git a/src/Listener/PodeHelpers.cs b/src/Listener/PodeHelpers.cs index 64e600cd3..bb68f6742 100644 --- a/src/Listener/PodeHelpers.cs +++ b/src/Listener/PodeHelpers.cs @@ -10,7 +10,7 @@ namespace Pode { public class PodeHelpers { - public static readonly string[] HTTP_METHODS = new string[] { "DELETE", "GET", "HEAD", "MERGE", "OPTIONS", "PATCH", "POST", "PUT", "TRACE" }; + public static readonly string[] HTTP_METHODS = new string[] { "CONNECT", "DELETE", "GET", "HEAD", "MERGE", "OPTIONS", "PATCH", "POST", "PUT", "TRACE" }; public const string WEB_SOCKET_MAGIC_KEY = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"; public readonly static char[] NEW_LINE_ARRAY = new char[] { '\r', '\n' }; public const string NEW_LINE = "\r\n"; diff --git a/src/Private/Context.ps1 b/src/Private/Context.ps1 index f6cceed77..5a8de9d18 100644 --- a/src/Private/Context.ps1 +++ b/src/Private/Context.ps1 @@ -278,6 +278,7 @@ function New-PodeContext # routes for pages and api $ctx.Server.Routes = @{ + 'connect' = [ordered]@{} 'delete' = [ordered]@{} 'get' = [ordered]@{} 'head' = [ordered]@{} diff --git a/src/Private/Middleware.ps1 b/src/Private/Middleware.ps1 index 6c36e0b9d..59b41ccb0 100644 --- a/src/Private/Middleware.ps1 +++ b/src/Private/Middleware.ps1 @@ -218,7 +218,7 @@ function Get-PodeRouteValidateMiddleware # if there's no route defined, it's a 404 - or a 405 if a route exists for any other method if ($null -eq $route) { # check if a route exists for another method - $methods = @('DELETE', 'GET', 'HEAD', 'MERGE', 'OPTIONS', 'PATCH', 'POST', 'PUT', 'TRACE') + $methods = @('CONNECT', 'DELETE', 'GET', 'HEAD', 'MERGE', 'OPTIONS', 'PATCH', 'POST', 'PUT', 'TRACE') $diff_route = @(foreach ($method in $methods) { $r = Find-PodeRoute -Method $method -Path $WebEvent.Path -EndpointName $WebEvent.Endpoint.Name if ($null -ne $r) { diff --git a/src/Private/Routes.ps1 b/src/Private/Routes.ps1 index 1b9229ead..c46ea6d8e 100644 --- a/src/Private/Routes.ps1 +++ b/src/Private/Routes.ps1 @@ -2,7 +2,7 @@ function Test-PodeRouteFromRequest { param( [Parameter(Mandatory=$true)] - [ValidateSet('DELETE', 'GET', 'HEAD', 'MERGE', 'OPTIONS', 'PATCH', 'POST', 'PUT', 'TRACE', 'STATIC', 'SIGNAL', '*')] + [ValidateSet('CONNECT', 'DELETE', 'GET', 'HEAD', 'MERGE', 'OPTIONS', 'PATCH', 'POST', 'PUT', 'TRACE', 'STATIC', 'SIGNAL', '*')] [string] $Method, @@ -27,7 +27,7 @@ function Find-PodeRoute { param( [Parameter(Mandatory=$true)] - [ValidateSet('DELETE', 'GET', 'HEAD', 'MERGE', 'OPTIONS', 'PATCH', 'POST', 'PUT', 'TRACE', 'STATIC', 'SIGNAL', '*')] + [ValidateSet('CONNECT', 'DELETE', 'GET', 'HEAD', 'MERGE', 'OPTIONS', 'PATCH', 'POST', 'PUT', 'TRACE', 'STATIC', 'SIGNAL', '*')] [string] $Method, diff --git a/src/Public/Middleware.ps1 b/src/Public/Middleware.ps1 index f3b2bddb8..b7a2b2e0e 100644 --- a/src/Public/Middleware.ps1 +++ b/src/Public/Middleware.ps1 @@ -228,7 +228,7 @@ function Initialize-PodeCsrf [CmdletBinding()] param ( [Parameter()] - [ValidateSet('Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace')] + [ValidateSet('Connect', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace')] [string[]] $IgnoreMethods = @('Get', 'Head', 'Options', 'Trace'), @@ -295,7 +295,7 @@ function Enable-PodeCsrfMiddleware [CmdletBinding()] param ( [Parameter()] - [ValidateSet('Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace')] + [ValidateSet('Connect', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace')] [string[]] $IgnoreMethods = @('Get', 'Head', 'Options', 'Trace'), diff --git a/src/Public/Routes.ps1 b/src/Public/Routes.ps1 index 775143e43..76ed89814 100644 --- a/src/Public/Routes.ps1 +++ b/src/Public/Routes.ps1 @@ -76,7 +76,7 @@ function Add-PodeRoute [CmdletBinding(DefaultParameterSetName='Script')] param( [Parameter(Mandatory=$true)] - [ValidateSet('Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] + [ValidateSet('Connect', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] [string[]] $Method, @@ -1214,7 +1214,7 @@ function Remove-PodeRoute [CmdletBinding()] param( [Parameter(Mandatory=$true)] - [ValidateSet('Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] + [ValidateSet('Connect', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] [string] $Method, @@ -1374,7 +1374,7 @@ function Clear-PodeRoutes [CmdletBinding()] param( [Parameter()] - [ValidateSet('', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] + [ValidateSet('', 'Connect', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] [string] $Method ) @@ -1484,7 +1484,7 @@ function ConvertTo-PodeRoute $Module, [Parameter()] - [ValidateSet('', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace')] + [ValidateSet('', 'Connect', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace')] [string] $Method, @@ -1813,7 +1813,7 @@ function Get-PodeRoute [CmdletBinding()] param( [Parameter()] - [ValidateSet('', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] + [ValidateSet('', 'Connect', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] [string] $Method, @@ -2104,7 +2104,7 @@ function Test-PodeRoute [CmdletBinding()] param( [Parameter(Mandatory=$true)] - [ValidateSet('Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] + [ValidateSet('Connect', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] [string] $Method, diff --git a/src/Public/Security.ps1 b/src/Public/Security.ps1 index 888468720..04c233375 100644 --- a/src/Public/Security.ps1 +++ b/src/Public/Security.ps1 @@ -1347,7 +1347,7 @@ function Set-PodeSecurityAccessControl $Origin, [Parameter()] - [ValidateSet('', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] + [ValidateSet('', 'Connect', 'Delete', 'Get', 'Head', 'Merge', 'Options', 'Patch', 'Post', 'Put', 'Trace', '*')] [string[]] $Methods = '', From 8f67ac367dcfdffa9211084b63dbcf0d63d28066 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sat, 21 Jan 2023 20:48:35 +0000 Subject: [PATCH 45/52] #1046: fix module loading order, and importing local pode module --- src/Private/AutoImport.ps1 | 16 +++-- src/Private/Context.ps1 | 6 +- src/Private/Helpers.ps1 | 124 +++++++++++++++++++++++++++++++++---- 3 files changed, 126 insertions(+), 20 deletions(-) diff --git a/src/Private/AutoImport.ps1 b/src/Private/AutoImport.ps1 index ba311f2da..49ba1a8cd 100644 --- a/src/Private/AutoImport.ps1 +++ b/src/Private/AutoImport.ps1 @@ -64,26 +64,32 @@ function Import-PodeModulesIntoRunspaceState return } - # load modules into runspaces, if allowed + # get modules currently loaded in session $modules = Get-Module | Where-Object { ($_.Name -ine 'pode') -and ($_.Name -inotlike 'microsoft.powershell.*') - } | Sort-Object -Unique + } | Select-Object -Unique + + # work out which order the modules need to be loaded + $modulesOrder = @(foreach ($module in $modules) { + Get-PodeModuleDependencies -Module $module + }) | Select-Object -Unique - foreach ($module in $modules) { + # load modules into runspaces, if allowed + foreach ($module in $modulesOrder) { # only exported modules? is the module exported? if ($PodeContext.Server.AutoImport.Modules.ExportOnly -and ($PodeContext.Server.AutoImport.Modules.ExportList -inotcontains $module.Name)) { continue } # import the module - $path = Find-PodeModuleFile -Name $module.Name + $path = Find-PodeModuleFile -Module $module if (($module.ModuleType -ieq 'Manifest') -or ($path.EndsWith('.ps1'))) { $PodeContext.RunspaceState.ImportPSModule($path) } else { - $PodeContext.Server.Modules[$module] = $path + $PodeContext.Server.Modules[$module.Name] = $path } } } diff --git a/src/Private/Context.ps1 b/src/Private/Context.ps1 index 5a8de9d18..a7280194c 100644 --- a/src/Private/Context.ps1 +++ b/src/Private/Context.ps1 @@ -88,7 +88,7 @@ function New-PodeContext $ctx.Server.Logic = $ScriptBlock $ctx.Server.LogicPath = $FilePath $ctx.Server.Interval = $Interval - $ctx.Server.PodeModulePath = (Get-PodeModulePath) + $ctx.Server.PodeModule = (Get-PodeModuleDetails) $ctx.Server.DisableTermination = $DisableTermination.IsPresent $ctx.Server.Quiet = $Quiet.IsPresent $ctx.Server.ComputerName = [System.Net.DNS]::GetHostName() @@ -411,7 +411,7 @@ function New-PodeContext } # modules - $ctx.Server.Modules = @{} + $ctx.Server.Modules = [ordered]@{} # setup security $ctx.Server.Security = @{ @@ -430,7 +430,7 @@ function New-PodeRunspaceState { # create the state, and add the pode module $state = [initialsessionstate]::CreateDefault() - $state.ImportPSModule($PodeContext.Server.PodeModulePath) + $state.ImportPSModule($PodeContext.Server.PodeModule.Path) # load the vars into the share state $session = New-PodeStateContext -Context $PodeContext diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index 58d6b42bb..c78575b33 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -928,6 +928,12 @@ function Add-PodePSDrives function Import-PodeModules { + # force re-import pode via psd1 if a local version of pode is being used - to fix the "version" + if (!$PodeContext.Server.PodeModule.InPath -and ![string]::IsNullOrEmpty($PodeContext.Server.PodeModule.DataPath) -and ((Get-Module -Name Pode).Version -ieq '0.0')) { + $null = Import-Module $PodeContext.Server.PodeModule.DataPath -DisableNameChecking -Scope Global -Force -ErrorAction Stop + } + + # import other modules in the session foreach ($path in $PodeContext.Server.Modules.Values) { $null = Import-Module $path -DisableNameChecking -Scope Global -ErrorAction Stop } @@ -1903,35 +1909,96 @@ function ConvertTo-PodeSslProtocols return [System.Security.Authentication.SslProtocols]($protos) } -function Get-PodeModulePath +function Get-PodeModuleDetails { # if there's 1 module imported already, use that $importedModule = @(Get-Module -Name Pode) if (($importedModule | Measure-Object).Count -eq 1) { - return (@($importedModule)[0]).Path + return (Convert-PodeModuleDetails -Module @($importedModule)[0]) } # if there's none or more, attempt to get the module used for 'engine' try { $usedModule = (Get-Command -Name 'Set-PodeViewEngine').Module if (($usedModule | Measure-Object).Count -eq 1) { - return $usedModule.Path + return (Convert-PodeModuleDetails -Module $usedModule) } } - catch { } + catch {} # if there were multiple to begin with, use the newest version if (($importedModule | Measure-Object).Count -gt 1) { - return (@($importedModule | Sort-Object -Property Version)[-1]).Path + return (Convert-PodeModuleDetails -Module @($importedModule | Sort-Object -Property Version)[-1]) } # otherwise there were none, use the latest installed - return (@(Get-Module -ListAvailable -Name Pode | Sort-Object -Property Version)[-1]).Path + return (Convert-PodeModuleDetails -Module @(Get-Module -ListAvailable -Name Pode | Sort-Object -Property Version)[-1]) +} + +function Convert-PodeModuleDetails +{ + param( + [Parameter(Mandatory=$true)] + [psmoduleinfo] + $Module + ) + + return @{ + Name = $Module.Name + Path = $Module.Path + BasePath = $Module.ModuleBase + DataPath = (Find-PodeModuleFile -Module $Module -DataOnly -CheckVersion) + InPath = (Test-PodeModuleInPath -Module $Module) + } +} + +function Test-PodeModuleInPath +{ + param( + [Parameter(Mandatory=$true)] + [psmoduleinfo] + $Module + ) + + $separator = ';' + if (Test-PodeIsUnix) { + $separator = ':' + } + + $paths = @($env:PSModulePath -split $separator) + + foreach ($path in $paths) { + if ($Module.Path.StartsWith($path)) { + return $true + } + } + + return $false +} + +function Get-PodeModuleDependencies +{ + param( + [Parameter(Mandatory=$true)] + [psmoduleinfo] + $Module + ) + + if (!$Module.RequiredModules) { + return $Module + } + + $mods = @() + foreach ($mod in $Module.RequiredModules) { + $mods += (Get-PodeModuleDependencies -Module $mod) + } + + return ($mods + $module) } function Get-PodeModuleRootPath { - return (Split-Path -Parent -Path $PodeContext.Server.PodeModulePath) + return (Split-Path -Parent -Path $PodeContext.Server.PodeModule.Path) } function Get-PodeModuleMiscPath @@ -2967,21 +3034,54 @@ function Use-PodeFolder function Find-PodeModuleFile { param( - [Parameter(Mandatory=$true)] + [Parameter(Mandatory=$true, ParameterSetName='Name')] [string] $Name, + [Parameter(Mandatory=$true, ParameterSetName='Module')] + [psmoduleinfo] + $Module, + + [switch] + $ListAvailable, + [switch] - $ListAvailable + $DataOnly, + + [switch] + $CheckVersion ) # get module and check psd1, then psm1 - $mod = (Get-Module -Name $Name -ListAvailable:$ListAvailable | Sort-Object -Property Version -Descending | Select-Object -First 1) + if ($null -eq $Module) { + $Module = (Get-Module -Name $Name -ListAvailable:$ListAvailable | Sort-Object -Property Version -Descending | Select-Object -First 1) + } # if the path isn't already a psd1 do this - $path = Join-Path $mod.ModuleBase "$($mod.Name).psd1" + $path = Join-Path $Module.ModuleBase "$($Module.Name).psd1" if (!(Test-Path $path)) { - $path = $mod.Path + # if we only want a psd1, return null + if ($DataOnly) { + $path = $null + } + else { + $path = $Module.Path + } + } + + # check the Version of the psd1 + elseif ($CheckVersion) { + $data = Import-PowerShellDataFile -Path $path -ErrorAction Stop + + $version = $null + if (![version]::TryParse($data.ModuleVersion, [ref]$version)) { + if ($DataOnly) { + $path = $null + } + else { + $path = $Module.Path + } + } } return $path From 35166fda454489afa41f3d7904db497e1e8305d9 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Sat, 21 Jan 2023 20:53:08 +0000 Subject: [PATCH 46/52] #1046: re-filter module list for pode/microsoft modules --- src/Private/AutoImport.ps1 | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/Private/AutoImport.ps1 b/src/Private/AutoImport.ps1 index 49ba1a8cd..b0d7fc6a7 100644 --- a/src/Private/AutoImport.ps1 +++ b/src/Private/AutoImport.ps1 @@ -73,7 +73,10 @@ function Import-PodeModulesIntoRunspaceState # work out which order the modules need to be loaded $modulesOrder = @(foreach ($module in $modules) { Get-PodeModuleDependencies -Module $module - }) | Select-Object -Unique + }) | + Where-Object { + ($_.Name -ine 'pode') -and ($_.Name -inotlike 'microsoft.powershell.*') + } | Select-Object -Unique # load modules into runspaces, if allowed foreach ($module in $modulesOrder) { From d257506e52ce49d3c5c99d18e8c3fec61986a58e Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Wed, 25 Jan 2023 18:29:02 +0000 Subject: [PATCH 47/52] #1046: add a Pode.Internal module/data file to load all functions in runspaces --- src/Pode.Internal.psd1 | 24 ++++++++++++++++++++++ src/Pode.Internal.psm1 | 17 +++++++++++++++ src/Pode.psm1 | 42 ++++++++++++++++++++------------------ src/Private/AutoImport.ps1 | 4 ++-- src/Private/Context.ps1 | 5 +++-- src/Private/Helpers.ps1 | 13 ++++++------ 6 files changed, 74 insertions(+), 31 deletions(-) create mode 100644 src/Pode.Internal.psd1 create mode 100644 src/Pode.Internal.psm1 diff --git a/src/Pode.Internal.psd1 b/src/Pode.Internal.psd1 new file mode 100644 index 000000000..1e295a37f --- /dev/null +++ b/src/Pode.Internal.psd1 @@ -0,0 +1,24 @@ +# +# Internal module manifest for module 'Pode' +# +# Generated by: Matthew Kelly (Badgerati) +# +# Generated on: 24/01/2023 +# + +@{ + # Script module or binary module file associated with this manifest. + RootModule = 'Pode.Internal.psm1' + + # Version number of this module. + ModuleVersion = '$version$' + + # ID used to uniquely identify this module + GUID = '86b48c1c-8b59-4f3c-80bb-936d6b3218f6' + + # Author of this module + Author = 'Matthew Kelly (Badgerati)' + + # Minimum version of the Windows PowerShell engine required by this module + PowerShellVersion = '5.0' +} \ No newline at end of file diff --git a/src/Pode.Internal.psm1 b/src/Pode.Internal.psm1 new file mode 100644 index 000000000..dcb516be0 --- /dev/null +++ b/src/Pode.Internal.psm1 @@ -0,0 +1,17 @@ +# root path +$root = Split-Path -Parent -Path $MyInvocation.MyCommand.Path + +# import everything +$sysfuncs = Get-ChildItem Function: + +# load private functions +Get-ChildItem "$($root)/Private/*.ps1" | ForEach-Object { . ([System.IO.Path]::GetFullPath($_)) } + +# load public functions +Get-ChildItem "$($root)/Public/*.ps1" | ForEach-Object { . ([System.IO.Path]::GetFullPath($_)) } + +# get functions from memory and compare to existing to find new functions added +$funcs = Get-ChildItem Function: | Where-Object { $sysfuncs -notcontains $_ } + +# export the module's public functions +Export-ModuleMember -Function ($funcs.Name) \ No newline at end of file diff --git a/src/Pode.psm1 b/src/Pode.psm1 index 702715c79..156cdf53b 100644 --- a/src/Pode.psm1 +++ b/src/Pode.psm1 @@ -1,35 +1,37 @@ # root path $root = Split-Path -Parent -Path $MyInvocation.MyCommand.Path -# load binaries -Add-Type -AssemblyName System.Web -Add-Type -AssemblyName System.Net.Http +# load assemblies +$assemblies = [System.AppDomain]::CurrentDomain.GetAssemblies().Location -# netstandard2 for <7.2 -if ($PSVersionTable.PSVersion -lt [version]'7.2.0') { - Add-Type -LiteralPath "$($root)/Libs/netstandard2.0/Pode.dll" -ErrorAction Stop +if (($assemblies -ilike '*System.Web.dll*').Length -eq 0) { + Add-Type -AssemblyName System.Web } -# net6 for =7.2 -elseif ($PSVersionTable.PSVersion -lt [version]'7.3.0') { - Add-Type -LiteralPath "$($root)/Libs/net6.0/Pode.dll" -ErrorAction Stop -} -# net7 for >7.2 -else { - Add-Type -LiteralPath "$($root)/Libs/net7.0/Pode.dll" -ErrorAction Stop + +if (($assemblies -ilike '*System.Net.Http.dll*').Length -eq 0) { + Add-Type -AssemblyName System.Net.Http } -# import everything if in a runspace -if ($PODE_SCOPE_RUNSPACE) { - $sysfuncs = Get-ChildItem Function: +if (($assemblies -ilike '*Pode.dll*').Length -eq 0) { + # netstandard2 for <7.2 + if ($PSVersionTable.PSVersion -lt [version]'7.2.0') { + Add-Type -LiteralPath "$($root)/Libs/netstandard2.0/Pode.dll" -ErrorAction Stop + } + # net6 for =7.2 + elseif ($PSVersionTable.PSVersion -lt [version]'7.3.0') { + Add-Type -LiteralPath "$($root)/Libs/net6.0/Pode.dll" -ErrorAction Stop + } + # net7 for >7.2 + else { + Add-Type -LiteralPath "$($root)/Libs/net7.0/Pode.dll" -ErrorAction Stop + } } # load private functions Get-ChildItem "$($root)/Private/*.ps1" | ForEach-Object { . ([System.IO.Path]::GetFullPath($_)) } -# only import public functions if not in a runspace -if (!$PODE_SCOPE_RUNSPACE) { - $sysfuncs = Get-ChildItem Function: -} +# only import public functions +$sysfuncs = Get-ChildItem Function: # load public functions Get-ChildItem "$($root)/Public/*.ps1" | ForEach-Object { . ([System.IO.Path]::GetFullPath($_)) } diff --git a/src/Private/AutoImport.ps1 b/src/Private/AutoImport.ps1 index b0d7fc6a7..d2d9def93 100644 --- a/src/Private/AutoImport.ps1 +++ b/src/Private/AutoImport.ps1 @@ -67,7 +67,7 @@ function Import-PodeModulesIntoRunspaceState # get modules currently loaded in session $modules = Get-Module | Where-Object { - ($_.Name -ine 'pode') -and ($_.Name -inotlike 'microsoft.powershell.*') + ($_.Name -inotin @('pode', 'pode.internal')) -and ($_.Name -inotlike 'microsoft.powershell.*') } | Select-Object -Unique # work out which order the modules need to be loaded @@ -75,7 +75,7 @@ function Import-PodeModulesIntoRunspaceState Get-PodeModuleDependencies -Module $module }) | Where-Object { - ($_.Name -ine 'pode') -and ($_.Name -inotlike 'microsoft.powershell.*') + ($_.Name -inotin @('pode', 'pode.internal')) -and ($_.Name -inotlike 'microsoft.powershell.*') } | Select-Object -Unique # load modules into runspaces, if allowed diff --git a/src/Private/Context.ps1 b/src/Private/Context.ps1 index a7280194c..515577feb 100644 --- a/src/Private/Context.ps1 +++ b/src/Private/Context.ps1 @@ -428,9 +428,10 @@ function New-PodeContext function New-PodeRunspaceState { - # create the state, and add the pode module + # create the state, and add the pode modules $state = [initialsessionstate]::CreateDefault() - $state.ImportPSModule($PodeContext.Server.PodeModule.Path) + $state.ImportPSModule($PodeContext.Server.PodeModule.DataPath) + $state.ImportPSModule($PodeContext.Server.PodeModule.InternalPath) # load the vars into the share state $session = New-PodeStateContext -Context $PodeContext diff --git a/src/Private/Helpers.ps1 b/src/Private/Helpers.ps1 index c78575b33..baefbede3 100644 --- a/src/Private/Helpers.ps1 +++ b/src/Private/Helpers.ps1 @@ -928,11 +928,6 @@ function Add-PodePSDrives function Import-PodeModules { - # force re-import pode via psd1 if a local version of pode is being used - to fix the "version" - if (!$PodeContext.Server.PodeModule.InPath -and ![string]::IsNullOrEmpty($PodeContext.Server.PodeModule.DataPath) -and ((Get-Module -Name Pode).Version -ieq '0.0')) { - $null = Import-Module $PodeContext.Server.PodeModule.DataPath -DisableNameChecking -Scope Global -Force -ErrorAction Stop - } - # import other modules in the session foreach ($path in $PodeContext.Server.Modules.Values) { $null = Import-Module $path -DisableNameChecking -Scope Global -ErrorAction Stop @@ -1943,13 +1938,17 @@ function Convert-PodeModuleDetails $Module ) - return @{ + $details = @{ Name = $Module.Name Path = $Module.Path BasePath = $Module.ModuleBase - DataPath = (Find-PodeModuleFile -Module $Module -DataOnly -CheckVersion) + DataPath = (Find-PodeModuleFile -Module $Module -CheckVersion) + InternalPath = $null InPath = (Test-PodeModuleInPath -Module $Module) } + + $details.InternalPath = $details.DataPath -ireplace 'Pode\.(ps[md]1)', 'Pode.Internal.$1' + return $details } function Test-PodeModuleInPath From 9de8799e2cbe3a88060b84ede2c8b6a62aeffad3 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Fri, 27 Jan 2023 15:42:35 +0000 Subject: [PATCH 48/52] #1046: update packaging to reference new Internal module --- packers/choco/tools/ChocolateyInstall.ps1 | 2 ++ pode.build.ps1 | 5 ++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/packers/choco/tools/ChocolateyInstall.ps1 b/packers/choco/tools/ChocolateyInstall.ps1 index 14adbfb1a..70babe97c 100644 --- a/packers/choco/tools/ChocolateyInstall.ps1 +++ b/packers/choco/tools/ChocolateyInstall.ps1 @@ -38,6 +38,8 @@ function Install-PodeModule($path, $version) # copy general files Copy-Item -Path ./Pode.psm1 -Destination $path -Force | Out-Null Copy-Item -Path ./Pode.psd1 -Destination $path -Force | Out-Null + Copy-Item -Path ./Pode.Internal.psm1 -Destination $path -Force | Out-Null + Copy-Item -Path ./Pode.Internal.psd1 -Destination $path -Force | Out-Null Copy-Item -Path ./LICENSE.txt -Destination $path -Force | Out-Null } finally { diff --git a/pode.build.ps1 b/pode.build.ps1 index 7ecf744cd..169caca37 100644 --- a/pode.build.ps1 +++ b/pode.build.ps1 @@ -116,6 +116,7 @@ function Invoke-PodeBuildDotnetBuild($target) # Synopsis: Stamps the version onto the Module task StampVersion { (Get-Content ./pkg/Pode.psd1) | ForEach-Object { $_ -replace '\$version\$', $Version } | Set-Content ./pkg/Pode.psd1 + (Get-Content ./pkg/Pode.Internal.psd1) | ForEach-Object { $_ -replace '\$version\$', $Version } | Set-Content ./pkg/Pode.Internal.psd1 (Get-Content ./packers/choco/pode.nuspec) | ForEach-Object { $_ -replace '\$version\$', $Version } | Set-Content ./packers/choco/pode.nuspec (Get-Content ./packers/choco/tools/ChocolateyInstall.ps1) | ForEach-Object { $_ -replace '\$version\$', $Version } | Set-Content ./packers/choco/tools/ChocolateyInstall.ps1 } @@ -268,6 +269,8 @@ task Pack -If (Test-PodeBuildIsWindows) Build, { # copy general files Copy-Item -Path ./src/Pode.psm1 -Destination $path -Force | Out-Null Copy-Item -Path ./src/Pode.psd1 -Destination $path -Force | Out-Null + Copy-Item -Path ./src/Pode.Internal.psm1 -Destination $path -Force | Out-Null + Copy-Item -Path ./src/Pode.Internal.psd1 -Destination $path -Force | Out-Null Copy-Item -Path ./LICENSE.txt -Destination $path -Force | Out-Null }, 7Zip, ChocoPack, DockerPack @@ -336,7 +339,7 @@ task DocsHelpBuild DocsDeps, { # build the function docs $path = './docs/Functions' - $map =@{} + $map = @{} (Get-Module Pode).ExportedFunctions.Keys | ForEach-Object { $type = [System.IO.Path]::GetFileNameWithoutExtension((Split-Path -Leaf -Path (Get-Command $_ -Module Pode).ScriptBlock.File)) From b299bc61ece13667c8ee98232cfc1f4883a34674 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Fri, 27 Jan 2023 15:51:46 +0000 Subject: [PATCH 49/52] #1046: minor tweak to version stamping --- pode.build.ps1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pode.build.ps1 b/pode.build.ps1 index 169caca37..3b59e8f82 100644 --- a/pode.build.ps1 +++ b/pode.build.ps1 @@ -272,7 +272,7 @@ task Pack -If (Test-PodeBuildIsWindows) Build, { Copy-Item -Path ./src/Pode.Internal.psm1 -Destination $path -Force | Out-Null Copy-Item -Path ./src/Pode.Internal.psd1 -Destination $path -Force | Out-Null Copy-Item -Path ./LICENSE.txt -Destination $path -Force | Out-Null -}, 7Zip, ChocoPack, DockerPack +}, StampVersion, 7Zip, ChocoPack, DockerPack <# From 43d6e00d01cb9b1ffdb0fa3e5516ac27fb1f8e02 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Wed, 1 Feb 2023 19:42:51 +0000 Subject: [PATCH 50/52] #1046: clean-up of code leftover while testing --- src/Pode.psm1 | 31 +++++++++++-------------------- 1 file changed, 11 insertions(+), 20 deletions(-) diff --git a/src/Pode.psm1 b/src/Pode.psm1 index 156cdf53b..8d2aea415 100644 --- a/src/Pode.psm1 +++ b/src/Pode.psm1 @@ -2,29 +2,20 @@ $root = Split-Path -Parent -Path $MyInvocation.MyCommand.Path # load assemblies -$assemblies = [System.AppDomain]::CurrentDomain.GetAssemblies().Location +Add-Type -AssemblyName System.Web +Add-Type -AssemblyName System.Net.Http -if (($assemblies -ilike '*System.Web.dll*').Length -eq 0) { - Add-Type -AssemblyName System.Web +# netstandard2 for <7.2 +if ($PSVersionTable.PSVersion -lt [version]'7.2.0') { + Add-Type -LiteralPath "$($root)/Libs/netstandard2.0/Pode.dll" -ErrorAction Stop } - -if (($assemblies -ilike '*System.Net.Http.dll*').Length -eq 0) { - Add-Type -AssemblyName System.Net.Http +# net6 for =7.2 +elseif ($PSVersionTable.PSVersion -lt [version]'7.3.0') { + Add-Type -LiteralPath "$($root)/Libs/net6.0/Pode.dll" -ErrorAction Stop } - -if (($assemblies -ilike '*Pode.dll*').Length -eq 0) { - # netstandard2 for <7.2 - if ($PSVersionTable.PSVersion -lt [version]'7.2.0') { - Add-Type -LiteralPath "$($root)/Libs/netstandard2.0/Pode.dll" -ErrorAction Stop - } - # net6 for =7.2 - elseif ($PSVersionTable.PSVersion -lt [version]'7.3.0') { - Add-Type -LiteralPath "$($root)/Libs/net6.0/Pode.dll" -ErrorAction Stop - } - # net7 for >7.2 - else { - Add-Type -LiteralPath "$($root)/Libs/net7.0/Pode.dll" -ErrorAction Stop - } +# net7 for >7.2 +else { + Add-Type -LiteralPath "$($root)/Libs/net7.0/Pode.dll" -ErrorAction Stop } # load private functions From aa59c6fa3676409a3074d240afa3749c01f41fb6 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Wed, 1 Feb 2023 21:12:06 +0000 Subject: [PATCH 51/52] add v2.8.0 release notes --- README.md | 3 ++- docs/index.md | 1 + docs/release-notes.md | 34 ++++++++++++++++++++++++++++++++++ docs/roadmap.md | 12 ++++++------ packers/choco/pode.nuspec | 1 + 5 files changed, 44 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 341e399e2..c2d0fdb58 100644 --- a/README.md +++ b/README.md @@ -70,6 +70,7 @@ Then navigate to `http://127.0.0.1:8000` in your browser. * Support for dynamically building Routes from Functions and Modules * Generate/bind self-signed certificates * Secret management support to load secrets from vaults +* Support for File Watchers * (Windows) Open the hosted server as a desktop application ## 📦 Install @@ -111,4 +112,4 @@ To work on issues you can fork Pode, and then open a Pull Request for approval. You can find a list of the features, enhancements and ideas that will hopefully one day make it into Pode [here in the documentation](https://badgerati.github.io/Pode/roadmap/). -There is also a [Project Board](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues and ideas. If you see any draft issues you wish to discuss, or have an idea for one, please dicuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. +There is also a [Project Board](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues and ideas. If you see any draft issues you wish to discuss, or have an idea for one, please discuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. diff --git a/docs/index.md b/docs/index.md index a469dd8c2..1060fb5c7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -42,6 +42,7 @@ Pode is a Cross-Platform framework to create web servers that host REST APIs, We * Support for dynamically building Routes from Functions and Modules * Generate/bind self-signed certificates * Secret management support to load secrets from vaults +* Support for File Watchers * (Windows) Open the hosted server as a desktop application ## 🏢 Companies using Pode diff --git a/docs/release-notes.md b/docs/release-notes.md index 23493283a..3ae1f7b17 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -1,5 +1,39 @@ # Release Notes +## v2.8.0 + +```plain +### Features +* #980: Adds support for Secret Management, either via the SecretManagement module or using custom logic +* #1063: Adds support for File Watchers, allowing you to run logic on file events +* #1067: Adds support for Mutexes and Semaphores + +### Enhancements +* #647: Adds a new helper function, `New-PodeCron`, to help with creating cron expressions for schedules +* #964: Adds a new `-IfExists` parameter for Routes, letting you now specify if Pode should overwrite a Route if it already exists +* #996: Multiple `-Method` values can now be passed for Routes +* #1036: Adds functions to reset and retrieve the current session's expiry +* #1071: Adds support for the `CONNECT` HTTP method + +### Bugs +* #1028: Fixes the QUIT command on the SMTP server to also return a "221 OK" response +* #1029: Resolves the "A drive with this name already exists" message +* #1041: Fixes a parsing error when sending form data from `Invoke-WebRequest` +* #1044: Fixes a duplicate key error when using the `multiple` attribute on HTML file inputs +* #1046: Fixes the version of Pode within its runspaces, so it's no "0.0" +* #1065: Fixes query string parsing when key is null (thanks @ili101!) + +### Documentation +* #1009: Adds clarification around password formats when using a file to store user authentication details (thanks @fatherofinvention!) +* #1054: Fixes rendering issue with `Write-PodeHtmlResponse` example +* #1056: Fixes typo in logging documentation (thanks @fatherofinvention!) + +### Packaging: +* #1050: Bump Dockerfiles to use PS7.3 +* #1051: Bump the PodeListener to use .NET7 +* #1052: Bump version of mkdocs and material theme +``` + ## v2.7.2 ```plain diff --git a/docs/roadmap.md b/docs/roadmap.md index 8057313ce..0a37bcfef 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,10 +1,10 @@ # Roadmap -This page lists the planned features and enhancements that will, hopefully, one day make it into Pode. There is no timeframe on when to expect them, some could be in-progress right now, and others in the future. +This page lists the planned features and enhancements that will, hopefully, one day make it into Pode. There is no time-frame on when to expect them, some could be in-progress right now, and others in the future. Where possible items listed here will have a link to any relevant issues in GitHub. -There is also a [Project Board](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues and ideas. If you see any draft issues you wish to discuss, or have an idea for one, please dicuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. +There is also a [Project Board](https://github.com/users/Badgerati/projects/2) in the beginnings of being setup for Pode, with milestone progression and current roadmap issues and ideas. If you see any draft issues you wish to discuss, or have an idea for one, please discuss it over on [Discord](https://discord.gg/fRqeGcbF6h) in the `#ideas` or `#pode` channel. ## 🎯 Goal @@ -14,9 +14,9 @@ The eventual goal is to have Pode be a central PowerShell module for a number of ## 🚢 Releases -Under normal circumstanes Pode releases approximately once every 2 months, where the following month is usually a Pode.Web release. +Under normal circumstances Pode releases approximately once every 2 months, where the following month is usually a Pode.Web release. -Sometimes there could be more, if patch releases are needed. But sometimes there could be fewer if peronsal time constraints prevent releases. +Sometimes there could be more, if patch releases are needed. But sometimes there could be fewer if personal time constraints prevent releases. ## 📃 Plan @@ -30,7 +30,7 @@ Sometimes there could be more, if patch releases are needed. But sometimes there - [ ] HTTP/2.0 support - [ ] HTTP/3.0 support - [ ] Inbuilt authorization support, on top the current authentications support [#992](https://github.com/Badgerati/Pode/issues/992) -- [ ] Secret management support [#980](https://github.com/Badgerati/Pode/issues/980) +- [x] Secret management support [#980](https://github.com/Badgerati/Pode/issues/980) - [ ] Some way of being able to merge authentication types [588](https://github.com/Badgerati/Pode/issues/588) - [ ] Improved garbage collection in runspaces, to help free up memory - [ ] A Session Pool that can be used to port/re-use PSSessions in Pode more easily @@ -39,7 +39,7 @@ Sometimes there could be more, if patch releases are needed. But sometimes there - [ ] Is it possible to implement an inbuilt SFTP server? - [ ] Inbuilt connectors for connecting to message brokers, like Kafka, RabbitMQ, etc. - [ ] Would is be possible to create an inbuilt pub/sub server? -- [ ] An inbuilt FIM server, so we can fun logic on FIM events +- [x] An inbuilt FIM server, so we can fun logic on FIM events ### Misc diff --git a/packers/choco/pode.nuspec b/packers/choco/pode.nuspec index 079c7fb38..6539329ef 100644 --- a/packers/choco/pode.nuspec +++ b/packers/choco/pode.nuspec @@ -37,6 +37,7 @@ Pode is a Cross-Platform framework for creating web servers to host REST APIs an * Support for dynamically building Routes from Functions and Modules * Generate/bind self-signed certificates * Secret management support to load secrets from vaults +* Support for File Watchers * (Windows) Open the hosted server as a desktop application From 32eec88a75d4d1dd3196091924febeba973d20d6 Mon Sep 17 00:00:00 2001 From: Matthew Kelly Date: Wed, 1 Feb 2023 21:21:19 +0000 Subject: [PATCH 52/52] minor tweak to release notes --- docs/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/release-notes.md b/docs/release-notes.md index 3ae1f7b17..3a8fe9f11 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -28,7 +28,7 @@ * #1054: Fixes rendering issue with `Write-PodeHtmlResponse` example * #1056: Fixes typo in logging documentation (thanks @fatherofinvention!) -### Packaging: +### Packaging * #1050: Bump Dockerfiles to use PS7.3 * #1051: Bump the PodeListener to use .NET7 * #1052: Bump version of mkdocs and material theme