Skip to content

Commit

Permalink
[Backport 1.x] Re-add ApiGenerator (#460)
Browse files Browse the repository at this point in the history
* Revert "Remove ApiGenerator (#156)" (#222)

* Revert "Remove ApiGenerator (#156)"

This reverts commit 7db02e9.

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Update packages.lock.json

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Bump Spectre.Console from 0.30.0 to 0.47.0 (#227)

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Implement parsing OpenAPI to internal model (#228)

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Bump Newtonsoft.Json from 13.0.1 to 13.0.3 (#232)

* Bump Newtonsoft.Json from 13.0.1 to 13.0.3

Bumps [Newtonsoft.Json](https://github.com/JamesNK/Newtonsoft.Json) from 13.0.1 to 13.0.3.
- [Release notes](https://github.com/JamesNK/Newtonsoft.Json/releases)
- [Commits](JamesNK/Newtonsoft.Json@13.0.1...13.0.3)

---
updated-dependencies:
- dependency-name: Newtonsoft.Json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog & packages.lock.json

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Upgrade RazorLight and use embedded resource project (#233)

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Remove support for the `net461` target (#256)

* Remove support for the `net461` target

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add changelog entry

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Bump Microsoft.CodeAnalysis.CSharp from 4.2.0 to 4.6.0 (#270)

* Bump Microsoft.CodeAnalysis.CSharp from 4.2.0 to 4.6.0

Bumps [Microsoft.CodeAnalysis.CSharp](https://github.com/dotnet/roslyn) from 4.2.0 to 4.6.0.
- [Release notes](https://github.com/dotnet/roslyn/releases)
- [Changelog](https://github.com/dotnet/roslyn/blob/main/docs/Breaking%20API%20Changes.md)
- [Commits](https://github.com/dotnet/roslyn/commits)

---
updated-dependencies:
- dependency-name: Microsoft.CodeAnalysis.CSharp
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog & packages.lock.json

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Remove packages.lock.json & fix CI caching (#265)

* Remove packages.lock.json & remove redundant caching

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-add caching

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Include all proj files in caching

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add ability to generate only a subset of operations (#278)

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Clean up old rest specification files (#319)

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate tasks namespace (#320)

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Bump NSwag.Core.Yaml from 13.19.0 to 13.20.0 (#336)

* Bump NSwag.Core.Yaml from 13.19.0 to 13.20.0

Bumps [NSwag.Core.Yaml](https://github.com/RicoSuter/NSwag) from 13.19.0 to 13.20.0.
- [Release notes](https://github.com/RicoSuter/NSwag/releases)
- [Changelog](https://github.com/RicoSuter/NSwag/blob/master/CHANGELOG.md)
- [Commits](RicoSuter/NSwag@v13.19.0...v13.20.0)

---
updated-dependencies:
- dependency-name: NSwag.Core.Yaml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Re-generate `dangling_indices` namespace (#333)

* Correct handling of deprecated parameters

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `dangling_indices` namespace

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add remarks about when cluster_manager_timeout is supported

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Delete missed old file

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `ingest` namespace (#340)

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `snapshot` namespace (#351)

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `nodes` namespace (#352)

* Use specification naming for enums

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Handle deprecated paths

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `nodes.hot_threads`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Handle overloaded param

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Better sorting

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `nodes.info`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `nodes.reload_secure_settings`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Sort URL parts

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `nodes.stats`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Correctly handle URL part enum options

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Make ApiUrls readonly

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `nodes.usage`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster` namespace (part 1) (#356)

* Re-generate `cluster.allocation_explain`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.delete_component_template`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.delete_voting_config_exclusions`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.exists_component_template`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.get_component_template`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.get_settings`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.health`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.pending_tasks`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Support deprecation and added versions for API methods in ApiGenerator (#402)

* Handle generating deprecation notices on high-level API methods

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Improve formatting

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Better URL path handling and support x-version-added on APIs

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add support for Point In Time APIs (#405)

* Add `CreatePit` & `DeletePit` methods

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add `DeleteAllPits` and `GetAllPits` methods

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add PIT search support

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add integration tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add docs

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add changelog entry

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Correct

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix changelog typo

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add tests for CreatePit

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add tests for DeletePit

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fixing tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Cleanup

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add tests for GetAllPits & DeleteAllPits

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* PitSearch tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* call isolated

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* writable cluster

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Revert breaking namespace changes (#425)

Reverts #200, #202, #203, #205, #206, #207, #208, #209

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Bump CSharpier.Core from 0.25.0 to 0.26.1 (#430)

* Bump CSharpier.Core from 0.25.0 to 0.26.1

Bumps [CSharpier.Core](https://github.com/belav/csharpier) from 0.25.0 to 0.26.1.
- [Release notes](https://github.com/belav/csharpier/releases)
- [Changelog](https://github.com/belav/csharpier/blob/main/CHANGELOG.md)
- [Commits](belav/csharpier@0.25.0...0.26.1)

---
updated-dependencies:
- dependency-name: CSharpier.Core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add support for component template APIs (#411)

* Generate `cluster.put_component_template`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Implement high-level DeleteComponentTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Implement high-level ComponentTemplateExists

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix license

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Remove old put_component_template

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Implement high-level GetComponentTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add tests for ComponentTemplateExists

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add tests for DeleteComponentTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add tests for GetComponentTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Implement high-level PutComponentTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add component template CRUD tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add naming convention exception for ComponentTemplateExists

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Update guide

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Formatting

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add changelog entry

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix license header

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix namespaces

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix license header

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add support for composable index templates (#437)

* Allow renaming URL path parts in generator

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Generate {delete,exists,get,put}_index_template as *ComposableTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add resp/req bodies for *ComposableIndexTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add ComposableIndexTemplateCrudTests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add ComposableIndexTemplateExists tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add DeleteComposableIndexTemplate tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add GetComposableIndexTemplate tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add PutComposableIndexTemplate tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Non-overlapping templates

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Test data_stream template mapping serialization

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Update guide and add sample

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add changelog entry

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Tidy generated code

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Review feedback

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster` namespace (part 2) (#385)

* Re-generate `cluster.post_voting_config_exclusions`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.put_settings`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.remote_info`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.reroute`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.state`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Re-generate `cluster.stats`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Generate missing cluster weighted routing & decommission awareness operations

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Bump Spectre.Console from 0.47.0 to 0.48.0 (#450)

* Bump Spectre.Console from 0.47.0 to 0.48.0

Bumps [Spectre.Console](https://github.com/spectreconsole/spectre.console) from 0.47.0 to 0.48.0.
- [Release notes](https://github.com/spectreconsole/spectre.console/releases)
- [Commits](spectreconsole/spectre.console@0.47.0...0.48.0)

---
updated-dependencies:
- dependency-name: Spectre.Console
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Bump CSharpier.Core from 0.26.1 to 0.26.3 (#448)

* Bump CSharpier.Core from 0.26.1 to 0.26.3

Bumps [CSharpier.Core](https://github.com/belav/csharpier) from 0.26.1 to 0.26.3.
- [Release notes](https://github.com/belav/csharpier/releases)
- [Changelog](https://github.com/belav/csharpier/blob/main/CHANGELOG.md)
- [Commits](belav/csharpier@0.26.1...0.26.3)

---
updated-dependencies:
- dependency-name: CSharpier.Core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add higher-level HTTP DSL methods (#447)

* Lay foundations

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Implement generation

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Run generator

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Update samples

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Update guide

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add changelog entry

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* PR comments

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix naming tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Why is the ordering of these statements load-bearing???

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
  • Loading branch information
3 people authored Dec 4, 2023
1 parent 1e7c4b3 commit 92c4b3e
Show file tree
Hide file tree
Showing 106 changed files with 39,018 additions and 0 deletions.
7 changes: 7 additions & 0 deletions OpenSearch.sln
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{3EA11364-051
src\_PublishArtifacts.Build.props = src\_PublishArtifacts.Build.props
EndProjectSection
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ApiGenerator", "src\ApiGenerator\ApiGenerator.csproj", "{CA508E92-50AE-4858-BD94-8637E88A8FAC}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "OpenSearch.Net.VirtualizedCluster", "src\OpenSearch.Net.VirtualizedCluster\OpenSearch.Net.VirtualizedCluster.csproj", "{CFE97627-8DD3-470B-B7CF-78B62E1D305D}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "build", "build", "{29E53C13-34F7-4F0D-8D28-41EF768793E7}"
Expand Down Expand Up @@ -124,6 +126,7 @@ Global
{81473437-5722-4829-A5CD-125B17CCA238} = {6C4A2627-AF22-4388-9DF7-7A9AEACFD635}
{E97CCF40-0BA6-43FE-9F2D-58D454134088} = {3EA11364-0513-44B7-AD6D-A675485E7448}
{072BA7DA-7B60-407D-8B6E-95E3186BE70C} = {3EA11364-0513-44B7-AD6D-A675485E7448}
{CA508E92-50AE-4858-BD94-8637E88A8FAC} = {3EA11364-0513-44B7-AD6D-A675485E7448}
{CFE97627-8DD3-470B-B7CF-78B62E1D305D} = {3EA11364-0513-44B7-AD6D-A675485E7448}
{D6997ADC-E933-418E-831C-DE1A78897493} = {29E53C13-34F7-4F0D-8D28-41EF768793E7}
{432D5575-2347-4D3C-BF8C-3E38410C46CA} = {29E53C13-34F7-4F0D-8D28-41EF768793E7}
Expand Down Expand Up @@ -189,6 +192,10 @@ Global
{81473437-5722-4829-A5CD-125B17CCA238}.Debug|Any CPU.Build.0 = Debug|Any CPU
{81473437-5722-4829-A5CD-125B17CCA238}.Release|Any CPU.ActiveCfg = Release|Any CPU
{81473437-5722-4829-A5CD-125B17CCA238}.Release|Any CPU.Build.0 = Release|Any CPU
{CA508E92-50AE-4858-BD94-8637E88A8FAC}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{CA508E92-50AE-4858-BD94-8637E88A8FAC}.Debug|Any CPU.Build.0 = Debug|Any CPU
{CA508E92-50AE-4858-BD94-8637E88A8FAC}.Release|Any CPU.ActiveCfg = Release|Any CPU
{CA508E92-50AE-4858-BD94-8637E88A8FAC}.Release|Any CPU.Build.0 = Release|Any CPU
{CFE97627-8DD3-470B-B7CF-78B62E1D305D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{CFE97627-8DD3-470B-B7CF-78B62E1D305D}.Debug|Any CPU.Build.0 = Debug|Any CPU
{CFE97627-8DD3-470B-B7CF-78B62E1D305D}.Release|Any CPU.ActiveCfg = Release|Any CPU
Expand Down
3 changes: 3 additions & 0 deletions build/scripts/Commandline.fs
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ Targets:
* benchmark [non-interactive] [url] [username] [password]
- Runs a benchmark from Tests.Benchmarking and indexes the results to [url] when provided.
If non-interactive runs all benchmarks without prompting
* codegen
- runs the code generator interactively
NOTE: both the `test` and `integrate` targets can be suffixed with `-all` to force the tests against all suported TFM's
Expand Down Expand Up @@ -193,6 +195,7 @@ Execution hints can be provided anywhere on the command line
| ["clean"]
| ["benchmark"]
| ["profile"] -> parsed
| "codegen" :: tail -> { parsed with RemainingArguments = tail }
| "rest-spec-tests" :: tail -> { parsed with RemainingArguments = tail }

| ["release"; version] -> { parsed with CommandArguments = SetVersion { Version = version; OutputLocation = None }; }
Expand Down
8 changes: 8 additions & 0 deletions build/scripts/ReposTooling.fs
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,14 @@ module ReposTooling =

Shell.deleteDir tempDir

let GenerateApi args =
//TODO allow branch name to be passed for CI
let folder = Path.getDirectory (Paths.ProjFile "ApiGenerator")
let timeout = TimeSpan.FromMinutes(120.)
// Building to make sure XML docs files are there, faster then relying on the ApiGenerator to emit these
// from a compilation unit
Tooling.DotNet.ExecInWithTimeout folder (["run"; "-c"; " Release"; "--" ] @ args) timeout |> ignore

let RestSpecTests args =
let folder = Path.getDirectory (Paths.TestProjFile "Tests.YamlRunner")
let timeout = TimeSpan.FromMinutes(120.)
Expand Down
3 changes: 3 additions & 0 deletions build/scripts/Targets.fs
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,9 @@ module Main =
command "cluster" [ "restore"; "full-build" ] <| fun _ ->
ReposTooling.LaunchCluster parsed

command "codegen" [ ] <| fun _ ->
ReposTooling.GenerateApi parsed.RemainingArguments

command "rest-spec-tests" [ ] <| fun _ ->
ReposTooling.RestSpecTests parsed.RemainingArguments

Expand Down
26 changes: 26 additions & 0 deletions src/ApiGenerator/ApiGenerator.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net6.0</TargetFramework>
<IsPackable>false</IsPackable>

<NoWarn>CS1591;NU1701</NoWarn>
<PreserveCompilationContext>true</PreserveCompilationContext>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CSharpier.Core" Version="0.26.3" />
<PackageReference Include="Glob" Version="1.1.9" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.3" />
<PackageReference Include="NSwag.Core.Yaml" Version="13.20.0" />
<PackageReference Include="SemanticVersioning" Version="2.0.2" />
<PackageReference Include="ShellProgressBar" Version="5.2.0" />
<PackageReference Include="Spectre.Console" Version="0.48.0" />
<PackageReference Include="System.CommandLine.DragonFruit" Version="0.3.0-alpha.20371.2" />
<PackageReference Include="RazorLight" Version="2.3.1" />
<PackageReference Include="System.Text.Encodings.Web" Version="7.0.0" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Views\**\*.cshtml" />
</ItemGroup>
</Project>
42 changes: 42 additions & 0 deletions src/ApiGenerator/CodeTemplatePage.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/
/*
* Modifications Copyright OpenSearch Contributors. See
* GitHub history for details.
*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

using System.Threading.Tasks;
using RazorLight;

namespace ApiGenerator;

public abstract class CodeTemplatePage<TModel> : TemplatePage<TModel>
{
protected new Task IncludeAsync(string key, object model = null)
=> base.IncludeAsync(key.Replace('/', '.'), model);

protected async Task IncludeLegacyGeneratorNotice() => await IncludeAsync("GeneratorNotice", true);

protected async Task IncludeGeneratorNotice() => await IncludeAsync("GeneratorNotice", false);
}
228 changes: 228 additions & 0 deletions src/ApiGenerator/Configuration/CodeConfiguration.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,228 @@
/* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/
/*
* Modifications Copyright OpenSearch Contributors. See
* GitHub history for details.
*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using GlobExpressions;

namespace ApiGenerator.Configuration
{
public static class CodeConfiguration
{
private static readonly Glob[] OperationsToInclude =
{
new("{create,delete}_pit"),
new("{delete,get}_all_pits"),

new("cluster.*"),
new("dangling_indices.*"),

new("indices.{delete,exists,get,put}_index_template"),

new("ingest.*"),
new("nodes.*"),
new("snapshot.*"),
new("tasks.*")
};

public static bool IncludeOperation(string name) => OperationsToInclude.Any(g => g.IsMatch(name));

/// <summary>
/// Map API default names for API's we are only supporting on the low level client first
/// </summary>
private static readonly Dictionary<string, string> LowLevelApiNameMapping = new()
{
};

/// <summary>
/// Scan all OSC source code files for Requests and look for the [MapsApi(filename)] attribute.
/// The class name minus Request is used as the canonical .NET name for the API.
/// </summary>
public static readonly Dictionary<string, string> HighLevelApiNameMapping =
(from f in new DirectoryInfo(GeneratorLocations.OpenSearchClientFolder).GetFiles("*.cs", SearchOption.AllDirectories)
let contents = File.ReadAllText(f.FullName)
let c = Regex.Replace(contents, @"^.+\[MapsApi\(""([^ \r\n]+)""\)\].*$", "$1", RegexOptions.Singleline)
where !c.Contains(" ") //filter results that did not match
select new { Value = f.Name.Replace("Request", ""), Key = c.Replace(".json", "") })
.DistinctBy(v => v.Key)
.ToDictionary(k => k.Key, v => v.Value.Replace(".cs", ""));

public static readonly HashSet<string> EnableHighLevelCodeGen = new HashSet<string>();

public static bool IsNewHighLevelApi(string apiFileName) =>
// no requests with [MapsApi("filename.json")] found
!HighLevelApiNameMapping.ContainsKey(apiFileName.Replace(".json", ""));

public static bool IgnoreHighLevelApi(string apiFileName)
{
//always generate already mapped requests

if (HighLevelApiNameMapping.ContainsKey(apiFileName.Replace(".json", ""))) return false;

return !EnableHighLevelCodeGen.Contains(apiFileName);
}

private static Dictionary<string, string> _apiNameMapping;

public static Dictionary<string, string> ApiNameMapping
{
get
{
if (_apiNameMapping != null) return _apiNameMapping;
lock (LowLevelApiNameMapping)
{
if (_apiNameMapping == null)
{
var mapping = new Dictionary<string, string>(HighLevelApiNameMapping);
foreach (var (k, v) in LowLevelApiNameMapping)
mapping[k] = v;
_apiNameMapping = mapping;
}
return _apiNameMapping;
}
}
}

private static readonly string ResponseBuilderAttributeRegex = @"^.+\[ResponseBuilderWithGeneric\(""([^ \r\n]+)""\)\].*$";
/// <summary>
/// Scan all OSC source code files for Requests and look for the [MapsApi(filename)] attribute.
/// The class name minus Request is used as the canonical .NET name for the API.
/// </summary>
public static readonly Dictionary<string, string> ResponseBuilderInClientCalls =
(from f in new DirectoryInfo(GeneratorLocations.OpenSearchClientFolder).GetFiles("*.cs", SearchOption.AllDirectories)
from l in File.ReadLines(f.FullName)
where Regex.IsMatch(l, ResponseBuilderAttributeRegex)
let c = Regex.Replace(l, @"^.+\[ResponseBuilderWithGeneric\(""([^ \r\n]+)""\)\].*$", "$1", RegexOptions.Singleline)
select new { Key = f.Name.Replace(".cs", ""), Value = c })
.DistinctBy(v => v.Key)
.ToDictionary(k => k.Key, v => v.Value);

public static readonly Dictionary<string, string> DescriptorGenericsLookup =
(from f in new DirectoryInfo(GeneratorLocations.OpenSearchClientFolder).GetFiles("*Request.cs", SearchOption.AllDirectories)
let name = Path.GetFileNameWithoutExtension(f.Name).Replace("Request", "")
let contents = File.ReadAllText(f.FullName)
let c = Regex.Replace(contents, $@"^.+class ({name}Descriptor(?:<[^>\r\n]+>)?[^ \r\n]*).*$", "$1", RegexOptions.Singleline)
let key = $"{name}Descriptor"
select new { Key = key, Value = Regex.Replace(c, @"^.*?(?:(\<.+>).*?)?$", "$1") })
.DistinctBy(v => v.Key)
.OrderBy(v => v.Key)
.ToDictionary(k => k.Key, v => v.Value);

/// <summary> Scan all OSC files for request interfaces and note any generics declared on them </summary>
private static readonly List<Tuple<string, string>> AllKnownRequestInterfaces = (
// find all files in OSC ending with Request.cs
from f in new DirectoryInfo(GeneratorLocations.OpenSearchClientFolder).GetFiles("*Request.cs", SearchOption.AllDirectories)
from l in File.ReadLines(f.FullName)
// attempt to locate all Request interfaces lines
where Regex.IsMatch(l, @"^.+interface [^ \r\n]+Request")
//grab the interface name including any generics declared on it
let c = Regex.Replace(l, @"^.+interface ([^ \r\n]+Request(?:<[^>\r\n]+>)?[^ \r\n]*).*$", "$1", RegexOptions.Singleline)
where c.StartsWith("I") && c.Contains("Request")
let request = Regex.Replace(c, "<.*$", "")
let generics = Regex.Replace(c, @"^.*?(?:(\<.+>).*?)?$", "$1")
select Tuple.Create(request, generics)
)
.OrderBy(v=>v.Item1)
.ToList();

public static readonly HashSet<string> GenericOnlyInterfaces = new HashSet<string>(AllKnownRequestInterfaces
.GroupBy(v => v.Item1)
.Where(g => g.All(v => !string.IsNullOrEmpty(v.Item2)))
.Select(g => g.Key)
.ToList());

public static readonly HashSet<string> DocumentRequests = new HashSet<string>((
// find all files in OSC ending with Request.cs
from f in new DirectoryInfo(GeneratorLocations.OpenSearchClientFolder).GetFiles("*Request.cs", SearchOption.AllDirectories)
from l in File.ReadLines(f.FullName)
// attempt to locate all Request interfaces lines
where Regex.IsMatch(l, @"^.+interface [^ \r\n]+Request")
where l.Contains("IDocumentRequest")
let c = Regex.Replace(l, @"^.+interface ([^ \r\n]+Request(?:<[^>\r\n]+>)?[^ \r\n]*).*$", "$1", RegexOptions.Singleline)
//grab the interface name including any generics declared on it
let request = Regex.Replace(c, "<.*$", "")
select request
)
.ToList());

public static readonly Dictionary<string, string> DescriptorConstructors = (
// find all files in OSC ending with Request.cs
from f in new DirectoryInfo(GeneratorLocations.OpenSearchClientFolder).GetFiles("*Request.cs", SearchOption.AllDirectories)
let descriptor = Path.GetFileNameWithoutExtension(f.Name).Replace("Request", "Descriptor")
let re = $@"^.+public {descriptor}\(([^\r\n\)]+?)\).*$"
from l in File.ReadLines(f.FullName)
where Regex.IsMatch(l, re)
let args = Regex.Replace(l, re, "$1", RegexOptions.Singleline)
where !string.IsNullOrWhiteSpace(args) && !args.Contains(": base")
select (Descriptor: descriptor, Args: args)
)
.ToDictionary(r => r.Descriptor, r => r.Args);

public static readonly Dictionary<string, string> RequestInterfaceGenericsLookup =
AllKnownRequestInterfaces
.GroupBy(v=>v.Item1)
.Select(g=>g.Last())
.ToDictionary(k => k.Item1, v => v.Item2);

/// <summary>
/// Some API's reuse response this is a hardcoded map of these cases
/// </summary>
private static Dictionary<string, (string, string)> ResponseReroute = new Dictionary<string, (string, string)>
{
{"UpdateByQueryRethrottleResponse", ("ListTasksResponse", "")},
{"DeleteByQueryRethrottleResponse", ("ListTasksResponse", "")},
{"MultiSearchTemplateResponse", ("MultiSearchResponse", "")},
{"ScrollResponse", ("SearchResponse", "<TDocument>")},
{"SearchTemplateResponse", ("SearchResponse", "<TDocument>")},

};


/// <summary> Create a dictionary lookup of all responses and their generics </summary>
public static readonly SortedDictionary<string, (string, string)> ResponseLookup = new SortedDictionary<string, (string, string)>(
(
// find all files in OSC ending with Request.cs
from f in new DirectoryInfo(GeneratorLocations.OpenSearchClientFolder).GetFiles("*Response.cs", SearchOption.AllDirectories)
from l in File.ReadLines(f.FullName)
// attempt to locate all Response class lines
where Regex.IsMatch(l, @"^.+public class [^ \r\n]+Response")
//grab the response name including any generics declared on it
let c = Regex.Replace(l, @"^.+public class ([^ \r\n]+Response(?:<[^>\r\n]+>)?[^ \r\n]*).*$", "$1", RegexOptions.Singleline)
where c.Contains("Response")
let response = Regex.Replace(c, "<.*$", "")
let generics = Regex.Replace(c, @"^.*?(?:(\<.+>).*?)?$", "$1")
select (response, (response, generics))
)
.Concat(ResponseReroute.Select(kv=>(kv.Key, (kv.Value.Item1, kv.Value.Item2))))
.ToDictionary(t=>t.Item1, t=>t.Item2));

}
}
Loading

0 comments on commit 92c4b3e

Please sign in to comment.