Support for versioning referenced projects

[hajekj]

Tasks

• Change PATCH version number in PCF manifest
  • Format should remain 0.0.<patch> (reference)
    • 0.0.<branch>YYMMDDXXX - example would be 0.0.1240402001 meaning built in 2nd April 2024 from develop
    • Branch:
      • 1 - develop
      • 2 - local
      • 3 - master
  • Simply take the produced version number and strip the dots (.) and use it as patch number
• Control solution version should reflect version including ProjectReferences
• Targets project shouldn't display any errors with VS Code
  • Caused by: Project does not contain 'Compile' target if there's an empty TFM dotnet/vscode-csharp#5786
• Rename all conflicting variables to Talxis*
  • We can simply inherit the variables from Microsoft's targets and re-use them, so we can then make our own internal ones where needed.
• ApplyVersionNumber should run AfterTargets="ProcessCdsProjectReferencesOutputs" - update docs
• Add required dependency Microsoft.PowerApps.MSBuild.Solution and lock it to the current version eg. 1.31.2
• PAC CLI steps
  • Require PAC CLI to be installed
  • Check if the required PAC version is installed 1.31.6
  • If not install the version and set it as used version
• OPTIONAL: Document debugging with breakpoints
  • https://stackoverflow.com/questions/357445/how-to-debug-msbuild-customtask
  • https://learn.microsoft.com/en-us/visualstudio/msbuild/diagnosing-task-failures?view=vs-2022#debug-a-custom-task

[TomProkop]

I don't understand how changing value of SolutionPackagerMetadataWorkingDirectory breaks the PCF build.
I intentionally wanted to use the existing variables to prevent creating multiple copies of the source file. Since I expect multiple types of sources/intermediate files to be present in the obj folder I wanted to change the default value and move the metadata into a specific subfolder. SolutionPackagerMetadataWorkingDirectory should only contain files which will be picked up by the SolutionPackager later in the process. In this folder we can generate files which are necessary during build.

We need to perform some operations (e.g. building the Relationships.xml file, packing Canvas app source into .msapp or replacing version numbers in plugin assembly references) on the source files before we run the SolutionPackager step. This should not be done over source files directly because we don't want to change the source during build and commit those changes. The best place for doing this manipulation is in the obj folder (IntermediateOutputPath) which is how the most MSBuild tasks work.

[hajekj]

There are quite few different things to how versions are applied by Microsoft and us:

• Plugins
  • Current
    • Plugin's .csproj is versioned the same as solution at build time in pipeline, resulting in same version applying everywhere
  • Future
    • ❓When will the plugin get its version and how will it be reflected in the XML files and solution version? Assuming it is the same repository, we can use the dependencies, however the XMLs will need to infer the version of the produced DLL or NUPKG.
• Controls
  • Current
    • Currently, we modify manifest's version in the repo before build being run, the version is 0.0.<run-number> and the solution is versioned in a standard way we use now
  • Future
    • Since the version number of the PCF is only used in the manifest and is not compiled into the final bundle.js or so, we can modify it when needed.
• Microsoft Power Apps Build Target's Approach
  • They copy everything into the intermediate folder, where they use PowerAppsPackage triggered after AfterBuild. They also run ProcessCdsProjectReferencesOutputs which moves the PCF/Plugin artifacts from referenced projects into the IntermediateOutputPath and then runs the rest of the stuff like versioning etc.
  • UPDATE: This works fine.

I believe we should hook on the AfterTarget of ProcessCdsProjectReferencesOutputs and perform versioning there. That would also remove the requirement of modifying the .cdsproj to add BeforeBuild - this is also problematic for PCFs since they weren't copied via ProcessCdsProjectReferencesOutputs yet.

As for versioning plugins, I would suggest that we build the plugin version based on plugin's folder + references commits and then build a separate version of the solution - this could easily however create a lot of confusion. I can also think of the following:

• When building a solution, run GenerateVersionNumber before anything else is being built. Infer the version generated by this step by all the referenced projects.
• This would require that plugins and PCFs also end up with their own build targets (like Microsoft has them), so we would be able to hook onto their events and do the versioning at the right time before build.

Additionally, we should lock into specific Microsoft's build target version, so in case they rename something (Cds for example), it will not break us and we will have control over the versions.

[TomProkop]

I believe we should hook on the AfterTarget of ProcessCdsProjectReferencesOutputs and perform versioning there. That would also remove the requirement of modifying the .cdsproj to add BeforeBuild - this is also problematic for PCFs since they weren't copied via ProcessCdsProjectReferencesOutputs yet.

I wanted this package to contain tasks and target definitions only. If someone references it they it doesn't mess up with the build until you explicitly call targets you are interested in. I didn't want to impose all the steps and behavior on everyone since they might be interested in a subset of functionality only. People can create their own NuGet packages and reference this package if they don't want to copy paste the explicit calls in their .csproj files.

[TomProkop]

@StepanHoudek wants to help with this.

Applying version numbers to a copy of source

The next step is to adjust our ApplyVersionNumber build target and make it work against a copy of solution files instead of the source files.
We want to do this to prevent version changes being committed (obj folder is git-ignored).

Solution source files are located in a folder defined by SolutionRootPath prop.

Before we make our own replacement for Microsoft.PowerApps.MSBuild.Solution targets we can rely on their CopyCdsSolutionContent target which copies source files to IntermediateOutputPath folder (resolves to obj/Debug/DataverseSolutionPackager/Metadata based on a selected build configuration) which is configured by SolutionPackagerMetadataWorkingDirectory prop.

Currently our targets use Item elements to locate individual solution source files. These items are defined in SolutionFileItems.props. I don't want to hard-code paths in C# tasks. The repo layout and formats will change with the recently introduced new source control format and I want to keep the paths in this .props file.

I propose that we introduce a new set if Items which will use SolutionPackagerMetadataWorkingDirectory instead of DataverseSolutionSourceFolderFullPath as the base path:

• SolutionXmlIntermediateOutputPath
• PluginAssembliesIntermediateOutputPath
• WorkflowMetadataIntermediateOutputPath
• SdkMessageProcessingStepsIntermediateOutputPath
• CustomControlsIntermediateOutputPath

I can think of a scenario where someone might need to commit the new version numbers. Therefore I would introduce a new boolean prop called ApplyVersionNumberToSource which would default to false. The target should evaluate this prop.

Build sequence

It is important to calculate versions and apply them in the correct step of the build process.

CopyCdsSolutionContent runs before ProcessCdsProjectReferencesOutputs and PowerAppsPackage targets.

Our GenerateVersionNumber target should run after ProcessCdsProjectReferencesOutputs so it can pick up and analyze dependencies (e.g. assembly versions) from ProjectReferences. It also needs to run before PowerAppsPackage target which invokes SolutionPackagerTask

As I described in my previous comment I want to keep TALXIS.DevKit.Build.Dataverse.Tasks NuGet package free of implicit changes to a consuming project's build process. This means that you should configure the order (BeforeTargets, AfterTargets, DependsOnTargets properties) outside of this package.

If people want to configure their desired build process and ship it with a NuGet package they should create their own package which calls our targets. We might provide a starter package example publicly later.

Testing

For testing purposes insert the following line to your .csproj which will disable (override) the target which deletes the working directory:
<Target Name="CleanUpIntermediateFiles" />