Building and Releasing

Building and Releasing

Master

All releases are cut off of master branch. No actual work is done here. Merges to this branch come from bug fix branches or develop.

Bugfix

Small production bug fixes are worked on bug fix oriented branches made off of master and named bugfix/whatever. When complete these are merged back on to master and the release number is advanced. According to Semanic Versioning, version numbers are formatted MAJOR.MINOR.BUGFIX so a bugfix release increases the BUGFIX element of the version number.

Develop

New development work is focused on the develop branch. But little actual work is done here. New features are worked on feature branches named with the prefix feature/. These are merged onto develop only when they individually pass Integration Tests. The "merge to develop" process is:

• Complete new feature development work on feature/* branch.

• Successfully run unit and integration tests.

• Merge onto develop and delete the feature/* branch.

• Rerun the unit and integration tests.

• Push merge to GitHub.

Directory to operate in

All command line commands below are to be run in the root of the project unless otherwise indicated.

Maven

Compiling XDS Toolkit on Ubuntu

Starting with XDS Toolkit Release 7.11.0 and JDK 17 or above, a shell environment variable needs to be set on some operating systems such as Ubuntu for the mvn command to work properly.

Use the following shell variable for Maven 3.6.0 and JDK 17+ on Ubuntu.

export JDK_JAVA_OPTIONS="--add-exports=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED"

Compiling on Windows

On Windows, usage of the JDK_JAVA_OPTIONS shell environment variable should not be necessary.

Note

Previous XDS Toolkit Releases 7.7.0 to 7.10.0 required a Java JDK 8 compiler.

You may encounter an issue with a failed test while building / compiling the sim-proxy module. The issue may involved an error such as: "Condition not satisfied: bparts[1].content == part2.bytes". In that case you will need to refresh your local repository with the git configuration setting core.autocrlf=false.

Build profiles

Maven allows for the collection of parameters for a build into a profile. Project profiles are collected into the module tk-deps. Under

src/main/filters

are various copies of toolkit.properties. The selection of a named profile in the build chooses the filter applied to toolkit.properties thus customizing the build. This is done by updating toolkit.properties with the contents of the profile file. A typical update is the location of the External Cache which is unique for each developer on their private machine.

An example is the profile Bill represented in src/main/filters by Bill.properties. It contains

External_Cache=/Users/bill/tmp/toolkit2a
Enable_SAML=true
Toolkit_Port=8888
Listener_Port_Range=5000,5020

This works in the build by starting with the toolkit.properties file under xdstools2/src/resources and overwriting the above properties. Note that the properties that can be updates are controlled by the xdstools2/pom.xml configuration.

New profiles can be created by simply adding them to this directory. They are referenced in the build as

mvn clean install -P MyProfileName

where MyProfileName is the filter file name without the .properties suffix.

Running unit tests

Unit tests are built into the maven process. Our typical build is

mvn clean install

which cleans out all results from prior builds and finishes by updating the local Maven repository. In between the clean and install the unit tests are run. If the unit tests fail then the entire build fails. This is usually run with the External Cache used during development. Note that the External Cache is controlled by selected profile as discussed above.

Running Integration Tests

Integration tests are global tests run by building the entire project and launching it under the web server Jetty. Where as unit tests are very local in nature, integration tests operate on the entire project. Integration tests are run by

mvn clean install -P Integration-Tests

The Integration-Tests profile does not specify an External Cache location to use so it must be specified by an addition profile (you can specify more than one) or the default will be used.

In my development work I use UbuntuIdeaBill which specifies

External_Cache=/home/tomcat/ec
Toolkit_Port=8080
Enable_SAML=true
Listener_Port_Range=5000,5020
Client_Cipher_Suites=TLS_RSA_WITH_AES_128_CBC_SHA
Client_SSL_Protocols=TLSv1.2

Before running Integration Tests the External Cache directory must exist, be writable by the account you are using, and should be empty so that Toolkit will initialize it at startup.

Running UI Tests

UI Tests are run by a separate Maven invocation. They depend on having the project already built. The test run finds the WAR file under xdstools2/target and uses that in the testing process. Since the build and run are independent we must assure through configuration that they are aligned. The most critical parameter is the web server port.

These tests operate by poking at the various UI elements (buttons, list boxes etc.) to simulate a user thus test the project from an all-encompassing point of view.

To build a compatible WAR with

mvn clean install -D skipTests -P UbuntuUITestsBill

This sets the expected port number to 8888 which is the configuration used in Jetty in the next step.

Clear the External Cache so it is built from the WAR - as configured the EC is at /home/bill/ec. You will likely have to change this for your system.

cd /home/bill/ec
rm -r *

Run the WEB UI tests:

cd webui-tests
mvn clean install

If this fails the detail messages can be found in

webui-tests/*stderrout.log

typically the last entry in the log indicates the failure but the prior entries indicate why it failed.

Run the UI tests before preparing the release since we insert an obvious bogus External Cache location into the toolkit.properties file as part of the release build.

Single command to build with all Maven profiles specified on a single line

mvn clean install -P YourProfileName,Integration-Tests,WebUI-Tests

Update the version for the release

All new release material is on the develop branch. Merge onto master. Back on develop advance version number to the next SNAPSHOT version number. Do this with

    sh update-version. MAJOR.MINOR.PATCH[-SNAPSHOT]

This is run in the project root directory. This updates all POM files. Commit these changes for later.

On the master branch also run update-version but with the release version. This will usually just remove the -SHAPSHOT. Commit these changes.

Making a release

Run the script

    sh build-release.sh <type>

Where type is a name found in the properties directory. Each sub-directory under properties/ holds a toolkit.properties file. This selection will cause this configuration file to be placed in the resulting WAR file.

For typical release which are to be uploaded to GitHub I use

sh build-release.sh release

If this script fails complaining about Offline mode and not having access to updates then.

    cd xdstools2
    mvn site

This will update the site preparation JARs. It will probably never complete - it does way to much work. After a while just stop it. Remember the first thing it will do is update JARS that it needs so you don’t have to wait very long. Then go back to the previous sh build-release. It should complete this time with BUILD SUCCESS displayed at the end. Ignore the stack traces. There are a lot of automatic features we do not use and do not have properly configured.

This will produce

    xdstools2/target/xdstoolsMAJOR.MINOR.PATCH.war

This file can be uploaded to GitHub as part of the release process.