Skip to content

Commit

Permalink
Merge pull request #67 from bitrise-io/ACI-3019
Browse files Browse the repository at this point in the history
Add write gradle verification command
  • Loading branch information
lpusok authored Dec 17, 2024
2 parents 8102e5c + 71e2420 commit 26afe97
Show file tree
Hide file tree
Showing 10 changed files with 214 additions and 14 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
bin/
.DS_Store
bitrise-build-cache-cli
.bitrise.secrets.yml
dist/
_tmp/
.idea/*
!.idea/codeStyles
.idea/codeStyles/*
!.idea/codeStyles/Project.xml
gradle_verification_sample/gradle/verification-metadata.xml
2 changes: 1 addition & 1 deletion .goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ release:
### Gradle verification
For reference, the verification metadata is available as an attached assset.
extra_files:
- glob: ./gradle_verification_sample/gradle/verification-metadata.xml
- glob: ./cmd/asset/verification-metadata.xml


before:
Expand Down
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,11 @@ When `enable-for gradle` or `enable-for bazel` is called:

## Release process

### 1: Create release
### 1: Update Gradle verification reference file

Run the `bitrise test` or `bitrise run check_gradle_verification` command locally, and commit changes (workflows will fail if there are any) to the repository. In case the CI workflow is still failing, copy the generated metadata from the `Generate Gradle verification reference` Step's log.

### 2: Create release

To release a new version we use [goreleaser](https://github.com/goreleaser/goreleaser).

Expand All @@ -121,7 +125,8 @@ This way you can test the new version by specifying it for the installer script,
looks good you can edit the release and change it to `Set as the latest release`,
so it'll be the version downloaded by the installer when the installer is called without a specified version.

### 2: Update downloader

### 3: Update downloader

Now that the new release is available run `godownloader` to update the
installer script.
Expand All @@ -136,7 +141,7 @@ godownloader .goreleaser.yaml > ./install/installer.sh
If only the timestamp is changed at the top of `./install/installer.sh` you don't have to commit
the change, just discard it. If something else also changes then commit and push the updated `installer.sh`.

### 3: Update depending repos (steps, ...)
### 4: Update depending repos (steps, ...)

The Bitrise Build Cache CLI is platform independent, but we have platform specific "adapters".
These adapters depend on the Bitrise Build Cache CLI, either by importing it as a Go
Expand Down
16 changes: 10 additions & 6 deletions bitrise.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,32 +118,36 @@ step_bundles:
go build -o /tmp/bin/bitrise-build-cache-cli
SAMPLE_PATH=$(realpath ./gradle_verification_sample)
GRADLE_VERIFICATION_REF_PATH=$SAMPLE_PATH/gradle/verification-metadata.xml
GRADLE_VERIFICATION_REF_TARGET_PATH=$(realpath ./cmd/asset/verification-metadata.xml)
# add plugin with CLI
cd "$SAMPLE_PATH"
/tmp/bin/bitrise-build-cache-cli add-gradle-deps
/tmp/bin/bitrise-build-cache-cli gradle-verification add-reference-deps
# generate xml
rm -f "$SAMPLE_PATH/gradle/verification-metadata.xml"
./gradlew --write-verification-metadata sha256 help
GRADLE_VERIFICATION_REF_PATH=$SAMPLE_PATH/gradle/verification-metadata.xml
cat $GRADLE_VERIFICATION_REF_PATH
echo "Generated verification xml: $GRADLE_VERIFICATION_REF_PATH"
envman add --key GRADLE_VERIFICATION_REF_PATH --value "$GRADLE_VERIFICATION_REF_PATH"
# Copy metadata to be included in the CLI binary
mv -f "$GRADLE_VERIFICATION_REF_PATH" "$GRADLE_VERIFICATION_REF_TARGET_PATH"
envman add --key GRADLE_VERIFICATION_REF_TARGET_PATH --value "$GRADLE_VERIFICATION_REF_TARGET_PATH"
- script@1:
title: Check if local updates to verification-metadata.xml are not commited
inputs:
- content: |-
set -x
git update-index --refresh | grep "verification-metadata.xml"
git update-index --refresh | grep "cmd/asset/verification-metadata.xml"
if [[ $? == 0 ]]; then
echo "Please manually commit the changes to the repo:"
echo "$GRADLE_VERIFICATION_REF_PATH"
echo "$GRADLE_VERIFICATION_REF_TARGET_PATH"
exit 1
else
echo "The files have no unstaged changes: $GRADLE_VERIFICATION_REF_PATH"
echo "The files have no unstaged changes: $GRADLE_VERIFICATION_REF_TARGET_PATH"
exit 0
fi
File renamed without changes.
8 changes: 4 additions & 4 deletions cmd/addGradleDeps.go → cmd/gradleVerificationAddDeps.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ import (
"github.com/spf13/cobra"
)

// addGradleDeps represents the gradle command
var addGradleDeps = &cobra.Command{ //nolint:gochecknoglobals
Use: "add-gradle-deps",
// addGradleVerificationReferenceDeps represents the gradle command
var addGradleVerificationReferenceDeps = &cobra.Command{ //nolint:gochecknoglobals
Use: "add-reference-deps",
Short: "Add Bitrise Build Cache plugins to the project (but do not enable it)",
Long: `Add Bitrise Build Cache plugins to the project (but do not enable it)
This command will:
Expand Down Expand Up @@ -44,7 +44,7 @@ This command will:
}

func init() {
rootCmd.AddCommand(addGradleDeps)
gradleVerification.AddCommand(addGradleVerificationReferenceDeps)
}

func addGradlePluginsFn(logger log.Logger, gradleHomePath string, envProvider func(string) string) error {
Expand Down
139 changes: 139 additions & 0 deletions cmd/gradleVerificationWrite.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
package cmd

import (
"bytes"
_ "embed"
"fmt"
"io"
"os"

"github.com/beevik/etree"
"github.com/bitrise-io/go-utils/v2/log"
"github.com/bitrise-io/go-utils/v2/pathutil"
"github.com/spf13/cobra"
)

//go:embed "asset/verification-metadata.xml"
var referenceVerificationDepsContent string

//nolint:gochecknoglobals
var metadataPath string

// gradleVerification ...
var gradleVerification = &cobra.Command{ //nolint:gochecknoglobals
Use: "gradle-verification",
Short: "Bitrise build cache support for projects using Gradle verification",
Long: `Bitrise build cache support for projects using Gradle verification
See https://docs.gradle.org/current/userguide/dependency_verification.html for more information.`,
SilenceUsage: true,
RunE: func(_ *cobra.Command, _ []string) error {
return nil
},
}

var writeGradleVerificationDeps = &cobra.Command{ //nolint:gochecknoglobals
Use: "write",
Short: "Add Bitrise build cache dependencies to verification-metadata.xml",
Long: `Add Bitrise build cache dependencies to verification-metadata.xml
Missing dependencies of Bitrise build cache are appended.
`,
SilenceUsage: true,
RunE: func(_ *cobra.Command, _ []string) error {
logger := log.NewLogger()

logger.TInfof("Adding Gradle verification dependencies")

projectMetadataPath, err := pathutil.NewPathModifier().AbsPath(metadataPath)
if err != nil {
return fmt.Errorf("expand metadata path (%s): %w", metadataPath, err)
}

if err := addGradleVerification(logger, projectMetadataPath, os.Getenv); err != nil {
return fmt.Errorf("adding Gradle verification: %w", err)
}

logger.TInfof("✅ Updated verification-metadata.xml")

return nil
},
}

func init() {
rootCmd.AddCommand(gradleVerification)
gradleVerification.AddCommand(writeGradleVerificationDeps)
writeGradleVerificationDeps.Flags().StringVar(&metadataPath, "metadata-path", "", "Path of verification-metadata.xml")
}

func addGradleVerification(logger log.Logger, projectMetadataPath string, _ func(string) string) error {
logger.Infof("(i) Checking parameters")
logger.Infof("(i) Metadata path: %s", projectMetadataPath)

referenceDepsReader := bytes.NewBufferString(referenceVerificationDepsContent)
projectDepsReader, err := os.Open(projectMetadataPath)
if err != nil {
return fmt.Errorf("failed to open project verification-metadata.xml: %w", err)
}

resultProjectMetadata, err := writeVerificationDeps(logger, referenceDepsReader, projectDepsReader)
if err != nil {
return err
}

if err = os.WriteFile(projectMetadataPath, []byte(resultProjectMetadata), os.ModeAppend); err != nil {
return fmt.Errorf("failed to write project verification-metadata.xml: %w", err)
}

logger.Debugf("Updated contents: %s", resultProjectMetadata)

return nil
}

func writeVerificationDeps(logger log.Logger, referenceDepsReader io.Reader, projectDepsReader io.Reader) (string, error) {
referenceDeps := etree.NewDocument()
if _, err := referenceDeps.ReadFrom(referenceDepsReader); err != nil {
return "", fmt.Errorf("failed to parse reference verification-metadata.xml: %w", err)
}

projectDeps := etree.NewDocument()
if _, err := projectDeps.ReadFrom(projectDepsReader); err != nil {
return "", fmt.Errorf("failed to parse project verification-metadata.xml: %w", err)
}

projectRoot := projectDeps.SelectElement("verification-metadata")
if projectRoot == nil {
return "", fmt.Errorf("project metadata missing verification-metadata")
}

projectComponents := projectRoot.SelectElement("components")
if projectComponents == nil {
return "", fmt.Errorf("project metadata missing components")
}

referenceRoot := referenceDeps.SelectElement("verification-metadata")
if referenceRoot == nil {
return "", fmt.Errorf("reference metadata missing verification-metadata")
}

referenceComponents := referenceRoot.SelectElement("components")
if referenceComponents == nil {
return "", fmt.Errorf("reference metadata missing components")
}

referenceComponentList := referenceComponents.SelectElements("component")
if referenceComponentList == nil {
return "", fmt.Errorf("reference metadata has no components")
}

for _, e := range referenceComponentList {
projectComponents.AddChild(e)
}

logger.Infof("Added %d dependecies to verification-metadata.xml", len(referenceComponentList))

result, err := projectDeps.WriteToString()
if err != nil {
return "", fmt.Errorf("failed to serialize updated verification-metadata.xml: %w", err)
}

return result, nil
}
47 changes: 47 additions & 0 deletions e2e/bitrise.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,53 @@ workflows:
- module: app
- variant: debug

test_gradle_verification_gradle7:
envs:
- TEST_APP_URL: https://github.com/bitrise-io/android-demo-app
- BRANCH: main
after_run:
- _setup
- _test_gradle_verification

test_gradle_verification_gradle8:
envs:
- TEST_APP_URL: https://github.com/bitrise-io/android-demo-app
- BRANCH: gradle8
after_run:
- _setup
- _test_gradle_verification

_test_gradle_verification:
steps:
- script:
title: build CLI
inputs:
- content: |-
set -ex
go build -o /tmp/bin/bitrise-build-cache-cli
- change-workdir:
title: Switch working dir to _tmp
inputs:
- path: ./_tmp
- script:
title: update metadata
inputs:
- content: |-
set -ex
if [[ -f ~/.gradle/init.d/bitrise-build-cache.init.gradle.kts ]]; then
rm ~/.gradle/init.d/bitrise-build-cache.init.gradle.kts
fi
./gradlew assemble --write-verification-metadata=sha256
METADATA_PATH=./gradle/verification-metadata.xml
/tmp/bin/bitrise-build-cache-cli gradle-verification add-reference-deps
/tmp/bin/bitrise-build-cache-cli gradle-verification write --metadata-path=$METADATA_PATH
./gradlew assemble
_setup:
steps:
- script:
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ go 1.22.3
require (
cloud.google.com/go/longrunning v0.6.0
github.com/bazelbuild/remote-apis v0.0.0-20240703191324-0d21f29acdb9
github.com/beevik/etree v1.4.1
github.com/bitrise-io/go-steputils/v2 v2.0.0-alpha.33
github.com/bitrise-io/go-utils v1.0.13
github.com/bitrise-io/go-utils/v2 v2.0.0-alpha.23
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ cloud.google.com/go/longrunning v0.6.0/go.mod h1:uHzSZqW89h7/pasCWNYdUpwGz3PcVWh
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/bazelbuild/remote-apis v0.0.0-20240703191324-0d21f29acdb9 h1:Ma3IQbZmIVd8ROR7TZNqN8YnSYKXWtW5t5uUc35pftI=
github.com/bazelbuild/remote-apis v0.0.0-20240703191324-0d21f29acdb9/go.mod h1:ry8Y6CkQqCVcYsjPOlLXDX2iRVjOnjogdNwhvHmRcz8=
github.com/beevik/etree v1.4.1 h1:PmQJDDYahBGNKDcpdX8uPy1xRCwoCGVUiW669MEirVI=
github.com/beevik/etree v1.4.1/go.mod h1:gPNJNaBGVZ9AwsidazFZyygnd+0pAU38N4D+WemwKNs=
github.com/bitrise-io/go-steputils/v2 v2.0.0-alpha.33 h1:BxTsbaY9hwC/3a2y4V8vHlKAu80DAEuv/rvEEAvaq8o=
github.com/bitrise-io/go-steputils/v2 v2.0.0-alpha.33/go.mod h1:Wu1/fiQqxr7A7a9aphh/Jxpof6WV+LEO6vGfBQ9T99A=
github.com/bitrise-io/go-utils v1.0.13 h1:1QENhTS/JlKH9F7+/nB+TtbTcor6jGrE6cQ4CJWfp5U=
Expand Down

0 comments on commit 26afe97

Please sign in to comment.