Upgrading Brightspot Gradle Plugins

Goals
Conventions
Example
0. Current build for verification
1. Upgrade Gradle Wrapper
2. Delete /build.gradle
3. Update settings.gradle
4. Update core/build.gradle
5. Update frontend/build.gradle
6. Update each bundle’s build.gradle
7. Update web/build.gradle
8. Update any other subproject build.gradle files
9. Verify the build
10. Other Configuration Changes
Further Reading

Goals

Establish and maintain Gradle best practices for all Brightspot projects
Improve build performance
Simplify future upgrades

Conventions

Set `$PROJECT` to the project name

Throughout this guide, $PROJECT will be used to refer to the project’s normalized kebab-case name.

✅ Right: PROJECT='example-co'
❌ Wrong: PROJECT='Example Co, Inc.'
🤡 Extra Wrong: PROJECT='$PROJECT'

In all cases, the string $PROJECT must be substituted for the project name in all file contents before committing changes to the file.

Before running any of the prescribed commands on your command line, export PROJECT='my-project-name'. This will automate the $PROJECT substitution in those cases.

Resolve `// XXX` comments

Comments prefixed with XXX are there for you, the reader of this guide, to review, resolve, and remove before committing changes to the file.

✅ Right: brightspotVersion = 4.5.15.8
❌ Wrong: brightspotVersion = 4.5.15.8 // XXX Copy this version number from the legacy build.gradle
🤡 Extra Wrong: brightspotVersion = 4.5.x // XXX Copy this version number from the legacy build.gradle

In all cases, any comment starting with // XXX must be deleted after satisfying the advice within.

Project Structure

Projects sometimes have different module naming conventions. For example, older projects have a site module that generates the war file, while newer projects call that the web module. Throughout this guide, we will use the “new” conventions, but this table should help you to map these names to those found in your project:

Name	New Path	Old Path
core	`./core`	`./core`
web	`./web`	`./site`
frontend	`./frontend`	`./frontend`
root styleguide	`./frontend/styleguide`	`./styleguide`
bundles	`./frontend/bundles`	`./themes`

Version Control Hygiene

Each step in this guide should be committed to the git repository. Suggested commit messages are detailed inline. The suggested command git commit -a adds and commits all changes in the entire repo, so make sure there aren’t any unintended changes included.

Command Line Working Directory

The working directory for all command line operations detailed in this guide is assumed to be the root directory of the project unless otherwise noted.

Example

Along with this guide, this pull request can be used as a reference.

0. Current build for verification

If a styleguide upgrade is necessary, do it first to simplify the verification of the gradle plugin upgrade.

Before changing anything, run a full build and set the gradle scan and war file aside for verification at the end.

The gradlew command will output a Gradle scan URL. Bookmark this URL to compare at the end.

If the current version of component-lib is before 4.2.188 or 4.5.23, upgrade to one of those versions before this step.

./gradlew clean build --no-build-cache --scan
cp web/build/libs/$PROJECT*.war /tmp/$PROJECT-before.war

1. Upgrade Gradle Wrapper

Remove gradle or gradle/ from the .gitignore file in the project root, if present.

./gradlew wrapper --gradle-version=8.4
./gradlew wrapper --gradle-version=8.4
git commit -a -m "Upgrade Gradle Wrapper to 8.4"

Be sure to run the ./gradlew wrapper command twice so it downloads the latest files.

2. Delete `/build.gradle`

Just set it aside for now, as we may need to copy some details from it later.

mv ./build.gradle /tmp/$PROJECT-build.gradle
git commit -a -m "Gradle plugin upgrade: Remove /build.gradle"

Take special note of any forced dependency constraints, substitutions, or exclusions. These should be converted to dependencyConstraints, dependencySubstitutions, or dependencyExclusions - see the plugin guide for details.

3. Update `settings.gradle`

Edit settings.gradle to match the following:

pluginManagement {
    repositories {
        gradlePluginPortal() {
            content {
                excludeGroupByRegex 'com\\.brightspot($|\\..+)'
                excludeGroupByRegex 'com\\.psddev($|\\..+)'
            }
        }
        maven {
            url 'https://artifactory.psdops.com/public'
        }
    }
}

plugins {
    id 'com.gradle.enterprise' version '3.15.1'
    id 'com.brightspot.gradle' version '3.0.1'
}

dependencyResolutionManagement {
    versionCatalogs {
        brightspotLibs {
            // XXX Copy the versions for these from existing build.gradle, if present.

            library('brightspot', 'com.psddev:brightspot-bom:4.5.15.19')
            library('brightspotDependencies', 'com.psddev:brightspot-dependencies:4.5.15.19')

            // XXX If BSP Go isn't in the existing build.gradle, leave these two lines out.
            library('brightspotGo', 'com.brightspot.go:bom:1.4.6')
            library('brightspotGoDependencies', 'com.brightspot.go:go-dependencies:1.4.6')

            library('brightspotSharedTests' , 'com.brightspot.shared-tests:bom:1.0.1')

            // XXX If Component Lib isn't in the existing build.gradle, leave these two lines out.
            // XXX If on version 4.2.188 or lower, or 4.5.23 or lower, you will need to upgrade to at least one of those versions.
            library('componentLib', 'com.psddev.component-lib:bom:4.5.26')
            library('componentLibDependencies', 'com.psddev.component-lib:cl-dependencies:4.5.26')

            // XXX Only include these if "com.psddev.migration:bom" is already included.
            // XXX (not "com.psddev:migration-bom" or "com.psddev.migration:migration-bom")
            // XXX Note that it may be in another build.gradle such as migration/build.gradle
            library('brightspotMigration', 'com.psddev.migration:bom:4.7.0.4')
            library('brightspotMigrationDependencies', com.psddev.migration:dependencies:4.7.0.4)
        }
    }
}

rootProject.name = '$PROJECT'

brightspot {
    projectGroup = 'com.$PROJECT' // XXX This is probably already set in the existing build.gradle. If so, use that value here.

    versions {
        java = 11 // XXX or 8 if the project is using Java 8
    }
}

// XXX Project-specific lines similar to the following (until XXX END) should already be present in the
// XXX existing settings.gradle. Leave them alone.

include(':core')
include(':web')
include(':frontend')
include(':bundle-default')

project(':core').projectDir = file('core')
project(':web').projectDir = file('web')
project(':frontend').projectDir = file('frontend')
project(':bundle-default').projectDir = file('frontend/bundles/bundle-default')

// XXX END

gradleEnterprise {
    buildScan {
        termsOfServiceUrl = "https://gradle.com/terms-of-service"
        termsOfServiceAgree = "yes"
    }
}

Most common dependency substitutions are now managed by the Brightspot Gradle plugins, but custom substitutions can be added. See dependencySubstitutions in the plugin guide for details.

Set the org.apache.solr:solr-solrj constraint like the example below. If your project is moving to cloud and a SOLR upgrade has been applied to the cluster, set the value to 8.11.1, otherwise use the value forced in {site/web}/build.gradle (likely 8.6.1). Any forced dependency constraints from the now-deleted /build.gradle must be added to dependencyConstraints in settings.gradle as well.

brightspot {
    dependencyConstraints = [
        'org.apache.solr:solr-solrj': '8.11.1', // XXX or forced value in {site/web}/build.gradle
    ]
}

git commit -a -m "Gradle plugin upgrade: Update settings.gradle"

4. Update `core/build.gradle`

Add the brightspot java plugin to the very top of the file:

plugins {
    id 'com.brightspot.java-core'
}

Standardize on using api rather than compile for all dependencies in core/build.gradle. Either use a text editor to find and replace, or do it on the command line like so:

sed -i '' 's/ compile / api /; s/constraints.compile(/constraints.api(/;' core/build.gradle

Check the file to ensure the changes are what you expect: All compile lines are changed to api.

Similarly, runtime should be replaced with runtimeOnly, testCompile with testImplementation, and testRuntime with testRuntimeOnly. compileOnly and testCompileOnly should remain as-is. More on deprecated configurations in the Gradle documentation

Testing dependencies have been updated. Remove these dependencies and run a build of the core project to ensure that no custom test work relied on these.

dependencies {
    testImplementation 'com.psddev.component-lib:test-utils'
    testImplementation 'com.psddev.component-lib:shared-unit-tests'
}

Assuming no build failures from the previous step, add shared unit tests to the dependencies to replace the old tests:

dependencies {
    sharedTest 'com.brightspot.shared-tests:pack-standard-backend-unit-tests'
}

Remove the com.psddev.component-lib:materialize dependency, if present.

Remove any lines like this:

  project.afterEvaluate {
      jar.dependsOn(test)
  }

If core/host.gradle exists and it is included into core/build.gradle via apply from: file("host.gradle"), remove the include and move the dependencies from core/host.gradle into the dependencies in core/build.gradle. Be sure to remove the core/host.gradle file as well.

Any instances of force = true in core/build.gradle need to be converted into dependency constraints in settings.gradle.

For projects using com.brightspot.gradle at version 1.1.1 or later, any dependency exclusions should be moved into settings.gradle. Refer to the Brightspot Gradle Plugins guide for proper syntax.

If the build fails with javadoc parsing errors, you can either fix them or skip javadoc processing like so:

tasks.withType(Javadoc).configureEach { enabled = false }

git commit -a -m "Gradle plugin upgrade: Update core/build.gradle"

5. Update `frontend/build.gradle`

frontend/build.gradle should only contain the following lines; anything else in this file can be deleted:

plugins {
    id 'com.brightspot.viewgen'
}

description = '$PROJECT: Frontend'

If the root styleguide directory is in the project root (rather than under the frontend/ directory), add the following to frontend/build.gradle:

viewgen {
    styleguideDir = '../styleguide'
}

Any files in frontend/src/main/webapp must be moved to web/src/main/webapp (or site/src/main/webapp). For example, frontend/src/main/webapp/WEB-INF/web.xml to web/src/main/webapp/WEB-INF/web.xml

Please pay special attention to web.xml. It is imperative that this file is located within the web (or site) directory listed above. If you encounter any errors pertaining to org.apache.jasper.JasperException or com.psddev.dari.web.NoCurrentWebRequestException please ensure that web.xml is in the correct location before troubleshooting further.

git commit -a -m "Gradle plugin upgrade: Update frontend/build.gradle"

6. Update each `bundle`’s `build.gradle`

Each frontend bundle’s build.gradle should contain the following:

plugins {
    id 'com.brightspot.bundle'
}

description = 'bundle-default' // XXX Just leave this description alone if it's set already,
                               // XXX otherwise set it to match the bundle directory name

bundle {
    yarnBuildCommand = 'build'
    styleguideOutputDir = 'styleguide'
    zipOutputPath = 'theme.zip'

    // for CdnCssTest
    cdnCssFiles = [
        'styles/default/All.min.css' // XXX Change this to the correct path(s) of All.min.css
    ]
}

node {
    yarnVersion = '1.22.19' // XXX Copy these versions from the existing /build.gradle
    version = '18.17.1' // XXX If the existing node version is earlier than 14, you'll also need to upgrade styleguide.
    download = true
}

dependencies {
    sharedTest 'com.brightspot.shared-tests:pack-standard-frontend-unit-tests'
}

If the bundle/theme’s build.gradle contains the test{} block in order to allocate more memory for the heap, you need to wrap it with an afterEvaluate{} since the BrightspotProjectPlugin adds dependency substitutions with its own afterEvaluate block and the test execution should be evaluated after that:

afterEvaluate {
    test {
        maxHeapSize = "4g"
    }
}

git commit -a -m "Gradle plugin upgrade: Update bundle build.gradle"

7. Update `web/build.gradle`

The build.gradle file that generates the war file should contain the following:

Some additional api, compile, or implementation dependencies, such as org.apache.solr:solr-solrj may be present in the existing web/build.gradle. These must be removed and added to core/build.gradle! Additionally, any instances of force = true in web/build.gradle need to be converted into dependency constraints in settings.gradle (see Update settings.gradle above for an example).

plugins {
    id 'com.brightspot.war'
}

description = '$PROJECT: Web'

dependencies {
    // XXX Project-specific lines similar to the following three lines should already be present in the
    // XXX existing web/build.gradle. Make sure they're using implementation instead of api, and otherwise leave them alone.
    implementation project(':frontend')
    implementation project(':core')
    implementation project(':bundle-default')

    // for expansion into the war
    compileOnly 'com.psddev:cms-tool-ui'

    // Tests
    sharedTest 'com.brightspot.go:lib-util-gradle-dynamic-test' // XXX Only if this is already present
    sharedTest 'com.brightspot.shared-tests:pack-standard-backend-integration-tests'
}

git commit -a -m "Gradle plugin upgrade: Update web/build.gradle"

8. Update any other subproject `build.gradle` files

An example of one of these would be a migration/build.gradle file.

These must be treated on a case-by-case basis but can probably follow the same process as core/build.gradle.

9. Verify the build

As before, run the build, bookmark the gradle scan, and set the war file aside for comparison.

./gradlew clean build --no-build-cache --scan
cp web/build/libs/$PROJECT*.war /tmp/$PROJECT-after.war

We’re going to execute a very simple test that compares the filenames in the “before” war file to the filenames in the “after” war file. This isn’t a perfect test, as the file contents could differ, but this simple test is sufficient for the purpose of this upgrade.

diff <(unzip -l /tmp/$PROJECT-before.war) <(unzip -l /tmp/$PROJECT-after.war)

The expected output from this command is something like this:

1c1
< Archive:  /tmp/$PROJECT-before.war
---
> Archive:  /tmp/$PROJECT-after.war

Anything else indicates that one or more files have been added or removed. The steps to remediate depend on the file that has changed, and are outside of the scope of this document.

The build scans taken in the first and last steps may be useful in troubleshooting this or other issues.

10. Other Configuration Changes

Move ignore test files after gradle upgrade

If the ignore files for the below tests are under site/src/test/resources/brightspot/, they need to move to core/src/test/resources/brightspot/ so those tests can continue to be ignored

AlterationClassAnnotationProcessorTest.ignored.txt
AlterationFieldAnnotationProcessorTest.ignored.txt
NoAlterationMethodsExistTest.ignored.txt
ModificationFieldInternalNamePrefixTest.ignored.txt
NoAfterCreateOutsideModificationsTest.ignored.txt
NoFinalFieldsInRecordsTest.ignored.txt
JobStatusTest.ignored.txt
RecordableHasNullaryConstructorTest.ignored.txt

docker-compose.yml

The war file name has changed as part of this update, so you’ll need to change docker-compose.yml to reflect this.

services: // XXX lines omitted for brevity
  tomcat:
    environment:
      - ROOT_WAR=/code/web/build/libs/${PROJECT}-web.war

gradle.properties

Double-check the file gradle.properties to ensure it contains a property like org.gradle.caching=true.

Upgrading Brightspot Gradle Plugins

Goals

Conventions

Set $PROJECT to the project name

Resolve // XXX comments