wttech / Gradle Aem Plugin

Table of contents

• About
  • Screenshot
  • Features
  • Compatibility
• Getting started
• Launcher
• Plugins
  • Plugins setup
    • Minimal plugins setup
    • Complete plugins setup
  • Plugins documentation
• How to's
  • Set AEM configuration properly for all / concrete project(s)
  • Understand why there are one or two plugins to be applied in build script
  • Work effectively on start and daily basis
  • Customize convention for CRX package and OSGi bundle names and paths
• Building
• Testing
  • Local instance tests
  • Debugging tests
• Contributing
• License

About

Swiss army knife for AEM related automation. Incremental build which takes seconds, not minutes. Developer who does not loose focus between build time gaps. Extend freely your build system directly in project.

AEM developer - it's time to meet Gradle! You liked or used plugin? Don't forget to star this project on GitHub :)

Be inspired by:

• watching live demos:

  • Sling AdaptTo 2020 Conference,
  • Sling AdaptTo 2018 Conference,

• reading blog articles:

  • Gradle-powered AEM archetypes by Krystian Panek,
  • AEM Dispatcher with ease - Gradle AEM Plugins by Damian Mierzwiński,
  • & more....

Looking for a dedicated version of plugin for Apache Sling? Check out Gradle Sling Plugin!

Demo

The example below presents building & deploying AEM package - all handled by Gradle. To review building AEM package by Maven but all the rest handled by Gradle/GAP see Enhancing Maven build section.

What is being done above by simply running super easy command sh gradlew <=> gw ?

• :env:instanceProvision - configuring AEM instances / performing provisioning steps like:
  • deploying dependent CRX packages,
  • enabling access to CRX DE (e.g on all environments except production),
  • configuring replication agents (AEM platform wide changes),
• :app:aem:all:packageDeploy -> building & deploying all-in-one CRX package to AEM instances in parallel, then awaiting for stable condition of AEM instances and built application.

Build is incremental which guarantees optimized time every time regardless of build command used. Only changed parts of application are processed again:

• Dependent CRX packages are installed only when they are not already installed on particular AEM instances.
• Provisioning steps are performed only once on each AEM instance (by default but customizable),
• CRX package is rebuild only when JCR content / files under jcr_root are changed.
• Java code is recompiled only when code in *.java files is changed.
• Front-end / Webpack build is run again only when code in *.scss and *.js etc files is changed.

Want to see it in action? Follow here!

Features

• Automated native AEM instances setup optimized for best development experience.
• Powerful AEM DSL scripting capabilities for performing JCR content migrations, managing AEM instances, integrating with Docker based tools.
• Advanced AEM instance(s) stability & health checking after CRX package deployment.
• Continuous AEM incident monitoring and interactive reporting (centralized log tailing of any AEM instances with no SSH).
• Easy parallel CRX package deployment to many remote group of instances.
• Fail-safe dependent CRX packages installation from local and remote sources using various protocols (SMB / SSH / HTTP / custom).
• Fast JCR content synchronization from running AEM instances with advanced content normalization.
• Composing CRX package from multiple separate JCR content roots, bundles.
• Validating CRX package using seamless integration with OakPAL tool.
• All-in-one CRX packages generation (assemblies), vault filters merging etc.
• Easy OSGi bundle customization with BND tool embedded.

Gradle AEM Plugin is following strategy convention over configuration. When following built-in conventions about project structure & naming, then only minimal configuration is required. Still all features are fully configurable.

Compatibility

Note that since GAP 14.4.22 default Java version used to compile and run AEM instances is Java 11. To instruct GAP to use Java 8, consider setting property:

javaSupport.version=8

Getting started

Most effective way to experience Gradle AEM Plugin is to use:

• GAP Launcher - recommended for adding Gradle/GAP support to existing AEM Maven builds (archetype-compatible),
• AEM Single-Project Example - recommended for application/library development,
• AEM Multi-Project Example - recommended for long-term project development,
• AEM Boot - only booting local AEM instances and AEM dispatcher automatically. Useful when building CRX packages is covered separately, e.g by Maven & Content Package Maven Plugin.

The only software needed on your machine to start using plugin is Java 8 or newer (also to setup local native AEM instances). Optionally, Docker is needed (when using automatic AEM dispatcher setup).

As a build command, it is recommended to use Gradle Wrapper (gradlew) instead of locally installed Gradle (gradle) to easily have same version of build tool installed on all environments. Only at first build time, wrapper will be automatically downloaded and installed, then reused.

Launcher

To use Gradle AEM Plugin it is not needed to have configured a regular Gradle project. By using a single bash command, to be able to:

• set up local AEM instances and be able to share command to others even on team chat,
• download and deploy to AEM instances dependent CRX package from any source by single command run in terminal,
• copy JCR content between instances using GAP run on continuous integration server,
• and more,

consider using standalone launcher as it could be the easiest and fastest way to use GAP.

Plugins

Plugins setup

Released versions of plugin are available on:

• Gradle Plugin Portal,
• Bintray (when using this repository proper URL need to be added into buildscript section).

Recommended way is to apply plugin using Gradle Plugin Portal and techniques described there.

Minimal plugins setup

Configuration assumes:

• building and deploying CRX packages to AEM instance(s) via command: gradlew packageDeploy.
• JCR content placed under directory src/main/content/jcr_root
• Vault filters located at path src/main/content/META-INF/vault/filter.xml,

Then the only thing needed to build CRX package is plugin application (all the rest is obtained automatically by convention):

File build.gradle.kts:

plugins {
    id("com.cognifide.aem.package") version "<version>"
}

Complete plugins setup

Illustrative configuration below assumes building and deploying on AEM instance(s) via command: gradlew (default tasks will be used). Intention of snippet below is to demonstrate:

• How particular values could be customized via Gradle AEM DSL,
• What are the default values and how they are determined.

plugins {
    id("com.cognifide.environment") version "<version>" // useful to setup AEM dispatcher running on Docker
    id("com.cognifide.aem.instance.local") version "<version>" // useful to setup local AEM instances running natively, skip '.local' to work with remote only
    id("org.jetbrains.kotlin.jvm") // needed when AEM code written in Kotlin, yes it could be :)
    id("com.cognifide.aem.bundle") version "<version>" // needed to built OSGi bundle
    id("com.cognifide.aem.package") version "<version>" // needed to build CRX package from JCR content and built OSGi bundle
}

group = "com.company.aem"
version = "1.0.0"
defaultTasks(":instanceProvision", ":packageDeploy")

aem {
    `package` { // built CRX package options
        contentDir.set(project.file("src/main/content"))
        appPath.set(when {
            project == project.rootProject -> "/apps/${project.rootProject.name}"
            else -> "/apps/${project.rootProject.name}/${projectName}"
        })
        nodeTypesSync("PRESERVE_AUTO")
        validator {
            enabled.set(prop.boolean("package.validator.enabled") ?: true)
            verbose.set(prop.boolean("package.validator.verbose") ?: true)
            planName.set(prop.string("package.validator.plan") ?: "plan.json")
            severity("MAJOR")
        }       
        // ...
    }
    instance { // AEM instances to work with
        local("http://localhost:4502") // local-author
        local("http://localhost:4503") // local-publish
        remote("http://192.168.100.101:4502", "int-author")
        remote("http://192.168.100.101:4503", "int-publish")
        // etc
        
        http { // allows to customize HTTP connection to AEM instances
            connectionTimeout.set(prop.int("instance.http.connectionTimeout") ?: 30000)
            connectionRetries.set(prop.boolean("instance.http.connectionRetries") ?: true)
            connectionIgnoreSsl.set(prop.boolean("instance.http.connectionIgnoreSsl") ?: true)
    
            proxyHost.set(prop.string("instance.http.proxyHost"))
            proxyPort.set(prop.int("instance.http.proxyPort"))
            proxyScheme.set(prop.string("instance.http.proxyScheme"))
        }

        provisioner { // configuring AEM instances in various circumstances (e.g only once)
            enableCrxDe()
            deployPackage("com.adobe.cq:core.wcm.components.all:[email protected]")
            deployPackage("com.neva.felix:search-webconsole-plugin:1.3.0")
            step("setup-replication-author") {
                condition { once() && instance.author }
                sync {
                    repository {
                        save("/etc/replication/agents.author/publish/jcr:content", mapOf(
                                "enabled" to true,
                                "userId" to instance.user,
                                "transportUri" to "http://localhost:4503/bin/receive?sling:authRequestLogin=1",
                                "transportUser" to instance.user,
                                "transportPassword" to instance.password
                        ))
                    }
                }
            }
            // ...
        }  
    }
    localInstance { // config for AEM instances to be created on local file system
        quickstart {
            jarUrl.set(prop.string("localInstance.quickstart.jarUrl"))
            licenseUrl.set(prop.string("localInstance.quickstart.licenseUrl"))
        }
        backup {
            uploadUrl.set(prop.string("localInstance.backup.uploadUrl"))
            downloadUrl.set(prop.string("localInstance.backup.downloadUrl"))
        }
        install { // CRX packages and OSGi bundles to be pre-installed on created AEM instances
            files(
                "http://.../package.zip" // CRX package downloaded over HTTP
                "group:name:version" // OSGi bundle from Maven repository
            )
        }
        init { // hook called once in scope of instance just created and up first time
            logger.info("Initializing instance '$name'")
            sync {
                // ...
            }
        }   
        rootDir.set(prop.string("localInstance.rootDir"))
        // ...
    }

    tasks {
        jar { 
            bundle {
                // customizing OSGi bundle manifest
                description = "Example application built by GAP"
                docUrl = "https://github.com/wttech/gradle-aem-example"
                exportPackage("com.company.example.aem.*") 
                slingModelPackages = "com.company.example.aem"
        
                // for checking OSGi component health on runtime
                javaPackage.set("com.company.example.aem")
    
                // other / more advanced options
                importPackageWildcard.set(true)
                // ...
            }
        }
        packageCompose { // customizing built CRX package
            nestPackageProject(":core")
            nestPackageProject(":config")
            
            archiveBaseName.set("example-for-changing-zip-name")
            
            vaultDefinition { // place for overriding CRX Package / Vault properties, defining hooks
                // ...
            }
        }
        // ... and all other tasks
    }
}

To see all available options and actual documentation, please follow to:

• aem - AemExtension
• package - PackageOptions
• instance - InstanceManager
• localInstance - LocalInstanceManager
• bundleCompose - BundleCompose
• packageCompose - PackageCompose
• instanceProvision - InstanceProvision
• ... - other tasks in similar way.

Plugins documentation

Gradle AEM Plugin to be more concise is now more like set of plugins. Each plugin has its own documentation:

• Common Plugin - defining common options like AEM instances available etc, base for custom Gradle tasks scripting for AEM,
• Package Plugin - building and deploying CRX package(s),
• Package Sync Plugin - synchronizing JCR content from running AEM instances into built CRX packages,
• Bundle Plugin - building and deploying OSGi bundle(s),
• Instance Plugin - managing remote AEM instance(s), automatic installation of service packs, performing provisioning actions,
• Local Instance Plugin - setting up local AEM instance(s),
• Environment Plugin - standalone plugin for setting AEM dispatcher running on Docker,
• Common Plugin - standalone plugin for transferring files over protocols SFTP/SMB/HTTP, e.g downloading AEM files, uploading AEM backups.

How to's

Set AEM configuration properly for all / concrete project(s)

Common configuration like root of content for JCR package, should be defined in allprojects section like below / e.g in root build.gradle.kts file:

allprojects {
  plugins.withId("com.cognifide.aem.common") {
    configure<AemExtension> {
        `package` {
            contentDir.set(project.file("src/main/aem")) // overrides default dir named 'content'
        }
    }
  }
  
  plugins.withId("com.cognifide.aem.bundle") {
    configure<AemExtension> {
        tasks {
            jar {
                bundle {
                    category = "example"
                    vendor = "Company"
                }
            }
        }
    }
  
    dependencies {
        "compileOnly"("com.adobe.aem:uber-jar:${Build.AEM_VERSION}:apis") // and more
    }
  }
}

For instance, subproject :aem:core specific configuration like OSGi bundle or CRX package options should be defined in aem/core/build.gradle.kts:

import com.cognifide.gradle.aem.bundle.tasks.bundle

plugins {
    id("com.cognifide.aem.bundle")
}

tasks {
    jar {
        bundle {
            javaPackage.set("com.company.example.aem.core")
        }
    }
    packageCompose {
        nestPackageProject(':content')
        archiveBaseName.set("example-core")
        duplicatesStrategy = DuplicatesStrategy.EXCLUDE
    }
}

Understand why there are one or two plugins to be applied in build script

Gradle AEM Plugin assumes separation of 5 plugins to properly fit into Gradle tasks structure correctly.

Most often, Gradle commands are being launched from project root and tasks are being run by their name e.g instanceStatus (which is not fully qualified, better if it will be :instanceStatus of root project). Let's imagine if task instanceStatus will come from package plugin, then Gradle will execute more than one instanceStatus (for all projects that have plugin applied), so that this is unintended behavior. Currently used plugin architecture solves that problem.

Work effectively on start and daily basis

Initially, to create fully configured local AEM instances simply run command gradlew instanceSetup. Later during development process, building and deploying to AEM should be done using the command: gradlew instanceProvision packageDeploy.

• Firstly dependent packages (like AEM hotfixes, Vanity URL Components etc) will be installed lazily (only when they are not installed yet).
• In next step application is being built and deployed to all configured AEM instances.
• Finally build awaits till all AEM instances and built application are stable.

Customize convention for CRX package and OSGi bundle names and paths

Because of bug related with regresion introduced in Gradle 5.1, there are some difficulties with setting archives base names. AEM Plugin is overriding default Gradle convention for not only having project name in archive base name, but also to having prefix - root project name when project is one of subprojects (multi-project build case as in Gradle AEM Multi). However overriding this convention might not be trivial and is not recommended as of AEM Plugin in most cases proposes good enough battle-tested convention.

Still, if it is really needed to be done - setting customized name for CRX packages and OSGi bundles built, use snippet:

subprojects {
    afterEvaluate {
        tasks {
            withType<AbstractArchiveTask>().configureEach {
                archiveBaseName.set("acme-${project.name}")
            }
        }
    }
}

Then, also common case is to customize paths in which OSGi bundles should be placed in built CRX package. As practice shows up, mostly desired snippet to be used is:

subprojects {
    plugins.withId("com.cognifide.aem.package") {
        configure<AemExtension> {
            `package` {
                installPath.set("/apps/acme/${project.name}/install")
            }
        }
    }
}

Building

1. Clone this project using command git clone https://github.com/wttech/gradle-aem-plugin.git
2. To build plugin, simply enter cloned directory run command: gradlew
3. To debug plugin under development in tests, use commands:
   • For functional tests: sh gradlew functionalTest --debug-jvm
   • For unit tests: sh gradlew test --debug-jvm
4. To debug built plugin in project when published to local Maven repository:
   • Append to any build command parameters --no-daemon -Dorg.gradle.debug=true
   • Run build, it will suspend, then connect remote at port 5005 by using IDE
   • Build will proceed and stop at previously set up breakpoint.

Testing

Local instance tests

Part of functional tests are using real AEM to ensure correctness of features. As of AEM is not available to the public, it needs to be provided externally from remote server or by providing local file path.

AEM files available locally:

gradlew functionalTest \
-DlocalInstance.jarUrl=/Users/krystian.panek/Servers/aem65/cq-quickstart-6.5.0.jar \
-DlocalInstance.licenseUrl=/Users/krystian.panek/Servers/aem65/license.properties

AEM files hosted externally:

gradlew functionalTest \
-DlocalInstance.jarUrl=https://my-company.com/cq/6.5.0/cq-quickstart-6.5.0.jar \
-DlocalInstance.licenseUrl=https://my-company.com/cq/6.5.0/license.properties \
-DfileTransfer.user=foo \
-DfileTransfer.password=pass

Debugging tests

To debug plugin source code while:

• running functional tests, append --debug-jvm -Porg.gradle.testkit.debug=true.
• project using plugin, append --no-daemon -Dorg.gradle.debug=true.

Gradle will stop for a moment and wait until remote connection at port 5005 will be established from e.g IDE.

Contributing

Issues reported or pull requests created will be very appreciated.

1. Fork plugin source code using a dedicated GitHub button.
2. Do code changes on a feature branch created from develop branch.
3. Create a pull request with a base of develop branch.

License

Gradle AEM Plugin is licensed under the Apache License, Version 2.0 (the "License")