How Updating to Kotlin 1.4 Broke Our Linter Rules

The Gradle build setup of PSPDFKit for Android is quite extensive, with many different modules, several custom Gradle plugins (also some contributed by a buildSrc project), a custom CMake integration, and a shared build cache for speeding up builds. While the setup is helpful for developing quickly and securely in a shared monorepo codebase, it’s not uncommon that build system upgrades cause regressions in complex build scripts like ours. Recently, we made the jump to the latest stable Android Gradle plugin (AGP), 4.1, as well as to Kotlin 1.4.10, which, among other things, broke the build of custom linter rules we ship in our framework.

Initial Inspection

In our build system upgrade branch, the automated CI checks failed with the following error:

> Task :pspdfkit:prepareLintJarForPublish FAILED

FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':pspdfkit:prepareLintJarForPublish'.
  A failure occurred while executing com.android.build.gradle.internal.tasks.Workers$ActionFacade
    Found more than one jar in the 'lintPublish' configuration. Only one file is supported. If using
    a separate Gradle project, make sure compilation dependencies are using compileOnly
...

This was curious, since we didn’t change anything pertaining to dependencies inside the integration branch. However, given the “move fast, break things” mentality of the Android tooling team, it wasn’t surprising to see this failure (even if the issue was something different in the end). So we started investigating.

Problem Analysis

The first step in figuring out the root cause was to inspect the custom linter module of our framework. Inside our :pspdfkit module (which is the primary Gradle module of our PSPDFKit for Android framework), the following dependency to our linter module was specified:

dependencies {
    // Package our custom lint rules.
    lintPublish project(':lint-checks')
    // ...
}

The lintPublish configuration is part of the Android Gradle plugin’s custom linter API, and it allows us to specify dependencies that contribute custom linter rules to the final AAR library file. This lets us ship linter rules to our customers, i.e. hints and warnings that make it easier to use PSPDFKit in a hassle-free way. A quick full-text search revealed that the lintPublish configuration wasn’t used anywhere else in the project, so this occurrence had to be the only contribution to the build issue. The dependency definition was correct though, and I figured it shouldn’t cause any issues.

In a second step, we inspected the :lint-checks module itself to see if anything about the module’s dependencies changed or was suspicious in any other way. The content of this module’s build.gradle file is short and simple:

apply plugin: 'java-library'
apply plugin: 'kotlin'
apply from: '../gradle/configuration.gradle'

dependencies {
    compileOnly "com.android.tools.lint:lint-api:$LINT_VERSION"
    compileOnly "com.android.tools.lint:lint-checks:$LINT_VERSION"

    testImplementation 'junit:junit:4.12'
    testImplementation "com.android.tools.lint:lint:$LINT_VERSION"
    testImplementation "com.android.tools.lint:lint-tests:$LINT_VERSION"
    testImplementation "com.android.tools:testutils:$LINT_VERSION"
}

sourceCompatibility = '1.8'
targetCompatibility = '1.8'

jar {
    manifest {
        attributes('Lint-Registry-v2': 'com.pspdfkit.lint.checks.PSPDFKitIssueRegistry')
    }
}

Looking at this file, it seemed there wasn’t anything explicitly wrong. So, since we couldn’t find the culprit in our own build script, we decided to take a look at the Android Gradle plugin sources themselves.

Inspecting the Android Gradle Plugin

Since the Android Gradle plugin codebase is large, it was necessary to narrow it down to a specific area of interest. Luckily, finding the failure point of any build is (most of the time) a simple task. Usually, if a Gradle build fails during the execution of a task, it’s due to an unhandled exception. To see the full stack trace of the exception that’s failing a build, you can run the build using the --stacktrace option. In our particular case, the exception causing the build to fail was this:

Caused by: java.lang.RuntimeException: Found more than one jar in the 'lintPublish' configuration. Only one file is supported. If using a separate Gradle project, make sure compilation dependencies are using compileOnly
        at com.android.build.gradle.internal.tasks.PublishLintJarWorkerRunnable.run(PublishLintJarWorkerRunnable.kt:33)
        at com.android.build.gradle.internal.tasks.Workers$ActionFacade.run(Workers.kt:242)

The failing PublishLintJarWorkerRunnable class is responsible for copying any JAR file generated by our lint rule module to the output for further processing into the final AAR file. You can find the full sources of that file online here. Here’s how lines 32–37 of that file read:

if (params.files.size > 1) {
    throw RuntimeException(
        "Found more than one jar in the '"
                + VariantDependencies.CONFIG_NAME_LINTPUBLISH
                + "' configuration. Only one file is supported. If using a separate Gradle project, make sure compilation dependencies are using compileOnly")
}

This meant PublishLintJarWorkerRunnable was indeed receiving too many input files: The task is programmed to copy a single JAR file that contains all the linter rules into the final AAR, but in our specific case, it received multiple input JAR files and failed. Since the error didn’t list the actual JAR files that were provided as input (imagine the “old man yells at cloud” meme here), we had to fire up a step debugger to inspect the running build.

Debugging Gradle

We quickly started a debugging session by attaching Android Studio’s step debugger to the running Gradle build. While this sounds like magic to many developers, it’s actually easy and can be done in two simple steps:

1. Start the Gradle build that should be inspected from the command line. Use the --no-daemon and -Dorg.gradle.debug=true options to put Gradle into debug mode, which will allow connections using a debug client.

2. Connect to the build using Android Studio’s step debugger by creating a remote debug configuration and hitting the Debug button.

ℹ️ Note: For an even simpler debugging experience (i.e. to directly open the relevant source file and place a breakpoint), it can be helpful to add the Android Gradle plugin to your buildSrc project’s dependencies. This will instruct Android Studio to index the AGP source files, making them browsable. If you don’t do this, you can still create a Java exception breakpoint inside Android Studio that will trigger as soon as the exception is thrown.

We opened the PublishLintJarWorkerRunnable.kt file inside Android Studio and placed a breakpoint right before where the exception was thrown. Once the breakpoint triggered, we had a closer look at the params.files property, which listed the following four files:

0 = {File@17651} "android/lint-checks/build/libs/lint-checks.jar"
1 = {File@17652} "~/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.4.10/ea29e063d2bbe695be13e9d044dcfb0c7add398e/kotlin-stdlib-1.4.10.jar"
2 = {File@17653} "~/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.4.10/6229be3465805c99db1142ad75e6c6ddeac0b04c/kotlin-stdlib-common-1.4.10.jar"
3 = {File@17654} "~/.gradle/caches/modules-2/files-2.1/org.jetbrains/annotations/13.0/919f0dfe192fb4e063e7dacadee7f8bb9a2672a9/annotations-13.0.jar"

While the first file was our own lint-checks.jar file that was there correctly, the remaining three files illegally contributed to the lint module’s outputs and shouldn’t have been there. We took another close look at our lint module’s build.gradle file, and we had a round of brainstorming to come up with ideas that could explain where these files came from. For those of you following along at home, I encourage you to do the same and to reread the build.gradle file at the top of this post. Maybe you already have an idea of what was going on.

The Solution. The Fix.

Since all three illegally contributed JAR files were related to Kotlin 1.4.10, we looked at the dependencies added by our :lint-checks module by running ./gradlew :lint-checks:dependencies from the command line. The dependencies task of any Gradle module collects all information about direct and transitive dependencies of the module, and it prints the information to the terminal. When inspecting the output, we saw all three Kotlin dependencies were the sole contributors to the apiDependenciesMetadata and runtimeClasspath configurations. Since our own build script didn’t add these dependencies, they were most likely contributed by the Kotlin plugin added to the Gradle module:

...
runtimeClasspath - Runtime classpath of compilation 'main' (target  (jvm)).
\--- org.jetbrains.kotlin:kotlin-stdlib:1.4.10
     +--- org.jetbrains.kotlin:kotlin-stdlib-common:1.4.10
     \--- org.jetbrains:annotations:13.0

...

To prevent those dependencies from leaking out of the :lint-checks module, we modified the module’s dependencies block to also hold org.jetbrains.kotlin:kotlin-stdlib explicitly inside the compileOnly configuration, which also instructs Gradle to remove that dependency from the public apiDependenciesMetadata and runtimeClasspath configurations:

dependencies {
     compileOnly "com.android.tools.lint:lint-api:$LINT_VERSION"
     compileOnly "com.android.tools.lint:lint-checks:$LINT_VERSION"
+    compileOnly "org.jetbrains.kotlin:kotlin-stdlib:$KOTLIN_VERSION"

     testImplementation "junit:junit:4.12"
     testImplementation "com.android.tools.lint:lint:$LINT_VERSION"
     testImplementation "com.android.tools.lint:lint-tests:$LINT_VERSION"
     testImplementation "com.android.tools:testutils:$LINT_VERSION"
 }

Verdict

After continued investigation (Kotlin’s Slack community to the rescue!), it turns out that, beginning with Kotlin 1.4, the kotlin plugin will automatically add the Kotlin standard library to a module’s dependencies. For us, this caused the standard library to leak to the lint JAR processing task, which in turn didn’t like the result. By explicitly declaring the Kotlin standard library as a compileOnly dependency, we reconfigured Gradle to keep the dependency internal and to stop it from leaking to the lint JAR processor’s inputs.

Maintaining a huge Gradle build setup can be complex and time consuming, but Gradle’s APIs and Android Studio’s build features help a lot with managing this (which our team really appreciates). We’re already waiting for the next challenge posed by the upcoming build system update, and we know it won’t disappoint.