GIT_ATTRIBUTES massive slowdown in Spotless 6.6.0+ #1527
Description
We have a custom Gradle plugin for spotless, bellow is a shortened (with relevant parts) version.
We use this custom plugin in other gradle projects. Once we upgraded spotless plugin to the version 6.6.0 onwards (including the latest 6.13.0) it causes us problems during build on Windows 11 platform. With the version 6.5.2 it works all right.
For example following command takes 1-5 seconds to finish (once all libraries are cached)
./gradlew buildEnvironment
But once we switch to Spotless version 6.6.0+ it takes 1-2 minutes to finish (even on repeated runs with everything cached)
The same happens with all build tasks like
./gradlew build -x test
Using:
Gradle 7.5.1 and Java 11
Windows 11 Pro - 22H2 (OS build: 22621.525), but happens on any Windows laptop (tested on 5 different Windows machines using Win 11)
NOTE1:
Now I found out that there is a delay even on Linux when switchnig from 5.5.2 to 6.6.0 version, but it is from 1 second to 7 seconds for the following task. On Windows however it takes minutes.
./gradlew buildEnvironment
NOTE2:
I was not able to see a big difference while running spotlessApply tasks.
NOTE3:
I tried to change jgit version versions (betweem 5.10 to latest) and it didn't help
I tried to inspect the gradle tasks using --debug option with spotless 6.6.0+ and bellow are some critical slow parts (there are big jumps between some of the log output):
... ... previous logs ... 2023-01-24T12:33:56.076+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Execute Project.afterEvaluate listener' completed 2023-01-24T12:33:56.076+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Execute Project.afterEvaluate listener' completed 2023-01-24T12:33:56.078+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Dev\mdm\.git\config, create new FileSnapshot: lastRead=2023-01-24 12:33:56.079005900, lastModified=2023-01-24 11:44:04.962093000, size=322, fileKey=java.lang.Object@a6f1ceb 2023-01-24T12:33:58.570+0100 [LIFECYCLE] [org.gradle.cache.internal.DefaultFileLockManager] 2023-01-24T12:33:58.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:33:58.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:33:58.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:33:58.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:33:58.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:33:58.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:34:08.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:34:08.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:34:08.572+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:34:08.572+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:34:08.572+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:34:08.572+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:33:57.611+0100 [LIFECYCLE] [org.gradle.internal.logging.progress.ProgressLoggerFactory] 2023-01-24T12:33:57.611+0100 [LIFECYCLE] [org.gradle.internal.logging.progress.ProgressLoggerFactory] > Configure project : 2023-01-24T12:34:14.586+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Dev\mdm\.git\config, create new FileSnapshot: lastRead=2023-01-24 12:34:14.587001900, lastModified=2023-01-24 11:44:04.962093000, size=322, fileKey=java.lang.Object@a6f1ceb 2023-01-24T12:34:14.587+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Dev\mdm\.git\config, create new FileSnapshot: lastRead=2023-01-24 12:34:14.587995600, lastModified=2023-01-24 11:44:04.962093000, size=322, fileKey=java.lang.Object@a6f1ceb 2023-01-24T12:34:14.587+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Dev\mdm\.git\config, create new FileSnapshot: lastRead=2023-01-24 12:34:14.587995600, lastModified=2023-01-24 11:44:04.962093000, size=322, fileKey=java.lang.Object@a6f1ceb 2023-01-24T12:34:14.587+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Users\temp\.config\jgit\config.lock, cannot be racily clean 2023-01-24T12:34:14.587+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Users\temp\.gitconfig, isRacyClean=false, read=2023-01-24 11:44:42.347606300, lastModified=1970-01-01 01:00:00.000000000, delta=1674557082347606300 ns, racy<=2500000000 ns 2023-01-24T12:34:14.587+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Users\temp\.gitconfig, is unmodified 2023-01-24T12:34:14.587+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Dev\mdm\.git\config, create new FileSnapshot: lastRead=2023-01-24 12:34:14.587995600, lastModified=2023-01-24 11:44:04.962093000, size=322, fileKey=java.lang.Object@a6f1ceb 2023-01-24T12:34:14.587+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Users\temp\.config\jgit\config.lock, cannot be racily clean 2023-01-24T12:34:14.587+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Users\temp\.gitconfig, isRacyClean=false, read=2023-01-24 11:44:42.347606300, lastModified=1970-01-01 01:00:00.000000000, delta=1674557082347606300 ns, racy<=2500000000 ns 2023-01-24T12:34:14.587+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Users\temp\.gitconfig, is unmodified ... ... another logs ... 2023-01-24T12:34:54.053+0100 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository] Found artifact 'klob-0.2.1.jar (com.github.shyiko.klob:klob:0.2.1)' in resolver cache: C:\Users\temp\.gradle\caches\modules-2\files-2.1\com.github.shyiko.klob\klob\0.2.1\f6bc8b98c3fcce8a559beb2b6e8ec58efadb5bbe\klob-0.2.1.jar 2023-01-24T12:34:54.053+0100 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository] Found artifact 'kotlin-compiler-embeddable-1.7.0.jar (org.jetbrains.kotlin:kotlin-compiler-embeddable:1.7.0)' in resolver cache: C:\Users\temp\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-compiler-embeddable\1.7.0\4d09852239f61842072b40b524882fa3caaed371\kotlin-compiler-embeddable-1.7.0.jar 2023-01-24T12:34:54.054+0100 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository] Found artifact 'picocli-4.6.3.jar (info.picocli:picocli:4.6.3)' in resolver cache: C:\Users\temp\.gradle\caches\modules-2\files-2.1\info.picocli\picocli\4.6.3\18177f4c3d65cc94e6d4039775ec5aed8089f8d0\picocli-4.6.3.jar 2023-01-24T12:34:54.055+0100 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Daemon worker Thread 4: acquired lock on state of project : 2023-01-24T12:34:54.055+0100 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Daemon worker Thread 4: acquired lock on worker lease 2023-01-24T12:34:54.055+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Resolve files of :spotless-1972453405' completed 2023-01-24T12:34:54.069+0100 [DEBUG] [org.eclipse.jgit.internal.storage.file.FileSnapshot] file=C:\Dev\mdm\.git\config, create new FileSnapshot: lastRead=2023-01-24 12:34:54.069859200, lastModified=2023-01-24 11:44:04.962093000, size=322, fileKey=java.lang.Object@a6f1ceb 2023-01-24T12:34:58.570+0100 [LIFECYCLE] [org.gradle.cache.internal.DefaultFileLockManager] 2023-01-24T12:34:58.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:34:58.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:34:58.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:34:58.571+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:34:58.571+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:34:58.571+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:35:08.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:35:08.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:35:08.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:35:08.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:35:08.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:35:08.570+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:35:18.571+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:35:18.571+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:35:18.571+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:35:18.571+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:35:18.571+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:35:18.572+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:35:28.572+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:35:28.572+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:35:28.572+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:35:28.572+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry. 2023-01-24T12:35:28.572+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2023-01-24T12:35:28.572+0100 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2023-01-24T12:35:31.157+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Execute container callback action' 2023-01-24T12:35:31.157+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Execute Project.afterEvaluate listener' 2023-01-24T12:35:31.157+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Execute Project.afterEvaluate listener' started 2023-01-24T12:35:31.157+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Execute Project.afterEvaluate listener' 2023-01-24T12:35:31.157+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Notify afterEvaluate listeners of :' 2023-01-24T12:34:54.108+0100 [LIFECYCLE] [org.gradle.internal.logging.progress.ProgressLoggerFactory] 2023-01-24T12:34:54.108+0100 [LIFECYCLE] [org.gradle.internal.logging.progress.ProgressLoggerFactory] > Configure project : 2023-01-24T12:35:31.157+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Execute container callback action' completed 2023-01-24T12:35:31.157+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Execute Project.afterEvaluate listener' completed 2023-01-24T12:35:31.157+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Execute Project.afterEvaluate listener' completed 2023-01-24T12:35:31.157+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Notify afterEvaluate listeners of :' completed 2023-01-24T12:35:31.158+0100 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Daemon worker Thread 4: released lock on state of project : 2023-01-24T12:35:31.158+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Configure project :' 2023-01-24T12:35:31.158+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Configure project :' completed 2023-01-24T12:35:31.158+0100 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Configure project :assembly' started class SpotlessDefaultsPlugin implements Plugin<Project> { @Override void apply(Project project) { // we want this only once, so we put it on root project project.rootProject.with { repositories { maven { url "$companyBaseRepoUrl/gradle-plugins" } } configurations { spotlessConfig } dependencies { spotlessConfig 'com.company.gradle:codestyle-config:+@zip' } } // and finally for the project which it was applied to project.with { apply plugin: SpotlessPlugin spotless { format 'misc', { target '**/*.gradle', '**/*.md', '**/.gitignore' trimTrailingWhitespace() endWithNewline() } java { target "**/java/**/*.java" targetExclude "**/generated/**/*.*", "**/generated-src/**/*.*", "**/out/**/classes/generated*/**/*.*", "**/gradle/**/*.*", "$rootDir/.codestyle/**/*.*" removeUnusedImports() eclipse('4.12.0').configFile "$rootDir/.codestyle/spotless.eclipseformat.xml" importOrderFile "$rootDir/.codestyle/spotless.eclipseimports.importorder" licenseHeaderFile "$rootDir/.codestyle/spotless.license.txt" } if (Boolean.valueOf(project.findProperty("spotlessNoCheck") ?: false)) { enforceCheck false } } } } }