-
Notifications
You must be signed in to change notification settings - Fork 2.3k
[rewrite] Add GradleBestPractices
#19885
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| name: Gradle Precommit | ||
| name: Gradle PreCommit | ||
| on: [pull_request] | ||
|
|
||
| jobs: | ||
|
|
@@ -21,8 +21,11 @@ jobs: | |
| java-version: ${{ matrix.java }} | ||
| distribution: temurin | ||
| cache: gradle | ||
| - name: Run Gradle (precommit) | ||
| - name: Run Gradle (pre-commit) | ||
| continue-on-error: ${{ matrix.experimental }} | ||
| shell: bash | ||
| run: | | ||
| ./gradlew javadoc precommit --parallel | ||
| run: ./gradlew javadoc precommit --parallel | ||
| - name: Run Gradle (sanity-check) | ||
| continue-on-error: ${{ matrix.experimental }} | ||
| shell: bash | ||
| run: ./gradlew rewriteDryRun -Dorg.gradle.jvmargs=-Xmx8G | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. assuming we can reuse cached compile phase somehow. is this actual true? kindly request @timtebeek 🍀 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't know; haven't tried. For large projects like this one I'd advise against running against every pull request, and instead run periodically against pre-built serialized LSTs through Moderne, as it can take a long time to run here.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for the update. So, this means we have to rely on an externally hosted Moderne instance, right? Yes, building on every commit doesn’t seem ideal. On the other hand, it would also be unpleasant to fix unrelated issues, and these kinds of diffs can be quite large since they don’t run on every increment. Of course, it’s very resource-intensive, but the tradeoff favors maintaining a clean state over minimizing CI costs. The good news is that it’s running quite fast now — around 15 minutes. Six months ago, it took about 45 minutes, so there’s already been significant improvement. The sanity check also runs much faster than the current pre-commit goal, wondering what in detail it does given the long execution time. Currently, this is handled similarly in Checkstyle and Spotless — we run and check on every increment, assuming we’re on the happy path with minimal or no changes. If that’s the case, it actually supports your approach of running these checks only occasionally rather than continuously. However, assuming we need to relying on external resources, I’d suggest taking the strict approach for now — checking on every increment — leaving any pipeline optimization or downsizing for the future. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| /* | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| * | ||
| * The OpenSearch Contributors require contributions made to | ||
| * this file be licensed under the Apache-2.0 license or a | ||
| * compatible open source license. | ||
| */ | ||
|
|
||
| project.apply plugin: 'org.openrewrite.rewrite' | ||
|
|
||
| rewrite { | ||
| activeRecipe('org.opensearch.openrewrite.SanityCheck') | ||
| exclusion('**ActionListener.java') | ||
| exclusion('**SearchAfterIT.java') | ||
| exclusion('**StarTreeMapper.java') | ||
| exclusion('**package-info.java') | ||
| setExportDatatables(true) | ||
| setFailOnDryRunResults(true) | ||
| } | ||
|
|
||
| dependencies { | ||
| rewrite(platform('org.openrewrite.recipe:rewrite-recipe-bom:3.17.0')) | ||
| rewrite('org.openrewrite.recipe:rewrite-migrate-java:3.20.0') | ||
| rewrite('org.openrewrite.recipe:rewrite-java-security:3.19.2') | ||
| rewrite('org.openrewrite.recipe:rewrite-rewrite:0.14.1') | ||
| rewrite('org.openrewrite.recipe:rewrite-static-analysis:2.20.0') | ||
| rewrite('org.openrewrite.recipe:rewrite-third-party:0.30.0') | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,8 +1,8 @@ | ||
| distributionBase=GRADLE_USER_HOME | ||
| distributionPath=wrapper/dists | ||
| distributionUrl=https\://services.gradle.org/distributions/gradle-9.2.0-all.zip | ||
| distributionUrl=https\://services.gradle.org/distributions/gradle-9.2.0-bin.zip | ||
| networkTimeout=10000 | ||
| validateDistributionUrl=true | ||
| zipStoreBase=GRADLE_USER_HOME | ||
| zipStorePath=wrapper/dists | ||
| distributionSha256Sum=16f2b95838c1ddcf7242b1c39e7bbbb43c842f1f1a1a0dc4959b6d4d68abcac3 | ||
| distributionSha256Sum=df67a32e86e3276d011735facb1535f64d0d88df84fa87521e90becc2d735444 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -124,14 +124,14 @@ public void testReadWritePaths() throws IOException, InterruptedException { | |
|
|
||
| public void testMaxProcesses() throws IOException, InterruptedException { | ||
| String limits = executeCommand("sudo su -c 'cat /proc/" + opensearchPid + "/limits'", "Failed to read process limits"); | ||
| assertTrue("Max processes limit should be 4096 or unlimited", | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. also spotless is not spotless, like anything else made by error-prone human being. mea culpa. |
||
| assertTrue("Max processes limit should be 4096 or unlimited", | ||
| limits.contains("Max processes 4096 4096") || | ||
| limits.contains("Max processes unlimited unlimited")); | ||
| } | ||
|
|
||
| public void testFileDescriptorLimit() throws IOException, InterruptedException { | ||
| String limits = executeCommand("sudo su -c 'cat /proc/" + opensearchPid + "/limits'", "Failed to read process limits"); | ||
| assertTrue("File descriptor limit should be at least 65535", | ||
| assertTrue("File descriptor limit should be at least 65535", | ||
| limits.contains("Max open files 65535 65535") || | ||
| limits.contains("Max open files unlimited unlimited")); | ||
| } | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| --- | ||
| type: specs.openrewrite.org/v1beta/recipe | ||
| name: org.opensearch.openrewrite.SanityCheck | ||
| displayName: Apply all Java & Gradle best practices | ||
| description: Comprehensive code quality recipe combining modernization, security, and best practices. | ||
| tags: | ||
| - java | ||
| - gradle | ||
| - static-analysis | ||
| - cleanup | ||
| recipeList: | ||
| - org.openrewrite.gradle.EnableGradleBuildCache | ||
| - org.openrewrite.gradle.EnableGradleParallelExecution | ||
| - org.openrewrite.gradle.GradleBestPractices | ||
| - org.openrewrite.java.format.NormalizeLineBreaks | ||
| - org.openrewrite.java.format.RemoveTrailingWhitespace | ||
| - org.openrewrite.staticanalysis.RemoveUnusedPrivateMethods | ||
| # TBD | ||
| # - org.openrewrite.java.RemoveUnusedImports | ||
| # - org.openrewrite.java.format.NormalizeFormat | ||
| # - org.openrewrite.java.migrate.UpgradeToJava17 | ||
| # - org.openrewrite.java.migrate.lang.FindVirtualThreadOpportunities # don't want to use: https://github.com/diffplug/spotless/pull/2684#discussion_r2433831887 | ||
| # - org.openrewrite.java.migrate.lang.StringRulesRecipes | ||
| # - org.openrewrite.java.migrate.util.JavaLangAPIs | ||
| # - org.openrewrite.java.migrate.util.JavaUtilAPIs | ||
| # - org.openrewrite.java.migrate.util.MigrateInflaterDeflaterToClose | ||
| # - org.openrewrite.java.migrate.util.ReplaceStreamCollectWithToList | ||
| # - org.openrewrite.java.migrate.util.SequencedCollection | ||
| # - org.openrewrite.java.recipes.JavaRecipeBestPractices | ||
| # - org.openrewrite.java.recipes.RecipeTestingBestPractices | ||
| # - org.openrewrite.java.security.JavaSecurityBestPractices | ||
| # - org.openrewrite.staticanalysis.BufferedWriterCreationRecipes | ||
| # - org.openrewrite.staticanalysis.CommonStaticAnalysis | ||
| # - org.openrewrite.staticanalysis.EqualsAvoidsNull | ||
| # - org.openrewrite.staticanalysis.JavaApiBestPractices | ||
| # - org.openrewrite.staticanalysis.LowercasePackage | ||
| # - org.openrewrite.staticanalysis.MissingOverrideAnnotation | ||
| # - org.openrewrite.staticanalysis.ModifierOrder | ||
| # - org.openrewrite.staticanalysis.NoFinalizer | ||
| # - org.openrewrite.staticanalysis.NoToStringOnStringType | ||
| # - org.openrewrite.staticanalysis.NoValueOfOnStringType | ||
| # - org.openrewrite.staticanalysis.RemoveUnusedLocalVariables | ||
| # - org.openrewrite.staticanalysis.RemoveUnusedPrivateFields | ||
| # - org.openrewrite.staticanalysis.ReplaceApacheCommonsLang3ValidateNotNullWithObjectsRequireNonNull | ||
| # - org.openrewrite.staticanalysis.SimplifyTernaryRecipes | ||
| # - org.openrewrite.staticanalysis.URLEqualsHashCodeRecipes | ||
| # - org.openrewrite.staticanalysis.UnnecessaryCloseInTryWithResources | ||
| # - org.openrewrite.staticanalysis.UnnecessaryExplicitTypeArguments | ||
| # - org.openrewrite.staticanalysis.UnnecessaryParentheses | ||
| # - org.openrewrite.staticanalysis.UnnecessaryReturnAsLastStatement | ||
| # - tech.picnic.errorprone.refasterrules.BigDecimalRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.CharSequenceRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.ClassRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.CollectionRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.ComparatorRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.EqualityRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.FileRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.MapRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.MicrometerRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.MockitoRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.NullRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.OptionalRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.PatternRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.PreconditionsRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.PrimitiveRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.StreamRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.StringRulesRecipes | ||
| # - tech.picnic.errorprone.refasterrules.TimeRulesRecipes | ||
| --- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could kind of suggest this, also to be aligned with new check.