Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[INFRA] Improve the java style checks log the errors to sbt console #3115

Merged
merged 3 commits into from
May 22, 2024

Conversation

tlm365
Copy link
Contributor

@tlm365 tlm365 commented May 18, 2024

Which Delta project/connector is this regarding?

  • Spark
  • Standalone
  • Flink
  • Kernel
  • Other (fill in here)

Description

Resolves #3067.

How was this patch tested?

On local machine, intentionally create checkstyle errors in module kernelDefaults (for experimental), then run the build/sbt compile and build/sbt kernelDefaults/test.

Does this PR introduce any user-facing changes?

No.

build.sbt Outdated Show resolved Hide resolved
Copy link
Collaborator

@vkorukanti vkorukanti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. One comment. Tried locally and it works great.

build.sbt Outdated
Comment on lines 1375 to 1380
if (errors.nonEmpty) {
errors.foreach { case (file, line, message) =>
log.error(s"- File: $file, Line: $line, Message: $message")
}
sys.error(s"Found checkstyle errors")
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good to me. The only request is, can we print the errors as part of the failure message? That way we don't need to scroll up to find the error.

    var errorMsg = "Found checkstyle errors"
    errors.foreach { case (file, line, message) =>
      val lineError = s"File: $file, Line: $line, Message: $message"
      log.error(lineError)
      errorMsg += ("\n" + lineError)
    }
    sys.error(errorMsg + "\n")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vkorukanti Thanks for reviewing.

build.sbt Outdated
checkstyleSeverityLevel := CheckstyleSeverityLevel.Error,
(Compile / compile) := ((Compile / compile) dependsOn (Compile / checkstyle)).value,
(Test / test) := ((Test / test) dependsOn (Test / checkstyle)).value
checkstyleSeverityLevel := CheckstyleSeverityLevel.Ignore,
Copy link
Collaborator

@vkorukanti vkorukanti May 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one naive qn: why is this changed from Error to Ignore? Is it because we want to control the part about throwing error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it because we want to control the part about throwing error?

@vkorukanti Yes, that's right. If we keep the Error severity, build/sbt will throw an error and immediately stop at the checkstyle phase (if error) -> never execute the check-report phase of checkstyle-report.xml and checkstyle-test-report.xml. We need to Ignore and throw error if exists when checking *report.xml.

build.sbt Outdated
@@ -1335,18 +1335,51 @@ lazy val scalaStyleSettings = Seq(
****************************
*/

lazy val compileJavastyle = taskKey[Unit]("compileJavastyle")
lazy val testJavastyle = taskKey[Unit]("testJavastyle")

def javaCheckstyleSettings(checkstyleFile: String): Def.SettingsDefinition = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this super helpful! Wondering if we should move all this code to a separate file since it's now more than just a few lines?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allisonport-db Thanks for reviewing. Your point sounds good, we have a quite complicated build.sbt now. But if isolating this checkstyle part to a separate file, where should we organize that file?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We keep some of the build-related code (e.g., Unidos) under the project directory. You can put it there.

…bt console

Signed-off-by: Tai Le Manh <manhtai.lmt@gmail.com>
Copy link
Collaborator

@vkorukanti vkorukanti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you for this change. It makes it easy to find the checkstyle errors.

I made a couple of minor edits. Let me know if the changes look fine to you.

@tlm365
Copy link
Contributor Author

tlm365 commented May 22, 2024

LGTM. Thank you for this change. It makes it easy to find the checkstyle errors.
I made a couple of minor edits. Let me know if the changes look fine to you.

@vkorukanti It's nice. TYSM!

@vkorukanti vkorukanti added this to the 3.3.0 milestone May 22, 2024
@vkorukanti vkorukanti merged commit 35c7536 into delta-io:master May 22, 2024
9 of 10 checks passed
longvu-db pushed a commit to longvu-db/delta that referenced this pull request May 28, 2024
…elta-io#3115)

## Description
Resolves delta-io#3067.

## How was this patch tested?
On local machine, intentionally create checkstyle errors in module
`kernelDefaults` (for experimental), then run the `build/sbt compile`
and `build/sbt kernelDefaults/test`.

Signed-off-by: Tai Le Manh <manhtai.lmt@gmail.com>
longvu-db pushed a commit to longvu-db/delta that referenced this pull request May 30, 2024
…elta-io#3115)

Resolves delta-io#3067.

On local machine, intentionally create checkstyle errors in module
`kernelDefaults` (for experimental), then run the `build/sbt compile`
and `build/sbt kernelDefaults/test`.

Signed-off-by: Tai Le Manh <manhtai.lmt@gmail.com>
@tlm365 tlm365 deleted the checkstyle branch August 21, 2024 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request][INFRA] Improve the java checkstyle checks to log the errors to the sbt console
3 participants