diff --git a/docs/modules/ROOT/images/comparisons/IntellijGradleResourcesClasses.png b/docs/modules/ROOT/images/comparisons/IntellijGradleResourcesClasses.png new file mode 100644 index 00000000000..d0e76264866 Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijGradleResourcesClasses.png differ diff --git a/docs/modules/ROOT/images/comparisons/IntellijGradleResourcesClassesDefinition.png b/docs/modules/ROOT/images/comparisons/IntellijGradleResourcesClassesDefinition.png new file mode 100644 index 00000000000..ec98b4d192f Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijGradleResourcesClassesDefinition.png differ diff --git a/docs/modules/ROOT/images/comparisons/IntellijMockitoGradleCompileOptions.png b/docs/modules/ROOT/images/comparisons/IntellijMockitoGradleCompileOptions.png new file mode 100644 index 00000000000..ab37d9a6313 Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijMockitoGradleCompileOptions.png differ diff --git a/docs/modules/ROOT/images/comparisons/IntellijMockitoGradleCompileOptions2.png b/docs/modules/ROOT/images/comparisons/IntellijMockitoGradleCompileOptions2.png new file mode 100644 index 00000000000..0deb22557b7 Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijMockitoGradleCompileOptions2.png differ diff --git a/docs/modules/ROOT/images/comparisons/IntellijMockitoMillCompile.png b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillCompile.png new file mode 100644 index 00000000000..d5aa21e1d7c Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillCompile.png differ diff --git a/docs/modules/ROOT/images/comparisons/IntellijMockitoMillCompileClasspath.png b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillCompileClasspath.png new file mode 100644 index 00000000000..d8e6df5459f Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillCompileClasspath.png differ diff --git a/docs/modules/ROOT/images/comparisons/IntellijMockitoMillCompileUsages.png b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillCompileUsages.png new file mode 100644 index 00000000000..ea7d5da7ba4 Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillCompileUsages.png differ diff --git a/docs/modules/ROOT/images/comparisons/IntellijMockitoMillJavacOptionsDef.png b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillJavacOptionsDef.png new file mode 100644 index 00000000000..21932299979 Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillJavacOptionsDef.png differ diff --git a/docs/modules/ROOT/images/comparisons/IntellijMockitoMillJavacOptionsDocs.png b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillJavacOptionsDocs.png new file mode 100644 index 00000000000..09973c6a367 Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillJavacOptionsDocs.png differ diff --git a/docs/modules/ROOT/images/comparisons/IntellijMockitoMillJavacOptionsParents.png b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillJavacOptionsParents.png new file mode 100644 index 00000000000..98f82592fab Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijMockitoMillJavacOptionsParents.png differ diff --git a/docs/modules/ROOT/images/comparisons/IntellijNettyAutocomplete.png b/docs/modules/ROOT/images/comparisons/IntellijNettyAutocomplete.png new file mode 100644 index 00000000000..1b94538f236 Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijNettyAutocomplete.png differ diff --git a/docs/modules/ROOT/images/comparisons/IntellijNettyPeekDocs.png b/docs/modules/ROOT/images/comparisons/IntellijNettyPeekDocs.png new file mode 100644 index 00000000000..905250e2070 Binary files /dev/null and b/docs/modules/ROOT/images/comparisons/IntellijNettyPeekDocs.png differ diff --git a/docs/modules/ROOT/pages/comparisons/gradle.adoc b/docs/modules/ROOT/pages/comparisons/gradle.adoc index 1c9a3d652b5..ebbf602d809 100644 --- a/docs/modules/ROOT/pages/comparisons/gradle.adoc +++ b/docs/modules/ROOT/pages/comparisons/gradle.adoc @@ -21,7 +21,9 @@ change any other files in the repository: == Completeness The Mill build for Mockito is not 100% complete, but it covers most of the major parts of Mockito: -compiling Java, running JUnit tests. For now, the Android, Kotlin, and OSGI tests are skipped. +compiling Java, running JUnit tests. For now, the Android, Kotlin, and OSGI tests are skipped, +as support for xref:javalib/android-examples.adoc[Building Android apps in Mill] +and xref:kotlinlib/intro.adoc[Kotlin with Mill] is still experimental. The goal of this exercise is not to be 100% feature complete enough to replace the Gradle build today. It is instead meant to provide a realistic comparison of how using Mill in a realistic, @@ -35,7 +37,9 @@ where the difference in the fixed overhead of the build tool is most noticeable. For comparison purposes, I disabled the Gradle subprojects that we did not fully implement in Mill (`groovyTest`, `groovyInlineTest`, `kotlinTest`, `kotlinReleaseCoroutinesTest`, `android`, -`osgi-test`, `java21-test`). +`osgi-test`, `java21-test`), and added the necessary flags to ensure caching/parallelism/etc. is +configured similarly for both tools. This ensures the comparison is fair with both builds compiling the +same code and running the same tests in the same way. For the benchmarks below, each provided number is the median wall time of three consecutive runs on my M1 Macbook Pro. While ad-hoc, these benchmarks are enough to give you a flavor of how @@ -86,6 +90,17 @@ Again to make it a fair comparison, we use `--no-build-cache` in Gradle and set single thread. Here we see Mill being about ~3.3x faster than Gradle, to do the equivalent amount of work. +As a point of reference, Java typically compiles at 10,000-50,000 lines per second on a +single thread, and the Mockito codebase is ~100,000 lines of code, so we would expect compile +to take 2-10 seconds without parallelism. +The 5-6s taken by Mill seems about what you would expect for a codebase of this size, +and the ~17s taken by Gradle is much more than what you would expect from simple Java compilation. + +It's actually not clear to me where the difference in execution time is coming from. Unlike +the xref:comparisons/maven.adoc[Mill v.s. Maven comparison], Gradle's command line output +doesn't show any obvious network requests or jar packing/unpacking/comparing going on. But +Gradle's CLI output is also much less verbose than Maven's, so it's possible things are going +on under the hood that I'm not aware of. === Parallel Clean Compile All @@ -95,14 +110,14 @@ $ ./gradlew clean; time ./gradlew classes testClasses --no-build-cache 12.3s 11.4s -$ ./mill clean; time ./mill -j 10 __.compile +$ ./mill clean; time ./mill __.compile 3.59s 3.57s 3.45s ``` This benchmark is identical to the <> benchmark above, but enables -parallelism: Gradle by default, Mill via `-j 10` to run on 10 cores (the number on my Macbook Pro). +parallelism: Gradle by default, Mill without `-j 1` to run on 10 cores (the number on my Macbook Pro). Neither Gradle nor Mill benefit hugely from parallelism: both show a moderate ~50% speedup, despite receiving 900% more CPUs. This likely indicates that the module dependency graph @@ -182,43 +197,202 @@ For both Mill and Gradle, we see small speedups relative to the <`: -image::comparisons/MockitoCompileGraph.svg[] +image::comparisons/IntellijMockitoGradleCompileOptions.png[] -Apart from the static dependency graph, another thing of interest may be the performance -profile and timeline: where the time is spent when you actually compile everything. With -Mill, when you run a compilation using `./mill -j 10 __.compile`, you automatically get a -`out/mill-chrome-profile.json` file that you can load into your `chrome://tracing` page and -visualize where your build is spending time and where the performance bottlenecks are: +But if you try to jump to definition or find out anything else about it you hit a wall: -image::comparisons/MockitoCompileProfile.png[] +image::comparisons/IntellijMockitoGradleCompileOptions2.png[] -If you want to inspect the tree of third-party dependencies used by any module, the -built in `ivyDepsTree` command lets you do that easily: +Often working with build configurations feels like hitting dead ends: if you don't have +`options.compilerArgs` memorized in your head, there is literally nothing you can do in your editor to +make progress to figure out what it is or what it is used for. That leaves you googling +for answers, which can be a frustrating experience that distracts you from the task at hand. -```bash -$ ./mill subprojects.junit-jupiter.ivyDepsTree -├─ org.junit.jupiter:junit-jupiter-api:5.10.3 -│ ├─ org.apiguardian:apiguardian-api:1.1.2 -│ ├─ org.junit.platform:junit-platform-commons:1.10.3 -│ │ └─ org.apiguardian:apiguardian-api:1.1.2 -│ └─ org.opentest4j:opentest4j:1.3.0 -└─ org.objenesis:objenesis:3.3 +The fundamental problem with tools like Gradle is that the code you write does not +actually perform the build: rather, you are just setting up some data structure that +is used to configure the _real_ build engine that runs later. Thus when you explore +the Gradle build in an IDE, the IDE can only explore the configuration logic (which +use usually un-interesting) and is unable to explore the actual build logic (which +is what you actually care about!) + +In comparison, Mill's `.mill` files are all statically typed, and as a result IntelliJ is easily able to +pull up the documentation for `def javacOptions`, even though it doesn't have any special support +for Mill built into the IDE: + +image::comparisons/IntellijMockitoMillJavacOptionsDocs.png[] + +Apart from static typing, the way Mill builds are structured also helps the IDE: Mill +code _actually performs your build_, rather than configuring some opaque build engine. +While that sounds academic, one concrete consequence is that IntelliJ is able to take +your `def javacOptions` override and +find the original definitions that were overridden, and show you where they are defined: + +image::comparisons/IntellijMockitoMillJavacOptionsParents.png[] + +image::comparisons/IntellijMockitoMillJavacOptionsDef.png[] + +Furthermore, because task dependencies in Mill are just normal method calls, IntelliJ is +able to _find usages_, showing you where the task is used. Below, we can see the method +call in the `def compile` task, which uses `javacOptions()` along with a number of other tasks: + +image::comparisons/IntellijMockitoMillCompile.png[] + +From there, if you are curious about any of the other tasks used alongside `javacOptions`, it's +easy for you to pull up _their_ documentation, jump to _their_ +definition, or find _their_ usages. For example we can pull up the docs of +`compileClasspath()` below: + +image::comparisons/IntellijMockitoMillCompileClasspath.png[] + +Or we can use _find usages_ on `def compile` to see where it is used, both in this build +and upstream in the Mill libraries: + +image::comparisons/IntellijMockitoMillCompileUsages.png[] + +Unlike most other build tools, Mill builds are extremely easy to explore interactively in your +IDE. If you do not know what something does, it's documentation, definition, or usages is always +one click away in IntelliJ or VSCode. That's not to say Mill builds aren't complex - as +we saw above, compilation has to deal with upstream outputs, classpaths, flags, reporters, and so on - +but at least in Mill your IDE can help you explore, understand and manage the complexity in a way +that no other build tool supports. + +Note that the IDE experience that Mill provides should already be very familiar to anyone writing +Java, Kotlin, or Scala: + +* _of course_ you can find the overridden definitions! +* _of course_ you can pull up the documentation in a click! +* _of course_ you can navigate around the codebase with your IDE, up and down + the call graph, to see who calls who! + +What Mill provides isn't rocket science, but rather it is just about taking your existing experience +and existing IDE tooling working with application codebases, and lets you use it to manage your build +system as well. + +Mill IDE support isn't perfect - you may have noticed the spurious red squigglies above - but it's +already better than most other build systems like Gradle or Maven. And that is with approximately +~zero custom integrations with the various IDEs: with some additional work, we can expect the Mill +IDE experience to improve even more over time. + +== Extensibility + +Another facet of Mill is that is worth exploring is the ease of making custom tasks or build steps. +For example, in Mill, overriding the resources to duplicate a file can be done as follows: + +```scala +def resources = Task { + os.copy( + compile().classes.path / "org/mockito/internal/creation/bytebuddy/inject/MockMethodDispatcher.class", + Task.dest / "org/mockito/internal/creation/bytebuddy/inject/MockMethodDispatcher.raw", + createFolders = true + ) + super.resources() ++ Seq(PathRef(Task.dest)) +} +``` + +In Gradle, it is written as: + +```scala +tasks.register('copyMockMethodDispatcher', Copy) { + dependsOn compileJava + + from "${sourceSets.main.java.classesDirectory.get()}/org/mockito/internal/creation/bytebuddy/inject/MockMethodDispatcher.class" + into layout.buildDirectory.dir("generated/resources/inline/org/mockito/internal/creation/bytebuddy/inject") + + rename '(.+)\\.class', '$1.raw' +} + +classes.dependsOn("copyMockMethodDispatcher") + +sourceSets.main { + resources { + output.dir(layout.buildDirectory.dir("generated/resources/inline")) + } +} ``` -None of these tools are rocket science, but Mill provides all of them out of the -box in a convenient package for you to use. Whether you want a visual graph layout, -a parallel performance profile, or a third-party dependency tree of your project, -Mill makes it easy and convenient without needing to fiddle with custom configuration -or third party plugins. This helps make it easy for you to explore, understand, and -take ownership of the build tool. +At a first glance, both of these snippets do the same thing, just with different syntaxes +and helper method names. However, on a deeper look, a few things are worth noting: + +1. In Mill, you do not need to manually add `dependsOn` clauses, unlike Gradle: + * In Mill, referencing the value of `compile()`, we both explicitly get access to the value of `compile` + and also add a dependency on it. In Gradle, you need to separately add `dependsOn compile` + to mark the dependency, and `rename '(.+)\\.class', '$1.raw'` to make use of it implicitly. + + * In Mill, overriding `def resources` is enough to make all tasks that previously depended on `resources` + now depend on the override (e.g. `run`, `test`, `jar`, `assembly`, etc.) as is the norm for object-oriented + ``override``s. In Gradle, you need to explicitly call `classes.dependsOn("copyMockMethodDispatcher")` to + make the downstream `classes` task depend on `copyMockMethodDispatcher`, and + `sourcesSets.main resources output.dir` to wire up the generated files to the resources of the module + +2. In Mill, the `resources` task is given a unique `Task.dest` folder that is unique to it. + In contrast, Gradle's `copyMockMethodDispatcher` puts things in a global `generated/` folder + * This means that in Mill, you do not need to worry about filesystem collisions, since every + task's `Task.dest` is unique. In contrast, in Gradle you need to make sure that no other + task in the entire build is scribbling over `generated/`, otherwise they could interfere + with one another in confusing ways + + * This also means that in Mill, you always know where the output of a particular task + is - `foo.bar.resources` writes to `out/foo/bar/resources.dest/` - so you can always easily + find the output of a particular task. In Gradle, you have to dig through the source code to + find where the task is implemented and see where it is writing to. + +3. Mill passes typed structured ``Path``s and ``PathRef``s between each other, while Gradle often + uses raw path strings + * In Mill, `def resources` returns a `PathRef(Task.dest)` for downstream tasks to use, + so downstream tasks can use it directly (similar to how it makes use of + `compile().classes.path` directly). This means different tasks can refer to each other + in a foolproof way without room for error + + * In Gradle, `sourcesSets.map resources output.dir` needs to refer to the path generated by + `copyMockMethodDispatcher` via it's string `"generated/resources/inline"`. That adds a lot of + room for error, since the strings can easily get out of sync accidentally. + +In general, although the two snippets aren't that different superficially, Mill makes it +easy to do the right thing by default: + +* Upstream task dependencies are recorded automatically when used +* Overridden definitions and automatically used by downstream tasks +* Every task is automatically assigned a place on disk so you don't need + to worry about collisions and can easily find outputs +* Tasks interact with each other via typed structured values - ``Path``s, ``PathRef``s, + etc. - rather than magic strings + +Although in Gradle it is possible for an expert to customize their build in a +way that mitigates these issues, Mill does it automatically and in a way that +is foolproof even for non-experts. This helps democratize the build so that +any engineer can contribute fixes or improvements without needing to be a +build-system expert and learn all the best practices first. + +Lastly, as mentioned earlier, the Gradle script has limited IDE support: it can +autocomplete things for you, but once you try to jump-to-definition or otherwise +navigate your build you hit a wall: it tells you some minimal documentation about +the identifier, but nothing about how it is implemented or where it is used: + +image::comparisons/IntellijGradleResourcesClasses.png[] +image::comparisons/IntellijGradleResourcesClassesDefinition.png[] + +In contrast, IntelliJ is able to navigate straight to the definition of `compile()` in the +Mill build (as we saw earlier in <<_ide_experience>>), and from there can continue to +traverse the build via _jump to definition_ (which we saw earlier) or _find usages_, +as we saw earlier: + +image::comparisons/IntellijMockitoMillCompile.png[] +image::comparisons/IntellijMockitoMillCompileClasspath.png[] +image::comparisons/IntellijMockitoMillCompileUsages.png[] + +Mill build scripts are written in Scala, but you do not need to be an expert in Scala +to use Mill, just like you do not need to be an expert in Groovy to use Gradle. Because +Mill has great IDE support, and does the right things by default, I hope it would be +much easier for a non-expert to contribute to a Mill build than it would be for a +non-expert to contribute to Gradle == Conclusion @@ -227,14 +401,17 @@ Both the Mill and Gradle builds we discussed in this case study do the same thin compile Java code and run tests. Sometimes they perform additional configuration, tweaking JVM arguments or doing ad-hoc classpath mangling. -Mill doesn't try to do _more_ than Gradle does, but it -tries to do it _better_: faster compiles, shorter and easier to read configs, easier -extensibility via libraries. +In general, building projects with Mill is significantly faster than Gradle, but the gap +is not as big as when comparing xref:comparisons/maven.adoc[Mill v.s. Maven]. Mill builds +do all the same things as gradle builds, and need to manage the same kind of complexity. +But where Mill shines over Gradle is just the understandability of the build: while Gradle is +https://news.ycombinator.com/item?id=25801986[famously confusing and opaque], Mill's great +IDE support allows the user to explore and understand their build as easily as any +application codebase, and its fool-proof approach to extensibility means non-experts can +confidently modify or add to their build system without worrying about getting it wrong. Again, the Mill build used in this comparison is for demonstration purposes, and more work would be necessary to make the Mill build production ready: publishing configuration, -code coverage integration, and so on. However, hopefully it demonstrates the potential value: -significantly improved performance, so that you spend less time waiting for your code to -compile and more time doing the work that actually matters, with builtin debugging tools -to help turn normally opaque "build config" into something that's transparent and -easily understandable. +code coverage integration, and so on. Furthermore, Mill is definitely not perfect, +and it is a work in progress to improve the user experience and iron out bugs. However, +hopefully this comparison demonstrates the potential value, and convinces you to give it a try! diff --git a/docs/modules/ROOT/pages/comparisons/maven.adoc b/docs/modules/ROOT/pages/comparisons/maven.adoc index 7a3a50c6429..7b07e6c2e40 100644 --- a/docs/modules/ROOT/pages/comparisons/maven.adoc +++ b/docs/modules/ROOT/pages/comparisons/maven.adoc @@ -59,7 +59,7 @@ Mill's performance compares to Maven: | <> | 1m 16.45s | 0m 09.95s | 7.7x | <> | 0m 19.62s | 0m 02.17s | 9.0x | <> | 0m 21.10s | 0m 00.54s | 39.1x -| <> | 0m 17.34s | 0m 00.47s | 39.1x +| <> | 0m 17.34s | 0m 00.47s | 36.9x |=== The column on the right shows the speedups of how much faster Mill is compared to the @@ -75,7 +75,7 @@ $ time ./mvnw -DskipTests -Dcheckstyle.skip -Denforcer.skip=true clean install 2m 27.58s 2m 31.12s -$ ./mill clean; time ./mill __.compile +$ ./mill clean; time ./mill -j1 __.compile 0m 29.14s 0m 22.19s 0m 20.79s @@ -85,7 +85,8 @@ This benchmark exercises the simple "build everything from scratch" workflow, wi artifacts already in the local cache. The actual files being compiled are the same in either case (as mentioned in the <> section). I have explicitly disabled the various linters and tests for the Maven build, to just focus -on the compilation of Java source code making it an apples-to-apples comparison. +on the compilation of Java source code making it an apples-to-apples comparison. As Mill +runs tasks in parallel by default, I have disabled parallelism explicitly via `-j1` As a point of reference, Java typically compiles at 10,000-50,000 lines per second on a single thread, and the Netty codebase is ~500,000 lines of code, so we would expect compile @@ -151,12 +152,12 @@ $ time ./mvnw -DskipTests -Dcheckstyle.skip -Denforcer.skip=true clean install 2m 27.58s 2m 31.12s -$ ./mill clean; time ./mill __.compile +$ ./mill clean; time ./mill -j1 __.compile 0m 29.14s 0m 22.19s 0m 20.79s -$ ./mill clean; time ./mill __.jar +$ ./mill clean; time ./mill -j1 __.jar 0m 32.58s 0m 24.90s 0m 23.35s @@ -171,19 +172,19 @@ whereas Mill directly uses the classfiles generated on disk to bypass all that w === Parallel Clean Compile All ```bash -$ time ./mvnw -T 4 -DskipTests -Dcheckstyle.skip -Denforcer.skip=true clean install +$ time ./mvnw -T 10 -DskipTests -Dcheckstyle.skip -Denforcer.skip=true clean install 1m 19.58s 1m 16.34s 1m 16.45s -$ ./mill clean; time ./mill -j 4 __.compile +$ ./mill clean; time ./mill __.compile 0m 14.80s 0m 09.95s 0m 08.83s ``` -This example compares Maven v.s. Mill, when performing the clean build on 4 threads. -Both build tools support parallelism (`-T 4` in Maven and `-j 4` in Mill), and both +This example compares Maven v.s. Mill, when performing the clean build on 10 threads. +Both build tools support parallelism (`-T 10` in Maven, by default in Mill), and both tools see a similar ~2x speedup for building the Netty project using 4 threads. Again, this tests a clean build using `./mvnw clean` or `./mill clean`. @@ -197,7 +198,7 @@ when performing a clean build of the Netty repository. $ time ./mvnw -pl common -DskipTests -Dcheckstyle.skip -Denforcer.skip=true clean install 0m 19.62s 0m 20.52s -0:19:50 +0m 19.50s $ ./mill clean common; time ./mill common.test.compile 0m 04.94s @@ -350,13 +351,13 @@ we saw with Maven, but due to Mill's minimal overhead, in the end the command finishes in less than half a second. -== Extensibility +== Extensibility & IDE Experience Even though Maven is designed to be declarative, in many real-world codebases you end up needing to run ad-hoc scripts and logic. This section will explore two such scenarios, so you can see how Mill differs from Maven in the handling of these requirements. -=== Groovy +=== JVM Libraries: Groovy The Maven build for the `common/` subproject uses a Groovy script for code generation. This is configured via: @@ -433,13 +434,26 @@ object common extends NettyModule{ While the number of lines of code _written_ is not that different, the Mill configuration is a lot more direct: rather than writing 35 lines of XML to configure an opaque third-party plugin, we instead write 25 lines of code to directly do what we want: import `groovy`, -configure a `GroovyShell`, and use it to evaluate our `codegen.groovy` script. +configure a `GroovyShell`, and use it to evaluate our `codegen.groovy` script. Although +you may not be familiar with the Scala language that Mill builds are written in, you could +probably skim the snippet above and guess what it is doing, and guess correctly. This direct control means you are not beholden to third party plugins: rather than being limited to what an existing plugin _allows_ you to do, Mill allows you to directly write -the code necessary to do what _you need to do_. +the code necessary to do what _you need to do_. In this case, if we need to invoke +https://github.com/apache/groovy[Groovy] and +https://github.com/groovy/groovy-core/blob/4c05980922a927b32691e4c3eba5633825cc01e3/subprojects/groovy-ant/src/spec/doc/groovy-ant-task.adoc[Groovy-Ant], +Mill allows us to direct xref:extending/import-ivy-plugins.adoc[import $ivy] the relevant +JVM artifacts from Maven Central and begin using them in our build code in a safe, +strongly-typed fashion, with full autocomplete and code assistance: -=== Calling Make +image::comparisons/IntellijNettyAutocomplete.png[] + +Mill gives you the full power of the JVM ecosystem to use in your build: any Java library +on Maven central is just an `import $ivy` away, and can be used with the full IDE support +and tooling experience you are used to in the JVM ecosystem. + +=== Subprocesses: Make The Maven build for the `transport-native-unix-common/` subproject needs to call `make` in order to compile its C code to modules that can be loaded into Java applications @@ -583,12 +597,19 @@ In Mill, we define the `makefile`, `cSources`, `cHeaders`, and `make` tasks. The of the logic is in `def make`, which prepares the `makefile` and C sources, resolves the `netty-jni-util` source jar and unpacks it with `jar xf`, and calls `make` with the given environment variables. Both `cHeaders` and the output of `make` are used -in downstream modules. +in downstream modules. In this case, `make` is a command-line utility rather than a JVM +library, so rather than importing it from Maven Central we use `os.proc.call` to invoke it. Again, the Maven XML and Mill code contains exactly the same logic, and neither is much more concise or verbose than the other. Rather, what is interesting is that it is much easier to work with this kind of _build logic_ via _concise type-checked code_, rather than configuring a bunch of third-party plugins to try and achieve what you want. +With Mill, you get your full IDE experience working with your build: autocomplete, code +assistance, navigation, and so on. Although working with the `os.proc.call` subprocess API +is not as right as working with the JVM libraries we saw earlier, it is still a much +richer experience than you typically get configuring XML files: + +image::comparisons/IntellijNettyPeekDocs.png[] == Debugging Tooling @@ -597,13 +618,14 @@ Another area that Mill does better than Maven is providing builtin tools for you what your build is doing. For example, the Netty project build discussed has 47 submodules and associated test suites, but how do these different modules depend on each other? With Mill, you can run `./mill visualize __.compile`, and it will show you how the -`compile` task of each module depends on the others: +`compile` task of each module depends on the others (right-click open-image-in-new-tab to see +at full size): image::comparisons/NettyCompileGraph.svg[] Apart from the static dependency graph, another thing of interest may be the performance profile and timeline: where the time is spent when you actually compile everything. With -Mill, when you run a compilation using `./mill -j 10 __.compile`, you automatically get a +Mill, when you run a compilation using `./mill __.compile`, you automatically get a `out/mill-chrome-profile.json` file that you can load into your `chrome://tracing` page and visualize where your build is spending time and where the performance bottlenecks are: @@ -639,13 +661,13 @@ C code or run `make` or Groovy. Mill doesn't try to do _more_ than Maven does, but it tries to do it _better_: faster compiles, shorter and easier to read configs, easier extensibility via libraries (e.g. `org.codehaus.groovy:groovy`) and subprocesses -(e.g. `make`). +(e.g. `make`), better IDE support for working with your build. Again, the Mill build used in this comparison is for demonstration purposes, and more work would be necessary to make the Mill build production ready: compatibility with different operating system architectures, Java versions, and so on. However, hopefully -it demonstrates the potential value: improved performance, conciseness of the build logic, -and easy extensibility so you can fine-tune your build logic to your requirements. +it demonstrates the potential value: greatly improved performance, easy extensibility, +and a much better IDE experience for working with your build. Mill provides builtin tools to help you navigate, visualize, and understand your build, turning a normally opaque "build config" into something that's transparent and easily understandable. \ No newline at end of file diff --git a/docs/modules/ROOT/pages/javalib/build-examples.adoc b/docs/modules/ROOT/pages/javalib/build-examples.adoc index 04502f9c9d2..d6e3072bdba 100644 --- a/docs/modules/ROOT/pages/javalib/build-examples.adoc +++ b/docs/modules/ROOT/pages/javalib/build-examples.adoc @@ -15,10 +15,6 @@ example projects. include::partial$example/javalib/builds/1-nested-modules.adoc[] -== Maven-Compatible Modules - -include::partial$example/javalib/builds/2-compat-modules.adoc[] - == Realistic Java Example Project diff --git a/docs/modules/ROOT/pages/javalib/builtin-commands.adoc b/docs/modules/ROOT/pages/javalib/builtin-commands.adoc index f90b1ae711c..df74210f205 100644 --- a/docs/modules/ROOT/pages/javalib/builtin-commands.adoc +++ b/docs/modules/ROOT/pages/javalib/builtin-commands.adoc @@ -5,4 +5,4 @@ :language-small: java -include::partial$example/javalib/basic/4-builtin-commands.adoc[] +include::partial$example/javalib/basic/5-builtin-commands.adoc[] diff --git a/docs/modules/ROOT/pages/javalib/intro.adoc b/docs/modules/ROOT/pages/javalib/intro.adoc index 771fc66c2a0..9fd6efd2a15 100644 --- a/docs/modules/ROOT/pages/javalib/intro.adoc +++ b/docs/modules/ROOT/pages/javalib/intro.adoc @@ -52,3 +52,7 @@ include::partial$example/javalib/basic/2-custom-build-logic.adoc[] include::partial$example/javalib/basic/3-multi-module.adoc[] include::partial$Intro_to_Mill_Footer.adoc[] + +== Maven-Compatible Modules + +include::partial$example/javalib/basic/4-compat-modules.adoc[] \ No newline at end of file diff --git a/docs/modules/ROOT/pages/kotlinlib/build-examples.adoc b/docs/modules/ROOT/pages/kotlinlib/build-examples.adoc index 4a79a0484c3..e9564255f16 100644 --- a/docs/modules/ROOT/pages/kotlinlib/build-examples.adoc +++ b/docs/modules/ROOT/pages/kotlinlib/build-examples.adoc @@ -16,10 +16,6 @@ example projects. include::partial$example/kotlinlib/builds/1-nested-modules.adoc[] -== Maven-Compatible Modules - -include::partial$example/kotlinlib/builds/2-compat-modules.adoc[] - == Realistic Kotlin Example Project diff --git a/docs/modules/ROOT/pages/kotlinlib/builtin-commands.adoc b/docs/modules/ROOT/pages/kotlinlib/builtin-commands.adoc index 8b53209f0a3..4a75450979a 100644 --- a/docs/modules/ROOT/pages/kotlinlib/builtin-commands.adoc +++ b/docs/modules/ROOT/pages/kotlinlib/builtin-commands.adoc @@ -1,4 +1,4 @@ = Built-in Commands :page-aliases: Kotlin_Builtin_Commands.adoc -include::partial$example/kotlinlib/basic/4-builtin-commands.adoc[] +include::partial$example/kotlinlib/basic/5-builtin-commands.adoc[] diff --git a/docs/modules/ROOT/pages/kotlinlib/intro.adoc b/docs/modules/ROOT/pages/kotlinlib/intro.adoc index e956a775784..e7a648b0a1c 100644 --- a/docs/modules/ROOT/pages/kotlinlib/intro.adoc +++ b/docs/modules/ROOT/pages/kotlinlib/intro.adoc @@ -56,3 +56,7 @@ include::partial$example/kotlinlib/basic/2-custom-build-logic.adoc[] include::partial$example/kotlinlib/basic/3-multi-module.adoc[] include::partial$Intro_to_Mill_Footer.adoc[] + +== Maven-Compatible Modules + +include::partial$example/kotlinlib/basic/4-compat-modules.adoc[] diff --git a/docs/modules/ROOT/pages/scalalib/build-examples.adoc b/docs/modules/ROOT/pages/scalalib/build-examples.adoc index ceb8c989d08..770f27f0eed 100644 --- a/docs/modules/ROOT/pages/scalalib/build-examples.adoc +++ b/docs/modules/ROOT/pages/scalalib/build-examples.adoc @@ -16,11 +16,6 @@ example projects. include::partial$example/scalalib/builds/1-nested-modules.adoc[] -== SBT-Compatible Modules - -include::partial$example/scalalib/builds/2-compat-modules.adoc[] - - == Cross-Scala-Version Modules include::partial$example/scalalib/builds/3-cross-scala-version.adoc[] diff --git a/docs/modules/ROOT/pages/scalalib/builtin-commands.adoc b/docs/modules/ROOT/pages/scalalib/builtin-commands.adoc index 3cd72f7bc8c..3183a96da50 100644 --- a/docs/modules/ROOT/pages/scalalib/builtin-commands.adoc +++ b/docs/modules/ROOT/pages/scalalib/builtin-commands.adoc @@ -5,4 +5,4 @@ :language-small: scala -include::partial$example/scalalib/basic/4-builtin-commands.adoc[] +include::partial$example/scalalib/basic/5-builtin-commands.adoc[] diff --git a/docs/modules/ROOT/pages/scalalib/intro.adoc b/docs/modules/ROOT/pages/scalalib/intro.adoc index 0e807800568..ca64f69a25e 100644 --- a/docs/modules/ROOT/pages/scalalib/intro.adoc +++ b/docs/modules/ROOT/pages/scalalib/intro.adoc @@ -85,3 +85,9 @@ include::partial$example/scalalib/basic/2-custom-build-logic.adoc[] include::partial$example/scalalib/basic/3-multi-module.adoc[] include::partial$Intro_to_Mill_Footer.adoc[] + + +== SBT-Compatible Modules + +include::partial$example/scalalib/basic/4-compat-modules.adoc[] + diff --git a/example/javalib/builds/2-compat-modules/bar/src/main/java/bar/Bar.java b/example/javalib/basic/4-compat-modules/bar/src/main/java/bar/Bar.java similarity index 100% rename from example/javalib/builds/2-compat-modules/bar/src/main/java/bar/Bar.java rename to example/javalib/basic/4-compat-modules/bar/src/main/java/bar/Bar.java diff --git a/example/javalib/builds/2-compat-modules/build.mill b/example/javalib/basic/4-compat-modules/build.mill similarity index 68% rename from example/javalib/builds/2-compat-modules/build.mill rename to example/javalib/basic/4-compat-modules/build.mill index 82fd1319733..49624650348 100644 --- a/example/javalib/builds/2-compat-modules/build.mill +++ b/example/javalib/basic/4-compat-modules/build.mill @@ -1,4 +1,10 @@ //// SNIPPET:ALL +// Mill's default folder layout of `foo/src/` and `foo/test/src` differs from that +// of Maven or Gradle's `foo/src/main/java/` and `foo/src/test/java/`. If you are +// migrating an existing codebase, you can use Mill's `MavenModule` and +// `MavenTests` as shown below to preserve filesystem compatibility with an existing +// Maven or Gradle build: + package build import mill._, javalib._ diff --git a/example/javalib/builds/2-compat-modules/foo/src/main/java/foo/Foo.java b/example/javalib/basic/4-compat-modules/foo/src/main/java/foo/Foo.java similarity index 100% rename from example/javalib/builds/2-compat-modules/foo/src/main/java/foo/Foo.java rename to example/javalib/basic/4-compat-modules/foo/src/main/java/foo/Foo.java diff --git a/example/javalib/builds/2-compat-modules/foo/src/test/java/foo/FooTests.java b/example/javalib/basic/4-compat-modules/foo/src/test/java/foo/FooTests.java similarity index 100% rename from example/javalib/builds/2-compat-modules/foo/src/test/java/foo/FooTests.java rename to example/javalib/basic/4-compat-modules/foo/src/test/java/foo/FooTests.java diff --git a/example/javalib/basic/4-builtin-commands/bar/src/bar/Bar.java b/example/javalib/basic/5-builtin-commands/bar/src/bar/Bar.java similarity index 100% rename from example/javalib/basic/4-builtin-commands/bar/src/bar/Bar.java rename to example/javalib/basic/5-builtin-commands/bar/src/bar/Bar.java diff --git a/example/javalib/basic/4-builtin-commands/bar/test/src/bar/BarTest.java b/example/javalib/basic/5-builtin-commands/bar/test/src/bar/BarTest.java similarity index 100% rename from example/javalib/basic/4-builtin-commands/bar/test/src/bar/BarTest.java rename to example/javalib/basic/5-builtin-commands/bar/test/src/bar/BarTest.java diff --git a/example/javalib/basic/4-builtin-commands/build.mill b/example/javalib/basic/5-builtin-commands/build.mill similarity index 100% rename from example/javalib/basic/4-builtin-commands/build.mill rename to example/javalib/basic/5-builtin-commands/build.mill diff --git a/example/javalib/basic/4-builtin-commands/foo/src/foo/Foo.java b/example/javalib/basic/5-builtin-commands/foo/src/foo/Foo.java similarity index 100% rename from example/javalib/basic/4-builtin-commands/foo/src/foo/Foo.java rename to example/javalib/basic/5-builtin-commands/foo/src/foo/Foo.java diff --git a/example/kotlinlib/builds/2-compat-modules/build.mill b/example/kotlinlib/basic/4-compat-modules/build.mill similarity index 68% rename from example/kotlinlib/builds/2-compat-modules/build.mill rename to example/kotlinlib/basic/4-compat-modules/build.mill index 69b87c7b1bf..8f0d51188d6 100644 --- a/example/kotlinlib/builds/2-compat-modules/build.mill +++ b/example/kotlinlib/basic/4-compat-modules/build.mill @@ -1,4 +1,10 @@ //// SNIPPET:ALL +// Mill's default folder layout of `foo/src/` and `foo/test/src` differs from that +// of Maven or Gradle's `foo/src/main/kotlin/` and `foo/src/test/kotlin/`. If you are +// migrating an existing codebase, you can use Mill's `KotlinMavenModule` and +// `KotlinMavenTests` as shown below to preserve filesystem compatibility with an existing +// Maven or Gradle build: + package build import mill._, kotlinlib._ @@ -6,7 +12,7 @@ object foo extends KotlinModule with KotlinMavenModule { def kotlinVersion = "1.9.24" - object test extends KotlinMavenModuleTests with TestModule.Junit5 { + object test extends KotlinMavenTests with TestModule.Junit5 { def ivyDeps = super.ivyDeps() ++ Agg( ivy"io.kotest:kotest-runner-junit5-jvm:5.9.1" ) diff --git a/example/kotlinlib/builds/2-compat-modules/foo/src/main/java/foo/Foo.kt b/example/kotlinlib/basic/4-compat-modules/foo/src/main/java/foo/Foo.kt similarity index 100% rename from example/kotlinlib/builds/2-compat-modules/foo/src/main/java/foo/Foo.kt rename to example/kotlinlib/basic/4-compat-modules/foo/src/main/java/foo/Foo.kt diff --git a/example/kotlinlib/builds/2-compat-modules/foo/src/main/java/foo/Foo2.java b/example/kotlinlib/basic/4-compat-modules/foo/src/main/java/foo/Foo2.java similarity index 100% rename from example/kotlinlib/builds/2-compat-modules/foo/src/main/java/foo/Foo2.java rename to example/kotlinlib/basic/4-compat-modules/foo/src/main/java/foo/Foo2.java diff --git a/example/kotlinlib/builds/2-compat-modules/foo/src/test/java/foo/FooTests.kt b/example/kotlinlib/basic/4-compat-modules/foo/src/test/java/foo/FooTests.kt similarity index 100% rename from example/kotlinlib/builds/2-compat-modules/foo/src/test/java/foo/FooTests.kt rename to example/kotlinlib/basic/4-compat-modules/foo/src/test/java/foo/FooTests.kt diff --git a/example/kotlinlib/basic/4-builtin-commands/bar/src/bar/Bar.kt b/example/kotlinlib/basic/5-builtin-commands/bar/src/bar/Bar.kt similarity index 100% rename from example/kotlinlib/basic/4-builtin-commands/bar/src/bar/Bar.kt rename to example/kotlinlib/basic/5-builtin-commands/bar/src/bar/Bar.kt diff --git a/example/kotlinlib/basic/4-builtin-commands/bar/test/src/bar/BarTests.kt b/example/kotlinlib/basic/5-builtin-commands/bar/test/src/bar/BarTests.kt similarity index 100% rename from example/kotlinlib/basic/4-builtin-commands/bar/test/src/bar/BarTests.kt rename to example/kotlinlib/basic/5-builtin-commands/bar/test/src/bar/BarTests.kt diff --git a/example/kotlinlib/basic/4-builtin-commands/build.mill b/example/kotlinlib/basic/5-builtin-commands/build.mill similarity index 100% rename from example/kotlinlib/basic/4-builtin-commands/build.mill rename to example/kotlinlib/basic/5-builtin-commands/build.mill diff --git a/example/kotlinlib/basic/4-builtin-commands/foo/src/foo/Foo.kt b/example/kotlinlib/basic/5-builtin-commands/foo/src/foo/Foo.kt similarity index 100% rename from example/kotlinlib/basic/4-builtin-commands/foo/src/foo/Foo.kt rename to example/kotlinlib/basic/5-builtin-commands/foo/src/foo/Foo.kt diff --git a/example/package.mill b/example/package.mill index 63d1989c25c..259d6ce977c 100644 --- a/example/package.mill +++ b/example/package.mill @@ -85,8 +85,6 @@ object `package` extends RootModule with Module { trait ExampleCrossModuleKotlin extends ExampleCrossModuleJava { override def lineTransform(line: String) = this.millModuleSegments.parts.last match { - case "4-builtin-commands" => - line.replace("compile.dest/zinc", "compile.dest/kotlin.analysis.dummy") case "1-test-suite" => line .replace("mill bar.test bar.BarTests.hello", "kotest_filter_tests='hello' kotest_filter_specs='bar.BarTests' ./mill bar.test") .replace("compiling 1 ... source...", "Compiling 1 ... source...") diff --git a/example/scalalib/builds/2-compat-modules/bar/src/main/scala-2.12/MinorVersionSpecific.scala b/example/scalalib/basic/4-compat-modules/bar/src/main/scala-2.12/MinorVersionSpecific.scala similarity index 100% rename from example/scalalib/builds/2-compat-modules/bar/src/main/scala-2.12/MinorVersionSpecific.scala rename to example/scalalib/basic/4-compat-modules/bar/src/main/scala-2.12/MinorVersionSpecific.scala diff --git a/example/scalalib/builds/2-compat-modules/bar/src/main/scala-2.13/MinorVersionSpecific.scala b/example/scalalib/basic/4-compat-modules/bar/src/main/scala-2.13/MinorVersionSpecific.scala similarity index 100% rename from example/scalalib/builds/2-compat-modules/bar/src/main/scala-2.13/MinorVersionSpecific.scala rename to example/scalalib/basic/4-compat-modules/bar/src/main/scala-2.13/MinorVersionSpecific.scala diff --git a/example/scalalib/builds/2-compat-modules/bar/src/main/scala/Bar.scala b/example/scalalib/basic/4-compat-modules/bar/src/main/scala/Bar.scala similarity index 100% rename from example/scalalib/builds/2-compat-modules/bar/src/main/scala/Bar.scala rename to example/scalalib/basic/4-compat-modules/bar/src/main/scala/Bar.scala diff --git a/example/scalalib/builds/2-compat-modules/build.mill b/example/scalalib/basic/4-compat-modules/build.mill similarity index 81% rename from example/scalalib/builds/2-compat-modules/build.mill rename to example/scalalib/basic/4-compat-modules/build.mill index e2211853b37..dc9d30c7294 100644 --- a/example/scalalib/builds/2-compat-modules/build.mill +++ b/example/scalalib/basic/4-compat-modules/build.mill @@ -1,4 +1,10 @@ //// SNIPPET:ALL +// Mill's default folder layout of `foo/src/` and `foo/test/src` differs from that +// of SBT's `foo/src/main/scala/` and `foo/src/test/scala/`. If you are +// migrating an existing codebase from SBT, you can use Mill's `SbtModule` and +// `SbtTests` as shown below to preserve filesystem compatibility with an existing +// SBT build: + package build import mill._, scalalib._ diff --git a/example/scalalib/builds/2-compat-modules/foo/src/main/scala/Foo.scala b/example/scalalib/basic/4-compat-modules/foo/src/main/scala/Foo.scala similarity index 100% rename from example/scalalib/builds/2-compat-modules/foo/src/main/scala/Foo.scala rename to example/scalalib/basic/4-compat-modules/foo/src/main/scala/Foo.scala diff --git a/example/scalalib/builds/2-compat-modules/foo/src/test/scala/FooTests.scala b/example/scalalib/basic/4-compat-modules/foo/src/test/scala/FooTests.scala similarity index 100% rename from example/scalalib/builds/2-compat-modules/foo/src/test/scala/FooTests.scala rename to example/scalalib/basic/4-compat-modules/foo/src/test/scala/FooTests.scala diff --git a/example/scalalib/basic/4-builtin-commands/bar/src/Bar.scala b/example/scalalib/basic/5-builtin-commands/bar/src/Bar.scala similarity index 100% rename from example/scalalib/basic/4-builtin-commands/bar/src/Bar.scala rename to example/scalalib/basic/5-builtin-commands/bar/src/Bar.scala diff --git a/example/scalalib/basic/4-builtin-commands/build.mill b/example/scalalib/basic/5-builtin-commands/build.mill similarity index 99% rename from example/scalalib/basic/4-builtin-commands/build.mill rename to example/scalalib/basic/5-builtin-commands/build.mill index 1df3c95423e..35a3dd4bf03 100644 --- a/example/scalalib/basic/4-builtin-commands/build.mill +++ b/example/scalalib/basic/5-builtin-commands/build.mill @@ -143,7 +143,7 @@ Inputs: > mill show foo.compile { - "analysisFile": ".../out/foo/compile.dest/zinc", + "analysisFile": ".../out/foo/compile.dest/...", "classes": ".../out/foo/compile.dest/classes" } */ diff --git a/example/scalalib/basic/4-builtin-commands/foo/src/Foo.scala b/example/scalalib/basic/5-builtin-commands/foo/src/Foo.scala similarity index 100% rename from example/scalalib/basic/4-builtin-commands/foo/src/Foo.scala rename to example/scalalib/basic/5-builtin-commands/foo/src/Foo.scala diff --git a/example/scalalib/basic/5-multiple-test-frameworks/build.mill b/example/scalalib/basic/6-multiple-test-frameworks/build.mill similarity index 100% rename from example/scalalib/basic/5-multiple-test-frameworks/build.mill rename to example/scalalib/basic/6-multiple-test-frameworks/build.mill diff --git a/example/scalalib/basic/5-multiple-test-frameworks/src/Foo.scala b/example/scalalib/basic/6-multiple-test-frameworks/src/Foo.scala similarity index 100% rename from example/scalalib/basic/5-multiple-test-frameworks/src/Foo.scala rename to example/scalalib/basic/6-multiple-test-frameworks/src/Foo.scala diff --git a/example/scalalib/basic/5-multiple-test-frameworks/test/src/FooScalaTests.scala b/example/scalalib/basic/6-multiple-test-frameworks/test/src/FooScalaTests.scala similarity index 100% rename from example/scalalib/basic/5-multiple-test-frameworks/test/src/FooScalaTests.scala rename to example/scalalib/basic/6-multiple-test-frameworks/test/src/FooScalaTests.scala diff --git a/example/scalalib/basic/5-multiple-test-frameworks/test/src/FooTests.scala b/example/scalalib/basic/6-multiple-test-frameworks/test/src/FooTests.scala similarity index 100% rename from example/scalalib/basic/5-multiple-test-frameworks/test/src/FooTests.scala rename to example/scalalib/basic/6-multiple-test-frameworks/test/src/FooTests.scala diff --git a/kotlinlib/src/mill/kotlinlib/KotlinMavenModule.scala b/kotlinlib/src/mill/kotlinlib/KotlinMavenModule.scala index 0365f74f417..66303b691a9 100644 --- a/kotlinlib/src/mill/kotlinlib/KotlinMavenModule.scala +++ b/kotlinlib/src/mill/kotlinlib/KotlinMavenModule.scala @@ -15,7 +15,7 @@ trait KotlinMavenModule extends KotlinModule with MavenModule { millSourcePath / "src/main/resources" } - trait KotlinMavenModuleTests extends KotlinTests with MavenTests { + trait KotlinMavenTests extends KotlinTests with MavenTests { override def intellijModulePath: os.Path = millSourcePath / "src/test" override def sources = T.sources(