Skip to content

Commit

Permalink
Add .mill.sc as an alternative file extension to .mill (#3461)
Browse files Browse the repository at this point in the history
Took a bit of fiddling because we had to stop using `os.Path#ext`
because it only takes the last segment, and also now one of the
extensions is a suffix of another. There will likely be a short tail of
bugs that arise because of this, but it shouldn't be too burdensome to
resolve them. Otherwise this functionality should be relatively cheap to
add and maintain since we already have the extensions and file names
centralized in `CodeGenConstants.java`.

This is a compromise between `.mill` and `.mill.sc`. I still believe
long term `.mill` is better, but I can believe that short term pain of
having everything not understand your `.mill` extensions is painful.
This lets people use `.mill.sc` where preferable, and `.mill` otherwise
by default. I expect long term `.mill` will be superior once all the
tooling has caught up, but until then people can use `.mill.sc` if they
want the generic `.sc` Scala script support that many tools use.

Having two extensions isn't that unusual, e.g. Bazel supports both
`BUILD` and `BUILD.bazel` as file extenions, supported indefinitely. The
cost of adding two extensions isn't significantly more than the cost of
adding one: in any case we need to send PRs to the various tools, and a
PR adding two extensions isn't any more difficult than a PR adding one.

Added an example test `13-helper-files-mill-sc`, variant of
`11-helper-files`, which should exercise most of the file
discovery/import/codegen/etc. logic that may be impacted by the
extension
  • Loading branch information
lihaoyi authored Sep 5, 2024
1 parent 4caadfb commit 75b7cfa
Show file tree
Hide file tree
Showing 80 changed files with 873 additions and 640 deletions.
97 changes: 41 additions & 56 deletions .github/workflows/actions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,80 +28,79 @@ concurrency:
cancel-in-progress: true

jobs:

build-linux:
uses: ./.github/workflows/run-mill-action.yml
with:
java-version: 11
millargs: __.compile
populate_cache: true
build-windows:
uses: ./.github/workflows/run-mill-action.yml
with:
os: windows-latest
java-version: 11
millargs: __.compile
populate_cache: true
itest:
needs: build-linux
strategy:
fail-fast: false
matrix:
include:
# bootstrap tests
- java-version: 11 # Have one job on oldest JVM
buildcmd: ci/test-mill-dev.sh
buildcmd: ci/test-mill-dev.sh && ci/test-mill-release.sh && ./mill -i -k __.ivyDepsTree && ./mill -i -k __.ivyDepsTree --withRuntime
- java-version: 17 # Have one job on default JVM
buildcmd: ci/test-mill-release.sh
- java-version: 17
buildcmd: ci/test-mill-bootstrap.sh
# Just some reporting to enable reasoning about library upgrades
- java-version: 11
buildcmd: |
./mill -i -k __.ivyDepsTree
./mill -i -k __.ivyDepsTree --withRuntime

uses: ./.github/workflows/run-mill-action.yml
with:
java-version: ${{ matrix.java-version }}
buildcmd: ${{ matrix.buildcmd }}

linux:
needs: build-linux
strategy:
fail-fast: false
matrix:
java-version: [11, 17]
millargs:
# Run unit and module tests on both oldest and newest Java versions
- '"{main,scalalib,testrunner,bsp}.__.test"'

include:
# For most tests, run them arbitrarily on Java 8 or Java 17 on Linux, and
# For most tests, run them arbitrarily on Java 11 or Java 17 on Linux, and
# on the opposite version on Windows below, so we get decent coverage of
# each test on each Java version and each operating system
# We also try to group tests together to manuaully balance out the runtimes of each jobs
- java-version: 17
millargs: "'{main,scalalib,testrunner,bsp,testkit}.__.testCached'"
- java-version: 11
millargs: '"scalajslib.__.test"'
- java-version: 11
millargs: '"scalanativelib.__.test"'
millargs: "'{scalajslib,scalanativelib}.__.testCached'"
- java-version: 17
millargs: "contrib._.test"
millargs: "contrib.__.testCached"

# Group these tests together to try and balance out the runtimes of each job
# Just running in `local` mode since they shouldn't depend on the mode
- java-version: 17
millargs: "'example.javalib.__.local.test'"
millargs: "'example.javalib.__.local.testCached'"
- java-version: 17
millargs: "'example.scalalib.__.local.test'"
millargs: "'example.scalalib.__.local.testCached'"
- java-version: 11
millargs: "'example.thirdparty[{mockito,acyclic,commons-io}].local.test'"
millargs: "'example.thirdparty[{mockito,acyclic,commons-io}].local.testCached'"
- java-version: 17
millargs: "'example.thirdparty[{fansi,jimfs,netty,gatling}].local.test'"
millargs: "'example.thirdparty[{fansi,jimfs,netty,gatling}].local.testCached'"
- java-version: 11
millargs: "'example.depth.__.local.test'"
- java-version: 17
millargs: "'example.extending.__.local.test'"
millargs: "'example.{depth,extending}.__.local.testCached'"

# Most of these integration tests should not depend on which mode they
# are run in, so just run them in `local`
- java-version: 11
millargs: "'integration.{failure,feature,ide}[_].local.test'"
millargs: "'integration.{failure,feature,ide}.__.local.testCached'"

# These invalidation tests need to be exercised in all three execution modes
# These invalidation tests need to be exercised in both execution modes
# to make sure they work with and without -i/--no-server being passed
- java-version: 17
millargs: "'integration.invalidation[_].local.test'"
- java-version: 17
millargs: "'integration.invalidation[_].fork.test'"
millargs: "'integration.invalidation.__.fork.testCached'"
- java-version: 17
millargs: "'integration.invalidation[_].server.test'"
millargs: "'integration.invalidation.__.server.testCached'"

# Check docsite compiles
- java-version: 17
- java-version: 11
millargs: docs.githubPages


Expand All @@ -111,32 +110,22 @@ jobs:
millargs: ${{ matrix.millargs }}

compiler-bridge:
needs: build-linux
uses: ./.github/workflows/run-mill-action.yml
with:
java-version: '8'
millargs: bridge.__.publishLocal
env-bridge-versions: 'essential'

format-check:
uses: ./.github/workflows/run-mill-action.yml
with:
java-version: '11'
millargs: mill.scalalib.scalafmt.ScalafmtModule/checkFormatAll __.sources

scalafix-check:
format-scalafix-bincompat:
needs: build-linux
uses: ./.github/workflows/run-mill-action.yml
with:
java-version: '11'
millargs: -i -k __.fix --check

bincompat-check:
uses: ./.github/workflows/run-mill-action.yml
with:
java-version: '11'
millargs: __.mimaReportBinaryIssues
continue-on-error: true
buildcmd: ./mill -i mill.scalalib.scalafmt.ScalafmtModule/checkFormatAll __.sources + __.mimaReportBinaryIssues + __.fix --check

windows:
needs: build-windows
strategy:
fail-fast: false
matrix:
Expand All @@ -145,16 +134,12 @@ jobs:
# the whole suite can take hours on windows v.s. half an hour on linux
- java-version: 11
millargs: '"{main,scalalib,bsp}.__.test"'
- java-version: 17
millargs: '"scalajslib.__.test"'
- java-version: 11
millargs: '"example.scalalib.basic.__.fork.test"'
millargs: '"example.scalalib.{basic,web}.__.fork.test"'
- java-version: 17
millargs: "'integration.feature[_].fork.test'"
millargs: "'integration.{feature,failure}[_].fork.test'"
- java-version: 11
millargs: "'integration.invalidation[_].server.test'"
- java-version: 17
millargs: "'integration.failure[_].fork.test'"
- java-version: 11
millargs: "contrib.__.test"

Expand All @@ -167,7 +152,7 @@ jobs:
publish-sonatype:
# when in master repo, publish all tags and manual runs on main
if: github.repository == 'com-lihaoyi/mill' && (startsWith( github.ref, 'refs/tags/') || (github.ref == 'refs/heads/main' && github.event_name == 'workflow_dispatch' ) )
needs: [linux, windows, compiler-bridge, format-check, bincompat-check, scalafix-check, itest]
needs: [linux, windows, compiler-bridge, format-scalafix-bincompat, itest]

runs-on: ubuntu-latest

Expand Down
30 changes: 28 additions & 2 deletions .github/workflows/run-mill-action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ on:
continue-on-error:
default: false
type: boolean
populate_cache:
default: false
type: boolean
timeout-minutes:
default: 60
type: number
Expand All @@ -38,6 +41,16 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
if: ${{ inputs.populate_cache }}

- uses: actions/download-artifact@v4
if: ${{ !inputs.populate_cache }}
with:
path: .
name: ${{ inputs.os }}-artifact

- name: chmod executable
run: "chmod -R +x ."

- uses: coursier/cache-action@v6

Expand All @@ -60,5 +73,18 @@ jobs:
if: inputs.millargs != '' && !startsWith(inputs.os, 'windows')

- name: Run Mill (on Windows) '${{ inputs.millargs }}'
run: cmd /C %GITHUB_WORKSPACE%\ci\mill.bat -i -d -k ${{ inputs.millargs }}
if: inputs.millargs != '' && startsWith(inputs.os, 'windows')
run: cmd /C %GITHUB_WORKSPACE%\ci\mill.bat -ij1 __.resolvedIvyDeps; cmd /C %GITHUB_WORKSPACE%\ci\mill.bat -i -j1 -k ${{ inputs.millargs }}
if: inputs.millargs != '' && startsWith(inputs.os, 'windows')

- name: Run Mill (on Windows) Worker Cleanup
run: 'taskkill -f -im java* && rm -rf out/mill-worker-*'
if: inputs.millargs != '' && startsWith(inputs.os, 'windows')
shell: bash
continue-on-error: true

- uses: actions/upload-artifact@v4.3.5
with:
path: .
name: ${{ inputs.os }}-artifact
include-hidden-files: true
if: ${{ inputs.populate_cache }}
2 changes: 1 addition & 1 deletion build.mill
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ object Deps {
val log4j2Core = ivy"org.apache.logging.log4j:log4j-core:2.23.1"
val osLib = ivy"com.lihaoyi::os-lib:0.10.5"
val pprint = ivy"com.lihaoyi::pprint:0.9.0"
val mainargs = ivy"com.lihaoyi::mainargs:0.7.2"
val mainargs = ivy"com.lihaoyi::mainargs:0.7.4"
val millModuledefsVersion = "0.11.0-M2"
val millModuledefsString = s"com.lihaoyi::mill-moduledefs:${millModuledefsVersion}"
val millModuledefs = ivy"${millModuledefsString}"
Expand Down
2 changes: 2 additions & 0 deletions ci/test-mill-dev.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,5 @@ test -d $EXAMPLE/out/foo/3.3.3/compile.dest
./mill -i dist.run $EXAMPLE show "bar[2.13.8].assembly"

test -f $EXAMPLE/out/bar/2.13.8/assembly.dest/out.jar

./mill -i dist.run $EXAMPLE shutdown
2 changes: 2 additions & 0 deletions ci/test-mill-release.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,5 @@ test -d $EXAMPLE/out/foo/3.3.3/compile.dest
(cd $EXAMPLE && ../../../../out/dist/assembly.dest/mill show "bar[2.13.8].assembly")

test -f $EXAMPLE/out/bar/2.13.8/assembly.dest/out.jar

(cd $EXAMPLE && ../../../../out/dist/assembly.dest/mill shutdown)
1 change: 1 addition & 0 deletions example/depth/large/11-helper-files/build.mill
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ object `package` extends RootModule with MyModule{
"MY_PROJECT_VERSION" -> versions.myProjectVersion,
)
}

/** See Also: util.mill */
/** See Also: foo/package.mill */
/** See Also: foo/versions.mill */
Expand Down
39 changes: 39 additions & 0 deletions example/depth/large/13-helper-files-mill-sc/build.mill.sc
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
package build
import mill._, scalalib._
import $file.foo.versions
import $file.util.MyModule
object `package` extends RootModule with MyModule{
def forkEnv = Map(
"MY_SCALA_VERSION" -> build.scalaVersion(),
"MY_PROJECT_VERSION" -> versions.myProjectVersion,
)
}
///** See Also: util.mill.sc */
///** See Also: foo/package.mill.sc */
///** See Also: foo/versions.mill.sc */


// Apart from having `package` files in subfolders to define modules, Mill
// also allows you to have helper code in any `*.mill` file in the same folder
// as your `build.mill` or a `package.mill`.
//
// Different helper scripts and ``build.mill``/``package`` files can all refer to
// each other using the `build` object, which marks the root object of your build.
// In this example:
//
// * `build.mill` can be referred to as simple `build`
// * `util.mill` can be referred to as simple `$file.util`
// * `foo/package` can be referred to as simple `build.foo`
// * `foo/versions.mill` can be referred to as simple `$file.foo.versions`

/** Usage
> ./mill run
Main Env build.util.myScalaVersion: 2.13.14
Main Env build.foo.versions.myProjectVersion: 0.0.1
> ./mill foo.run
Foo Env build.util.myScalaVersion: 2.13.14
Foo Env build.foo.versions.myProjectVersion: 0.0.1
*/
10 changes: 10 additions & 0 deletions example/depth/large/13-helper-files-mill-sc/foo/package.mill.sc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package build.foo
import mill._, scalalib._
import $file.util
import $file.foo.versions.myProjectVersion
object `package` extends RootModule with build_.util.MyModule {
def forkEnv = Map(
"MY_SCALA_VERSION" -> util.myScalaVersion,
"MY_PROJECT_VERSION" -> myProjectVersion
)
}
8 changes: 8 additions & 0 deletions example/depth/large/13-helper-files-mill-sc/foo/src/Foo.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package foo

object Foo {
def main(args: Array[String]): Unit = {
println("Foo Env build.util.myScalaVersion: " + sys.env("MY_SCALA_VERSION"))
println("Foo Env build.foo.versions.myProjectVersion: " + sys.env("MY_PROJECT_VERSION"))
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
package build.foo

def myProjectVersion = "0.0.1"
6 changes: 6 additions & 0 deletions example/depth/large/13-helper-files-mill-sc/src/Main.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
object Main {
def main(args: Array[String]): Unit = {
println("Main Env build.util.myScalaVersion: " + sys.env("MY_SCALA_VERSION"))
println("Main Env build.foo.versions.myProjectVersion: " + sys.env("MY_PROJECT_VERSION"))
}
}
9 changes: 9 additions & 0 deletions example/depth/large/13-helper-files-mill-sc/util.mill.sc
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
package build

import mill._, scalalib._

def myScalaVersion = "2.13.14"

trait MyModule extends ScalaModule {
def scalaVersion = myScalaVersion
}
2 changes: 1 addition & 1 deletion example/javalib/web/2-hello-spring-boot/build.mill
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ object `package` extends RootModule with JavaModule {
> mill test
...com.example.HelloSpringBootTest#shouldReturnDefaultMessage() finished...

> mill runBackground; sleep 10 # give time for server to start
> mill runBackground; sleep 15 # give time for server to start

> curl http://localhost:8086
...<h1>Hello, World!</h1>...
Expand Down
2 changes: 1 addition & 1 deletion example/javalib/web/3-todo-spring-boot/build.mill
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ object `package` extends RootModule with JavaModule {
...com.example.TodomvcIntegrationTests#homePageLoads() finished...
...com.example.TodomvcIntegrationTests#addNewTodoItem() finished...

> mill test.runBackground; sleep 10 # give time for server to start
> mill test.runBackground; sleep 15 # give time for server to start

> curl http://localhost:8087
...<h1>todos</h1>...
Expand Down
2 changes: 1 addition & 1 deletion example/javalib/web/4-hello-micronaut/build.mill
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ trait MicronautModule extends MavenModule{
> mill test
...example.micronaut.HelloControllerTest#testHello()...

> mill runBackground; sleep 2 # give time for server to start
> mill runBackground; sleep 5 # give time for server to start

> curl http://localhost:8088/hello
...Hello World...
Expand Down
2 changes: 1 addition & 1 deletion example/javalib/web/5-todo-micronaut/build.mill
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ trait MicronautModule extends MavenModule{
...example.micronaut.TodoItemControllerTest...
...example.micronaut.HtmxWebJarsTest...

> mill runBackground; sleep 2 # give time for server to start
> mill runBackground; sleep 5 # give time for server to start

> curl http://localhost:8088
...<h1>todos</h1>...
Expand Down
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
package mill.integration

import mill.testkit.IntegrationTestSuite
import mill.testkit.UtestIntegrationTestSuite

import utest._

object BuildFileInSubfolderTests extends IntegrationTestSuite {
object BuildFileInSubfolderTests extends UtestIntegrationTestSuite {
val tests: Tests = Tests {
initWorkspace()

test("success") {
val res = eval(("resolve", "_"))
test("success") - integrationTest { tester =>
val res = tester.eval(("resolve", "_"))
assert(res.isSuccess == false)
assert(res.err.contains("Mill build.mill files can only be in the project root"))
}
Expand Down
Loading

0 comments on commit 75b7cfa

Please sign in to comment.