Fixes: #3862; Added an example Python Support case study

docs/modules/ROOT/pages/extending/new-language.adoc

@@ -36,4 +36,32 @@ programming language toolchain. It would take significantly more work to flesh o
 the featureset and performance of `TypeScriptModule` to be usable in a real world
 build. But this should be enough to get you started working with Mill to add support
 to any language you need: whether it's TypeScript or some other language, most programming
-language toolchains have similar concepts of `compile`, `run`, `bundle`, etc.
+language toolchains have similar concepts of `compile`, `run`, `bundle`, etc.
+
+
+== Python Support
+
+This section walks through the process of adding support for Python
+language to Mill. We will be adding a small `trait PythonModule` with the
+ability to resolve dependencies, typecheck local code, and optimize a final
+bundle.
+
+The Python integration here is not intended for production usage, but is
+instead meant for illustration purposes of the techniques typically used in
+implementing language toolchains.
+
+include::partial$example/extending/newlang/5-hello-python.adoc[]
+
+include::partial$example/extending/newlang/6-python-modules.adoc[]
+
+include::partial$example/extending/newlang/7-python-module-deps.adoc[]
+
+include::partial$example/extending/newlang/8-python-libs-bundle.adoc[]
+
+
+
+As mentioned earlier, the `PythonModule` examples on this page are meant for
+demo purposes: to show what it looks like to add support in Mill for a new
+programming language toolchain. It would take significantly more work to flesh out
+the featureset and performance of `PythonModule` to be usable in a real world
+build.

example/extending/newlang/5-hello-python/build.mill

@@ -0,0 +1,158 @@
+// == Basic Python Pipeline 
+
+// This Example demonstrates integration of https://www.python.org[Python]
+// compilation into a Mill build to compile https://www.python.org[Python] Scripts. Mill
+// does not come bundled with https://www.python.org[Python] integration, so here we begin setting
+// one up from first principles using the Basic `Python3`(Already Present in OS) command Line.
+
+// === Python Setup
+//
+// First, we need to use the `Python3` CLI tool to create virtual environment and
+// install `mypy` for Type Checking using the official Python Virtual Environment
+// https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments[Documentation].
+
+// `MyPy` is a static type checker that uses Python `type hints` to enforce `type correctness` and `catch errors` during development.
+
+package build
+import mill._
+
+def createVenv: T[PathRef] = Task {
+  val venvDir = T.dest / "venv"
+
+  os.call(
+    Seq(
+      "python3",
+      "-m",
+      "venv",
+      venvDir.toString
+    )
+  )
+
+  PathRef(venvDir)
+}
+
+def setup: T[PathRef] = Task {
+  // creating virtual environment
+  val pythonVenv = createVenv().path / "bin" / "python3"
+
+  // installing mypy for Type Checking
+  os.call(
+    Seq(
+      pythonVenv.toString,
+      "-m",
+      "pip",
+      "install",
+      "mypy"
+    )
+  )
+  PathRef(pythonVenv)
+}
+
+// The `createVenv` task ensures the Python virtual environment creation process using the official documentaion.
+// After this, we need to install python `mypy` library for Type Checking and Error Checking before actual run of the code, 
+// using the offical https://mypy.readthedocs.io/en/stable[Documentation] for `mypy` 
+
+// === Defining our Sources
+
+// Next, We define the `sources` task to help gather source files for processing. 
+// `sources` sets a base directory for Python source files and enable easy access to Python files in a project,
+// making it straightforward to retrieve all relevant code files for tasks like building, testing, or analysis.
+
+def sources: T[PathRef] = Task.Source(millSourcePath / "src")
+
+// === Type Checking
+
+// The `mainFileName` Task provides the name for the main python script file for running and 
+// The `typeCheck` Task  ensures that the python setup is set and Checks the main file for error free code.
+// This task is considered as a part of code which is responsible for Error Checking of the Main Script 
+// and setting up setup for python script runable environment.
+
+def mainFileName: T[String] = Task { "main.py" }
+def typeCheck: T[PathRef] = Task {
+  val pythonVenv = setup().path
+
+  os.call(
+    Seq(
+      pythonVenv.toString,
+      "-m",
+      "mypy",
+      "--strict",
+      s"${sources().path / mainFileName()}"
+    ),
+    stdout = os.Inherit
+  )
+
+  PathRef(pythonVenv)
+}
+
+// At this point, we have a minimal working build, with a build graph that looks like this:
+//
+// ```graphviz
+// digraph G {
+//   rankdir=LR
+//   node [shape=box width=0 height=0 style=filled fillcolor=white]
+//   createVenv -> setup -> typeCheck
+//   sources -> typeCheck
+//   mainFileName -> typeCheck
+// }
+// ```
+
+// When `typeCheck` task sets up the Setup for Virtual Environment and checks the main file for error free code 
+// then we have `main.py` and running this file will result in either a generic greeting 
+// or a personalized one based on the command-line arguments passed, allowing for flexible interaction with the user.
+
+/** See Also: src/main.py */
+
+
+// === Running
+
+// The `run` function executes the main Python file (`main.py`) and accepts command-line arguments from the user. 
+// It uses `os.call` to run the Python interpreter(from Virtual Environment) with the main file's path 
+// and any additional arguments. The output is shown in the console, allowing for real-time interaction.
+
+def run(args: mill.define.Args) = Task.Command {
+  val pythonVenv = typeCheck().path
+
+  os.call(
+    Seq(
+      pythonVenv.toString,
+      s"${sources().path / mainFileName()}"
+    ) ++ args.value,
+    stdout = os.Inherit
+  )
+}
+
+// Note that we use `stdout = os.Inherit` since we want to display any output to the user,
+// rather than capturing it for use in our command.
+
+// ```graphviz
+// digraph G {
+//   rankdir=LR
+//   node [shape=box width=0 height=0 style=filled fillcolor=white]
+//   createVenv -> setup -> typeCheck -> run
+//   sources -> typeCheck
+//   mainFileName -> typeCheck
+//   sources -> run
+//   mainFileName -> run
+//   mainFileName [color=green, penwidth=3]
+//   run [color=green, penwidth=3]
+// }
+// ```
+
+// Running run command will return the result to the console.
+
+/** Usage
+
+> ./mill typeCheck
+Success: no issues found in 1 source file
+
+> ./mill run Mill Python
+Hello, Mill Python!
+
+*/
+
+// Uncomment the Result variable line(Line Number:13) in the `src/main.py` code to see `typeCheck` error
+
+// So that's a minimal example of implementing a `Basic Python
+// pipeline` locally. Next, we will look at turning it into a `PythonModule` that
+// can be re-used

example/extending/newlang/5-hello-python/src/main.py

@@ -0,0 +1,24 @@
+import sys
+
+def add(a: int, b: int) -> int:
+    """Adds two integers."""
+    return a + b
+
+def main() -> None:
+    # Get command-line arguments, skipping the first one (script name)
+    args = sys.argv[1:]
+
+    # Example of using the add function with incorrect types
+    # Uncomment this line to see the error
+    # result = add("5", 10)  # This will cause a TypeError, handled in the function
+
+    # Check if any arguments were provided
+    if not args:
+        print("Hello, World!")
+    else:
+        # Join the arguments with spaces and print the greeting
+        greeting = "Hello, " + " ".join(args) + "!"
+        print(greeting)
+
+if __name__ == "__main__":
+    main()

example/extending/newlang/6-python-modules/build.mill

@@ -0,0 +1,168 @@
+// == Re-usable PythonModule
+// This Example demonstrates the use of a `PythonModule` trait to manage Python Scripts within
+// different objects in a Mill build.
+
+// ==== Using Same PythonModule for Mutiple Objects
+
+// The `PythonModule` trait simplifies Python project management within the `Mill` build tool by providing essential
+// functionalities such as creating `virtual environment`, managing `source files`, `Checking code for Error`, and `executing Python` scripts.
+// It ensures that users have a streamlined workflow by automating the setup of necessary tools 
+// and organizing project files effectively. 
+
+package build
+import mill._
+
+/** PythonModule Trait for Basic Python Pipeline in Mill Build Tool */
+trait PythonModule extends Module {
+
+  /** Returns the path to the source directory containing Python files. */
+  def sources: T[PathRef] = Task.Source(millSourcePath / "src")
+
+  /** Returns the name of the main Python file. */
+  def mainFileName: T[String] = Task { "main.py" }
+
+  /** Creates a virtual environment using the installed Python version. */
+  def createVenv: T[PathRef] = Task {
+    val venvDir = T.dest / "venv"
+
+    os.call(
+      Seq(
+        "python3",
+        "-m",
+        "venv",
+        venvDir.toString
+      )
+    )
+
+    PathRef(venvDir)
+  }
+
+  /** This minimal setup provide basic Virtual Environment and Type Checking. */
+  def setup: T[PathRef] = Task {
+    // creating virtual environment
+    val pythonVenv = createVenv().path / "bin" / "python3"
+
+    // installing mypy for Type Checking
+    os.call(
+      Seq(
+        pythonVenv.toString,
+        "-m",
+        "pip",
+        "install",
+        "mypy"
+      )
+    )
+    PathRef(pythonVenv)
+  }
+
+  /** Creates Virtual env and check main file. */
+  def typeCheck: T[PathRef] = Task {
+    val pythonVenv = setup().path
+
+    os.call(
+      Seq(
+        pythonVenv.toString,
+        "-m",
+        "mypy",
+        "--strict",
+        s"${sources().path / mainFileName()}"
+      ),
+      stdout = os.Inherit
+    )
+
+    PathRef(pythonVenv)
+  }
+
+  /** Runs the checked main Python file with given command-line arguments. */
+  def run(args: mill.define.Args) = Task.Command {
+    val pythonVenv = typeCheck().path
+
+    os.call(
+      Seq(
+        pythonVenv.toString,
+        s"${sources().path / mainFileName()}"
+      ) ++ args.value,
+      stdout = os.Inherit
+    )
+  } 
+
+}
+
+// ==== Example
+
+// The script defines three key `objects` that extend the `PythonModule` trait, each object encapsulating a distinct runtime configuration.
+// By allowing objects like `foo`, `bar`, and `qux` to extend the `PythonModule`, it promotes code reuse and modular design, 
+// enabling easy access to common functionalities while facilitating customization for specific needs.
+
+object foo extends PythonModule{
+  override def mainFileName: T[String] = Task { "foo.py" }
+  object bar extends PythonModule{
+    override def mainFileName: T[String] = Task { "bar.py" }
+    // Inherits all functionalities of PythonModule
+  }
+}
+
+object qux extends PythonModule{
+  override def mainFileName: T[String] = Task { "qux.py" }
+  // Functions independently but has access to PythonModule methods
+}
+
+// We have used three different Python Scripts `foo/src/foo.py`, `foo/bar/src/bar.py`, `qux/src/qux.py` 
+
+/** See Also: foo/src/foo.py */
+/** See Also: foo/bar/src/bar.py */
+/** See Also: qux/src/qux.py */
+
+
+// Run the following commands to run each module, displaying unique outputs based on the configuration of each object:
+
+/** Usage
+
+> ./mill foo.run Mill 
+Hello, Mill Foo!
+
+> ./mill foo.bar.run Mill 
+Hello, Mill Foo Bar!
+
+> ./mill qux.run Mill
+Hello, Mill Qux!
+
+*/
+
+// The Final working build, with a build graph that looks like this:
+
+// ```graphviz
+// digraph G {
+//   rankdir=LR
+//   node [shape=box width=0 height=0 style=filled fillcolor=white]
+//   subgraph cluster_3 {
+//     style=dashed
+//     label=qux
+//     "qux.createVenv" -> "qux.setup" -> "qux.typeCheck" -> "qux.run"
+//     "qux.sources" -> "qux.typeCheck"
+//     "qux.mainFileName" -> "qux.typeCheck"
+//     "qux.sources" -> "qux.run"
+//     "qux.mainFileName" -> "qux.run"
+//   }
+//   subgraph cluster_1 {
+//     subgraph cluster_2 {
+//       style=dashed
+//       label=bar
+//       "bar.createVenv" -> "bar.setup" -> "bar.typeCheck" -> "bar.run"
+//       "bar.sources" -> "bar.typeCheck"
+//       "bar.mainFileName" -> "bar.typeCheck"
+//       "bar.sources" -> "bar.run"
+//       "bar.mainFileName" -> "bar.run"
+//     }
+//     style=dashed
+//     label=foo
+//     "foo.createVenv" -> "foo.setup" -> "foo.typeCheck" -> "foo.run"
+//     "foo.sources" -> "foo.typeCheck"
+//     "foo.mainFileName" -> "foo.typeCheck"
+//     "foo.sources" -> "foo.run"
+//     "foo.mainFileName" -> "foo.run"
+//   }
+// }
+// ```
+
+// Next, we will look at how to Manage `Module Dependencies` for `Python` Using `Mill`.