Skip to content

Commit

Permalink
Reviews Changes Updated
Browse files Browse the repository at this point in the history
  • Loading branch information
himanshumahajan138 committed Nov 7, 2024
1 parent 04b4db7 commit 8346ba2
Show file tree
Hide file tree
Showing 13 changed files with 113 additions and 284 deletions.
22 changes: 9 additions & 13 deletions docs/modules/ROOT/pages/extending/new-language.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,12 @@ language toolchains have similar concepts of `compile`, `run`, `bundle`, etc.

== Python Support

This section walks through the process of adding support for Python
language to Mill. We will be adding a small `trait PythonModule` with the
ability to resolve dependencies, typecheck local code, and optimize a final
bundle.

The Python integration here is not intended for production usage, but is
instead meant for illustration purposes of the techniques typically used in
implementing language toolchains.
This section demonstrates how to integrate `Python` support into `Mill`.
We will define a simple `PythonModule` trait that can resolve dependencies,
perform type checking on local code, and optimize the final bundle.

`Note`: This integration is for `educational purposes only`, showcasing common technique
used in building language toolchains, and is not intended for production use.

include::partial$example/extending/newlang/5-hello-python.adoc[]

Expand All @@ -60,8 +58,6 @@ include::partial$example/extending/newlang/8-python-libs-bundle.adoc[]



As mentioned earlier, the `PythonModule` examples on this page are meant for
demo purposes: to show what it looks like to add support in Mill for a new
programming language toolchain. It would take significantly more work to flesh out
the featureset and performance of `PythonModule` to be usable in a real world
build.
As mentioned, The `PythonModule` examples here demonstrate
how to add support for a new language toolchain in Mill.
A production-ready version would require more work to enhance features and performance.
74 changes: 35 additions & 39 deletions example/extending/newlang/5-hello-python/build.mill
Original file line number Diff line number Diff line change
@@ -1,54 +1,42 @@
// == Python Integration in Mill

// This example demonstrates the integration of https://www.python.org[Python]
// compilation into a Mill build to compile Python scripts. Mill does not come
// bundled with Python integration, so here we will set one up from scratch
// using the basic `python3` command line, which is already present in the OS.
// This Example shows how to integrate https://www.python.org[Python] into a Mill build, enabling Python script compilation.
// Mill does not come bundled with Python support, so we will set up integration using `python3`, which is generally pre-installed.

// === Python installation Setup
// === Python initialization Setup
//
// First, we need to use the `Python3` CLI tool to create virtual environment and
// install `mypy` for Type Checking using the official Python Virtual Environment
// https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments[Documentation].
// First, The `setup` task creates a Python virtual environment and installs `mypy` for type-checking.

// `MyPy` is a static type checker that uses Python `type hints` to enforce `type correctness` and `catch errors` during development.
// `mypy` verifies type correctness using Python's `type hints`, helping to catch errors in development.

// Guides:
// https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments[Python virtual environment]
// & https://mypy.readthedocs.io/en/stable[mypy]

package build
import mill._

def setup: T[PathRef] = Task {
// creating virtual environment
os.call(("python3","-m","venv",Task.dest / "venv"))

val pythonVenv = Task.dest / "venv" / "bin" / "python3"

// installing mypy for Type Checking
os.call(("python3","-m","venv",Task.dest / "venv"))
os.call((pythonVenv,"-m","pip","install","mypy"))

PathRef(pythonVenv)
}

// The `setup` task ensures the Python virtual environment creation process using the official
// https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments[Documentation].
// After this, we need to install python `mypy` library for Type Checking and Error Checking before actual run of the code,
// using the offical https://mypy.readthedocs.io/en/stable[Documentation] for `mypy`

// === Defining our Sources

// Next, We define the `sources` task to help gather source files for processing.
// `sources` sets a base directory for Python source files and enable easy access to Python files in a project,
// making it straightforward to retrieve all relevant code files for tasks like building, testing, or analysis.
// The `sources` task specifies the directory for Python source files (`src` folder).
// This setup facilitates organizing and accessing Python files needed for building, testing, or analysis.

def sources: T[PathRef] = Task.Source(millSourcePath / "src")

// === Type Checking

// The `mainFileName` Task provides the name for the main python script file for running and
// The `typeCheck` Task ensures that the python setup is set and Checks the main file for error free code.
// This task is considered as a part of code which is responsible for Error Checking of the Main Script
// and setting up setup for python script runable environment.
// The `typeCheck` task verifies that the code in the main Python file passes type checks.
// It checks for errors before execution to ensure a reliable setup for running Python scripts.

def mainFileName: T[String] = Task { "main.py" }
def typeCheck: T[PathRef] = Task {
val pythonVenv = setup().path

Expand All @@ -60,19 +48,29 @@ def typeCheck: T[PathRef] = Task {
PathRef(pythonVenv)
}

// When `typeCheck` task sets up the Setup for Virtual Environment and checks the main file for error free code
// then we have `main.py` and running this file will result in either a generic greeting
// or a personalized one based on the command-line arguments passed, allowing for flexible interaction with the user.
// At this point, we have a minimal working build, with a build graph that looks like this:
//
// ```graphviz
// digraph G {
// rankdir=LR
// node [shape=box width=0 height=0 style=filled fillcolor=white]
// setup -> typeCheck
// sources -> typeCheck
// mainFileName -> typeCheck
// }
// ```

// Here is the `main.py` file
/** See Also: src/main.py */


// === Running

// The `run` function executes the main Python file (`main.py`) and accepts command-line arguments from the user.
// It uses `os.call` to run the Python interpreter(from Virtual Environment) with the main file's path
// and any additional arguments. The output is shown in the console, allowing for real-time interaction.

// The `mainFileName` task defines the name of the main Python script (`main.py`).

// The `run` function runs the main file with user-provided command-line arguments.
// It uses the virtual environment's Python interpreter to execute the script, with output displayed in the console.

def mainFileName: T[String] = Task { "main.py" }
def run(args: mill.define.Args) = Task.Command {
val pythonVenv = typeCheck().path

Expand Down Expand Up @@ -108,11 +106,9 @@ Success: no issues found in 1 source file

> ./mill run Mill Python
Hello, Mill Python!
15

*/

// Uncomment the Result variable line(Line Number:13) in the `src/main.py` code to see `typeCheck` error

// So that's a minimal example of implementing a Basic `Python
// Integration` in Mill Build Tool. Next, we will look at turning it into a `PythonModule` that
// can be re-used
// This completes a basic `Python integration in Mill`.
// Next steps could involve transforming this into a `reusable PythonModule`
24 changes: 3 additions & 21 deletions example/extending/newlang/5-hello-python/src/main.py
Original file line number Diff line number Diff line change
@@ -1,24 +1,6 @@
import sys

def add(a: int, b: int) -> int:
"""Adds two integers."""
return a + b

def main() -> None:
# Get command-line arguments, skipping the first one (script name)
args = sys.argv[1:]

# Example of using the add function with incorrect types
# Uncomment this line to see the error
# result = add("5", 10) # This will cause a TypeError, handled in the function

# Check if any arguments were provided
if not args:
print("Hello, World!")
else:
# Join the arguments with spaces and print the greeting
greeting = "Hello, " + " ".join(args) + "!"
print(greeting)

def add(a: int, b: int) -> int: return a + b
def main() -> None: print("Hello, " + " ".join(sys.argv[1:]) + "!")
if __name__ == "__main__":
main()
print(add(5, 10)) # Error Example: add("5", 10) will cause a TypeError
38 changes: 12 additions & 26 deletions example/extending/newlang/6-python-modules/build.mill
Original file line number Diff line number Diff line change
@@ -1,40 +1,29 @@
// == Re-usable PythonModule
// This Example demonstrates the use of a `PythonModule` trait to manage Python Scripts within
// different objects in a Mill build.
// This example illustrates the `PythonModule` trait for managing Python scripts within multiple Mill objects.

// ==== Using Same PythonModule for Mutiple Objects
// ==== Using Same PythonModule for Multiple Objects

// The `PythonModule` trait simplifies Python project management within the `Mill` build tool by providing essential
// functionalities such as creating `virtual environment`, managing `source files`, `Checking code for Error`, and `executing Python` scripts.
// It ensures that users have a streamlined workflow by automating the setup of necessary tools
// and organizing project files effectively.
// `PythonModule` automates essential tasks in Python project management within Mill,
// such as setting up virtual environments, managing source files, performing type checks, and running scripts.

package build
import mill._

/** PythonModule Trait for Basic Python Pipeline in Mill Build Tool */
/** `PythonModule`: Trait for basic Python support in Mill */
trait PythonModule extends Module {

/** Returns the path to the source directory containing Python files. */
def sources: T[PathRef] = Task.Source(millSourcePath / "src")

/** Returns the name of the main Python file. */
def mainFileName: T[String] = Task { "main.py" }

/** This minimal setup provide basic Virtual Environment and Type Checking. */
def setup: T[PathRef] = Task {
// creating virtual environment
os.call(("python3","-m","venv",Task.dest / "venv"))

val pythonVenv = Task.dest / "venv" / "bin" / "python3"

// installing mypy for Type Checking
os.call(("python3","-m","venv",Task.dest / "venv"))
os.call((pythonVenv,"-m","pip","install","mypy"))

PathRef(pythonVenv)
}

/** Creates Virtual env and check main file. */
def typeCheck: T[PathRef] = Task {
val pythonVenv = setup().path

Expand All @@ -46,7 +35,6 @@ trait PythonModule extends Module {
PathRef(pythonVenv)
}

/** Runs the checked main Python file with given command-line arguments. */
def run(args: mill.define.Args) = Task.Command {
val pythonVenv = typeCheck().path

Expand All @@ -60,21 +48,19 @@ trait PythonModule extends Module {

// ==== Example

// The script defines three key `objects` that extend the `PythonModule` trait, each object encapsulating a distinct runtime configuration.
// By allowing objects like `foo`, `bar`, and `qux` to extend the `PythonModule`, it promotes code reuse and modular design,
// enabling easy access to common functionalities while facilitating customization for specific needs.
// Below are three objects extending `PythonModule`, each representing a unique configuration.

object foo extends PythonModule{
override def mainFileName: T[String] = Task { "foo.py" }
object bar extends PythonModule{
override def mainFileName: T[String] = Task { "bar.py" }
// Inherits all functionalities of PythonModule
// Inherits PythonModule features
}
}

object qux extends PythonModule{
override def mainFileName: T[String] = Task { "qux.py" }
// Functions independently but has access to PythonModule methods
// Independent, but uses PythonModule methods
}

// We have used three different Python Scripts `foo/src/foo.py`, `foo/bar/src/bar.py`, `qux/src/qux.py`
Expand All @@ -84,7 +70,7 @@ object qux extends PythonModule{
/** See Also: qux/src/qux.py */


// Run the following commands to run each module, displaying unique outputs based on the configuration of each object:
// Use the following commands to run each module, displaying unique outputs per configuration:

/** Usage

Expand Down Expand Up @@ -135,4 +121,4 @@ Hello, Mill Qux!
// }
// ```

// Next, we will look at how to Manage `Module Dependencies` for `Python` Using `Mill`.
// Next, we will look at how to Manage `Module Dependencies` for `Python` in `Mill`.
16 changes: 2 additions & 14 deletions example/extending/newlang/6-python-modules/foo/bar/src/bar.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,4 @@
import sys

def main() -> None:
# Get command-line arguments, skipping the first one (script name)
args = sys.argv[1:]

# Check if any arguments were provided
if not args:
print("Hello, World Foo Bar!")
else:
# Join the arguments with spaces and print the greeting
greeting = "Hello, " + " ".join(args) + " Foo Bar!"
print(greeting)

def main() -> None: print("Hello, " + " ".join(sys.argv[1:]) + " Foo Bar!")
if __name__ == "__main__":
main()
main()
16 changes: 2 additions & 14 deletions example/extending/newlang/6-python-modules/foo/src/foo.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,4 @@
import sys

def main() -> None:
# Get command-line arguments, skipping the first one (script name)
args = sys.argv[1:]

# Check if any arguments were provided
if not args:
print("Hello, World Foo!")
else:
# Join the arguments with spaces and print the greeting
greeting = "Hello, " + " ".join(args) + " Foo!"
print(greeting)

def main() -> None: print("Hello, " + " ".join(sys.argv[1:]) + " Foo!")
if __name__ == "__main__":
main()
main()
16 changes: 2 additions & 14 deletions example/extending/newlang/6-python-modules/qux/src/qux.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,4 @@
import sys

def main() -> None:
# Get command-line arguments, skipping the first one (script name)
args = sys.argv[1:]

# Check if any arguments were provided
if not args:
print("Hello, World Qux!")
else:
# Join the arguments with spaces and print the greeting
greeting = "Hello, " + " ".join(args) + " Qux!"
print(greeting)

def main() -> None: print("Hello, " + " ".join(sys.argv[1:]) + " Qux!")
if __name__ == "__main__":
main()
main()
Loading

0 comments on commit 8346ba2

Please sign in to comment.