Skip to content

Commit

Permalink
[mypyc] Using built-in mypyc and just showing fib example (#99)
Browse files Browse the repository at this point in the history
  • Loading branch information
sureshjoshi authored May 9, 2024
1 parent 6ac20ee commit 4da0d92
Show file tree
Hide file tree
Showing 12 changed files with 40 additions and 385 deletions.
15 changes: 15 additions & 0 deletions examples/python/hellofib/BUILD.pants
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
python_sources(name="libhellofib", sources=["**/*.py"])

python_distribution(
name="hellofib-dist",
dependencies=[":libhellofib"],
provides=python_artifact(name="hellofib-dist", version="0.0.1"),
generate_setup=False,
sdist=False,
uses_mypyc=True,
)

pex_binary(
name="hellofib-mypyc",
entry_point="hellofib.main",
dependencies=[":hellofib-dist"],
)

pex_binary(
name="hellofib-pex",
entry_point="hellofib.main",
Expand Down
12 changes: 12 additions & 0 deletions examples/python/hellofib/hellofib/lib.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import time


def fib() -> None:
def fib(n: int) -> int:
if n <= 1:
return n
return fib(n - 2) + fib(n - 1)

start_time = time.time()
fib(34)
print(f"Calculating fibs took {time.time() - start_time} seconds")
16 changes: 2 additions & 14 deletions examples/python/hellofib/hellofib/main.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,6 @@
import time


def main() -> None:
def fib(n: int) -> int:
if n <= 1:
return n
return fib(n - 2) + fib(n - 1)

start_time = time.time()
fib(34)
print(f"Calculating fibs took {time.time() - start_time} seconds")

from hellofib.lib import fib

# In .bzl config, setting python_config.run_module = "hellofib.main" should cause this to run as the entry point
if __name__ == "__main__":
print("Launching HelloFib from __main__")
main()
fib()
8 changes: 8 additions & 0 deletions examples/python/hellofib/setup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
from mypyc.build import mypycify # pants: no-infer-dep
from setuptools import setup

setup(
name="hellofib",
packages=["hellofib"],
ext_modules=mypycify(["hellofib/__init__.py", "hellofib/lib.py"]),
)
14 changes: 0 additions & 14 deletions examples/python/hellofib/setup.py.bak

This file was deleted.

1 change: 0 additions & 1 deletion pants-plugins/experimental/mypyc/BUILD

This file was deleted.

73 changes: 2 additions & 71 deletions pants-plugins/experimental/mypyc/README.md
Original file line number Diff line number Diff line change
@@ -1,74 +1,5 @@
# pants-mypyc-plugin

## How to use
# Similar solution mainlined in 2.13 via [PR #15380](https://github.com/pantsbuild/pants/pull/15380) by Benjy

1. This plugin requires changes to the Pants source code, so you'll need to use the [pants_from_sources](https://www.pantsbuild.org/docs/running-pants-from-sources#running-pants-from-sources-in-other-repos) approach against https://github.com/sureshjoshi/pants/tree/mypyc-support

2. Add `mypyc` support to the setuptools process:

```toml
# pants.toml
[setuptools]
extra_requirements = ["wheel", "mypy"]
lockfile = "build-support/setuptools.txt"
```

3. In your `BUILD` file, use the new `mypyc_python_distribution` target (which is identical to `python_distribution` with a new name)
```python
mypyc_python_distribution(
name="hellofib-dist",
dependencies=[":libhellofib"],
wheel=True,
sdist=False,
provides=setup_py(
name="hellofib-dist",
version="0.0.1",
description="A distribution for the hello fib library.",
),
# Setting this True or False depends on the next step
generate_setup = True,
)
```

4. If you want to test using your own `setup.py`, place one in your source root and set `generate_setup = False` in the `BUILD` file
```python
# setup.py
from setuptools import setup

from mypyc.build import mypycify

setup(
name="hellofib",
packages=["hellofib"],
ext_modules=mypycify(
[
"hellofib/__init__.py",
"hellofib/main.py",
]
),
)
```

5. If you want the plugin to auto-generate your `setup.py`, set `generate_setup = True` in the `BUILD` file (or remove the line, since `True` is the default). The plugin will pass all of the source files from the dependencies into `mypycify`

## Examples/Libraries to test

All of the examples that can compile with `mypyc_python_distribution` have that target applied. There are some outstanding examples which fail when `mypy` tries to compile them.

To quickly see which examples are supported, type the following: `./pants_from_sources filter --target-type=mypyc_python_distribution ::`

For example, with `hellofib`:
```bash
./pants_from_sources --version
./pants_from_sources package hellofib:hellofib-dist

pip install dist/hellofib-{whatever}-.whl --force-reinstall
python -c "from hellofib.main import main; main()"
```

## Next Steps

1. Add support for multiple dependency targets (only tested/working with one `python_sources` dependency)
2. Handle use case where `ext_modules` are already specified in the SetupKwargs
3. Figure out better API for SetupPyContentRequest - it feels a bit hacky to expect a certain key from another method, where there might be a better, more holistic solution
4. ~Test on imported libraries and add libraries to pants deps~
Refer to [examples/python/hellofib](https://github.com/sureshjoshi/pants-plugins/tree/main/examples/python/hellofib) for example usage (`pants run examples/python/hellofib:hellofib-mypyc`).
Empty file.
10 changes: 0 additions & 10 deletions pants-plugins/experimental/mypyc/register.py

This file was deleted.

Loading

0 comments on commit 4da0d92

Please sign in to comment.