move to extras

.github/workflows/test.yml

@@ -116,7 +116,7 @@ jobs:
           pip install jupyter
           sed -i.bak 's/OUTPUT_TIMEOUT = 10/OUTPUT_TIMEOUT = 100/g' \
             $(python -c 'import site; print(site.getsitepackages()[0])')/jupyter_client/runapp.py
-          BRANCH=${{ github.ref_name }} jupyter run examples/mlir_python_utils.ipynb
+          BRANCH=${{ github.ref_name }} jupyter run examples/mlir_python_extras.ipynb
 
   mlir-bindings-aarch64:
 

.github/workflows/test_pypi.yml

@@ -22,7 +22,7 @@ jobs:
 
       - uses: actions/upload-artifact@v3
         with:
-          path: ./wheelhouse/mlir_python_utils*.whl
+          path: ./wheelhouse/mlir_python_extras*.whl
 
   build_sdist:
     name: Build source distribution

README.md

@@ -1,4 +1,4 @@
-# mlir-python-utils
+# mlir-python-extras
 
 The missing pieces (as far as boilerplate reduction goes) of the MLIR python bindings.
 
@@ -77,7 +77,7 @@ The few main features/affordances:
 1. `region_op`s (like `@func` above)
    \
     
-   1. These are decorators around ops (bindings for MLIR operations) that have regions (e.g., [in_parallel](https://github.com/makslevental/mlir-python-utils/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_utils/dialects/ext/scf.py#L185)). 
+   1. These are decorators around ops (bindings for MLIR operations) that have regions (e.g., [in_parallel](https://github.com/makslevental/mlir-python-extras/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_extras/dialects/ext/scf.py#L185)). 
    They turn decorated functions, by executing them "eagerly", into an instance of such an op, e.g., 
       ```python
       @func
@@ -87,7 +87,7 @@ The few main features/affordances:
       becomes `func.func @foo(%arg0: i32) { }`; if the region carrying op produces a result, the identifier for the python function (`foo`) becomes the corresponding `ir.Value` of the result (if the op doesn't produce a result then the identifier becomes the corresponding `ir.OpView`).
       \
       \
-      See [mlir_utils.util.op_region_builder](https://github.com/makslevental/mlir-python-utils/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_utils/util.py#L123) for details.
+      See [mlir_extras.util.op_region_builder](https://github.com/makslevental/mlir-python-extras/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_extras/util.py#L123) for details.
       \
        
 2. `@canonicalize` (like `@canonicalize(using=scf)` above)
@@ -96,16 +96,16 @@ The few main features/affordances:
    1. These are decorators that **rewrite the python AST**. They transform a select few forms (basically only `if`s) into a more "canonical" form, in order to more easily map to MLIR. If that scares you, fear not; they are not essential and all target MLIR can still be mapped to without using them (by using the slightly more verbose `region_op`).
       \
       \
-      See [mlir_utils.ast.canonicalize](https://github.com/makslevental/mlir-python-utils/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_utils/ast/canonicalize.py) for details.
+      See [mlir_extras.ast.canonicalize](https://github.com/makslevental/mlir-python-extras/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_extras/ast/canonicalize.py) for details.
       \
        
-3. `mlir_utils.types` (like `T.memref(K, K, T.i64)` above)
+3. `mlir_extras.types` (like `T.memref(K, K, T.i64)` above)
    \
     
-   1. These are just convenient wrappers around upstream type constructors. Note, because MLIR types are uniqued to a `ir.Context`, these are all actually functions that return the type (yes, even `T.i64`, which uses [`__getattr__` on the module](https://github.com/makslevental/mlir-python-utils/blob/2ca62e9c1540b1624c302bc9efb4666ff5d1c133/mlir_utils/types.py#L98)).
+   1. These are just convenient wrappers around upstream type constructors. Note, because MLIR types are uniqued to a `ir.Context`, these are all actually functions that return the type (yes, even `T.i64`, which uses [`__getattr__` on the module](https://github.com/makslevental/mlir-python-extras/blob/2ca62e9c1540b1624c302bc9efb4666ff5d1c133/mlir_extras/types.py#L98)).
       \
       \
-      See [mlir_utils.types](https://github.com/makslevental/mlir-python-utils/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_utils/types.py) for details.
+      See [mlir_extras.types](https://github.com/makslevental/mlir-python-extras/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_extras/types.py) for details.
       \
        
 4. `Pipeline()`
@@ -114,8 +114,8 @@ The few main features/affordances:
    1. This is just a (generated) wrapper around available **upstream** passes; it can be used to build pass pipelines (by `str(Pipeline())`). It is mainly convenient with IDEs/editors that will tab-complete the available methods on the `Pipeline` class (which correspond to passes), Note, if your host bindings don't register some upstream passes, then this will generate "illegal" pass pipelines.
       \
       \
-      See [mlir_utils._configuration.generate_pass_pipeline.py](https://github.com/makslevental/mlir-python-utils/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_utils/_configuration/generate_pass_pipeline.py) for details on generation
-      [mlir_utils.runtime.passes.py](https://github.com/makslevental/mlir-python-utils/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_utils/runtime/passes.py#L80) for the passes themselves.
+      See [mlir_extras._configuration.generate_pass_pipeline.py](https://github.com/makslevental/mlir-python-extras/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_extras/_configuration/generate_pass_pipeline.py) for details on generation
+      [mlir_extras.runtime.passes.py](https://github.com/makslevental/mlir-python-extras/blob/a9885db18096a610d29a26293396d860d40ad213/mlir_extras/runtime/passes.py#L80) for the passes themselves.
       \
        
 
@@ -133,7 +133,7 @@ Practically speaking that means you need to have *some* package installed that i
 So
 
 ```shell
-$ YOUR_HOST_MLIR_PYTHON_PACKAGE_PREFIX=<YOUR_HOST_MLIR_PYTHON_PACKAGE_PREFIX> pip install git+https://github.com/makslevental/mlir-python-utils
+$ YOUR_HOST_MLIR_PYTHON_PACKAGE_PREFIX=<YOUR_HOST_MLIR_PYTHON_PACKAGE_PREFIX> pip install git+https://github.com/makslevental/mlir-python-extras
 ```
 
 where `YOUR_HOST_MLIR_PYTHON_PACKAGE_PREFIX` is (as it says) the package prefix for your chosen host bindings.
@@ -148,7 +148,7 @@ $ pip install mlir-python-bindings -f https://makslevental.github.io/wheels/
 and then
 
 ```shell
-$ pip install git+https://github.com/makslevental/mlir-python-utils
+$ pip install git+https://github.com/makslevental/mlir-python-extras
 ```
 
 ## Examples/Demo

examples/mlir_python_utils.ipynb → examples/mlir_python_extras.ipynb

@@ -6,9 +6,9 @@
     "id": "MVpw-wdNOFv0"
    },
    "source": [
-    "# Welcome to `mlir-python-utils` enjoy your stay!\n",
+    "# Welcome to `mlir-python-extras` enjoy your stay!\n",
     "\n",
-    "more at https://github.com/makslevental/mlir-python-utils"
+    "more at https://github.com/makslevental/mlir-python-extras"
    ]
   },
   {
@@ -19,7 +19,7 @@
    },
    "outputs": [],
    "source": [
-    "!pip install mlir-python-utils[mlir] -f https://github.com/makslevental/mlir-python-utils.git@$BRANCH &> /dev/null"
+    "!pip install mlir-python-extras[mlir] -f https://github.com/makslevental/mlir-python-extras.git@$BRANCH &> /dev/null"
    ]
   },
   {
@@ -30,7 +30,7 @@
    },
    "outputs": [],
    "source": [
-    "!mlir-python-utils-generate-all-upstream-trampolines &> /dev/null"
+    "!mlir-python-extras-generate-all-upstream-trampolines &> /dev/null"
    ]
   },
   {
@@ -52,19 +52,19 @@
    "source": [
     "import numpy as np\n",
     "\n",
-    "import mlir.utils.types as T\n",
-    "from mlir.utils.ast.canonicalize import canonicalize\n",
-    "from mlir.utils.context import MLIRContext, mlir_mod_ctx\n",
-    "from mlir.utils.dialects.ext.arith import constant\n",
-    "from mlir.utils.dialects.ext.memref import load, store, S\n",
-    "from mlir.utils.dialects.ext.func import func\n",
-    "from mlir.utils.dialects.ext.scf import canonicalizer as scf, range_ as range\n",
-    "from mlir.utils.runtime.passes import Pipeline, run_pipeline\n",
-    "from mlir.utils.runtime.refbackend import LLVMJITBackend\n",
+    "import mlir.extras.types as T\n",
+    "from mlir.extras.ast.canonicalize import canonicalize\n",
+    "from mlir.extras.context import MLIRContext, mlir_mod_ctx\n",
+    "from mlir.extras.dialects.ext.arith import constant\n",
+    "from mlir.extras.dialects.ext.memref import load, store, S\n",
+    "from mlir.extras.dialects.ext.func import func\n",
+    "from mlir.extras.dialects.ext.scf import canonicalizer as scf, range_ as range\n",
+    "from mlir.extras.runtime.passes import Pipeline, run_pipeline\n",
+    "from mlir.extras.runtime.refbackend import LLVMJITBackend\n",
     "\n",
     "# you need this to register the memref value caster\n",
     "# noinspection PyUnresolvedReferences\n",
-    "import mlir.utils.dialects.ext.memref\n",
+    "import mlir.extras.dialects.ext.memref\n",
     "\n",
     "ctx_man = mlir_mod_ctx()\n",
     "ctx = ctx_man.__enter__()\n",
@@ -663,7 +663,7 @@
     "ranked_memref_kxk_f32 = T.memref(K, K, T.f32)\n",
     "ranked_memref_dxd_f32 = T.memref(D, D, T.f32, layout=((K, 1), S))\n",
     "\n",
-    "from mlir.utils.dialects import linalg\n",
+    "from mlir.extras.dialects import linalg\n",
     "\n",
     "@func\n",
     "@canonicalize(using=scf)\n",

examples/mwe.py

@@ -2,15 +2,15 @@
 
 # you need this to register the memref value caster
 # noinspection PyUnresolvedReferences
-import mlir.utils.dialects.ext.memref
-import mlir.utils.types as T
-from mlir.utils.ast.canonicalize import canonicalize
-from mlir.utils.context import MLIRContext, mlir_mod_ctx
-from mlir.utils.dialects.ext.arith import constant
-from mlir.utils.dialects.ext.func import func
-from mlir.utils.dialects.ext.scf import canonicalizer as scf, range_ as range
-from mlir.utils.runtime.passes import Pipeline
-from mlir.utils.runtime.refbackend import LLVMJITBackend
+import mlir.extras.dialects.ext.memref
+import mlir.extras.types as T
+from mlir.extras.ast.canonicalize import canonicalize
+from mlir.extras.context import MLIRContext, mlir_mod_ctx
+from mlir.extras.dialects.ext.arith import constant
+from mlir.extras.dialects.ext.func import func
+from mlir.extras.dialects.ext.scf import canonicalizer as scf, range_ as range
+from mlir.extras.runtime.passes import Pipeline
+from mlir.extras.runtime.refbackend import LLVMJITBackend
 
 
 def setting_memref(ctx: MLIRContext, backend: LLVMJITBackend):

mlir/utils/_configuration/generate_trampolines.py → mlir/extras/_configuration/generate_trampolines.py

@@ -349,20 +349,20 @@ def generate_all_upstream_trampolines():
             generate_linalg(f"{mlir_dialects.__name__}.linalg")
         elif mod.name == "transform":
             transform_module_path = Path(mlir_dialects.__path__[0]) / mod.name
-            transform_module_path_mlir_utils = mlir_mod_path / mod.name
-            if not transform_module_path_mlir_utils.exists():
-                os.makedirs(transform_module_path_mlir_utils)
+            transform_module_path_mlir_extras = mlir_mod_path / mod.name
+            if not transform_module_path_mlir_extras.exists():
+                os.makedirs(transform_module_path_mlir_extras)
 
             generate_trampolines(
                 f"{mlir_dialects.__name__}.{mod.name}",
-                transform_module_path_mlir_utils,
+                transform_module_path_mlir_extras,
                 "__init__",
                 skips={"1kOp"},
             )
             for mod in pkgutil.iter_modules([str(transform_module_path)]):
                 generate_trampolines(
                     f"{mlir_dialects.__name__}.transform.{mod.name}",
-                    transform_module_path_mlir_utils,
+                    transform_module_path_mlir_extras,
                     mod.name,
                     skips={"1kOp"},
                 )

mlir/utils/util.py → mlir/extras/util.py

@@ -30,19 +30,19 @@
 
 
 def get_user_code_loc(user_base: Optional[Path] = None):
-    from .. import utils
+    from .. import extras
 
     if Context.current is None:
         return
 
-    mlir_utils_root_path = Path(utils.__path__[0])
+    mlir_extras_root_path = Path(extras.__path__[0])
 
     prev_frame = inspect.currentframe().f_back
     if user_base is None:
         user_base = Path(prev_frame.f_code.co_filename)
 
     while prev_frame.f_back and (
-        Path(prev_frame.f_code.co_filename).is_relative_to(mlir_utils_root_path)
+        Path(prev_frame.f_code.co_filename).is_relative_to(mlir_extras_root_path)
         or Path(prev_frame.f_code.co_filename).is_relative_to(sys.prefix)
         or Path(prev_frame.f_code.co_filename).is_relative_to(user_base)
     ):

setup.py

@@ -7,7 +7,7 @@
 HOST_MLIR_PYTHON_PACKAGE_PREFIX = os.environ.get(
     "HOST_MLIR_PYTHON_PACKAGE_PREFIX", "mlir"
 )
-PACKAGE_NAME = f"{HOST_MLIR_PYTHON_PACKAGE_PREFIX.replace('.', '-').replace('_', '-')}-python-utils"
+PACKAGE_NAME = f"{HOST_MLIR_PYTHON_PACKAGE_PREFIX.replace('.', '-').replace('_', '-')}-python-extras"
 
 
 def load_requirements(fname):
@@ -30,6 +30,6 @@ def load_requirements(fname):
     python_requires=">=3.10",
     # lhs is package namespace, rhs is path (relative to this setup.py)
     package_dir={
-        f"{HOST_MLIR_PYTHON_PACKAGE_PREFIX}.utils": "mlir/utils",
+        f"{HOST_MLIR_PYTHON_PACKAGE_PREFIX}.extras": "mlir/extras",
     },
 )

tests/test_async.py

@@ -6,11 +6,11 @@
 
 import pytest
 
-from mlir.utils.runtime.passes import Pipeline
-from mlir.utils.runtime.refbackend import LLVMJITBackend
+from mlir.extras.runtime.passes import Pipeline
+from mlir.extras.runtime.refbackend import LLVMJITBackend
 
 # noinspection PyUnresolvedReferences
-from mlir.utils.testing import mlir_ctx as ctx, filecheck, MLIRContext, backend
+from mlir.extras.testing import mlir_ctx as ctx, filecheck, MLIRContext, backend
 
 # needed since the fix isn't defined here nor conftest.py
 pytest.mark.usefixtures("ctx")
@@ -21,7 +21,7 @@
     platform.machine() == "aarch64", reason="https://github.com/numba/numba/issues/9109"
 )
 @pytest.mark.skipif(
-    platform.system() == "Windows", reason="windows can't load runner utils"
+    platform.system() == "Windows", reason="windows can't load runner extras"
 )
 def test_simple_parfor(ctx: MLIRContext, backend: LLVMJITBackend):
     module = ctx.module.parse(

tests/test_func.py

@@ -4,13 +4,13 @@
 
 import pytest
 
-import mlir.utils.types as T
-from mlir.utils.context import mlir_mod_ctx
-from mlir.utils.dialects.ext.arith import constant
-from mlir.utils.dialects.ext.func import func
+import mlir.extras.types as T
+from mlir.extras.context import mlir_mod_ctx
+from mlir.extras.dialects.ext.arith import constant
+from mlir.extras.dialects.ext.func import func
 
 # noinspection PyUnresolvedReferences
-from mlir.utils.testing import mlir_ctx as ctx, filecheck, MLIRContext
+from mlir.extras.testing import mlir_ctx as ctx, filecheck, MLIRContext
 
 # needed since the fix isn't defined here nor conftest.py
 pytest.mark.usefixtures("ctx")