Add DocxodusEngine as a second comparison engine

JSv4 · JSv4 · commit 48269d94b325 · 2026-02-10T00:52:11.000-05:00
Introduce Docxodus (a modernized .NET 8.0 fork of Open-XML-PowerTools with
better move detection) as an alternative engine alongside XmlPowerToolsEngine.

- Extract BaseEngine class with shared binary extraction and subprocess logic
- XmlPowerToolsEngine and DocxodusEngine are thin subclasses setting 3 constants
- Add Docxodus as a git submodule at docxodus/
- Refactor build_differ.py into reusable build_engine() function (also fixes
  missing win-arm64 compression)
- Update CI workflow for submodules and .NET SDK
- Add integration tests and parametrized contract tests for both engines
diff --git a/.github/workflows/python-publish.yml b/.github/workflows/python-publish.yml
@@ -14,10 +14,16 @@ jobs:
 
     steps:
     - uses: actions/checkout@v3
+      with:
+        submodules: recursive
     - name: Set up Python
       uses: actions/setup-python@v3
       with:
         python-version: '3.x'
+    - name: Setup .NET
+      uses: actions/setup-dotnet@v3
+      with:
+        dotnet-version: '8.0.x'
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
@@ -26,4 +32,4 @@ jobs:
       run: hatch build
     - name: Publish package
       run: |
-        hatch publish -u "__token__" -a ${{ secrets.PYPI_API_TOKEN }}
+        hatch publish -u "__token__" -a ${{ secrets.PYPI_API_TOKEN }}
diff --git a/.gitignore b/.gitignore
@@ -6,6 +6,8 @@ __pycache__/
 # C# Build Dirs
 csproj/bin/*
 csproj/obj/*
+docxodus/**/bin/*
+docxodus/**/obj/*
 
 # C extensions
 *.so
diff --git a/.gitmodules b/.gitmodules
@@ -0,0 +1,3 @@
+[submodule "docxodus"]
+	path = docxodus
+	url = https://github.com/JSv4/Docxodus.git
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -0,0 +1,73 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Python-Redlines is a Python wrapper around compiled C# binaries that generate `.docx` redline/tracked-changes documents by comparing two Word files. The Python layer handles platform detection, binary extraction, temp file management, and subprocess execution.
+
+Two comparison engines are available:
+- **XmlPowerToolsEngine** — wraps Open-XML-PowerTools WmlComparer (original engine)
+- **DocxodusEngine** — wraps Docxodus, a modernized .NET 8.0 fork with better move detection
+
+## Commands
+
+```bash
+# Run tests
+hatch run test
+
+# Run a single test
+hatch run test tests/test_openxml_differ.py::test_run_redlines_with_real_files
+
+# Run tests with coverage
+hatch run cov
+
+# Type checking
+hatch run types:check
+
+# Build C# binaries for all platforms (requires .NET 8.0 SDK)
+hatch run build
+
+# Build Python package (triggers C# build via custom hook)
+hatch build
+
+# Initialize Docxodus submodule (required before building)
+git submodule update --init --recursive
+```
+
+## Architecture
+
+The system uses a two-layer wrapper pattern with a shared base class:
+
+1. **Python layer** (`src/python_redlines/engines.py`):
+   - `BaseEngine` — shared logic for binary extraction, subprocess invocation, and temp file management
+   - `XmlPowerToolsEngine(BaseEngine)` — sets constants for the Open-XML-PowerTools binary (`dist/`, `bin/`, `redlines`)
+   - `DocxodusEngine(BaseEngine)` — sets constants for the Docxodus binary (`dist_docxodus/`, `bin_docxodus/`, `redline`)
+
+   Both engines share the same CLI argument format: `<author_tag> <original.docx> <modified.docx> <output.docx>`
+
+2. **C# binaries**:
+   - `csproj/Program.cs` — Open-XML-PowerTools CLI tool
+   - `docxodus/tools/redline/Program.cs` — Docxodus CLI tool (git submodule)
+
+Pre-compiled binaries for 6 platform targets (linux/win/osx x x64/arm64) are stored as archives in `src/python_redlines/dist/` and `src/python_redlines/dist_docxodus/`, included in the wheel. The build script `build_differ.py` compiles both engines using `dotnet publish`.
+
+## Key Files
+
+- `src/python_redlines/engines.py` — BaseEngine, XmlPowerToolsEngine, and DocxodusEngine classes
+- `src/python_redlines/__init__.py` — Exports all engine classes
+- `src/python_redlines/__about__.py` — Single source of truth for package version
+- `csproj/Program.cs` — Open-XML-PowerTools C# comparison utility
+- `docxodus/` — Docxodus git submodule (tools/redline/ contains the CLI)
+- `build_differ.py` — Cross-platform C# build orchestration for both engines
+- `hatch_run_build_hook.py` — Hatch build hook that triggers C# compilation
+- `tests/fixtures/` — Test `.docx` files (original, modified, expected_redline)
+
+## Testing Notes
+
+Tests must be run from the project root (fixtures use relative paths like `tests/fixtures/original.docx`). The XmlPowerToolsEngine integration test validates that comparing the fixture documents produces exactly 9 revisions. Docxodus uses a different stdout format (`"revision(s) found"` vs `"Revisions found: 9"`).
+
+## Stdout Format Differences
+
+- **XmlPowerToolsEngine**: `"Revisions found: 9"`
+- **DocxodusEngine**: `"Redline complete: 9 revision(s) found"`
diff --git a/build_differ.py b/build_differ.py
@@ -4,6 +4,16 @@
 import zipfile
 
 
+RIDS = [
+    ("linux-x64", ".tar.gz"),
+    ("linux-arm64", ".tar.gz"),
+    ("win-x64", ".zip"),
+    ("win-arm64", ".zip"),
+    ("osx-x64", ".tar.gz"),
+    ("osx-arm64", ".tar.gz"),
+]
+
+
 def get_version():
     """
     Extracts the version from the specified __about__.py file.
@@ -50,57 +60,47 @@ def cleanup_old_builds(dist_dir, current_version):
             print(f"Deleted old build file: {file}")
 
 
-def main():
-    version = get_version()
-    print(f"Version: {version}")
-
-    dist_dir = "./src/python_redlines/dist/"
-
-    # Build for Linux x64
-    print("Building for Linux x64...")
-    run_command('dotnet publish ./csproj -c Release -r linux-x64 --self-contained')
-
-    # Build for Linux ARM64
-    print("Building for Linux ARM64...")
-    run_command('dotnet publish ./csproj -c Release -r linux-arm64 --self-contained')
-
-    # Build for Windows x64
-    print("Building for Windows x64...")
-    run_command('dotnet publish ./csproj -c Release -r win-x64 --self-contained')
-
-    # Build for Windows ARM64
-    print("Building for Windows ARM64...")
-    run_command('dotnet publish ./csproj -c Release -r win-arm64 --self-contained')
-
-    # Build for macOS x64
-    print("Building for macOS x64...")
-    run_command('dotnet publish ./csproj -c Release -r osx-x64 --self-contained')
-
-    # Build for macOS ARM64
-    print("Building for macOS ARM64...")
-    run_command('dotnet publish ./csproj -c Release -r osx-arm64 --self-contained')
+def build_engine(csproj_path, dist_dir, version):
+    """
+    Builds a C# engine for all platform targets, compresses the output, and cleans up old builds.
 
-    # Compress the Linux x64 build
-    linux_x64_build_dir = './csproj/bin/Release/net8.0/linux-x64'
-    compress_files(linux_x64_build_dir, f"{dist_dir}/linux-x64-{version}.tar.gz")
+    :param csproj_path: Path to the .csproj directory (e.g. './csproj' or './docxodus/tools/redline')
+    :param dist_dir: Path to the distribution directory for compressed binaries
+    :param version: Version string for archive naming
+    """
+    # Build for each RID
+    for rid, _ in RIDS:
+        print(f"Building {csproj_path} for {rid}...")
+        run_command(f'dotnet publish {csproj_path} -c Release -r {rid} --self-contained')
+
+    # Determine the build output base directory
+    # dotnet publish outputs to <csproj_path>/bin/Release/net8.0/<rid>
+    build_base = os.path.join(csproj_path, 'bin', 'Release', 'net8.0')
+
+    # Compress each build
+    for rid, ext in RIDS:
+        build_dir = os.path.join(build_base, rid)
+        archive_path = os.path.join(dist_dir, f"{rid}-{version}{ext}")
+        print(f"Compressing {rid} to {archive_path}...")
+        compress_files(build_dir, archive_path)
 
-    # Compress the Linux ARM64 build
-    linux_arm64_build_dir = './csproj/bin/Release/net8.0/linux-arm64'
-    compress_files(linux_arm64_build_dir, f"{dist_dir}/linux-arm64-{version}.tar.gz")
+    cleanup_old_builds(dist_dir, version)
 
-    # Compress the Windows x64 build
-    windows_build_dir = './csproj/bin/Release/net8.0/win-x64'
-    compress_files(windows_build_dir, f"{dist_dir}/win-x64-{version}.zip")
 
-    # Compress the macOS x64 build
-    macos_x64_build_dir = './csproj/bin/Release/net8.0/osx-x64'
-    compress_files(macos_x64_build_dir, f"{dist_dir}/osx-x64-{version}.tar.gz")
+def main():
+    version = get_version()
+    print(f"Version: {version}")
 
-    # Compress the macOS ARM64 build
-    macos_arm64_build_dir = './csproj/bin/Release/net8.0/osx-arm64'
-    compress_files(macos_arm64_build_dir, f"{dist_dir}/osx-arm64-{version}.tar.gz")
+    # Build the XmlPowerTools engine (original)
+    build_engine('./csproj', './src/python_redlines/dist/', version)
 
-    cleanup_old_builds(dist_dir, version)
+    # Build the Docxodus engine (if submodule is available)
+    docxodus_csproj = './docxodus/tools/redline'
+    if os.path.exists(os.path.join(docxodus_csproj, 'redline.csproj')):
+        build_engine(docxodus_csproj, './src/python_redlines/dist_docxodus/', version)
+    else:
+        print("WARNING: Docxodus submodule not found at docxodus/tools/redline/redline.csproj — skipping Docxodus build.")
+        print("Run 'git submodule update --init --recursive' to initialize the submodule.")
 
     print("Build and compression complete.")
 
diff --git a/docxodus b/docxodus
@@ -0,0 +1 @@
+Subproject commit 0a0b8c6586566088d80c12de79aded04b349c29b
diff --git a/pyproject.toml b/pyproject.toml
@@ -13,7 +13,9 @@ artifacts = [
 [tool.hatch.build.targets.sdist]
 include = [
   "python_redlines/dist",
+  "python_redlines/dist_docxodus",
   "python_redlines/bin",
+  "python_redlines/bin_docxodus",
 ]
 
 # Build hook to build the binaries for distribution...
diff --git a/src/python_redlines/__init__.py b/src/python_redlines/__init__.py
@@ -1,3 +1,7 @@
 # SPDX-FileCopyrightText: 2024-present U.N. Owen <void@some.where>
 #
 # SPDX-License-Identifier: MIT
+
+from .engines import XmlPowerToolsEngine, DocxodusEngine, BaseEngine
+
+__all__ = ["XmlPowerToolsEngine", "DocxodusEngine", "BaseEngine"]
diff --git a/src/python_redlines/bin_docxodus/.gitignore b/src/python_redlines/bin_docxodus/.gitignore
@@ -0,0 +1,2 @@
+*
+!.gitignore
diff --git a/src/python_redlines/dist_docxodus/.gitignore b/src/python_redlines/dist_docxodus/.gitignore
@@ -0,0 +1,2 @@
+*
+!.gitignore
diff --git a/src/python_redlines/engines.py b/src/python_redlines/engines.py
@@ -13,35 +13,45 @@
 logger = logging.getLogger(__name__)
 
 
-class XmlPowerToolsEngine(object):
+class BaseEngine(object):
+    """
+    Base class for redline comparison engines. Subclasses must define:
+      - DIST_DIR_NAME: directory name under src/python_redlines/ holding compressed binaries
+      - BIN_DIR_NAME: directory name under src/python_redlines/ for extracted binaries
+      - BINARY_BASE_NAME: the name of the executable (without .exe extension)
+    """
+    DIST_DIR_NAME: str = NotImplemented
+    BIN_DIR_NAME: str = NotImplemented
+    BINARY_BASE_NAME: str = NotImplemented
+
     def __init__(self, target_path: Optional[str] = None):
         self.target_path = target_path
-        self.extracted_binaries_path = self.__unzip_binary()
+        self.extracted_binaries_path = self._unzip_binary()
 
-    def __unzip_binary(self):
+    def _unzip_binary(self):
         """
         Unzips the appropriate C# binary for the current platform.
         """
         base_path = os.path.dirname(__file__)
-        binaries_path = os.path.join(base_path, 'dist')
-        target_path = self.target_path if self.target_path else os.path.join(base_path, 'bin')
+        binaries_path = os.path.join(base_path, self.DIST_DIR_NAME)
+        target_path = self.target_path if self.target_path else os.path.join(base_path, self.BIN_DIR_NAME)
 
         if not os.path.exists(target_path):
             os.makedirs(target_path)
 
         # Get the binary name and zip name based on the OS and architecture
-        binary_name, zip_name = self.__get_binaries_info()
+        binary_name, zip_name = self._get_binaries_info()
 
         # Check if the binary already exists. If not, extract it.
         full_binary_path = os.path.join(target_path, binary_name)
 
         if not os.path.exists(full_binary_path):
             zip_path = os.path.join(binaries_path, zip_name)
-            self.__extract_binary(zip_path, target_path)
+            self._extract_binary(zip_path, target_path)
 
         return os.path.join(target_path, binary_name)
 
-    def __extract_binary(self, zip_path: str, target_path: str):
+    def _extract_binary(self, zip_path: str, target_path: str):
         """
         Extracts the binary from the zip file based on the extension. Supports .zip and .tar.gz files
         :parameter
@@ -56,7 +66,7 @@ def __extract_binary(self, zip_path: str, target_path: str):
             with tarfile.open(zip_path, 'r:gz') as tar_ref:
                 tar_ref.extractall(target_path)
 
-    def __get_binaries_info(self):
+    def _get_binaries_info(self):
         """
         Returns the binary name and zip name based on the OS and architecture
         :return
@@ -75,15 +85,15 @@ def __get_binaries_info(self):
 
         if os_name == 'linux':
             zip_name = f"linux-{arch}-{__version__}.tar.gz"
-            binary_name = f'linux-{arch}/redlines'
+            binary_name = f'linux-{arch}/{self.BINARY_BASE_NAME}'
 
         elif os_name == 'windows':
             zip_name = f"win-{arch}-{__version__}.zip"
-            binary_name = f'win-{arch}/redlines.exe'
+            binary_name = f'win-{arch}/{self.BINARY_BASE_NAME}.exe'
 
         elif os_name == 'darwin':
             zip_name = f"osx-{arch}-{__version__}.tar.gz"
-            binary_name = f'osx-{arch}/redlines'
+            binary_name = f'osx-{arch}/{self.BINARY_BASE_NAME}'
 
         else:
             raise EnvironmentError("Unsupported OS")
@@ -93,7 +103,7 @@ def __get_binaries_info(self):
     def run_redline(self, author_tag: str, original: Union[bytes, Path], modified: Union[bytes, Path]) \
             -> Tuple[bytes, Optional[str], Optional[str]]:
         """
-        Runs the redlines binary. The 'original' and 'modified' arguments can be either bytes or file paths.
+        Runs the redline binary. The 'original' and 'modified' arguments can be either bytes or file paths.
         Returns the redline output as bytes.
         """
         temp_files = []
@@ -134,3 +144,15 @@ def _write_to_temp_file(self, data):
         temp_file.write(data)
         temp_file.close()
         return temp_file.name
+
+
+class XmlPowerToolsEngine(BaseEngine):
+    DIST_DIR_NAME = 'dist'
+    BIN_DIR_NAME = 'bin'
+    BINARY_BASE_NAME = 'redlines'
+
+
+class DocxodusEngine(BaseEngine):
+    DIST_DIR_NAME = 'dist_docxodus'
+    BIN_DIR_NAME = 'bin_docxodus'
+    BINARY_BASE_NAME = 'redline'
diff --git a/tests/test_docxodus_engine.py b/tests/test_docxodus_engine.py
@@ -0,0 +1,32 @@
+import pytest
+
+from python_redlines.engines import DocxodusEngine
+
+
+def load_docx_bytes(file_path):
+    with open(file_path, 'rb') as file:
+        return file.read()
+
+
+@pytest.fixture
+def original_docx():
+    return load_docx_bytes('tests/fixtures/original.docx')
+
+
+@pytest.fixture
+def modified_docx():
+    return load_docx_bytes('tests/fixtures/modified.docx')
+
+
+def test_run_docxodus_with_real_files(original_docx, modified_docx):
+    wrapper = DocxodusEngine()
+
+    author_tag = "TestAuthor"
+
+    redline_output, stdout, stderr = wrapper.run_redline(author_tag, original_docx, modified_docx)
+
+    assert redline_output is not None
+    assert isinstance(redline_output, bytes)
+    assert len(redline_output) > 0
+    assert stderr is None
+    assert "revision(s) found" in stdout
diff --git a/tests/test_engine_contract.py b/tests/test_engine_contract.py

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	`+[submodule "docxodus"]`
	`2`	`+ path = docxodus`
	`3`	`+ url = https://github.com/JSv4/Docxodus.git`
Original file line number	Diff line number	Diff line change
`@@ -13,7 +13,9 @@ artifacts = [`
`13`	`13`	`[tool.hatch.build.targets.sdist]`
`14`	`14`	`include = [`
`15`	`15`	`"python_redlines/dist",`
	`16`	`+ "python_redlines/dist_docxodus",`
`16`	`17`	`"python_redlines/bin",`
	`18`	`+ "python_redlines/bin_docxodus",`
`17`	`19`	`]`
`18`	`20`
`19`	`21`	`# Build hook to build the binaries for distribution...`
Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,7 @@`
`1`	`1`	`# SPDX-FileCopyrightText: 2024-present U.N. Owen <void@some.where>`
`2`	`2`	`#`
`3`	`3`	`# SPDX-License-Identifier: MIT`
	`4`	`+`
	`5`	`+from .engines import XmlPowerToolsEngine, DocxodusEngine, BaseEngine`
	`6`	`+`
	`7`	`+__all__ = ["XmlPowerToolsEngine", "DocxodusEngine", "BaseEngine"]`