fix(mode): ensure degenerate modes are orthogonal from mode solver

[dmarek-flex]

Remaining considerations:

• bi-orthogonality for fully anisotropic media?
• better way to identify degenerate modes
• should we make the user aware of detected degenerate modes?
• how does bi-orthogonality interact with bend or angled transforms

Greptile Overview

Greptile Summary

This PR fixes a mathematical issue with degenerate modes in the mode solver. When eigenmodes have identical or nearly identical eigenvalues (degenerate modes), the eigensolver doesn't guarantee orthogonality between them. This PR ensures degenerate modes respect the bi-orthogonality condition by:

• Adding TOL_DEGENERATE constant for identifying degenerate modes
• Implementing _identify_degenerate_modes() to group modes with similar effective indices
• Adding _dot() and _outer_dot() for computing bi-orthogonal overlap integrals
• Adding _normalize_modes() to normalize modes to unit self-overlap
• Implementing _make_orthogonal_basis_for_degenerate_modes() using a complex-symmetric eigendecomposition approach

The implementation correctly handles the complex-symmetric (not Hermitian) nature of the bi-orthogonal inner product, using unconjugated dot products for eigenvector normalization.

Confidence Score: 4/5

• This PR is safe to merge with minor considerations around edge case handling.
• The implementation is mathematically sound and follows the correct approach for bi-orthogonal mode orthogonalization. The test verifies the core functionality. Minor concern about potential numerical edge cases with near-zero eigenvalues in the orthogonalization transformation.
• tidy3d/components/mode/solver.py - the _make_orthogonal_basis_for_degenerate_modes method could benefit from a check for near-zero eigenvalues.

Important Files Changed

File Analysis

Filename	Score	Overview
CHANGELOG.md	5/5	Added changelog entry for degenerate mode handling fix under the "Fixed" section, following project conventions.
tests/test_plugins/test_mode_solver.py	5/5	Added test test_degenerate_mode_processing that verifies degenerate modes are bi-orthogonal by checking off-diagonal overlap values are below threshold.
tidy3d/components/mode/solver.py	4/5	Added TOL_DEGENERATE constant and implemented degenerate mode orthogonalization via _identify_degenerate_modes, _dot, _outer_dot, _normalize_modes, and _make_orthogonal_basis_for_degenerate_modes methods. The implementation handles complex symmetric matrices correctly.

Sequence Diagram

sequenceDiagram
    participant User
    participant ModeSolver
    participant EigSolver
    participant compute_modes

    User->>ModeSolver: data_raw
    ModeSolver->>compute_modes: compute_modes(...)
    compute_modes->>EigSolver: solver_em()
    EigSolver-->>compute_modes: E, H, neff, keff
    
    Note over compute_modes: Transform back to original axes
    
    compute_modes->>EigSolver: _normalize_modes(E, H, dl_f, dl_b, all_modes)
    Note over EigSolver: Normalize each mode to unit self-overlap
    EigSolver-->>compute_modes: E, H (normalized)
    
    compute_modes->>EigSolver: _identify_degenerate_modes(n_complex, TOL_DEGENERATE)
    Note over EigSolver: Group modes with similar n_complex
    EigSolver-->>compute_modes: degenerate_sets: [{0,1}, ...]
    
    compute_modes->>EigSolver: _make_orthogonal_basis_for_degenerate_modes(...)
    
    loop For each degenerate group
        EigSolver->>EigSolver: _outer_dot() → compute overlap matrix S
        EigSolver->>EigSolver: np.linalg.eig(S) → diagonalize
        EigSolver->>EigSolver: Renormalize eigenvectors (v.T @ v = 1)
        EigSolver->>EigSolver: W = V @ Λ^(-1/2) @ V.T
        EigSolver->>EigSolver: Transform E, H by W
    end
    
    EigSolver-->>compute_modes: E, H (orthogonalized)
    compute_modes-->>ModeSolver: fields, n_complex, eps_spec
    ModeSolver-->>User: ModeSolverData

Loading

[greptile-apps]

_{3 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

style: The eigenvalues from np.linalg.eig(S) could theoretically be complex (since S is a complex symmetric matrix, not Hermitian). Taking np.sqrt(eigvals) of complex eigenvalues will produce complex results, which is mathematically correct for this complex-symmetric decomposition. However, if any eigenvalue is zero or very close to zero, 1.0 / np.sqrt(eigvals) will cause a division by zero or produce very large values. Consider adding a check for near-zero eigenvalues similar to the eigenvector self-orthogonality check above.

Prompt To Fix With AI

This is a comment left during a code review.
Path: tidy3d/components/mode/solver.py
Line: 1257:1257

Comment:
**style:** The eigenvalues from `np.linalg.eig(S)` could theoretically be complex (since S is a complex symmetric matrix, not Hermitian). Taking `np.sqrt(eigvals)` of complex eigenvalues will produce complex results, which is mathematically correct for this complex-symmetric decomposition. However, if any eigenvalue is zero or very close to zero, `1.0 / np.sqrt(eigvals)` will cause a division by zero or produce very large values. Consider adding a check for near-zero eigenvalues similar to the eigenvector self-orthogonality check above.

How can I resolve this? If you propose a fix, please make it concise.

[greptile-apps]

style: The commented-out # , conductivity=0.1) should be removed to keep the code clean.

Context Used: Rule from dashboard - Remove commented-out or obsolete code; rely on version control for history. (source)

Prompt To Fix With AI

This is a comment left during a code review.
Path: tests/test_plugins/test_mode_solver.py
Line: 1517:1517

Comment:
**style:** The commented-out `# , conductivity=0.1)` should be removed to keep the code clean.

**Context Used:** Rule from `dashboard` - Remove commented-out or obsolete code; rely on version control for history. ([source](https://app.greptile.com/review/custom-context?memory=0ee31792-33ef-48e0-8328-48277c2ef438))

How can I resolve this? If you propose a fix, please make it concise.

[caseyflex]

Still testing with EME, sent you a slack message. Just a preliminary comment, will give more feedback tomorrow. This is great though, thanks @dmarek-flex !

[caseyflex]

shouldn't this be conjugate=False?