feat: add EMESimulationData.coeffs (FXC-4275) #3049

caseyflex · 2025-12-02T17:44:39Z

Replaces #3024.

This PR extends EMECoefficientDataset to store additional data:

cell interface S matrices
mode overlaps
n_complex and flux of EME modes

Furthermore, instead of using a monitor, this data is always stored in EMESimulationData.coeffs (it is always computed and is not very large) as long as EMESimulation.store_coeffs is True (the default). So EMECoefficientMonitor is deprecated.

Greptile Overview

Greptile Summary

This PR extends EMECoefficientDataset to store additional EME solver data including interface S-matrices, mode overlaps, complex propagation indices, and mode flux. Instead of using a monitor, this data is now always stored in EMESimulationData.coeffs when EMESimulation.store_coeffs=True (the default), deprecating EMECoefficientMonitor.

Key Changes:

Added EMEInterfaceSMatrixDataArray and EMEFluxDataArray data array classes
Created EMEInterfaceSMatrixDataset and EMEOverlapDataset to organize interface data
Extended EMECoefficientDataset with optional fields: n_complex, flux, interface_smatrices, overlaps
Implemented normalized_copy cached property for flux-normalized coefficients and S-matrices
Added store_coeffs boolean field to EMESimulation (defaults to True)
Deprecated EMECoefficientMonitor with appropriate warning message
Added coeffs field to EMESimulationData to store the coefficient dataset
Comprehensive test coverage for new functionality including normalization tests

The implementation follows established patterns in the codebase and provides useful debugging/optimization data to users without requiring explicit monitor configuration.

Confidence Score: 5/5

This PR is safe to merge with minimal risk
The implementation is clean, follows established patterns, has comprehensive test coverage, and properly handles backward compatibility through deprecation warnings. All previously reported issues have been addressed, and the changes are well-documented with appropriate type hints and docstrings.
No files require special attention

Important Files Changed

File Analysis

Filename	Score	Overview
tidy3d/components/data/data_array.py	5/5	Added two new EME data array classes (`EMEInterfaceSMatrixDataArray` and `EMEFluxDataArray`) with proper dimensions and units. Clean implementation following existing patterns.
tidy3d/components/eme/data/dataset.py	4/5	Extended `EMECoefficientDataset` with new optional fields (interface S-matrices, overlaps, n_complex, flux) and added `normalized_copy` property for flux normalization. Implementation is solid with proper null checks.
tidy3d/components/eme/simulation.py	5/5	Added `store_coeffs` boolean field (defaults to True) and deprecation warning for `EMECoefficientMonitor`. Simple and appropriate changes.
tidy3d/components/eme/data/sim_data.py	5/5	Added optional `coeffs` field to store `EMECoefficientDataset`. Straightforward addition with proper typing and documentation.
tests/test_components/test_eme.py	5/5	Added comprehensive test coverage for new data arrays and datasets, including normalization tests. Tests follow existing patterns and verify the new functionality properly.

Sequence Diagram

sequenceDiagram
    participant User
    participant EMESimulation
    participant Solver as EME Solver
    participant EMESimulationData
    participant EMECoefficientDataset
    
    User->>EMESimulation: Create simulation with store_coeffs=True
    User->>EMESimulation: Run simulation
    EMESimulation->>Solver: Execute EME solver
    
    Note over Solver: Compute mode coefficients (A, B)
    Note over Solver: Compute interface S-matrices
    Note over Solver: Compute mode overlaps
    Note over Solver: Compute n_complex and flux
    
    Solver->>EMECoefficientDataset: Create dataset with all coefficients
    EMECoefficientDataset->>EMECoefficientDataset: Store A, B coefficients
    EMECoefficientDataset->>EMECoefficientDataset: Store interface_smatrices
    EMECoefficientDataset->>EMECoefficientDataset: Store overlaps
    EMECoefficientDataset->>EMECoefficientDataset: Store n_complex, flux
    
    Solver->>EMESimulationData: Create simulation data
    EMESimulationData->>EMESimulationData: Store smatrix
    EMESimulationData->>EMESimulationData: Store coeffs (if store_coeffs=True)
    EMESimulationData->>EMESimulationData: Store port_modes
    EMESimulationData->>EMESimulationData: Store monitor data
    
    EMESimulationData-->>User: Return results
    
    alt User wants normalized coefficients
        User->>EMECoefficientDataset: Access normalized_copy property
        EMECoefficientDataset->>EMECoefficientDataset: Normalize A, B by sqrt(flux)
        EMECoefficientDataset->>EMECoefficientDataset: Normalize S12, S21 by sqrt(flux ratio)
        EMECoefficientDataset->>EMECoefficientDataset: Set flux=None to prevent double normalization
        EMECoefficientDataset-->>User: Return normalized dataset
    end

greptile-apps

_{7 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

tidy3d/components/data/data_array.py

github-actions · 2025-12-02T18:19:07Z

Diff Coverage

Diff: origin/develop...HEAD, staged and unstaged changes

tidy3d/components/data/data_array.py (100%)
tidy3d/components/eme/data/dataset.py (100%)
tidy3d/components/eme/data/sim_data.py (100%)
tidy3d/components/eme/simulation.py (100%)

Summary

Total: 57 lines
Missing: 0 lines
Coverage: 100%

caseyflex · 2025-12-02T19:08:10Z

@greptile

greptile-apps

_{8 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

caseyflex · 2025-12-02T23:15:56Z

@greptile

greptile-apps

_{9 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

tidy3d/components/eme/data/dataset.py

tests/test_components/test_eme.py

tidy3d/components/eme/data/dataset.py

caseyflex · 2025-12-02T23:26:14Z

@greptile

greptile-apps

_{9 files reviewed, 4 comments}

_{Edit Code Review Agent Settings | Greptile}

tidy3d/components/eme/data/dataset.py

tests/test_components/test_eme.py

caseyflex · 2025-12-02T23:33:45Z

@greptile

greptile-apps

_{9 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

tidy3d/components/eme/data/dataset.py

momchil-flex

Nice!

yaugenst-flex

Thanks looks good! Just some questions on compatibility.

tidy3d/components/eme/data/dataset.py

tidy3d/components/eme/simulation.py

caseyflex · 2025-12-09T21:20:14Z

@greptile

greptile-apps

_{9 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

momchil-flex · 2025-12-10T09:42:42Z

tidy3d/components/data/data_array.py

+    """
+
+    __slots__ = ()
+    _dims = ("f", "sweep_index", "eme_cell_index", "mode_index_out", "mode_index_in")


How big could this reasonably get? Also, f is over all requested frequencies or over the interpolation frequencies only?

If it's the latter, then something like 100 x 100 x 100 x 20 x 20 could be something that a user could still reasonably try? And that's 400M complex numbers, and we s tore multiple such arrays by default with no option to turn it off. Sounds like it could be a problem?

Thanks for taking another look.

first, I did add the option to turn it off via the Boolean sim.store_coeffs

second, it is over all frequencies, but whether sweep_index appears depends on the exact sweep. For example the overlaps only vary for the deprecated EMEFreqSweep. The interface s matrices don’t vary for EMELengthSweep.

so yes in theory it is possible to request GB of data here, but I think it’s far from typical?

I guess I can add warning / validator here like we do for monitors

Or, have this off by default. What do you think?

I will say store_port_modes is on by default and that can be even more of an issue

that is handled by existing validators

I guess we could go back to the old api of having the coeffs monitor (so it only selects desired cell indices too)

It’s just that most of the size of this data isn’t from spatial extent, it’s from many non-spatial dimensions combined

I would say have it on by default but add validators sounds good?

greptile-apps bot reviewed Dec 2, 2025

View reviewed changes

tidy3d/components/data/data_array.py Outdated Show resolved Hide resolved

tidy3d/components/data/data_array.py Show resolved Hide resolved

tidy3d/components/data/data_array.py Outdated Show resolved Hide resolved

caseyflex force-pushed the casey/emelocaldata branch from 9bff29c to 98b5483 Compare December 2, 2025 19:07

caseyflex force-pushed the casey/emelocaldata branch from 98b5483 to be32d2a Compare December 2, 2025 19:09

greptile-apps bot reviewed Dec 2, 2025

View reviewed changes

caseyflex force-pushed the casey/emelocaldata branch 3 times, most recently from d4e40a8 to 54f45d3 Compare December 2, 2025 23:15

greptile-apps bot reviewed Dec 2, 2025

View reviewed changes

tidy3d/components/eme/data/dataset.py Outdated Show resolved Hide resolved

tests/test_components/test_eme.py Outdated Show resolved Hide resolved

tidy3d/components/eme/data/dataset.py Outdated Show resolved Hide resolved

caseyflex force-pushed the casey/emelocaldata branch from 54f45d3 to e16d683 Compare December 2, 2025 23:25

greptile-apps bot reviewed Dec 2, 2025

View reviewed changes

tidy3d/components/eme/data/dataset.py Outdated Show resolved Hide resolved

tidy3d/components/eme/data/dataset.py Outdated Show resolved Hide resolved

tidy3d/components/eme/data/dataset.py Outdated Show resolved Hide resolved

tests/test_components/test_eme.py Outdated Show resolved Hide resolved

caseyflex force-pushed the casey/emelocaldata branch from e16d683 to ad087ee Compare December 2, 2025 23:33

greptile-apps bot reviewed Dec 2, 2025

View reviewed changes

tidy3d/components/eme/data/dataset.py Show resolved Hide resolved

caseyflex requested review from momchil-flex and yaugenst-flex December 2, 2025 23:40

momchil-flex approved these changes Dec 5, 2025

View reviewed changes

yaugenst-flex approved these changes Dec 5, 2025

View reviewed changes

tidy3d/components/eme/data/dataset.py Show resolved Hide resolved

tidy3d/components/eme/simulation.py Show resolved Hide resolved

caseyflex force-pushed the casey/emelocaldata branch 2 times, most recently from bbdef0b to d0f925f Compare December 9, 2025 20:59

feat: add EMESimulationData.coeffs (FXC-4275)

171b8b5

caseyflex force-pushed the casey/emelocaldata branch from d0f925f to 171b8b5 Compare December 9, 2025 21:14

greptile-apps bot reviewed Dec 9, 2025

View reviewed changes

momchil-flex reviewed Dec 10, 2025

View reviewed changes

feat: add EMESimulationData.coeffs (FXC-4275) #3049

Are you sure you want to change the base?

feat: add EMESimulationData.coeffs (FXC-4275) #3049

Uh oh!

Conversation

caseyflex commented Dec 2, 2025 • edited by greptile-apps bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Overview

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Diff Coverage

Diff: origin/develop...HEAD, staged and unstaged changes

Summary

Uh oh!

caseyflex commented Dec 2, 2025

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

caseyflex commented Dec 2, 2025

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

caseyflex commented Dec 2, 2025

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

caseyflex commented Dec 2, 2025

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

momchil-flex left a comment

Choose a reason for hiding this comment

Uh oh!

yaugenst-flex left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

caseyflex commented Dec 9, 2025

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

caseyflex commented Dec 2, 2025 •

edited by greptile-apps bot

Loading

github-actions bot commented Dec 2, 2025 •

edited

Loading