feat(config): introduce profile-based configuration system

Author: yaugenst-flex

CHANGELOG.md

@@ -24,6 +24,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - A new type of doping box has been introduced, `CustomDoping` which accepts a `SpatialDataArray` to define doping concentration. Unlike in the case where a `SpatialDataArray`, custom doping defined with `CustomDoping` have additive behavior, i.e., one can add other doping on top. This deprecates the `SpatialDataArray` as direct input for `N_a` and `N_d`.
 - Non-isothermal Charge simulations are now available. One can now run this type of simulations by using the `SteadyChargeDCAnalysis` as the `analysis_spec` of a `HeatChargeSimulation`. This type of simulations couple the heat equation with the drift-diffusion equations which allow to account for self heating behavior.
 - Because non-isothermal Charge simulations are now supported, new models for the effective density of states and bandgap energy have been introduced. These models are the following: `ConstantEffectiveDOS`, `IsotropicEffectiveDOS`, `MultiValleyEffectiveDOS`, `DualValleyEffectiveDOS`.
+- Introduced a profile-based configuration manager with TOML persistence and runtime overrides exposed via `tidy3d.config`.
 
 ### Changed
 - `LayerRefinementSpec` defaults to assuming structures made of different materials are interior-disjoint for more efficient mesh generation.

docs/api/configuration.rst

@@ -0,0 +1,43 @@
+Configuration API
+=================
+
+.. currentmodule:: tidy3d.config
+
+The objects and helpers below expose the public configuration interface.
+
+Manager and Helpers
+-------------------
+
+.. autosummary::
+   :toctree: _autosummary/
+   :template: module.rst
+
+   ConfigManager
+   get_manager
+   reload_config
+   config
+
+Legacy Compatibility
+--------------------
+
+.. autosummary::
+   :toctree: _autosummary/
+   :template: module.rst
+
+   LegacyConfigWrapper
+   Env
+   Environment
+   EnvironmentConfig
+
+Registration Utilities
+----------------------
+
+.. autosummary::
+   :toctree: _autosummary/
+   :template: module.rst
+
+   register_section
+   register_plugin
+   register_handler
+   get_sections
+   get_handlers

docs/api/constants.rst

@@ -35,7 +35,7 @@ Tidy3D Configuration
    :toctree: _autosummary/
    :template: module.rst
 
-   tidy3d.config.Tidy3dConfig
+   tidy3d.config.config
 
 Default Absorber Parameters
 ----------------------------
@@ -92,4 +92,4 @@ Precision & Comparator Values
    tidy3d.constants.fp_eps
    tidy3d.constants.pec_val
    tidy3d.constants.LARGE_NUMBER
-   tidy3d.constants.GLANCING_CUTOFF
+   tidy3d.constants.GLANCING_CUTOFF

docs/api/index.rst

@@ -18,6 +18,7 @@ API |:computer:|
     output_data
     analytic_beams
     utilities
+    configuration
     mesh/index
     heat/index
     charge/index

docs/configuration/index.rst

@@ -0,0 +1,175 @@
+Configuration Guide |:gear:|
+============================
+
+.. highlight:: python
+
+Working with cloud simulations usually requires a handful of settings such as
+your API key, the active environment, and any local tweaks you make while
+experimenting. The ``tidy3d.config`` module keeps all of this in one place
+through a single object called ``config``. This page explains how it behaves,
+where values are stored, and how to keep your changes consistent across
+sessions.
+
+Getting Started
+---------------
+
+Most users only need the following import::
+
+    from tidy3d.config import config
+
+You can then read or update sections just like attributes::
+
+    # read values
+    print(config.web.api_endpoint)
+    print(config.logging.level)
+
+    # update values
+    config.logging.level = "DEBUG"
+    config.web.timeout = 60
+    config.save()
+
+The ``save()`` call writes your edits to disk so the same settings load the
+next time you import ``tidy3d``.
+
+Where Settings Are Stored
+-------------------------
+
+Tidy3D chooses a configuration directory the first time you import the module.
+The location depends on your operating system:
+
+.. list-table:: Default configuration directory
+   :widths: 30 70
+   :header-rows: 1
+
+   * - Platform
+     - Path
+   * - macOS / Linux
+     - ``~/.config/tidy3d``
+   * - Windows
+     - ``C:\\Users\\<you>\\.config\\tidy3d``
+
+You can override this by setting the ``TIDY3D_BASE_DIR`` environment variable
+*before* importing ``tidy3d``. When it is present, the config files are kept in
+``<base>/.tidy3d``. If the chosen location is not writable, Tidy3D falls back to
+a temporary directory and warns that the settings will not persist.
+
+Files Inside the Directory
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- ``config.toml`` – base settings shared by all profiles.
+- ``profiles/<name>.toml`` – optional overrides for custom profiles. Each file
+  only contains the differences from the base settings.
+
+Priority Order
+--------------
+
+Whenever you read ``config.<section>.<field>``, the value comes from the highest
+priority source in the list below. Lower entries only apply when the ones above
+them do not set a value.
+
+1. Runtime changes you make in the current Python session.
+2. Environment variables (``TIDY3D_<SECTION>__<FIELD>``).
+3. Profile overrides from ``profiles/<name>.toml``.
+4. The base ``config.toml`` file.
+5. Built-in profiles (for example ``prod`` and ``dev``) bundled with Tidy3D.
+6. Default values defined by the software.
+
+This means environment variables always win over ``config.toml``, and any
+attribute you set in code wins over everything else until you discard it or
+call ``save()``.
+
+Making Changes That Last
+------------------------
+
+Runtime Updates
+~~~~~~~~~~~~~~~
+
+Assignments like ``config.logging.level = "INFO"`` apply immediately but only
+live in memory. They affect new simulations started in the same interpreter but
+disappear when the process exits.
+
+Saving to Disk
+~~~~~~~~~~~~~~
+
+Call ``config.save()`` to write the current profile to disk. The method removes
+environment-variable overrides automatically so you never persist an API key or
+other secret that was loaded from the shell. To store the full set of values,
+including defaults, pass ``include_defaults=True``::
+
+    config.save(include_defaults=True)
+
+Profiles
+--------
+
+Tidy3D ships with built-in profiles such as ``prod`` and ``dev``. Switch between
+them with::
+
+    config.switch_profile("dev")
+
+To create your own profile, switch to a new name, edit settings, then call
+``save()``::
+
+    config.switch_profile("customer")
+    config.web.api_endpoint = "https://example.com"
+    config.save()
+
+This writes ``profiles/customer.toml`` containing only the adjustments you
+made. Use ``config.profiles.list()`` to discover available built-in and user
+profiles.
+
+Environment Variables
+---------------------
+
+Environment variables let you override individual options without touching any
+files. The naming pattern is ``TIDY3D_<SECTION>__<FIELD>``, for example::
+
+    export TIDY3D_LOGGING__LEVEL=WARNING
+
+Supported variables take effect the next time you import ``tidy3d``. Remove a
+variable or clear the shell environment to restore the lower priority setting.
+
+Plugin Settings
+---------------
+
+Plugins can register their own sections so their options appear under
+``config.plugins``. A typical plugin exposes a model decorated with
+``@register_plugin``::
+
+    from tidy3d.config.sections import ConfigSection
+    from tidy3d.config import register_plugin
+
+    @register_plugin("sample")
+    class SamplePluginConfig(ConfigSection):
+        enabled: bool = False
+        threshold: float = 0.5
+
+After the plugin is imported, you can read or modify its settings with::
+
+    config.plugins.sample.enabled = True
+    config.save()
+
+If the plugin registers later in the session, previously saved values are
+loaded automatically.
+
+Command Line Helpers
+--------------------
+
+Use ``tidy3d configure`` to store your API key in ``config.toml``. The command
+creates the directory if it is missing and updates only the ``web`` section.
+
+If you have older files in ``~/.tidy3d``, run ``tidy3d config migrate`` to move
+them into the new location described above.
+
+Legacy Access Points
+--------------------
+
+Older code paths such as ``tidy3d.config.logging_level`` and ``tidy3d.config.Env``
+still work. They emit a ``DeprecationWarning`` each time you use them to help
+you transition to the modern interface. See :doc:`migration` for advice on
+updating scripts that depend on these names.
+
+Next Steps
+----------
+
+- :doc:`migration`
+- :doc:`../api/configuration`

docs/configuration/migration.rst

@@ -0,0 +1,48 @@
+Upgrading Existing Setups
+=========================
+
+This short note highlights the differences you may notice when moving from
+earlier versions of Tidy3D to the current configuration manager.
+
+File Locations
+--------------
+
+- Previous releases stored settings in ``~/.tidy3d`` on all platforms. The new
+  manager now prefers the platform-specific paths described in
+  :doc:`index`.
+- Your existing ``~/.tidy3d/config.toml`` is still respected. Run
+  ``tidy3d config migrate`` if you would like to copy it into the new directory.
+
+Environment Switching
+---------------------
+
+- The ``Env`` helper remains available. Calls such as ``Env.dev.active()`` now
+  forward to the new manager and produce a ``DeprecationWarning`` to encourage
+  the modern API, e.g. ``config.switch_profile("dev")``.
+
+Legacy Attributes
+-----------------
+
+- Shorthand properties ``config.logging_level``, ``config.log_suppression``,
+  and ``config.use_local_subpixel`` still work and set the equivalent fields in
+  ``config.logging`` or ``config.simulation``. Each call raises a warning so
+  you can update scripts at your own pace.
+
+Working Safely With Existing Scripts
+------------------------------------
+
+- Prefer the new attribute paths (``config.logging.level``) in fresh code.
+- When editing older notebooks, import ``warnings`` and silence specific
+  deprecations temporarily if needed::
+
+      import warnings
+      warnings.filterwarnings("ignore", category=DeprecationWarning)
+
+- After updating your scripts, remove the filter so you notice future changes.
+
+Need Help?
+----------
+
+Reach out through the `Tidy3D Discussions board <https://github.com/flexcompute/tidy3d/discussions>`_
+or contact support@flexcompute.com if you hit any issues while upgrading.
+

docs/extras/index.rst

@@ -66,6 +66,9 @@ You can check whether local subpixel averaging is turned on::
 
    tidy3d.packaging.tidy3d_extras["use_local_subpixel"]
 
+For a broader overview of configuration options and how they are stored, see
+:doc:`../configuration/index`.
+
 Licenses
 --------
 

docs/index.rst

@@ -52,9 +52,9 @@ Get Started
 
            tidy3d configure --apikey=XXX
 
-        And enter your API key when prompted.
-
-        For more detailed installation instructions, see `this page <./install.html>`_.
+        For more detailed installation instructions, see `this page <./install.html>`_,
+        and refer to :doc:`configuration/index` if you would like to fine-tune your
+        settings.
 
     .. group-tab:: On Windows |:window:|
 
@@ -78,19 +78,9 @@ Get Started
         If you're running into trouble, you may need to manually set the API key directly in the configuration file where Tidy3D looks for it.
         You need to place the ``$HOME/.tidy3d/config`` file in your home directory such as ``C:\Users\username\`` (where ``username`` is your username).
 
-        The API key must be in a file called ``$HOME/.tidy3d/config`` located in your home directory, with the following contents
-
-        .. code-block:: bash
-
-            apikey = "XXX"
-
-        You can manually set up your file like this, or do it through the command line line:
-
-        .. code-block:: bash
-
-            echo 'apikey = "XXX"' > ~/.tidy3d/config
-
-        Note the quotes around `XXX`.
+        The ``tidy3d configure`` command stores the API key for you. If you prefer
+        to manage the file yourself, see :doc:`configuration/index` for the current
+        location and format on each platform.
 
     .. group-tab:: In the Cloud |:cloud:|
 
@@ -249,6 +239,7 @@ Contents
   :maxdepth: 2
 
   install
+  configuration/index
   lectures/index
   notebooks/docs/index
   faq/docs/index
@@ -260,5 +251,3 @@ Contents
   About our Solver <https://www.flexcompute.com/tidy3d/solver/>
 
 
-
-

docs/install.rst

@@ -78,23 +78,9 @@ Alternatively, the API key can be set up using the environment variable ``SIMCLO
 
     export SIMCLOUD_APIKEY="XXX"
 
-Finally, one may manually set the API key directly in the configuration file where Tidy3D looks for it.
-
-The API key must be in a file called ``.tidy3d/config`` located in your home directory, with the following contents
-
-.. code-block:: python
-
-    apikey = "XXX"
-
-You can manually set up your file like this, or do it through the command line line:
-
-.. code-block:: python
-
-    echo 'apikey = "XXX"' > ~/.tidy3d/config
-
-Note the quotes around `XXX`.
-
-Note that Windows users will most likely need to place the ``.tidy3d/config`` file in their ``C:\Users\username\`` directory (where ``username`` is your username).
+Finally, one may manually set the API key directly in the configuration file
+where Tidy3D looks for it. The path and file format differ slightly between
+platforms; see :doc:`configuration/index` for the up-to-date layout.