v0.29.0: keep undefined keys in dict & support custom object path in deserialization

HISTORY.rst

@@ -2,6 +2,17 @@
 History
 =======
 
+0.29.0 (2024-11-23)
+-------------------
+
+**Features and Improvements**
+
+- *Nested JSON Mapping* (:issue:`60`): Map nested JSON keys to dataclass fields using helper functions :func:`KeyPath` or :func:`json_field`.
+- *Catch-All Keys* (:issue:`57`): Save unknown JSON keys with ease.
+- *Cleaner Codebase*: Remove comments and type annotations for Python files with ``.pyi`` counterparts.
+- *Enhanced Debugging*: ``debug_enabled`` now supports ``bool | int | str``, allowing flexible logging levels.
+- *Documentation Updates*: Improved and expanded docs!
+
 0.28.0 (2024-11-15)
 -------------------
 

README.rst

@@ -27,11 +27,12 @@ Full documentation is available at `Read The Docs`_. (`Installation`_)
 
 
 
-This library provides a set of simple, yet elegant *wizarding* tools for
-interacting with the Python ``dataclasses`` module.
+Dataclass Wizard offers simple, elegant, *wizarding* tools for
+interacting with Python's ``dataclasses``.
 
-    The primary use is as a fast serialization framework that enables dataclass instances to
-    be converted to/from JSON; this works well in particular with a *nested dataclass* model.
+    It excels at lightning-fast de/serialization, converting dataclass
+    instances to/from JSON effortlessly -- perfect for *nested dataclass*
+    models!
 
 -------------------
 
@@ -74,34 +75,32 @@ interacting with the Python ``dataclasses`` module.
 Installation
 ------------
 
-The Dataclass Wizard library is available `on PyPI`_, and can be installed with ``pip``:
+Dataclass Wizard is available on `PyPI`_. Install with ``pip``:
 
 .. code-block:: shell
 
     $ pip install dataclass-wizard
 
-Alternatively, this library is available `on conda`_ under the `conda-forge`_ channel:
+Also available on `conda`_ via `conda-forge`_. Install with ``conda``:
 
 .. code-block:: shell
 
     $ conda install dataclass-wizard -c conda-forge
 
-The ``dataclass-wizard`` library officially supports **Python 3.9** or higher.
+This library supports **Python 3.9** or higher.
 
-.. _on conda: https://anaconda.org/conda-forge/dataclass-wizard
+.. _PyPI: https://pypi.org/project/dataclass-wizard/
+.. _conda: https://anaconda.org/conda-forge/dataclass-wizard
 .. _conda-forge: https://conda-forge.org/
 
 Features
 --------
 
-Here are the supported features that ``dataclass-wizard`` currently provides:
+Here are the key features that ``dataclass-wizard`` offers:
 
--  *JSON/YAML (de)serialization*: marshal dataclasses to/from JSON, YAML, and Python
-   ``dict`` objects.
--  *Field properties*: support for using properties with default
-   values in dataclass instances.
--  *JSON to Dataclass generation*: construct a dataclass schema with a JSON file
-   or string input.
+-   *Flexible (de)serialization*: Marshal dataclasses to/from JSON, TOML, YAML, or ``dict``.
+-  *Field properties*: Use properties with default values in dataclass instances.
+-  *JSON to Dataclass generation*: Auto-generate a dataclass schema from a JSON file or string.
 
 
 Wizard Mixins
@@ -478,6 +477,70 @@ Example below:
     # Assert we get the same dictionary object when serializing the instance.
     assert c.to_dict() == d
 
+Mapping Nested JSON Keys
+------------------------
+
+The ``dataclass-wizard`` library lets you map deeply nested JSON keys to dataclass fields using custom path notation. This is ideal for handling complex or non-standard JSON structures.
+
+You can specify paths to JSON keys with the ``KeyPath`` or ``path_field`` helpers. For example, the deeply nested key ``data.items.myJSONKey`` can be mapped to a dataclass field, such as ``my_str``:
+
+.. code:: python3
+
+    from dataclasses import dataclass
+    from dataclass_wizard import path_field, JSONWizard
+
+    @dataclass
+    class MyData(JSONWizard):
+        my_str: str = path_field('data.items.myJSONKey', default="default_value")
+
+    input_dict = {'data': {'items': {'myJSONKey': 'Some value'}}}
+    data_instance = MyData.from_dict(input_dict)
+    print(data_instance.my_str)  # Output: 'Some value'
+
+Custom Paths for Complex JSON
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can use `custom paths to access nested keys`_ and map them to specific fields, even when keys contain special characters or follow non-standard conventions.
+
+Example with nested and complex keys:
+
+.. code:: python3
+
+    from dataclasses import dataclass
+    from typing import Annotated
+    from dataclass_wizard import JSONWizard, path_field, KeyPath
+
+
+    @dataclass
+    class NestedData(JSONWizard):
+        my_str: str = path_field('data[0].details["key with space"]', default="default_value")
+        my_int: Annotated[int, KeyPath('data[0].items[3.14].True')] = 0
+
+
+    input_dict = {
+        'data': [
+            {
+                'details': {'key with space': 'Another value'},
+                'items': {3.14: {True: "42"}}
+            }
+        ]
+    }
+
+    # Deserialize JSON to dataclass
+    data = NestedData.from_dict(input_dict)
+    print(data.my_str)  # Output: 'Another value'
+
+    # Serialize back to JSON
+    output_dict = data.to_dict()
+    print(output_dict)  # {'data': {0: {'details': {'key with space': 'Another value'}, 'items': {3.14: {True: 42}}}}}
+
+    # Verify data consistency
+    assert data == NestedData.from_dict(output_dict)
+
+    # Handle empty input gracefully
+    data = NestedData.from_dict({'data': []})
+    print(repr(data))  # NestedData(my_str='default_value', my_int=0)
+
 Extending from ``Meta``
 -----------------------
 
@@ -535,6 +598,10 @@ a full list of available settings can be found in the `Meta`_ section in the doc
 Debug Mode
 ##########
 
+.. admonition:: **Added in v0.28.0**
+
+   There is now `Easier Debug Mode`_.
+
 Enables additional (more verbose) log output. For example, a message can be
 logged whenever an unknown JSON key is encountered when
 ``from_dict`` or ``from_json`` is called.
@@ -569,17 +636,19 @@ an unknown JSON key is encountered in the  *load* (de-serialization) process.
     from dataclass_wizard import JSONWizard
     from dataclass_wizard.errors import UnknownJSONKey
 
-
     # Sets up application logging if we haven't already done so
-    logging.basicConfig(level='INFO')
+    logging.basicConfig(level='DEBUG')
 
 
     @dataclass
     class Container(JSONWizard):
 
         class _(JSONWizard.Meta):
             # True to enable Debug mode for additional (more verbose) log output.
-            debug_enabled = True
+            #
+            # Pass in a `str` to `int` to set the minimum log level:
+            #   logging.getLogger('dataclass_wizard').setLevel('INFO')
+            debug_enabled = logging.INFO
             # True to raise an class:`UnknownJSONKey` when an unmapped JSON key is
             # encountered when `from_dict` or `from_json` is called. Note that by
             # default, this is also recursively applied to any nested dataclasses.
@@ -616,6 +685,67 @@ an unknown JSON key is encountered in the  *load* (de-serialization) process.
         print('Successfully de-serialized the JSON object.')
         print(repr(c))
 
+See the section on `Handling Unknown JSON Keys`_ for more info.
+
+Save or "Catch-All" Unknown JSON Keys
+######################################
+
+When calling ``from_dict`` or ``from_json``, any unknown or extraneous JSON keys
+that are not mapped to fields in the dataclass are typically ignored or raise an error.
+However, you can capture these undefined keys in a catch-all field of type ``CatchAll``,
+allowing you to handle them as needed later.
+
+For example, suppose you have the following dictionary::
+
+    dump_dict = {
+        "endpoint": "some_api_endpoint",
+        "data": {"foo": 1, "bar": "2"},
+        "undefined_field_name": [1, 2, 3]
+    }
+
+You can save the undefined keys in a catch-all field and process them later.
+Simply define a field of type ``CatchAll`` in your dataclass. This field will act
+as a dictionary to store any unmapped keys and their values. If there are no
+undefined keys, the field will default to an empty dictionary.
+
+.. code:: python
+
+    from dataclasses import dataclass
+    from typing import Any
+    from dataclass_wizard import CatchAll, JSONWizard
+
+    @dataclass
+    class UnknownAPIDump(JSONWizard):
+        endpoint: str
+        data: dict[str, Any]
+        unknown_things: CatchAll
+
+    dump_dict = {
+        "endpoint": "some_api_endpoint",
+        "data": {"foo": 1, "bar": "2"},
+        "undefined_field_name": [1, 2, 3]
+    }
+
+    dump = UnknownAPIDump.from_dict(dump_dict)
+    print(f'{dump!r}')
+    # > UnknownAPIDump(endpoint='some_api_endpoint', data={'foo': 1, 'bar': '2'},
+    #       unknown_things={'undefined_field_name': [1, 2, 3]})
+
+    print(dump.to_dict())
+    # > {'endpoint': 'some_api_endpoint', 'data': {'foo': 1, 'bar': '2'}, 'undefined_field_name': [1, 2, 3]}
+
+.. note::
+    - When using a "catch-all" field, it is strongly recommended to define exactly **one** field of type ``CatchAll`` in the dataclass.
+
+    - ``LetterCase`` transformations do not apply to keys stored in the ``CatchAll`` field; the keys remain as they are provided.
+
+    - If you specify a default (or a default factory) for the ``CatchAll`` field, such as
+      ``unknown_things: CatchAll = None``, the default value will be used instead of an
+      empty dictionary when no undefined parameters are present.
+
+    - The ``CatchAll`` functionality is guaranteed only when using ``from_dict`` or ``from_json``.
+      Currently, unknown keyword arguments passed to ``__init__`` will not be written to a ``CatchAll`` field.
+
 Date and Time with Custom Patterns
 ----------------------------------
 
@@ -955,7 +1085,6 @@ This package was created with Cookiecutter_ and the `rnag/cookiecutter-pypackage
 
 .. _Read The Docs: https://dataclass-wizard.readthedocs.io
 .. _Installation: https://dataclass-wizard.readthedocs.io/en/latest/installation.html
-.. _on PyPI: https://pypi.org/project/dataclass-wizard/
 .. _Cookiecutter: https://github.com/cookiecutter/cookiecutter
 .. _`rnag/cookiecutter-pypackage`: https://github.com/rnag/cookiecutter-pypackage
 .. _`Contributing`: https://dataclass-wizard.readthedocs.io/en/latest/contributing.html
@@ -982,3 +1111,6 @@ This package was created with Cookiecutter_ and the `rnag/cookiecutter-pypackage
 .. _`Cyclic or "Recursive" Dataclasses`: https://dataclass-wizard.readthedocs.io/en/latest/common_use_cases/cyclic_or_recursive_dataclasses.html
 .. _as milestones: https://github.com/rnag/dataclass-wizard/milestones
 .. _longstanding issue: https://github.com/rnag/dataclass-wizard/issues/62
+.. _Easier Debug Mode: https://dataclass-wizard.readthedocs.io/en/latest/common_use_cases/easier_debug_mode.html
+.. _Handling Unknown JSON Keys: https://dataclass-wizard.readthedocs.io/en/latest/common_use_cases/handling_unknown_json_keys.html
+.. _custom paths to access nested keys: https://dataclass-wizard.readthedocs.io/en/latest/common_use_cases/nested_key_paths.html

benchmarks/complex.py

@@ -115,15 +115,17 @@ def data():
         'the_answer_to_life': '42',
         'people': [
             {
-                'name': ('Roberto', 'Fuirron'),
+                # I want to make this into a Tuple - ('Roberto', 'Fuirron') -
+                # but `dataclass-factory` doesn't seem to like that.
+                'name': {'first': 'Roberto', 'last': 'Fuirron'},
                 'age': 21,
                 'birthdate': '1950-02-28T17:35:20Z',
                 'gender': 'M',
                 'occupation': ['sailor', 'fisher'],
                 'hobbies': {'M-F': ('chess', '123', 'reading'), 'Sat-Sun': ['parasailing']}
             },
             {
-                'name': ('Janice', 'Darr', 'Dr.'),
+                'name': {'first': 'Janice', 'last': 'Darr', 'salutation': 'Dr.'},
                 'age': 45,
                 # `jsons` doesn't support this format (not sure how to fix?)
                 # 'birthdate': '1971-11-05 05:10:59',

dataclass_wizard/__init__.py

@@ -64,7 +64,7 @@
 For full documentation and more advanced usage, please see
 <https://dataclass-wizard.readthedocs.io>.
 
-:copyright: (c) 2021 by Ritvik Nag.
+:copyright: (c) 2021-2025 by Ritvik Nag.
 :license: Apache 2.0, see LICENSE for more details.
 """
 
@@ -89,20 +89,23 @@
     # Models
     'json_field',
     'json_key',
+    'path_field',
+    'KeyPath',
     'Container',
     'Pattern',
     'DatePattern',
     'TimePattern',
     'DateTimePattern',
+    'CatchAll',
 ]
 
 import logging
 
 from .bases_meta import LoadMeta, DumpMeta
 from .dumpers import DumpMixin, setup_default_dumper, asdict
 from .loaders import LoadMixin, setup_default_loader, fromlist, fromdict
-from .models import (json_field, json_key, Container,
-                     Pattern, DatePattern, TimePattern, DateTimePattern)
+from .models import (json_field, json_key, path_field, KeyPath, Container,
+                     Pattern, DatePattern, TimePattern, DateTimePattern, CatchAll)
 from .property_wizard import property_wizard
 from .serial_json import JSONSerializable
 from .wizard_mixins import JSONListWizard, JSONFileWizard, TOMLWizard, YAMLWizard

dataclass_wizard/__version__.py

@@ -3,12 +3,12 @@
 """
 
 __title__ = 'dataclass-wizard'
-__description__ = 'Marshal dataclasses to/from JSON. Use field properties ' \
-                  'with initial values. Construct a dataclass schema with ' \
-                  'JSON input.'
+__description__ = ('Effortlessly marshal dataclasses to/from JSON. '
+                   'Leverage field properties with default values. '
+                   'Generate dataclass schemas from JSON input.')
 __url__ = 'https://github.com/rnag/dataclass-wizard'
 __version__ = '0.28.0'
 __author__ = 'Ritvik Nag'
-__author_email__ = 'rv.kvetch@gmail.com'
+__author_email__ = 'me@ritviknag.com'
 __license__ = 'Apache 2.0'
-__copyright__ = 'Copyright 2021 Ritvik Nag'
+__copyright__ = 'Copyright 2021-2025 Ritvik Nag'