Merge remote-tracking branch 'origin/main' into feature_convert_gener…

…ator

.pre-commit-config.yaml

@@ -8,18 +8,18 @@ repos:
       - id: end-of-file-fixer
       - id: debug-statements
   - repo: https://github.com/crate-ci/typos
-    rev: v1.20.9
+    rev: v1.21.0
     hooks:
       - id: typos
         exclude: ^tests/|.xsd|xsdata/models/datatype.py$
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.4.1
+    rev: v0.4.4
     hooks:
       - id: ruff
         args: [ --fix, --show-fixes]
       - id: ruff-format
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v1.9.0
+    rev: v1.10.0
     hooks:
       - id: mypy
         files: ^(xsdata/)

CHANGES.md

@@ -1,3 +1,23 @@
+## 24.5 (2024-05-07)
+
+**Features**
+
+- Rewrite TreeSerializer, drop support for native python ElementTree
+  ([#1032](https://github.com/tefra/xsdata/pull/1032))
+- Validate fields fixed values ([#1013](https://github.com/tefra/xsdata/pull/1013))
+- Detect optional fields in dict mapper
+- Refactor typing annotations analyze process
+- Generate ForwardRef() instead of Type[]
+
+**Fixes**
+
+- Allow soap client config subclassing
+  ([#1010](https://github.com/tefra/xsdata/pull/1010))
+- Avoid recursive error on nested group references
+  ([#1016](https://github.com/tefra/xsdata/pull/1016))
+- Add warning for same module designation
+  ([#1018](https://github.com/tefra/xsdata/pull/1018))
+
 ## 24.4 (2024-04-01)
 
 **Features**

README.md

@@ -5,7 +5,6 @@
 [![image](https://github.com/tefra/xsdata/workflows/tests/badge.svg)](https://github.com/tefra/xsdata/actions)
 [![image](https://readthedocs.org/projects/xsdata/badge)](https://xsdata.readthedocs.io/)
 [![image](https://codecov.io/gh/tefra/xsdata/branch/main/graph/badge.svg)](https://codecov.io/gh/tefra/xsdata)
-[![image](https://img.shields.io/github/languages/top/tefra/xsdata.svg)](https://xsdata.readthedocs.io/)
 [![image](https://www.codefactor.io/repository/github/tefra/xsdata/badge)](https://www.codefactor.io/repository/github/tefra/xsdata)
 [![image](https://img.shields.io/pypi/pyversions/xsdata.svg)](https://pypi.org/pypi/xsdata/)
 [![image](https://img.shields.io/pypi/v/xsdata.svg)](https://pypi.org/pypi/xsdata/)
@@ -75,44 +74,22 @@ Check the [documentation](https://xsdata.readthedocs.io) for more ✨✨✨
 - Support xinclude statements and unknown properties
 - Customize behaviour through config
 
-## Changelog: 24.3.1 (2024-03-10)
-
-**Fixes**
-
-- Unnest class with circular reference
-  ([#974](https://github.com/tefra/xsdata/pull/974))
-
-## Changelog: 24.4 (2024-04-01)
+## Changelog: 24.5 (2024-05-07)
 
 **Features**
 
-- Add xml and lxml tree serializers ([#975](https://github.com/tefra/xsdata/pull/975))
-- Capture namespace prefixes in user dicts
-  ([#978](https://github.com/tefra/xsdata/pull/978))
-- Add cli option to generate wrapper fields
-  ([#982](https://github.com/tefra/xsdata/pull/982))
-- Support wrapper fields in JSON data bindings
-  ([#982](https://github.com/tefra/xsdata/pull/982))
-- Use abstract suffixes to resolve class name conflicts
-  ([#985](https://github.com/tefra/xsdata/pull/985))
-- Add the version number in the cli cache key
-  ([#990](https://github.com/tefra/xsdata/pull/990))
-- Use unicodedata.name for attrs with only special characters
-  ([#993](https://github.com/tefra/xsdata/pull/993))
-- Add src code excerpts on ruff errors
-  ([#996](https://github.com/tefra/xsdata/pull/996))
-- Detect circular imports and raise appropriate error
-  ([#999](https://github.com/tefra/xsdata/pull/999))
-- Add support for Python 3.13 ([#1001](https://github.com/tefra/xsdata/pull/1001))
-- Add cli debug messages with performance stats
+- Rewrite TreeSerializer, drop support for native python ElementTree
+  ([#1032](https://github.com/tefra/xsdata/pull/1032))
+- Validate fields fixed values ([#1013](https://github.com/tefra/xsdata/pull/1013))
+- Detect optional fields in dict mapper
+- Refactor typing annotations analyze process
+- Generate ForwardRef() instead of Type[]
 
 **Fixes**
 
-- Use deepcopy to clone codegen models
-  ([#980](https://github.com/tefra/xsdata/pull/980))
-- Generate type hints for compound fields with token elements
-  ([#997](https://github.com/tefra/xsdata/pull/997))
-- Protect prohibited attrs from turning into lists
-  ([#998](https://github.com/tefra/xsdata/pull/998))
-- Convert child attr to list when parent is list
-  ([#998](https://github.com/tefra/xsdata/pull/998))
+- Allow soap client config subclassing
+  ([#1010](https://github.com/tefra/xsdata/pull/1010))
+- Avoid recursive error on nested group references
+  ([#1016](https://github.com/tefra/xsdata/pull/1016))
+- Add warning for same module designation
+  ([#1018](https://github.com/tefra/xsdata/pull/1018))

docs/codegen/config.md

@@ -85,6 +85,13 @@ The output package for the generated code, e.g. `code.models`
 
 **CLI Option:** `-p, --package TEXT`
 
+!!! Warning
+
+    Formatting and linting is executed on any directory that contains a Python file
+    created during generation, including other Python files in the same directory prior
+    to generation. As such it is recommended that the directories represented by this
+    option do not include any previously created files.
+
 ### Format
 
 The output format for the generated code, e.g. `code.models`

docs/data_binding/tree_serializing.md

@@ -1,41 +1,26 @@
 # Element Tree Serializer
 
-The element tree serializers will render an object into an element tree, that you can
-use to run XPATH evaluations or XSLT transformations.
+The tree serializer will render an object into an element tree, that you can use to run
+XPATH evaluations or XSLT transformations.
 
-There are two implementations based on lxml
-[LxmlTreeSerializer][xsdata.formats.dataclass.serializers.LxmlTreeSerializer] and native
-python [XmlTreeSerializer][xsdata.formats.dataclass.serializers.XmlTreeSerializer].
+The [TreeSerializer][xsdata.formats.dataclass.serializers.TreeSerializer] depends on
+lxml. There is no native python ElementTree implementation because of limitations with
+namespaces.
 
-## xml.etree.ElementTree.Element
-
-```python
->>> from xml.etree import ElementTree
->>> from tests.fixtures.books.fixtures import books
->>> from xsdata.formats.dataclass.serializers import XmlTreeSerializer
-...
->>> serializer = XmlTreeSerializer()
->>> result = serializer.render(books)
-...
->>> result.find(".//title").text
-'The First Book'
-
-```
-
-## lxml.etree.Element
+## Example
 
 ```python
 >>> from lxml import etree
 >>> from tests.fixtures.books.fixtures import books
->>> from xsdata.formats.dataclass.serializers import LxmlTreeSerializer
+>>> from xsdata.formats.dataclass.serializers import TreeSerializer
 ...
->>> serializer = LxmlTreeSerializer()
->>> result = serializer.render(books)
+>>> serializer = TreeSerializer()
+>>> serializer.config.indent = "  "
+>>> result = serializer.render(books, ns_map={'bk': "urn:books"})
 ...
->>> etree.indent(result)
 >>> actual = etree.tostring(result)
 >>> print(actual.decode())
-<ns0:books xmlns:ns0="urn:books">
+<bk:books xmlns:bk="urn:books">
   <book id="bk001" lang="en">
     <author>Hightower, Kim</author>
     <title>The First Book</title>
@@ -52,6 +37,6 @@ python [XmlTreeSerializer][xsdata.formats.dataclass.serializers.XmlTreeSerialize
     <pub_date>2001-01-10</pub_date>
     <review>A masterpiece of the fine art of gossiping.</review>
   </book>
-</ns0:books>
+</bk:books>
 
 ```

docs/models/types.md

@@ -25,6 +25,7 @@ collections are also supported.
 | Case           | Example                                                                           |
 | -------------- | --------------------------------------------------------------------------------- |
 | List           | `value: List[str] = field(default_factory=list)`                                  |
+| Optional List  | `value: Optional[List[str]] = field(default=None)`                                |
 | List Union     | `value: List[Union[str, int]] = field(default_factory=list)`                      |
 | Tokens List    | `value: List[str] = field(default_factory=list, metadata={"tokens": True})`       |
 | List of Tokens | `value: List[List[str]] = field(default_factory=list, metadata={"tokens": True})` |
@@ -36,6 +37,7 @@ collections are also supported.
 | Case            | Example                                                                                        |
 | --------------- | ---------------------------------------------------------------------------------------------- |
 | Tuple           | `value: Tuple[str, ...] = field(default_factory=tuple)`                                        |
+| Optional Tuple  | `value: Optional[Tuple[str, ...]] = field(default=None)`                                       |
 | Tuple Union     | `value: Tuple[Union[str, int], ...] = field(default_factory=tuple)`                            |
 | Tokens Tuple    | `value: Tuple[str, ...] = field(default_factory=tuple, metadata={"tokens": True})`             |
 | Tuple of Tokens | `value: Tuple[Tuple[str, ...], ...] = field(default_factory=tuple, metadata={"tokens": True})` |

pyproject.toml

@@ -31,7 +31,7 @@ classifiers = [
 keywords = ["xsd", "wsdl", "schema", "dtd", "binding", "xml", "json", "dataclasses", "generator", "cli"]
 requires-python = ">=3.8"
 dependencies = [
-    "typing-extensions",
+    "typing-extensions>=4.7.0",
 ]
 dynamic = ["version"]
 

tests/codegen/handlers/test_create_compound_fields.py

@@ -118,7 +118,7 @@ def test_group_fields(self):
         )
         expected_res = Restrictions(min_occurs=0, max_occurs=20)
 
-        self.processor.group_fields(target, list(target.attrs))
+        self.processor.group_fields(target, target.attrs.copy())
         self.assertEqual(1, len(target.attrs))
         self.assertEqual(expected, target.attrs[0])
         self.assertEqual(expected_res, target.attrs[0].restrictions)
@@ -134,7 +134,7 @@ def test_group_fields_with_effective_choices_sums_occurs(self):
 
         expected_res = Restrictions(min_occurs=4, max_occurs=6)
 
-        self.processor.group_fields(target, list(target.attrs))
+        self.processor.group_fields(target, target.attrs.copy())
         self.assertEqual(1, len(target.attrs))
         self.assertEqual(expected_res, target.attrs[0].restrictions)
 

tests/codegen/handlers/test_disambiguate_choices.py

@@ -144,7 +144,6 @@ def test_disambiguate_choice_with_circular_ref(self):
 
     def test_find_ambiguous_choices_ignore_wildcards(self):
         """Wildcards are merged."""
-
         attr = AttrFactory.create()
         attr.choices.append(AttrFactory.any())
         attr.choices.append(AttrFactory.any())