Skip to content

Commit

Permalink
Quix legacy SERDES support for SDF (#208)
Browse files Browse the repository at this point in the history
# Legacy Quix SerDes additions
Added/changed several things around serdes to accomodate parsing of Quix legacy (aka pre-header) formats (`quixstreams` <0.6.0) in a fully backwards-compatible manner.

## Quix Deserialization 
- Deserialization now has one universal class, `QuixDeserialzer`
  - other Quix deserializer classes removed/consolidated
- You can now handle multiple quix message types upon consume instead of just 1
  - Includes `EventData`, `EventData[]`,  and`TimeSeries`/`ParameterData` 
  - Ignore list remains the same
  - If your app needs access to different messages in different formats, you will need to do some sort of conversion with SDF manually to properly convert them to the same data structure post-deserialization
- Legacy formats of `ParameterData`, `EventData`, and `EventData[]` are now supported (<0.6.0) and fully backwards-compatible
  - "Split" messages remain NOT supported
  - Messages with multiple "rows" (lists of entries) are still parsed/handled as individual rows (just like the current SDF format).
  
  
## Quix Serialization
- You can now produce legacy messages by passing the boolean argument `is_legacy` (default `True`) to either the `QuixTimeseriesSerealizer` or `QuixEventsSerealizer`
  • Loading branch information
tim-quix authored and daniil-quix committed Nov 7, 2023
1 parent 0e3484a commit c6037b7
Show file tree
Hide file tree
Showing 8 changed files with 477 additions and 276 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,5 @@
from .quix import (
QuixEventsSerializer,
QuixTimeseriesSerializer,
QuixTimeseriesDeserializer,
QuixEventsDeserializer,
QuixDeserializer,
)
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,16 @@
class JSONSerializer(Serializer):
def __init__(
self,
dumps: Callable[[Any, None], Union[str, bytes]] = json.dumps,
dumps: Optional[Callable[[Any, None], Union[str, bytes]]] = None,
dumps_kwargs: Optional[Mapping] = None,
):
"""
Serializer that returns data in json format.
:param dumps: a function to serialize objects to json. Default - `json.dumps`
:param dumps_kwargs: a dict with keyword arguments for `dumps()` function.
"""
self._dumps = dumps
self._dumps_kwargs = dumps_kwargs or {}
self._dumps = dumps or json.dumps
self._dumps_kwargs = {"separators": (",", ":"), **(dumps_kwargs or {})}

def __call__(self, value: Any, ctx: SerializationContext) -> Union[str, bytes]:
return self._to_json(value)
Expand All @@ -35,9 +35,9 @@ class JSONDeserializer(Deserializer):
def __init__(
self,
column_name: Optional[str] = None,
loads: Callable[
[Union[str, bytes, bytearray], None], Union[List, Mapping]
] = json.loads,
loads: Optional[
Callable[[Union[str, bytes, bytearray], None], Union[List, Mapping]]
] = None,
loads_kwargs: Optional[Mapping] = None,
):
"""
Expand All @@ -49,7 +49,7 @@ def __init__(
:param loads_kwargs: dict with named arguments for `loads` function.
"""
super().__init__(column_name=column_name)
self._loads = loads
self._loads = loads or json.loads
self._loads_kwargs = loads_kwargs or {}

def __call__(
Expand Down
Loading

0 comments on commit c6037b7

Please sign in to comment.