Quix legacy SERDES support for SDF #208
Conversation
src/StreamingDataFrames/streamingdataframes/models/serializers/quix.py
Outdated
Show resolved
Hide resolved
src/StreamingDataFrames/streamingdataframes/models/serializers/quix.py
Outdated
Show resolved
Hide resolved
src/StreamingDataFrames/streamingdataframes/models/serializers/quix.py
Outdated
Show resolved
Hide resolved
src/StreamingDataFrames/streamingdataframes/models/serializers/quix.py
Outdated
Show resolved
Hide resolved
src/StreamingDataFrames/streamingdataframes/models/serializers/quix.py
Outdated
Show resolved
Hide resolved
| if as_legacy is None: | ||
| as_legacy = True | ||
| self.as_legacy = as_legacy | ||
| super().__init__(dumps=dumps, dumps_kwargs=dumps_kwargs) | ||
| if self.as_legacy: | ||
| self._legacy = { | ||
| QModelKey.HEADER_NAME: QModelKey.PARAMETERDATA, | ||
| QCodecId.HEADER_NAME: QCodecId.JSON_TYPED, | ||
| } | ||
| else: | ||
| self.extra_headers = { | ||
| QModelKey.HEADER_NAME: QModelKey.TIMESERIESDATA, | ||
| QCodecId.HEADER_NAME: QCodecId.JSON_TYPED, | ||
| } |
There was a problem hiding this comment.
Could you please clarify how does this part play with the descendant classes?
I see it changes self._as_legacy here, but the descendants also change it
Looks like this base class can be replaced by a simple function _as_quix_legacy(value: bytes, model_key: str, codec_id: str) that will wrap the serialized JSON into some envelope
There was a problem hiding this comment.
I have cleaned up the approach (and removed stuff that I thought I cleaned up, sorry, I should have double-checked)...wanted you to feel Halloween in code form apparently, haha!!
While I probably could remove the inheritance if needed, please take a look at my latest commit and see if you are happy with my improved approach...i think it should make a lot more sense now.
Basically, I still have a QuixSerializer base class for the sake of not repeating stuff across both the Timeseries and EventData serializers.
Also, just in case you were wondering what I was doing with the headers, I am basically making sure I do not include the extra_headers if it's in the legacy format (in case that wasn't clear). I want to replicate the original format as closely as possible so the C# client doesn't know any different.
# Legacy Quix SerDes additions Added/changed several things around serdes to accomodate parsing of Quix legacy (aka pre-header) formats (`quixstreams` <0.6.0) in a fully backwards-compatible manner. ## Quix Deserialization - Deserialization now has one universal class, `QuixDeserialzer` - other Quix deserializer classes removed/consolidated - You can now handle multiple quix message types upon consume instead of just 1 - Includes `EventData`, `EventData[]`, and`TimeSeries`/`ParameterData` - Ignore list remains the same - If your app needs access to different messages in different formats, you will need to do some sort of conversion with SDF manually to properly convert them to the same data structure post-deserialization - Legacy formats of `ParameterData`, `EventData`, and `EventData[]` are now supported (<0.6.0) and fully backwards-compatible - "Split" messages remain NOT supported - Messages with multiple "rows" (lists of entries) are still parsed/handled as individual rows (just like the current SDF format). ## Quix Serialization - You can now produce legacy messages by passing the boolean argument `is_legacy` (default `True`) to either the `QuixTimeseriesSerealizer` or `QuixEventsSerealizer`
…iles-into-state-directory Add a state management how to
Legacy Quix SerDes additions
Added/changed several things around serdes to accomodate parsing of Quix legacy (aka pre-header) formats (
quixstreams<0.6.0) in a fully backwards-compatible manner.Quix Deserialization
QuixDeserialzerEventData,EventData[], andTimeSeries/ParameterDataParameterData,EventData, andEventData[]are now supported (<0.6.0) and fully backwards-compatibleQuix Serialization
QuixLegacyTimeseriesSerializerandQuixLegacyEventSerializer