test_msgpack.txt

Msgpack Marshaller
******************

  >>> from cromlech.marshallers import MsgpackMarshaller
  >>> marshaller = MsgpackMarshaller()

The `msgpack` marshaller handles binary data :

  >>> marshaller.binary
  True


Straightforward dump/load
=========================

Respecting the conventions set by the `marshal` and `json` native
packages, we use the `dumps` and `loads` to handle string or bytes.

  >>> marshaller.dumps([1, "two", 3])
  b'\x93\x01\xa3two\x03'

  >>> struct = {
  ... "Accept-Language": "en-US,en;q=0.8",
  ... "Host": "headers.jsontest.com",
  ... "Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.3",
  ... "Accept": "text/html,application/xhtml+xml;q=0.9,*/*;q=0.8"
  ... }

  >>> data =  marshaller.dumps(struct)
  >>> assert marshaller.loads(data) == struct


Non-basic types
---------------

As for now, the `msgpack` marshaller can convert datetime and date.

  >>> from datetime import datetime
  >>> mydate = datetime(2017, 12, 25)
  >>> mydatetime = datetime(2017, 12, 19, 11, 30, 25)

  >>> structure = {
  ...    'date': mydate,
  ...    'datetime': mydatetime,
  ... }

  >>> data = marshaller.dumps(structure)
  >>> assert marshaller.loads(data) == {
  ...     'datetime': datetime(2017, 12, 19, 11, 30, 25),
  ...     'date': datetime(2017, 12, 25, 0, 0)
  ... }


Decorator
---------

We can use the marshaller as a decorator

  >>> @marshaller.wraps
  ... def my_renderer():
  ...     return [1, 2, "three", 12.145]

  >>> my_renderer
  <function my_renderer at ...>

  >>> my_renderer()
  b'\x94\x01\x02\xa5three\xcb@(J=p\xa3\xd7\n'

  >>> marshaller.loads(my_renderer())
  [1, 2, 'three', 12.145]


Streams and files
=================

Each marshaller has 4 methods devoted to handling file or stream.
`dump` and `load` work with streams. `dump_to` and `load_from` work
with filesystem paths.


Files
-----

  >>> import os
  >>> path = str(getfixture('tmpdir'))
  >>> filepath = os.path.join(path, 'data.msg')

  >>> struct = [1, "two", 3.00001]
  >>> marshaller.dump_to(struct, filepath)

The `dump_to` method is in charge of writing the data in the file.
We can load the content of this file using the `load_from` method :

  >>> marshaller.load_from(filepath)
  [1, 'two', 3.00001]

We can check manually the content of the file :

  >>> with open(filepath, 'rb') as fd:
  ...     fd.read()
  b'\x93\x01\xa3two\xcb@\x08\x00\x05>-b9'


Streams
--------

The `dump` method is in charge writing the data in the stream.
We can interpret the content of this stream using the `load` method.
We use a binary stream since our marshaller handles binary data.

  >>> from io import BytesIO
  >>> stream = BytesIO()

  >>> marshaller.dump(struct, stream)
  >>> stream.getvalue()
  b'\x93\x01\xa3two\xcb@\x08\x00\x05>-b9'

We now return to the offset 0 of the file and we can load it:

  >>> assert stream.seek(0) == 0
  >>> marshaller.load(stream)
  [1, 'two', 3.00001]


Concurrency and timeout
-----------------------

Concurrency is handled via a file lock. We use the cross-os `portalocker`
package.

For our test, let's simulate a lock in place :

  >>> import portalocker
  >>> lock = portalocker.Lock(filepath)
  >>> assert lock.acquire()


When we try to write to this file, we get an `OSError` raised by our
marshaller in response of the lock timeout :

  >>> import pytest
  >>> with pytest.raises(IOError) as timeout:
  ...     marshaller.dump_to(struct, filepath, timeout=0)
  >>> timeout.value
  OSError('Resource is busy and could not be freed.')


Once we release the problematic timeout, we can write :

  >>> lock.release()
  >>> marshaller.dump_to(struct, filepath, timeout=0)


The same happens with the read :

  >>> assert lock.acquire()
  >>> with pytest.raises(IOError) as timeout:
  ...     marshaller.load_from(filepath, timeout=0)
  >>> timeout.value
  OSError('Resource is busy and could not be freed.')

  >>> lock.release()
  >>> assert marshaller.load_from(filepath, timeout=0) == struct

Please note that the read/write are tied by the same lock. You can't read
and write at the same time.