test_marshal.txt

Marshal Marshaller
******************

  >>> from cromlech.marshallers import MarshalMarshaller
  >>> marshaller = MarshalMarshaller()

The `msgpack` marshaller handles binary data :

  >>> marshaller.binary
  True


Straightforward dump/load
=========================

Respecting the conventions set by the `marshal` and `json` native
packages, we use the `dumps` and `loads` to handle bytes.

  >>> data = marshaller.dumps({'test': 'this is a test'})
  >>> data
  b'\xfb\xda\x04test\xfa\x0ethis is a test0'

  >>> marshaller.loads(data)
  {'test': 'this is a test'}


Non-basic types
---------------

Marshal can handle only the most basic python types.

  >>> from datetime import datetime
  >>> mydatetime = datetime(2017, 12, 19, 11, 30, 25)

  >>> import pytest
  >>> with pytest.raises(ValueError) as exc:
  ...     data = marshaller.dumps(mydatetime)

  >>> exc.value
  ValueError('unmarshallable object')


Decorator
---------

We can use the marshaller as a decorator

  >>> @marshaller.wraps
  ... def my_renderer():
  ...     return {"three": 12.145}

  >>> my_renderer
  <function my_renderer at ...>

  >>> my_renderer()
  b'\xfb\xda\x05three\xe7\n\xd7\xa3p=J(@0'

  >>> marshaller.loads(my_renderer())
  {'three': 12.145}


Streams and files
=================

Each marshaller has 4 methods devoted to handling file or stream.
`dump` and `load` work with streams. `dump_to` and `load_from` work
with filesystem paths.


Files
-----

  >>> import os
  >>> path = str(getfixture('tmpdir'))
  >>> filepath = os.path.join(path, 'data.mhr')

  >>> struct = [1, "two", 3.00001]
  >>> marshaller.dump_to(struct, filepath)

The `dump_to` method is in charge of writing the data in the file.
We can load the content of this file using the `load_from` method :

  >>> marshaller.load_from(filepath)
  [1, 'two', 3.00001]

We can check manually the content of the file :

  >>> with open(filepath, 'rb') as fd:
  ...     fd.read()
  b'\xdb\x03\x00\x00\x00\xe9\x01\x00\x00\x00Z\x03twog9b->\x05\x00\x08@'


Streams
--------

The `dump` method is in charge writing the data in the stream.
We can interpret the content of this stream using the `load` method.
We use a binary stream since our marshaller handles binary data.

  >>> from io import BytesIO
  >>> stream = BytesIO()

  >>> marshaller.dump(struct, stream)
  >>> stream.getvalue()
  b'\xdb\x03\x00\x00\x00\xe9\x01\x00\x00\x00Z\x03twog9b->\x05\x00\x08@'

We now return to the offset 0 of the file and we can load it:

  >>> assert stream.seek(0) == 0
  >>> marshaller.load(stream)
  [1, 'two', 3.00001]


Concurrency and timeout
-----------------------

Concurrency is handled via a file lock. We use the cross-os `portalocker`
package.

For our test, let's simulate a lock in place :

  >>> import portalocker
  >>> lock = portalocker.Lock(filepath)
  >>> assert lock.acquire()


When we try to write to this file, we get an `OSError` raised by our
marshaller in response of the lock timeout :

  >>> import pytest
  >>> with pytest.raises(IOError) as timeout:
  ...     marshaller.dump_to(struct, filepath, timeout=0)
  >>> timeout.value
  OSError('Resource is busy and could not be freed.')


Once we release the problematic timeout, we can write :

  >>> lock.release()
  >>> marshaller.dump_to(struct, filepath, timeout=0)


The same happens with the read :

  >>> assert lock.acquire()
  >>> with pytest.raises(IOError) as timeout:
  ...     marshaller.load_from(filepath, timeout=0)
  >>> timeout.value
  OSError('Resource is busy and could not be freed.')

  >>> lock.release()
  >>> assert marshaller.load_from(filepath, timeout=0) == struct

Please note that the read/write are tied by the same lock. You can't read
and write at the same time.