Loading error when using recorded .log file

[mikihiroikura]

Issue

When I tried to load .log file recorded by following this procedure camera_dumping.md, the reshape error of numpy.array happened.
The main reason seems to be (importIitYarp.py:723) outDict = decodeEvents(np.reshape(np.concatenate(eventsToDecode), (-1, 2)))

Environment

bimvee=1.0.23

Error message

---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
File ~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:622, in importIitYarpDataLog(**kwargs)
    [621](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:621) with open(kwargs['filePathOrName'], 'r') as inFile:
--> [622](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:622)     content = inFile.readline()  # Look at first line of file
    [623](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:623)     patternForVicon = re.compile('(\d+) (\d+\.\d+) \((.*)\)')

File /usr/lib/python3.10/codecs.py:322, in BufferedIncrementalDecoder.decode(self, input, final)
    [321](https://file+.vscode-resource.vscode-cdn.net/usr/lib/python3.10/codecs.py:321) data = self.buffer + input
--> [322](https://file+.vscode-resource.vscode-cdn.net/usr/lib/python3.10/codecs.py:322) (result, consumed) = self._buffer_decode(data, self.errors, final)
    [323](https://file+.vscode-resource.vscode-cdn.net/usr/lib/python3.10/codecs.py:323) # keep undecoded input until the next call

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 37: invalid continuation byte

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In[5], [line 2](vscode-notebook-cell:?execution_count=5&line=2)
      [1](vscode-notebook-cell:?execution_count=5&line=1) log_path: pathlib.Path = pathlib.Path('/home/mikura-iit.local/data/recorded_data/calib_board_tcp/ATIS/data.log')
----> [2](vscode-notebook-cell:?execution_count=5&line=2) importAe(filePathOrName=log_path.__str__(), template=None)

File ~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importAe.py:143, in importAe(**kwargs)
    [141](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importAe.py:141)         kwargs['filePathOrName'] = os.path.dirname(kwargs['filePathOrName'])
    [142](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importAe.py:142)     from .importIitYarp import importIitYarp
--> [143](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importAe.py:143)     importedData = importIitYarp(**kwargs)
    [144](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importAe.py:144) elif fileFormat in ['rpgdvsros', 'rosbag', 'rpg', 'ros', 'bag', 'rpgdvs']:
    [145](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importAe.py:145)     from .importRpgDvsRos import importRpgDvsRos

File ~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:833, in importIitYarp(**kwargs)
    [831](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:831) def importIitYarp(**kwargs):
    [832](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:832)     """Import data in IIT Yarp format."""
--> [833](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:833)     importedDicts = importIitYarpRecursive(**kwargs)
    [834](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:834)     if kwargs.get('zeroTime', kwargs.get('zeroTimestamps', True)):
    [835](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:835)         # Optional: start the timestamps at zero for the first event
    [836](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:836)         # This is done collectively for all the concurrent imports
    [837](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:837)         rezeroTimestampsForImportedDicts(importedDicts)

File ~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:798, in importIitYarpRecursive(**kwargs)
    [796](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:796)     importedDicts.append(importIitYarpBinaryDataLog(**kwargs))
    [797](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:797) if file == 'data.log':
--> [798](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:798)     importedDicts.append(importIitYarpDataLog(**kwargs))
    [799](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:799) if file == 'info.log':
    [800](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:800)     tsOffset = importIitYarpInfoLog(**kwargs)

File ~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:723, in importIitYarpDataLog(**kwargs)
    [721](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:721)     timestamps += [float(ts) / 0.000001]*len(bitStrings)
    [722](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:722) if with_ts:
--> [723](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:723)     outDict = decodeEvents(np.reshape(np.concatenate(eventsToDecode), (-1, 2)))
    [724](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:724) else:
    [725](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/bimvee/importIitYarp.py:725)     outDict = decodeEvents(np.vstack((timestamps, np.concatenate(eventsToDecode))).swapaxes(0, 1).astype(int))

File ~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:328, in reshape(a, shape, order, newshape, copy)
    [326](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:326) if copy is not None:
    [327](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:327)     return _wrapfunc(a, 'reshape', shape, order=order, copy=copy)
--> [328](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:328) return _wrapfunc(a, 'reshape', shape, order=order)

File ~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:57, in _wrapfunc(obj, method, *args, **kwds)
     [54](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:54)     return _wrapit(obj, method, *args, **kwds)
     [56](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:56) try:
---> [57](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:57)     return bound(*args, **kwds)
     [58](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:58) except TypeError:
     [59](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:59)     # A TypeError occurs if the object does have such a method in its
     [60](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:60)     # class, but its signature is not identical to that of NumPy's. This
   (...)
     [64](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:64)     # Call _wrapit from within the except clause to ensure a potential
     [65](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:65)     # exception has a traceback chain.
     [66](https://file+.vscode-resource.vscode-cdn.net/home/mikura-iit.local/Github/Line_Segment_Analysis/notebook/~/Github/Line_Segment_Analysis/.venv/lib/python3.10/site-packages/numpy/_core/fromnumeric.py:66)     return _wrapit(obj, method, *args, **kwds)

ValueError: cannot reshape array of size 2837469 into shape (2)

Test code

from bimvee.importAe import importAe
import pathlib
import numpy as np
log_path: pathlib.Path = pathlib.Path('/home/mikura-iit.local/data/recorded_data/calib_board_tcp/ATIS/data.log')
importAe(filePathOrName=log_path.__str__(), template=None)

Test data

OneDrive