Skip to content

HDS Toolbox Details

Jump to bottom Edit New page
jwagenaar edited this page Nov 29, 2012 · 9 revisions

Memory structure layout

HDSManagedData

  • BasePath Path to the base of the data tree. The folder that this path points to should contain the HDSConfig.mat file for the data-tree and the host-object.
  • TreeConst
    1. HostId The unique object ID of the host of the data tree. This unique number indicates which tree is stored in the struct.
    2. SaveID ID number indicating the version that is saved to disk. This number increments with each save action.
    3. isdNr The total number of objects belonging to tree. This number is used to generate object IDS and incremented with each addition of an object to the tree.
    4. Offset Offset of the HDSconfig.mat file with respect to the ‘real’ basePath’.
  • TreeIniteTime DateNumber generated when the first object of tree was registered in memory each session. Used to determine the session time.
  • IsSaving Boolean indicating whether the tree is currently being saved.
  • ObjBools
    1. SaveObj Boolean indicating whether the object has changed
    2. DataChanged Boolean indication whether one of the data properties in the object has changed.
    3. ObjOnDisk Boolean indicating whether the object has previously been saved to disk.
    4. DataLoaded Boolean indicating whether the dataproperties of the object have been loaded in memory.
  • ObjIds
    1. ObjectID Unique number identifying the object that is managed.
    2. ClassID The index in the ‘Classes’ vector that this object belongs to.
    3. ClasIdx Index of the object in the array of the class to which the object belongs.
    4. FileLoc The index in the ‘FileLoc’ array that this object belongs to.
    5. TimeStamp TimeStamp when the object is last used.
    6. ObjectSize Number of bytes the object takes in memory.
  • Classes Cell array of strings with the classes that are currently managed.
  • LocArray Array with all file locations that are currently managed. Each column is a location. If no base offfset: vector is always an uneven number of indeces. If offset, vector is always an even number of indeces. 1.classId of object 2.objectID of parent 3. classID of parent 4. ... etc.
  • RemIds 3 x n array of singles indicating which indices from which files have been removed
    1. Index in FileLoc
    2. ObjectID of removed object.

Object Layout

  • IbjIds (uint32) Vector with ids for various related objects
    1. objId The ID of the current object
    2. objClassId The ID of the class of the current object (1 if not registered).
    3. parentId The ID of the parent of the object (0 if no parent is present)
    4. hostId The ID of the hoste object of the data tree.
    5. treeOffset Depth of the object in the tree.
  • linkProps (cell-array) Property names with linked objects.
  • linkPropIds* (uint32)
    1. classID of linked objects.
    2. Boolean indicating if property is child of object
  • linkLocs (uint32) LinkLocs is an array of locations to links. First index is the objId of the parent object. Last index is the class of the host. General chain: objId-class-objId-class-objId-class.
  • linkIds (uint32)
    1. Object ID
    2. Property ID
    3. Location of link if present
  • dPropSize
    1. n x k array where n is the number of dataproperties. k = [isRepos classId l1 l2 l3 …]
    2. classId: 1)int8 2)char 3)int16 4)int32 5)single 6)double. (same as nc def)
  • dataInMem
    1. Load Indicator : 0: not accessed ; 1: partially loaded from disk ; 2: fully loaded from disk ; 3:changed from disk
    2. min
    3. max
    4. ... continue min and max for each dimension
  • treeNr 0 when not managed, index otherwise
  • saveStatus 0-onDisk, 1-objSave, 2-dataSave, 3-updated.

Bin Files

There is one .bin file per class per folder.

  1. 1 - Boolean indicating changes in children (not fully implemented)
  2. 1 - 1e6 * hostID
  3. 1 - Current Class ID
  4. 1 - Length of folder ID(linkID)
  5. linkID - link ID to folder
  6. 1 - Number of parent object (np)
  7. np - Object Ids of parents
  8. np - File Changed since last update search tables (not fully implemented)
  9. np - Number of object in parent property (Npp)
  10. Npp - Object Ids of objects in Mat-files corresponding to current class ID.

.NC files

Properties in .nc file

  1. var: chIDfile: Vector n*1 long —> sets the file id that contains the channel data for each index.
  2. file names are incremented as follows: *.nc — *.002 — *.003 — *.nnn

Threshold for starting new sub-file

  1. File size at the time of saving new variable > 3GB
  2. Number of dimensions in file exceeds 1000.
  3. Number of variables in file exceeds 2000.

Events when things need to be changed in data-files

  1. During saving. Sub file index can increase. Also, for each variable, save function should find first sub-file that has enough space to fill as it is possible that a file that was full is now empty.
  2. After deleting objects the data files should be updated.
  3. After sorting objects, the names of the variables and their locations should be updated.
Add a custom footer
Add a custom sidebar
Clone this wiki locally