Skip to content

Commit

Permalink
migration to github
Browse files Browse the repository at this point in the history
  • Loading branch information
pfoerdie committed Dec 12, 2024
1 parent f016064 commit 8ec8bb0
Show file tree
Hide file tree
Showing 17 changed files with 3,068 additions and 122 deletions.
33 changes: 33 additions & 0 deletions .github/workflows/npm-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# This workflow will run tests using node and then publish a package to GitHub Packages when a release is created
# For more information see: https://docs.github.com/en/actions/publishing-packages/publishing-nodejs-packages

name: Node.js Package

on:
release:
types: [created]

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npm test

publish-npm:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
registry-url: https://registry.npmjs.org/
- run: npm ci
- run: npm publish --access public
env:
NODE_AUTH_TOKEN: ${{secrets.npm_token}}
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
/.*/
/node_modules/
/package-lock.json
/.vscode/
/.idea/
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2024 nicos Research & Development

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
99 changes: 1 addition & 98 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,98 +1 @@
# @nrd/fua.module.space

### Thoughts (SPE):

From the current implementation it is unclear, what the purpose and the functional principle of this module is. It seems
to be a wrapper around a _Map_, called the _graph_. It has some extended methods to store data in this graph and get it
back. Data is meant to be in the format of JSON objects with an _@id_ attribute. Maybe the map is just a substitute for
a real _persistence store_, but it cannot show the mechanics that are involved in updating and retrieving data in a
database. Also, the differentiation between _map_, _weaktypes_ and _weaknodes_ is a bit confusing.

The assumption of storing all data in RAM will just not work. In fact the assumption has to be made, that barely any
data can be held in RAM. In my opinion, _types_ are the only kind of data suited to be held in a _dataset_
rather than being held in a _store_.

Currently, the _model_ methods from _IM_ are responsible for creating and structuring the _nodes_ that should be managed
by the _space_, although I cannot see the connection, where those nodes have the ability to store or retrieve any data,
as they have no reference to the space at all. They can only be added by other methods and would update automatically in
the map if changed, which again is not the case with databases.

Also, the only method for reading actual data from disk is the _load_ function, but it seems that the dataset, as the
main point of interest here, is never used anywhere to create nodes and has no connection to the models. I consider the
space a really complex module, because it plays an important role and has many duties. That is the reason why I do not
want to judge the current implementation, though it has many flaws. To sum up the responsibilities the space has in my
opinion:

- Manage a persistence store. (The store itself could be a manager of stores, but has the same interface.)
- Manage a dataset with permanently loaded types. (Probably read once from disk.)
- Use a standardized format for loading data. (Load scripts should also avoid to make up random terms.)
- Differentiate between data and types.
- Manage the creation of resources/nodes with the appropriate type and data. (Maybe with models.)
- Be the entrypoint for every CRUD operation on resources in the store.
- Implement a transition from convenient resource nodes to/from quads for datasets and stores.
- Cache nodes, merge changes and organize access and TTLs.
- Have a consistent data model to prevent inconsistent behaviour. (Do not accept custom nodes.)
- Be able to update single data points as well as huge datasets. (Minimize redundant operations.)
- Notify resources and other listeners via events on data updates.
- Prevent memory leaks and RAM overload. (Also reduce RAM usage as much as possible.)
- Have documented and easy to understand code without unnecessary extras. (Also a ton of mocha tests.)

All in all, if we want to have a chance of getting the space right in whatever way we want or decide, we must definitely
describe our interface and the reasons for our decision in detail, before we start implementing anything. Otherwise, no
one will be able to rely on this module and maintain it in the future.

#### Questions to answer:

- What is the lifecycle of a resource node?
- On what occasion in the process of the app will they be created?
- Nodes must be created, if not already existing, at least on usage of another application.
- What is the purpose of these nodes, where will they be used?
- The purpose is an easy and convenient usage of the data store.
- Applications like PEP, LDP, AMEC or TRACL should be able to access or update data.
- The usage of these nodes should make data handling more consistent, convenient and reduce errors.
- Will they be cached and what is the condition for creating them?
- If possible, the internal garbage collector should handle removing of not needed nodes.
- Making references weak might be the key to archive garbage collection.
- How long will they stay or how long can they be used?
- What is the condition or mechanism to remove them?
- How is it possible to reference another resource node?
- What data is needed to create them and where does the data come from?
- Data might come directly from the store to enrich the node.
- Data might not be available because the node does not exist yet. In that case the node must be created empty.
- How can a node be manipulated and how are updates synchronized with the data then?
- With `process.nextTick()` the current synchronous changes could be determined and synced after that.
- Synchronizing should only occur on an explizit update call to bundle changes together.
- If no update gets called, the synchronous changes might get discarded.
- The node cannot be manipulated directly, but with methods designed with sync mechanics in mind.
- Manipulation of nodes cannot be synchronous only, because relied on data might get queried late.
- But maybe changes can be tracked synchronous without knowing relied on data and then synced later.
- Does the current state of a node reflect the state of the data and how to tell the difference?
- The nodes need to track the currently persistent state and the changes that needs to be made.
- Does a resource node need access to the space to update its data?
- How to differentiate between actual data nodes, which have a corresponding resource or entry in a database, and
temporary nodes, which are created by the process to eventually make them persist?
- Can resource nodes be created outside the space and later be added?
- I do not think it would be good practice to created nodes outside this model and add them later. It would only
introduce potential for more errors.
- In that case, a solution would not be dependent on spezial nodes with customized functions and could only
implement parsers to/from dataset representation. But this might introduce race conditions which would be
impossible to debug.
- If a resource node has references to a blank node, how will the data be merged on an update?
- If the data has been loaded earlier, the blank node id would be known. In the case that a blank node update on a
node occurs without having created this node beforehand, a merge would be indistinguishable from a new blank node.

## Interface

```ts
interface Space {

};

interface Node {

};

interface Literal {

};
```
# @fua/module.space
2 changes: 1 addition & 1 deletion back/2020/src/module.Space.beta.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
const
construct_space_load = require('./module.Space.beta.load.js'),
{join: joinPath, isAbsolute} = require('path'),
{shaclValidate} = require('@nrd/fua.module.rdf');
{shaclValidate} = require('@fua/module.rdf');

module.exports = () => {

Expand Down
2 changes: 1 addition & 1 deletion back/2020/src/module.Space.beta.load.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
const
{createReadStream} = require('fs'),
{parseStream} = require('@nrd/fua.module.rdf');
{parseStream} = require('@fua/module.rdf');

/**
* This is a constructor for the load method, because it still needs other methods.
Expand Down
41 changes: 41 additions & 0 deletions back/2021/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,46 @@
# module.space

### Questions to answer:

- What is the lifecycle of a resource node?
- On what occasion in the process of the app will they be created?
- Nodes must be created, if not already existing, at least on usage of another application.
- What is the purpose of these nodes, where will they be used?
- The purpose is an easy and convenient usage of the data store.
- Applications like PEP, LDP, AMEC or TRACL should be able to access or update data.
- The usage of these nodes should make data handling more consistent, convenient and reduce errors.
- Will they be cached and what is the condition for creating them?
- If possible, the internal garbage collector should handle removing of not needed nodes.
- Making references weak might be the key to archive garbage collection.
- How long will they stay or how long can they be used?
- What is the condition or mechanism to remove them?
- How is it possible to reference another resource node?
- What data is needed to create them and where does the data come from?
- Data might come directly from the store to enrich the node.
- Data might not be available because the node does not exist yet. In that case the node must be created empty.
- How can a node be manipulated and how are updates synchronized with the data then?
- With `process.nextTick()` the current synchronous changes could be determined and synced after that.
- Synchronizing should only occur on an explizit update call to bundle changes together.
- If no update gets called, the synchronous changes might get discarded.
- The node cannot be manipulated directly, but with methods designed with sync mechanics in mind.
- Manipulation of nodes cannot be synchronous only, because relied on data might get queried late.
- But maybe changes can be tracked synchronous without knowing relied on data and then synced later.
- Does the current state of a node reflect the state of the data and how to tell the difference?
- The nodes need to track the currently persistent state and the changes that needs to be made.
- Does a resource node need access to the space to update its data?
- How to differentiate between actual data nodes, which have a corresponding resource or entry in a database, and
temporary nodes, which are created by the process to eventually make them persist?
- Can resource nodes be created outside the space and later be added?
- I do not think it would be good practice to created nodes outside this model and add them later. It would only
introduce potential for more errors.
- In that case, a solution would not be dependent on spezial nodes with customized functions and could only
implement parsers to/from dataset representation. But this might introduce race conditions which would be
impossible to debug.
- If a resource node has references to a blank node, how will the data be merged on an update?
- If the data has been loaded earlier, the blank node id would be known. In the case that a blank node update on a
node occurs without having created this node beforehand, a merge would be indistinguishable from a new blank node.


## Interface

```ts
Expand Down
2 changes: 1 addition & 1 deletion back/2021/src/module.space.Resource.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
const
_ = require('./module.space.util.js'),
{Dataset} = require('@nrd/fua.module.persistence');
{Dataset} = require('@fua/module.persistence');

class Resource {

Expand Down
6 changes: 3 additions & 3 deletions back/2021/src/module.space.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
const
_ = require('./module.space.util.js'),
{loadDataFiles} = require('@nrd/fua.module.rdf'),
{loadDataFiles} = require('@fua/module.rdf'),
Resource = require('./module.space.Resource.js'),
{DataFactory, Dataset, DataStore} = require('@nrd/fua.module.persistence'),
InmemoryStore = require('@nrd/fua.module.persistence.inmemory');
{DataFactory, Dataset, DataStore} = require('@fua/module.persistence'),
InmemoryStore = require('@fua/module.persistence.inmemory');

/**
* @typedef {Resource} SpaceResource
Expand Down
2 changes: 1 addition & 1 deletion back/2021/src/module.space.util.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
const
_ = exports,
util = require('@nrd/fua.core.util'),
util = require('@fua/core.util'),
MODULE_NAME = 'module.space';

_.assert = new util.Assert(MODULE_NAME);
Expand Down
4 changes: 2 additions & 2 deletions lab/model/util.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
const
_util = require('@nrd/fua.core.util'),
_space = require('@nrd/fua.module.space');
_util = require('@fua/core.util'),
_space = require('@fua/module.space');

exports = module.exports = {
..._util,
Expand Down
Loading

0 comments on commit 8ec8bb0

Please sign in to comment.