Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BAG extraction app for data-modules #2

Open
wants to merge 23 commits into
base: master
Choose a base branch
from

Conversation

reinvantveer
Copy link

No fully functional module yet, just checking whether it is on the right track. Some issues:

  • Haven't tested with the real BAG yet. But most ingredients are in place
  • Haven't dealt with the address points yet. Will get to that.
  • I don't plan on creating relations between BAG objects just yet, since I don't have a strategy yet. It could involve inserting everyting into Neo4j and run a join in there. It would require either a spatial join, or extraction of the "Verblijfsobjecten" which lists the connection between buildings and addresses.

@reinvantveer
Copy link
Author

Oh yeah, now it's not too late yet, I would propose to do the chaining of dataset module execution through the resolving of promises. I think it would be better than passing in a write object, and let the module be responsible for its own writing. If anything, a configuration object could be passed in, but coupling the data modules to the ingestion process makes for hard debugging and difficult unit testing.

Of course this requires rewriting some of https://github.com/histograph/data

@reinvantveer reinvantveer changed the title Preliminary tests on building extraction Tests on building extraction Feb 8, 2016
@reinvantveer reinvantveer changed the title Tests on building extraction BAG extraction app for data-modules Mar 5, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant