Talk at w3g

Simple OS MasterMap GML/GZ loader

Matt Walker, Astun Technology Ltd. @_walkermatt

https://github.com/AstunTechnology/Loader/

What?

A very simple OS MasterMap loader written in Python that makes use of OGR 1.8. Source data can be in GML or GZip (.gz) format and can be output to any of the formats supported by OGR.

• OS MasterMap - Large scale topographic data for GB (building outlines, curb lines, boundaries etc.)
• GML (Geographic Markup Language) - XML grammar for expressing features and geometry (open standard)
• OGR - Library and utilities for reading and writing vector data (it's awesome)
• Python - Dynamic programming language (also awesome)

Why?

• Need to load OS MasterMap for client projects
• Easier to take the raw data
• Made sense to have an open loader to allow adoption of OS MasterMap under PMA
• OGR had got better at handling GML :)

How?

• Read GZ or GML
• Manipulate it
  • Make life easy on OGR (expose the fid, update SRS, define output schema)
  • Some 'value add' (profiles for specific use cases or software)
• Have OGR do the loading (broad format support)

Who?

• Astun Technology - several clients inc. Eden, Surrey Heath, DORIC Partnership etc.
• Faunalia's QGIS Plugin
• Ordnance Survey
• GeoPlace

National Load Stats

Tim Martin at Ordnance Survey loaded:

• 1297 GZ files equating to 35.9GB
• Running 6 instances in parallel
• Using OGR PGDump driver to output SQL
• Loaded SQL into PostgreSQL via psql utility

Total time 40 hours for national cover :)

What's next?

• Change of name to OS GML Loader (or something equally catchy)
• VectorMap Local - already done but needs some testing
• Classification of features
• Change only Updates
• Repository of styles for rendering (SLD, Mapfile etc.)