OpenStreetMap PBF history parser

This Rust program:

• parses OSM history files from Geofabrik
• then creates a postgres osm.history table with the following fields:
  • id (negative IDs are for relations)
  • timestamps
  • changesets
  • first_timestamps
  • last_timestamps
  • users_number
  • versions_number

Configure your rust environment

Tools

Install rust and cargo with:

curl https://sh.rustup.rs -sSf | sh

Useful commands

Documentation

Building the executable file

from the osm/pbf_history_reader subfolder

cargo build

It creates an executable file in ./target/debug/ directory (by default, cargo builds in debug mode)

Building & running the executable file

cargo run

This command builds then run the executable file.

Building for release

cargo build --release

It creates an executable file in ./target/release/ directory. With this option, the program is optimized to run faster.

Cross-compilation

• Install a cross toolchain:

On mac:

brew tap SergioBenitez/osxct
brew install x86_64-unknown-linux-gnu

then specify it in cargo config file .cargo/config.toml:

# .cargo/config.toml
[target.x86_64-unknown-linux-gnu]
linker = "/opt/homebrew/bin/x86_64-unknown-linux-gnu-gcc"

• Specify the compilation target:

rustup target add x86_64-unknown-linux-gnu

• Compile (in release):

cargo build --target x86_64-unknown-linux-gnu -v --release

The executable file is created in osm/pbf_history_reader/target/x86_64-unknown-linux-gnu/release directory

pbf_history_reader needs several environment variables to work:

• DB_NAME
• DB_PORT
• DB_USER

(DB_NAME, DB_USER and DB_PORT have default values defined in osm/pbf_history_reader/src/main.rs file, but you can override them (especially if you are working locally))

• DB_HOST
• DB_PASSWORD
• OSM_ACCOUNT_PASSWORD (for history files)
• OSM_ACCOUNT_USER

They can be provided with env.local or .envrc files (don't forget to pass them to the container in docker-compose.yml file)