Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generating .json from planet.osm.pbf results in process being 'killed'. #129

Open
NMC92 opened this issue Nov 1, 2021 · 1 comment
Open

Comments

@NMC92
Copy link

NMC92 commented Nov 1, 2021

When I try to generate a .json file based on either unfiltered or filtered by boundaries/admin_level planet.osm.pbf the process gets 'killed' after a while. I'll leave more info below and attached in the image.

I'm using WSL2 and the steps I'm doing are just:
1 - Get planet.osm.pbf file(in my case I got it from the torrent version);
2 - Filter pbf file by either boundary or admin_level, which would result in a considerably smaller file. Additionally I'm also filtering some other info out. The commands I'm using are these:
osmium tags-filter planet-latest.osm.pbf r/admin_level --overwrite -o planet-admin.osm.pbf
osmconvert planet-admin.osm.pbf -o=planet-admin.osm
osmfilter planet-admin.osm --drop-tags="barrier= building= highway= landuse= office= place= waterway=" -o=planet-admin-noplace.osm
osmconvert planet-admin-noplace.osm -o=planet-admin-noplace.osm.pbf;

3 - Build cosmogony with command:
cargo run --release -- generate -i ../../planet-admin-noplace.osm.pbf
It's in this command that the process is getting "killed".

error_gen_json_from_planetosmpbf

(Note: This happens with unfiltered planet.osm.pbf as well. I also don't know if this could be related to #118 ...)

@NMC92
Copy link
Author

NMC92 commented Nov 2, 2021

I've been researching, and I found that my system might be killing the process after a possible OOM Panic. I've tried to set overcommit memory to 2, meaning it will try not to do it. I've also changed vm.oom-kill to 0. Let's see what'll happen now...

Edit: I was able to fully process and import unfiltered Poland-latest .osm.pbf abd run the cosmogony-explorer with it without the process being killed. I was also able to do the same but to asia osm.pbf file filtered by r/admin_level and without additional unnecessary tags(like in the command above), but when I tried to do the same with either planet or europe osm.pbf files it gets killed. I thought it could be due to the docker OOM Killswitch so I turned docker off, but now it still kills it(with RAM set to 8GB and CPU to 3/6 in the wslconfig file). I'll try with CPU 6/6 and RAM 14/16.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant