Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add country geoip lookup for endpoint addresses #6

Merged
merged 4 commits into from
Apr 9, 2024

Conversation

furmur
Copy link
Contributor

@furmur furmur commented Mar 26, 2024

  • add CLI option 'geoip_path' to specify Country MMDB path
  • add attribute 'endpoint_country' to the 'wireguard_peer_endpoint' metric if 'geoip_path' was specified

* add CLI option 'geoip_path' to specify Country MMDB path
* add attribute 'endpoint_country' to the 'wireguard_peer_endpoint'
  metric if 'geoip_path' was specified
@kbknapp
Copy link
Owner

kbknapp commented Mar 27, 2024

Very cool idea but two general comments:

  • I wonder what the performance cost is of doing lookups on every update tick; perhaps there is a way to amortize that or cache IPs that have already been looked up? Although I haven't tested or looked at it, my worry would be looking up the same peers over and over when their IP hasn't changed and thus the country wouldn't either.
  • It'd probably be better to store the maxminddb_reader inside Metrics instead of having to re-pass the reader to Metrics::update over and over.

@kbknapp
Copy link
Owner

kbknapp commented Mar 27, 2024

Also looks like this PR bumps the MSRV to 1.58.1 which is fine, but we need to set that new value in CI.

@furmur
Copy link
Contributor Author

furmur commented Apr 8, 2024

thank you for reviewing. fixed issues you mentioned.

strace for ./wireguard_exporter -p8080 -l127.0.0.1 -g/opt/geoip/GeoLite2-Country.mmdb:

openat(AT_FDCWD, "/opt/geoip/GeoLite2-Country.mmdb", O_RDONLY|O_CLOEXEC) = 9
statx(9, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=5652356, ...}) = 0
mmap(NULL, 5652480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdc4abdc000
read(9, "\0\0\1\16\30(\0\0\2\16\0301\0\0\3\5\3723\0\0\4\16\0301\0\0\5\5\372\26\0\0"..., 5652356) = 5652356
read(9, "", 32)                         = 0
close(9)

maxminddb crate loads the whole database into the memory and then all lookups are performed within the memory over the binary search tree. so no performance issues here and caching would be excessive, but i can rewrite to use cache if you consider it is necessary.

@kbknapp
Copy link
Owner

kbknapp commented Apr 9, 2024

Ah ok, good to know. It's possible that if we're looking at the same several IPs over-and-over some kind of caching layer would be a better long term approach as the caching layer, even naively implemented as a linear search would be O(n) versus the maxminddb O(log n), however the in this case I think the actual values of n matter and a linear search over ~100 items will absolutely beat a binary search just due to how good CPUs are going through linear items in low cache levels.

However, I think that extra complication is fine to do without at least until we have some actual numbers and know if it'd be worth it or not.

I'm good with this merge, and I really appreciate your taking the time to put this together. I'll get a new version released hopefully tonight but feel free to ping me if you don't see one in the next few days.

@kbknapp kbknapp merged commit 4be2870 into kbknapp:main Apr 9, 2024
4 checks passed
@furmur furmur deleted the feature/mmdb_country_lookup branch April 9, 2024 19:02
@dmitry-sinina
Copy link

@kbknapp Could you release new version with this feature?

@kbknapp
Copy link
Owner

kbknapp commented Apr 15, 2024

@dmitry-sinina thanks for the ping - v0.3.0 is out now on both crates.io and this project's release page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants