Simpler HTTP client to send requests #51
Labels
enhancement
Improving existing functionality
I3
Minimal impact
performance
More of something per second
S2
Regular significance
U4
Nothing urgent
We're using viper at the moment and it's just made for a different problem, it manages connection pool to optimize connections to different servers while we only have one target server. It spends some time on URL parsing to see where the request should be sent, but we don't need that. It can have multiple connections to one server, while we don't need that either, we run multiple instances for that.
What we really need as a "worker" is a dumb TCP connection with minimal HTTP 1.1 handling on top of it. One worker is one permanent connection, POST and get response. Simple as that. Unfortunately, I wasn't able to find any real HTTP library for request creation/result parsing and it's very annoying. Of course our requests are rather simple and we can use some static header, but responses need to be parsed properly anyway and it's better to use some library for that.
It'd be nice to rework this part, I think it'll give us more predictable behavior (as we had some problems with workers already) and minimize bench overhead (giving more resources to nodes).
The text was updated successfully, but these errors were encountered: