Offload TLS negotiation to I/O threads #1338
Conversation
Signed-off-by: Uri Yagelnik <[email protected]>
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## unstable #1338 +/- ##
============================================
+ Coverage 70.71% 70.75% +0.03%
============================================
Files 115 117 +2
Lines 63159 63348 +189
============================================
+ Hits 44666 44824 +158
- Misses 18493 18524 +31
|
There was a problem hiding this comment.
This implementation basically breaks the connection abstraction, since it has the TLS implementation calling functions related to IO threading (which is supposed to be agnostic to the connection type). I was sort of expecting we would offload the event handler.
src/io_threads.c
Outdated
| if (server.io_threads_num <= 1) { | ||
| return C_ERR; | ||
| } | ||
|
|
||
| if (!(conn->flags & CONN_FLAG_CLIENT)) { | ||
| return C_ERR; | ||
| } | ||
|
|
||
| client *c = connGetPrivateData(conn); | ||
| if (c->io_read_state != CLIENT_IDLE) { | ||
| return C_OK; | ||
| } | ||
|
|
||
| if (server.active_io_threads_num <= 1) { | ||
| return C_ERR; | ||
| } |
There was a problem hiding this comment.
| if (server.io_threads_num <= 1) { | |
| return C_ERR; | |
| } | |
| if (!(conn->flags & CONN_FLAG_CLIENT)) { | |
| return C_ERR; | |
| } | |
| client *c = connGetPrivateData(conn); | |
| if (c->io_read_state != CLIENT_IDLE) { | |
| return C_OK; | |
| } | |
| if (server.active_io_threads_num <= 1) { | |
| return C_ERR; | |
| } | |
| if (server.active_io_threads_num <= 1) { | |
| return C_ERR; | |
| } | |
| if (!(conn->flags & CONN_FLAG_CLIENT)) { | |
| return C_ERR; | |
| } | |
| client *c = connGetPrivateData(conn); | |
| if (c->io_read_state != CLIENT_IDLE) { | |
| return C_OK; | |
| } |
There was a problem hiding this comment.
It seems everywhere else we just check for active threads.
There was a problem hiding this comment.
Consider a scenario where the main thread sends a job to the IO thread. After the IO thread completes the job, the main thread deactivates all threads. In this case, we want to ensure the main thread processes the returned job from the IO thread before performing an accept operation even if the io threads are not active.
To achieve this:
First, check if IO threads are enabled at all (less expensive check)
Then, verify the read_state is not idle
Finally, check the active state
Unlike read/write where we anyway check for the read/write states, the accept flow operates at the connection layer rather than the client layer, so we can't check the read state there.
Signed-off-by: Uri Yagelnik <[email protected]>
We still don't check in any point for the connection type, since the TLS code calls the IO threads to offload the negotiation with a supplied callback, not the other way around. Maybe we can rename it to 'accept' instead of 'tls_negotiate' to be more abstract.
Not sure I get this, would you please elaborate. |
…abstraction Signed-off-by: Uri Yagelnik <[email protected]>
uriyage
force-pushed
the
tls_negotiation_offload
branch
from
cd86e38
to
c0cf818
Compare
Signed-off-by: Uri Yagelnik <[email protected]>
Signed-off-by: Uri Yagelnik <[email protected]>
TLS Negotiation Offloading to I/O Threads
Overview
This PR introduces the ability to offload TLS handshake negotiations to I/O threads, significantly improving performance under high TLS connection loads.
Key Changes
Performance Impact
Testing with 650 clients with SET commands and 160 new TLS connections per second in the background:
Throughput Impact of new TLS connections
New Connection Rate
Implementation Details
Main Thread:
I/O Thread:
Related issue:#761