You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
haproxy-1.5.18 tcp mode occasionally occur cD message item in haproxy.log, but backend return response quickly, and frontend service also not slow. below is the detail:
#1
Open
timusketeers opened this issue
Oct 12, 2017
· 0 comments
Haproxy-1.5.18 tcp mode occasionally occur cD message item in haproxy.log, but backend return response quickly, and frontend service also not slow. below is the detail:
Our message flow as below:
172.30.49.193 172.30.49.18 172.30.49.29
UI-Console ---> Haproxy ----> Tomcat
alias: (A) (B) (C)
note:
message come from Internet Browser to UI-Console application(marked as A), then UI-Console
redispatch it to Haproxy(marked as B), and Haproxy transmit it to backend service who using
tomcat as web container(marked as C).
We find that occasionally the request will response slow, as a result the UI-Console sometimes can not get response in time, thus can not show the response result on UI (Internet Browser). we analysis our log, and find that service C return response quickly, logs as below:
172.30.49.18 - - [29/Sep/2017:15:43:27 +0200] "GET /network/v1/publicIps HTTP/1.1" 200 3568 462
172.30.49.18 - - [29/Sep/2017:15:43:32 +0200] "GET /network/v1/publicIps HTTP/1.1" 200 3568 446
172.30.49.18 - - [29/Sep/2017:15:43:36 +0200] "GET /network/v1/publicIps HTTP/1.1" 200 3568 387
172.30.49.18 - - [29/Sep/2017:15:43:38 +0200] "GET /network/v1/publicIps HTTP/1.1" 200 3568 417
172.30.49.18 - - [29/Sep/2017:15:43:39 +0200] "GET /network/v1/publicIps HTTP/1.1" 200 3568 422
172.30.49.18 - - [29/Sep/2017:15:43:40 +0200] "GET /network/v1/publicIps HTTP/1.1" 200 3568 433
172.30.49.18 - - [29/Sep/2017:15:43:42 +0200] "GET /network/v1/publicIps HTTP/1.1" 200 3568 429
172.30.49.18 - - [29/Sep/2017:15:43:43 +0200] "GET /network/v1/publicIps HTTP/1.1" 200 3568 479
172.30.49.18 - - [29/Sep/2017:15:43:44 +0200] "GET /network/v1/publicIps HTTP/1.1" 200 3568 434
172.30.49.18 - - [29/Sep/2017:15:43:45 +0200] "GET /network/v1/publicIps HTTP/1.1" 200 3568 405
note:
The last column is the response time of a request.
Then we analysis haproxy.log and find cD terminal code in haproxy.log as below:
Sep 29 15:44:10 localhost haproxy[67984]: 172.30.49.193:58298[29/Sep/2017:15:43:27.244] VPC VPC/VPC-172.30.49.29 1/0/120193 40293 cD 7/1/1/0/0 0/0
It seems the problem occur below Service A and B, considering that TCP packets should be transmitted and switched directly via A to B, the murderer seems B. Is haproxy delay the response? To check it we use tcpdump tool to capture tcp packages.
From captured tcp packages, we find Service A not close the socket immediately when A send packages over, and then after 20s Haproxy send a FIN-ACK package closed the socket. one question is that why haproxy close the socket after 20s, is there any configure item to config it.
Also we discover another problem, when haproxy send packages over to backend service C, sometimes haproxy does not close the socket immediately, then service C will close the socket after 20s(because Tomcat connector's connectTimeout is configed as 20s in server.xml on tomcat). so another question is that why sometimes haproxy does not close the socket immediately when haproxy send packages over to backend service C. is there any configuration could cause the delay? plus, below is our haproxy configuration:
haproxy.cfg:
global
log 127.0.0.1 local2 debug
maxconn 250000
user root
group root
daemon
nbproc 4
defaults
log global
mode tcp
option tcplog
option dontlognull
retries 3
option redispatch
maxconn 4000
timeout connect 30000
timeout client 120000
timeout server 120000
listen AS-Server 172.30.49.153:8443
mode tcp
maxconn 4000
balance roundrobin
server AS-Server-172.30.49.54 172.30.49.54:7443 check inter 500 fall 2 rise 2
server AS-Server-172.30.49.132 172.30.49.132:7443 check inter 500 fall 2 rise 2
listen VPC 172.30.49.152:8443
mode tcp
maxconn 4000
balance roundrobin
server VPC-172.30.49.29 172.30.49.29:7443 check inter 500 fall 2 rise 2
server VPC-172.30.49.30 172.30.49.30:7443 check inter 500 fall 2 rise 2
server VPC-172.30.49.233 172.30.49.233:7443 check inter 500 fall 2 rise 2
The text was updated successfully, but these errors were encountered:
The text was updated successfully, but these errors were encountered: