zriadkavy problem s naviazanim TCP spojenia

[Marian Cerny]

Zdravim,

zacalo to tym, ked som si vsimol, ze sa mi v logoch z casu na cas
objavuje warning:

PHP Warning:  mysql_connect(): Can't connect to MySQL server on 'claire' (4)

Chyba (4) znamena EINT, ktoru mysql pouziva, ak vyprsi timeout na
pripojenie (u mna v PHP 30 sekund).

Spojenie na databazu sa vytvara v PHP skriptoch na front-end serveri
(mohinder, FreeBSD 6.2R, pf) a databaza je na back-end serveri (claire,
FreeBSD 6.3R, pf).

Snazil som sa tento problem analyzovat. Vytvoril som si testovaci
skript, ktory sa snazi opakovane pripajat na MySQL server. Zistil som,
ze bezne trva pripojenie 3-4 ms, ale obcas pripojenie trva niekolko
sekund (1-30 sekund) a niekedy sa spojenie vobec nepodari (timeout po 30
sekundach). Napriklad z 2626 spojeni (kazdu sekundu) trvalo 7 spojeni
viac ako 0.1 sekundy (3, 3, 9, 9, 9, 9, 13, 23 sekund) a jedno spojenie
sa nepodarilo (timeoutovalo po 31 sekundach). Vypadkom konektivity to
nie je - pocas trvania timeoutu je mozne vytvorit nove spojenie.

Tak som pustil tcpdump na backend serveri a zachytil taketo dlhotrvajuce
spojenie a timeoutujuce spojenie. Zistil som, ze k tomuto problemu
dochadza, ked sa pouzije rovnaky zdrojovy port, aky bol pouzity pred
chvilou. 

Ja TCP na tolko nerozumiem, aby som dokazal posudit, ci ten druhy pokus
o spojenie je OK, alebo je v nom nejaky zadrhel. Predpokladam ale, ze
doslo k jednemu z nasledujucich problemov:
1) zdrojovy port este nemal byt pouzity (teda chyba zdroja), alebo
2) na cieli nebolo spojenie korektne ukoncene (chyba cliela).

Prosim o radu. Nizsie este posielam tcp dump zo spominanych spojeni.

Dlhotrvajuce spojenie:

09:45:41.954994 IP mohinder.55451 > claire.3306: S 1444121530:1444121530(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 1764542986 0,sackOK,eol>
09:45:41.955020 IP claire.3306 > mohinder.55451: S 156661697:156661697(0) ack 1444121531 win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 592413577 1764542986,sackOK,eol>
09:45:41.955869 IP mohinder.55451 > claire.3306: . ack 1 win 33304 <nop,nop,timestamp 1764542987 592413577>
09:45:41.956086 IP claire.3306 > mohinder.55451: P 1:57(56) ack 1 win 33304 <nop,nop,timestamp 592413578 1764542987>
09:45:41.957244 IP mohinder.55451 > claire.3306: P 1:74(73) ack 57 win 33304 <nop,nop,timestamp 1764542988 592413578>
09:45:41.957286 IP claire.3306 > mohinder.55451: P 57:68(11) ack 74 win 33304 <nop,nop,timestamp 592413579 1764542988>
09:45:41.958367 IP mohinder.55451 > claire.3306: P 74:79(5) ack 68 win 33304 <nop,nop,timestamp 1764542989 592413579>
09:45:41.958369 IP mohinder.55451 > claire.3306: F 79:79(0) ack 68 win 33304 <nop,nop,timestamp 1764542989 592413579>
09:45:41.958385 IP claire.3306 > mohinder.55451: . ack 80 win 33301 <nop,nop,timestamp 592413580 1764542989>
09:45:41.958406 IP claire.3306 > mohinder.55451: F 68:68(0) ack 80 win 33301 <nop,nop,timestamp 592413580 1764542989>
09:45:41.959367 IP mohinder.55451 > claire.3306: . ack 69 win 33303 <nop,nop,timestamp 1764542990 592413580>
---------------------------------------------------------------------------------------------------------------------
09:46:59.666161 IP mohinder.55451 > claire.3306: S 1522880049:1522880049(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 1764618075 0,sackOK,eol>
09:47:02.777386 IP mohinder.55451 > claire.3306: S 1522880049:1522880049(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 1764621075 0,sackOK,eol>
09:47:06.101989 IP mohinder.55451 > claire.3306: S 1522880049:1522880049(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 1764624275 0,sackOK,eol>
09:47:09.426717 IP mohinder.55451 > claire.3306: S 1522880049:1522880049(0) win 65535 <mss 1460,sackOK,eol>
09:47:12.743325 IP mohinder.55451 > claire.3306: S 1522880049:1522880049(0) win 65535 <mss 1460,sackOK,eol>
09:47:16.056437 IP mohinder.55451 > claire.3306: S 1522880049:1522880049(0) win 65535 <mss 1460,sackOK,eol>
09:47:22.492138 IP mohinder.55451 > claire.3306: S 1522880049:1522880049(0) win 65535 <mss 1460,sackOK,eol>
09:47:22.492176 IP claire.3306 > mohinder.55451: S 3586133292:3586133292(0) ack 1522880050 win 65535 <mss 1460,sackOK,eol>
09:47:22.492762 IP mohinder.55451 > claire.3306: . ack 1 win 65535
09:47:22.493034 IP claire.3306 > mohinder.55451: P 1:57(56) ack 1 win 65535
09:47:22.494012 IP mohinder.55451 > claire.3306: P 1:74(73) ack 57 win 65535
09:47:22.494061 IP claire.3306 > mohinder.55451: P 57:68(11) ack 74 win 65535
09:47:22.495261 IP mohinder.55451 > claire.3306: P 74:79(5) ack 68 win 65535
09:47:22.495264 IP mohinder.55451 > claire.3306: F 79:79(0) ack 68 win 65535
09:47:22.495285 IP claire.3306 > mohinder.55451: . ack 80 win 65535
09:47:22.495322 IP claire.3306 > mohinder.55451: F 68:68(0) ack 80 win 65535
09:47:22.496135 IP mohinder.55451 > claire.3306: . ack 69 win 96

Timeoutujuce spojenie:

10:01:33.329972 IP mohinder.63014 > claire.3306: S 2027815492:2027815492(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 1765461966 0,sackOK,eol>
10:01:33.329998 IP claire.3306 > mohinder.63014: S 205274919:205274919(0) ack 2027815493 win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 593329653 1765461966,sackOK,eol>
10:01:33.330977 IP mohinder.63014 > claire.3306: . ack 1 win 33304 <nop,nop,timestamp 1765461966 593329653>
10:01:33.331127 IP claire.3306 > mohinder.63014: P 1:57(56) ack 1 win 33304 <nop,nop,timestamp 593329654 1765461966>
10:01:33.332221 IP mohinder.63014 > claire.3306: P 1:74(73) ack 57 win 33304 <nop,nop,timestamp 1765461968 593329654>
10:01:33.332264 IP claire.3306 > mohinder.63014: P 57:68(11) ack 74 win 33304 <nop,nop,timestamp 593329655 1765461968>
10:01:33.333220 IP mohinder.63014 > claire.3306: F 79:79(0) ack 68 win 33304 <nop,nop,timestamp 1765461969 593329655>
10:01:33.333223 IP mohinder.63014 > claire.3306: P 74:79(5) ack 68 win 33304 <nop,nop,timestamp 1765461969 593329655>
10:01:33.333240 IP claire.3306 > mohinder.63014: . ack 74 win 33304 <nop,nop,timestamp 593329656 1765461968>
10:01:33.333250 IP claire.3306 > mohinder.63014: . ack 80 win 33301 <nop,nop,timestamp 593329656 1765461969>
10:01:33.333280 IP claire.3306 > mohinder.63014: F 68:68(0) ack 80 win 33301 <nop,nop,timestamp 593329656 1765461969>
10:01:33.334344 IP mohinder.63014 > claire.3306: . ack 69 win 33303 <nop,nop,timestamp 1765461970 593329656>
10:01:33.334348 IP mohinder.63014 > claire.3306: . ack 69 win 33303 <nop,nop,timestamp 1765461970 593329656>
10:01:33.334369 IP claire.3306 > mohinder.63014: R 205274988:205274988(0) win 0
---------------------------------------------------------------------------------------------------------------------
10:02:45.281172 IP mohinder.63014 > claire.3306: S 2100771279:2100771279(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 1765531540 0,sackOK,eol>
10:02:48.389774 IP mohinder.63014 > claire.3306: S 2100771279:2100771279(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 1765534540 0,sackOK,eol>
10:02:51.715501 IP mohinder.63014 > claire.3306: S 2100771279:2100771279(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 1765537740 0,sackOK,eol>
10:02:55.030111 IP mohinder.63014 > claire.3306: S 2100771279:2100771279(0) win 65535 <mss 1460,sackOK,eol>
10:02:58.342096 IP mohinder.63014 > claire.3306: S 2100771279:2100771279(0) win 65535 <mss 1460,sackOK,eol>
10:03:01.649709 IP mohinder.63014 > claire.3306: S 2100771279:2100771279(0) win 65535 <mss 1460,sackOK,eol>
10:03:08.074044 IP mohinder.63014 > claire.3306: S 2100771279:2100771279(0) win 65535 <mss 1460,sackOK,eol>
10:03:16.376682 IP mohinder.63014 > claire.3306: F 2100771280:2100771280(0) win 65535

Marian