Jake zelezo na router (UP nebo DP)

[Petr Rehor]

2006/2/27, Petr Bezděk <freebsd at ada-net.cz>:
> Pres den bezne dosahujem kolem 5 000 paketu/s. S lokalnim provozem
> kratkodobe o dost vic. Stavajici zelezo (Athlon 2500+, FE a GE sitovka,
> na GE sitovce nekolik VLANu), hlavne vykon procesoru, prestava stihat.
>
> Kamen urazu je v poctu pravidel firewallu IPFW (cca 3000) pro
> internetovy provoz. V nejhorsim pripade se prochazi az nekolik set
> pravidel. Snizovat pocet prochazenych pravidel uz moc nejde, pouzivam
> skoky, kde to jen jde. Dale v kazdem smeru paket musi projit pres 3
> pipe/queue kvuli shapingu rychlosti. A jeste k tomu se musi provadet NAT
> pomoci natd.

Taky jsem musel postelovat router na vykon, bylo to sice na mensi tok
(# Mbps a 600 pps prumer, 5 Mbps a 1000 pps spicky), ale taky na
pomalejsim stroji (celeron - jestli se dobre pamatuju tak 300 MHz).
Dopadlo to takhle (400 zakazniku, 1200 pravidel):

IPFW pravidla se hned na zacatku rozdeli podle interfejsu a smeru:

05000 12754950 7899297849 skipto 10000 ip from any to any in via fxp0
05010 13047286 7663124721 skipto 20000 ip from any to any out via fxp0
05020 13037914 7681360139 skipto 30000 ip from any to any in via fxp1
05030 12619978 7868841507 skipto 40000 ip from any to any out via fxp1

Z internetu do routeru

10040    24768    2193394 allow tcp from any to me dst-port 22
10050 12729452 7897065558 divert 8668 ip from any to any
10150 12686485 7873627831 allow ip from any to any

Z routeru do Internetu

20000    37912    4609953 allow tcp from me 22 to any
20010 13009374 7658514768 divert 8668 ip from any to any
20140 12953598 7635047943 allow ip from any to any

Od zakazniku do routeru (400 pravidel)

31050       92       5804 skipto 51040 ip from 192.168.254.101 to any

Z routeru k zakaznikum  (400 pravidel)

41050       92      14245 skipto 51045 ip from any to 192.168.254.101

Takhle se jeden zakaznik strka do front (net.inet.ip.fw.one_pass=0)

51040        0          0 queue 41 tcp from any to any tcpflags ack iplen 52
51041     4257     242922 queue 43 ip from any to any
51045        0          0 queue 42 tcp from any to any tcpflags ack iplen 52
51046     1353     181209 queue 44 ip from any to any

A takhle jsou nakonfigurovany jeho fronty

/sbin/ipfw pipe 41 config bw 32k gred 0.002/30/80/0.1 queue 600k
/sbin/ipfw pipe 42 config bw 32k gred 0.002/30/80/0.1 queue 600k
/sbin/ipfw queue 41 config gred 0.002/30/80/0.1 pipe 41 weight 10
/sbin/ipfw queue 42 config gred 0.002/30/80/0.1 pipe 42 weight 10
/sbin/ipfw queue 43 config gred 0.002/30/80/0.1 pipe 41 weight 1 mask
src-ip 0xffffffff
/sbin/ipfw queue 44 config gred 0.002/30/80/0.1 pipe 42 weight 1 mask
dst-ip 0xffffffff

Kdyz ma vic lidi sdilet jednu pipe tak se fronty vyrabej samy pres
parametr mask.

Dalsi optamlizace, ktery dost pomohly:
- zvysit HZ na 1000 (pri prekladu, zobrazit pomoci sysctl kern.clockrate)
  na Athlon 2.5 GHz by slo urcite jit i vys, hodne to pomohlo efektivite
  zpracovani traffic shapingu
- skompilovat kernel s options POLLING a zapnout pomoci sysctl
  kern.polling.enable=1
- zjednodusit traffic shaping, aby paket prochazel jenom jednou frontou

Na routeru navic bezi jenom sshd a sber dat pro rrdtool. Kdyz je klid
(jako treba ted :-) tak to vypada takhle:

CPU states:  5.4% user,  0.0% nice,  5.1% system,  8.9% interrupt, 80.5% idle

ve spickach pada idle na 50%, ale nejsou zadne skoky, kdy by spadl idle k nule.

P.