ZFS + Samba performance

[Jan Dušátko]

Ahoj,
Po upgrade samba z predchozi verze mi drasticky poklesla vykonnost serveru.
Puvodnich dosahovanych 80MB/s je v nenavratnu,  momentalni rychlost je okolo
2MB/s. Problem neni na filesystemu, CPU nebo pameti, ale nekde v Sambe.

# diskinfo -tv /dev/da0
/dev/da0
        512             # sectorsize
        14002857967616  # mediasize in bytes (12T)
        27349331968     # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        1702417         # Cylinders according to firmware.
        255             # Heads according to firmware.
        63              # Sectors according to firmware.
        4152615147574235        # Disk ident.

Seek times:
        Full stroke:      250 iter in   4.833226 sec =   19.333 msec
        Half stroke:      250 iter in   4.454185 sec =   17.817 msec
        Quarter stroke:   500 iter in   3.755233 sec =    7.510 msec
        Short forward:    400 iter in   1.323541 sec =    3.309 msec
        Short backward:   400 iter in   3.371633 sec =    8.429 msec
        Seq outer:       2048 iter in   0.599982 sec =    0.293 msec
        Seq inner:       2048 iter in   0.335802 sec =    0.164 msec
Transfer rates:
        outside:       102400 kbytes in   0.254936 sec =   401669 kbytes/sec
        middle:        102400 kbytes in   0.540097 sec =   189596 kbytes/sec
        inside:        102400 kbytes in   0.520260 sec =   196825 kbytes/sec

Nebo i jinym zpusobem primo nad ZFS:
# dd if=/dev/zero of=test2 bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 5.904555 secs (181 849 743 bytes/sec)
# dd if=test2 of=/dev/zero bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 2.850506 secs (376 684 630 bytes/sec)

Pokles vykonnosti pri zapisu je vzhledem k RAID5 ocekavatelny (4 operace na
zapis jednoho bloku oproti jedne operaci na cteni jednoho bloku se musi
projevit). Ale i tak je system schopny plne saturovat sitovy interface. I
pri provozu mam priblizne 1GB ze 4GB instalovanych volny, zadne swapovani.
Prikladem vytizeni rozhrani je napr. FTP prenos, kde se dokazu dostat na
123MB/s-124MB/s. Pri tom stroj stiha bez problemu pracovat jako mail/web
server. Vzhledem k nizke efektivite SMB protokolu bych se spokojil s
nejakymi 75%, tedy okolo 80MB/s-90MB/s.
Zkousim ladit sambu co to jde, bohuzel spise to nejde nez jde. Prikladam
seznam zmeny konfiguracniho souboru ze sekce [global]:
...
        bind interfaces only = yes
        prefered master = yes
        domain master = yes
        local master = yes
        domain logons = yes
        wins support = yes
        wins proxy = yes
        oplock = no
        level2 oplocks = no
        client ntlmv2 auth = yes
        max protocol = smb2
        name resolve order = host wins lmhosts bcast
...
        max log size = 1000
        log file = /var/log/samba/log.%m
        syslog = 0
...
        security = user
        null passwords = yes
        passdb backend = tdbsam
        inherit acls = yes
...
        socket options = SO_KEEPALIVE TCP_NODELAY IPTOS_LOWDELAY
IPTOS_THROUGHPUT SO_SNDBUF=131072 SO_RCVBUF=131072
        min receivefile size = 16384
        aio read size = 16384
#       aio write size = 16384
        aio write behind = true
        read raw = yes
        write raw = no
        use sendfile = true
...

Verze Samby (instalovane package):
# pkg_info | grep samba
samba34-libsmbclient-3.4.14 Shared libs from the samba package
samba35-3.5.11      A free SMB and CIFS client and server for UNIX

Napada vas nekoho nejaky trik, co bych mohl udelat pro zvyseni vykonnosti?

Diky

Honza