[tech] Murasoi dropouts

[Nick Bannon]

On Fri, Jun 26, 2015 at 09:39:12AM +0800, James Andrewartha wrote:
[...]
> Some quick googling returns 
> http://blog.bradiceanu.net/2010/11/28/netdev-watchdog-eth0-transmit-timed-out/ 
> which suggests building a more recent version of the e1000 driver (which 
> is 8.0.35 vs 3.16's 7.3.21-k8-NAPI) and setting in modprobe.d:
> options e1000 ignore_64bit_dma=1

It seemed to start happening (often) when we went from Linux
3.2.0-4-amd64 to Linux 3.16.0-4-amd64 . Linux 4.1 still seems to have
e1000 7.3.21-k8-NAPI , so no ignore_64bit_dma option.

On Wed, Sep 23, 2015 at 11:59:18AM +0800, David Adam wrote:
> Those cards didn't fit, but this worked:
> # ethtool -K eth1 tso off
> Performance still seems OK.
> [DAA]

eth0 (the non-uplink interface) wasn't hanging as often, but it needs
it too, now set for eth0, eth1 in murasoi:/etc/network/interfaces :
auto eth0
iface eth0 inet stati
	up ethtool -K eth0 tso off

With TSO enabled, short iperf's like "iperf -n5M -c motsugo" or "iperf
-t1 -c motsugo" from murasoi seems to trigger a Hang pretty easily.

With TSO disabled, performance is still most of a gigabit with default
MSS or with a small MSS.

motsugo> iperf -m -c murasoi -d
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to murasoi, TCP port 5001
TCP window size: 93.5 KByte (default)
------------------------------------------------------------
[  5] local 130.95.13.7 port 32852 connected with 130.95.13.1 port 5001
[  4] local 130.95.13.7 port 5001 connected with 130.95.13.1 port 59797
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec  1.09 GBytes   934 Mbits/sec
[  5] MSS size 1448 bytes (MTU 1500 bytes, ethernet)
[  4]  0.0-10.0 sec  1.09 GBytes   933 Mbits/sec
[  4] MSS size 1448 bytes (MTU 1500 bytes, ethernet)
motsugo> iperf -M536 -c murasoi -d
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to murasoi, TCP port 5001
TCP window size:  207 KByte (default)
------------------------------------------------------------
[  5] local 130.95.13.7 port 32861 connected with 130.95.13.1 port 5001
[  4] local 130.95.13.7 port 5001 connected with 130.95.13.1 port 59798
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   914 MBytes   766 Mbits/sec
[  4]  0.0-10.0 sec   924 MBytes   773 Mbits/sec

...and for routing machineroom <-> clubroom (I wonder how it ever gets a
10 seconds test above 1Gbps/2?):

motsugo> iperf  -c clownfish -d
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to clownfish, TCP port 5001
TCP window size:  357 KByte (default)
------------------------------------------------------------
[  5] local 130.95.13.7 port 57023 connected with 130.95.13.89 port 5001
[  4] local 130.95.13.7 port 5001 connected with 130.95.13.89 port 47360
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   734 MBytes   615 Mbits/sec
[  4]  0.0-10.0 sec   368 MBytes   308 Mbits/sec

[  5] local 130.95.13.7 port 57158 connected with 130.95.13.89 port 5001
[  4] local 130.95.13.7 port 5001 connected with 130.95.13.89 port 47376
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   412 MBytes   345 Mbits/sec
[  4]  0.0-10.0 sec   418 MBytes   350 Mbits/sec

Nick.

-- 
   Nick Bannon   | "I made this letter longer than usual because
nick-sig at rcpt.to | I lack the time to make it shorter." - Pascal