-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Troy Dawson wrote:
> Hello,
> No, I won't change the default SL network tuning parameters.
> But I'm not against making that a SL rpm.  I think that would be good
> idea if everyone could agree on a batch of settings to put into an rpm.

OK - that seems perfectly reasonable.

I'm guessing that all that would be required is to put some stuff in
/etc/sysctl.conf and run sysctl -p after.

Are you able to make conditional decisions when installing an RPM?  For
example, are you able to make a list of Linux versions that have known
bugs for certain things and change parameters based on that?  For example:

$congestion_alg = "default";
if ($KERNEL_VERSION == 2.6.18) {
	$congestion_alg = "htcp";
}
(.... other tests ...)
if ($congestion_alg ne "default") {
	InstallCongestionAlg($congestion_alg);
}

where InstallCongestionAlg inserts the line
"net.ipv4.tcp_congestion_control = $congestion_alg" into /etc/sysctl.conf

Linux TCP autotuning is pretty good these days.  All that is really
needed is to give it enough room to move.  So, for most stuff the
following are fine:

# increase TCP max buffer size
net.core.rmem_max = 4194304
net.core.wmem_max = 4194304
# increase Linux autotuning TCP buffer limits
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 65536 4194304

This allows autotuning to go up to 4MB of window.  Some thoughts on
this...these are some approximate latencies of interest:

80msec = SF Bay Area to New York (US continental width)
170msec = SF Bay Area to CERN across the Atlantic
280msec = US East Coast (NY) to China (Beijing) across the Pacific

With a 4MB TCP window, we get the following per-flow bandwidth limits:

80msec with 4MB max window = ~420Mbps max per-flow bandwidth
170msec with 4MB max window = ~200Mbps max per-flow bandwidth
280msec with 4MB max window = ~120Mbps max per-flow bandwidth

Given that GridFTP and friends routinely use parallel streams, I think
that these numbers are reasonable for 1G-attached hosts (which are
largely the norm in environments without lots of network tuning
expertise, which are the target market for such an RPM).  Going 4-way to
8-way parallel will fill a 1G pipe with the above config (as far as
network stack tuning is concerned).

		--eli


> Troy
> 
> Eli Dart wrote:
> Hi all,
> 
> I hope this is the right list for discussion of this topic...
> 
> It appears that Scientific Linux is used by many science communities,
> but in particular by the HEP community.  The science community often has
> significant bulk data movement requirements that are outside the
> capabilities of the default network tuning parameters of most Linux
> distributions.
> 
> Would the Scientific Linux community consider changing the network
> tuning defaults for future releases?
> 
> ESnet maintains a site that explains network performance tuning and how
> to increase network performance - please see http://fasterdata.es.net/
> 
> However, we have recently seen several sites where the first thing that
> is needed is to change the network stack parameters so that
> high-performance wide-area data transfers are possible.
> 
> Note that with today's TCP autotuning and modern congestion recovery
> algorithms, one need not set up particular TCP parameters on a
> per-destination basis.  One need only give TCP autotuning enough buffer
> space to do its work and ensure that a modern congestion recovery
> algorithm is used (the default in Linux 2.6 has been cubic for a while,
> though 2.6.18 has bugs in cubic that significantly damage performance so
> for 2.6.18 one should use htcp instead).
> 
> Please see http://fasterdata.es.net/tuning.html and
> http://fasterdata.es.net/TCP-tuning/linux.html for linux-specific
> information.
> 
> Thoughts?  Comments welcome....
> 
> Many thanks,
> 
> 
>         --eli
> 
> 

- --
Eli Dart                                            NOC: (510) 486-7600
ESnet Network Engineering Group                          (800) 333-7638
Lawrence Berkeley National Laboratory
PGP Key fingerprint = C970 F8D3 CFDD 8FFF 5486 343A 2D31 4478 5F82 B2B3
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpXnVoACgkQLTFEeF+CsrMV5gCgmaK0Qrf8REdLmYj0Yl3Pu451
RsQAoJcTEAUfBcv3RYq8Wv6c2Q9Sc+0H
=ZeiT
-----END PGP SIGNATURE-----