-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Troy Dawson wrote: > Hello, > No, I won't change the default SL network tuning parameters. > But I'm not against making that a SL rpm. I think that would be good > idea if everyone could agree on a batch of settings to put into an rpm. OK - that seems perfectly reasonable. I'm guessing that all that would be required is to put some stuff in /etc/sysctl.conf and run sysctl -p after. Are you able to make conditional decisions when installing an RPM? For example, are you able to make a list of Linux versions that have known bugs for certain things and change parameters based on that? For example: $congestion_alg = "default"; if ($KERNEL_VERSION == 2.6.18) { $congestion_alg = "htcp"; } (.... other tests ...) if ($congestion_alg ne "default") { InstallCongestionAlg($congestion_alg); } where InstallCongestionAlg inserts the line "net.ipv4.tcp_congestion_control = $congestion_alg" into /etc/sysctl.conf Linux TCP autotuning is pretty good these days. All that is really needed is to give it enough room to move. So, for most stuff the following are fine: # increase TCP max buffer size net.core.rmem_max = 4194304 net.core.wmem_max = 4194304 # increase Linux autotuning TCP buffer limits net.ipv4.tcp_rmem = 4096 87380 4194304 net.ipv4.tcp_wmem = 4096 65536 4194304 This allows autotuning to go up to 4MB of window. Some thoughts on this...these are some approximate latencies of interest: 80msec = SF Bay Area to New York (US continental width) 170msec = SF Bay Area to CERN across the Atlantic 280msec = US East Coast (NY) to China (Beijing) across the Pacific With a 4MB TCP window, we get the following per-flow bandwidth limits: 80msec with 4MB max window = ~420Mbps max per-flow bandwidth 170msec with 4MB max window = ~200Mbps max per-flow bandwidth 280msec with 4MB max window = ~120Mbps max per-flow bandwidth Given that GridFTP and friends routinely use parallel streams, I think that these numbers are reasonable for 1G-attached hosts (which are largely the norm in environments without lots of network tuning expertise, which are the target market for such an RPM). Going 4-way to 8-way parallel will fill a 1G pipe with the above config (as far as network stack tuning is concerned). --eli > Troy > > Eli Dart wrote: > Hi all, > > I hope this is the right list for discussion of this topic... > > It appears that Scientific Linux is used by many science communities, > but in particular by the HEP community. The science community often has > significant bulk data movement requirements that are outside the > capabilities of the default network tuning parameters of most Linux > distributions. > > Would the Scientific Linux community consider changing the network > tuning defaults for future releases? > > ESnet maintains a site that explains network performance tuning and how > to increase network performance - please see http://fasterdata.es.net/ > > However, we have recently seen several sites where the first thing that > is needed is to change the network stack parameters so that > high-performance wide-area data transfers are possible. > > Note that with today's TCP autotuning and modern congestion recovery > algorithms, one need not set up particular TCP parameters on a > per-destination basis. One need only give TCP autotuning enough buffer > space to do its work and ensure that a modern congestion recovery > algorithm is used (the default in Linux 2.6 has been cubic for a while, > though 2.6.18 has bugs in cubic that significantly damage performance so > for 2.6.18 one should use htcp instead). > > Please see http://fasterdata.es.net/tuning.html and > http://fasterdata.es.net/TCP-tuning/linux.html for linux-specific > information. > > Thoughts? Comments welcome.... > > Many thanks, > > > --eli > > - -- Eli Dart NOC: (510) 486-7600 ESnet Network Engineering Group (800) 333-7638 Lawrence Berkeley National Laboratory PGP Key fingerprint = C970 F8D3 CFDD 8FFF 5486 343A 2D31 4478 5F82 B2B3 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkpXnVoACgkQLTFEeF+CsrMV5gCgmaK0Qrf8REdLmYj0Yl3Pu451 RsQAoJcTEAUfBcv3RYq8Wv6c2Q9Sc+0H =ZeiT -----END PGP SIGNATURE-----