-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Brett Viren wrote:
> Eli Dart <[log in to unmask]> writes:
> 
>> However, the high-order bit from my perspective is to remove the barrier
>> to high-performance network utilization that default stack tuning
>> parameters represent.....that is why I was asking about changing the
>> defaults.  
> 
> What downsides, if any, are there to the tunes you suggest?  

Depending on how Linux allocates TCP memory, it could be that for hosts
with large numbers of concurrent connections (e.g. web servers with
hundreds of concurrent clients) could run out of memory.  However, some
documentation appears to imply that under memory pressure the system
will not scale up TCP memory utilization.  Other than that, I can't
think of any downsides - the changes we advocate on fasterdata.es.net
change the default maximum values not the default starting values, and
we rely on TCP autotuning to increase the window to the appropriate size
during the transfer.  I do _not_ recommend setting the default starting
window size high, since you can hurt LAN performance that way.  But by
setting the default maximum window size, you give TCP autotuning enough
room to help you in a high-performance networking environment.

There are three values set in each of the following:
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 65536 4194304

They are min, default, max.  So, by only cranking up the max, you start
all connections off with relatively low values (i.e. sane defaults) and
allow TCP autotuning to open up for high performance if such is needed.

> Or, to ask
> another way, why not lobby on the Linux kernel mailing list?

Science traffic has a fundamentally different profile than most
commodity Internet traffic.

Web, email, IM, etc. are all low-bandwidth applications, and most
computers on the Internet do not have access to high-performance
research and education networks.  In fact, they have the opposite
problem - they are typically bandwidth-constrained, and the networks
they use often have significant packet loss (e.g. the wireless at your
local Starbucks).

In contrast, the traffic on research and education networks (e.g. ESnet)
is dominated in terms of volume by high-bandwidth science flows.  These
are large data transfers that move many gigabytes per flow.   This is
fundamentally different than the typical web/email/IM traffic profile of
the commodity Internet, where there are billions of tiny flows.

I think that conservatism is warranted when talking about system
defaults for all computers (though Microsoft has already taken the
plunge and given Vista a default max TCP buffer of 16MB).

However, it is my perception that Scientific Linux is largely used in
environments with access to high-speed research and education networks.
 I'm also guessing that a significant number of Scientific Linux users
have data movement needs that are significantly greater than watching
the odd YouTube movie, or downloading a TV show from Hulu.

So, I think it is worth considering changing the defaults for Scientific
Linux, which is why I asked.  As I've said, I respect the desire to keep
the defaults as they are.   However, I'd love to hear about ways in
which we could all help our scientists to get reasonable TCP tuning
parameters configured on their data transfer machines.

Thanks,

		--eli





> 
> 
> -Brett.
> 
> PS: thanks for what you are doing, it was news to me and has explained
> some odd things I've come across.
> 

- --
Eli Dart                                            NOC: (510) 486-7600
ESnet Network Engineering Group                          (800) 333-7638
Lawrence Berkeley National Laboratory
PGP Key fingerprint = C970 F8D3 CFDD 8FFF 5486 343A 2D31 4478 5F82 B2B3
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpc8ysACgkQLTFEeF+CsrMYmQCcD8CX4nfnOgVIvhZeplAk3jLE
pF0AoIwa/D/zmvEdoYNz/94ksTcsGw23
=ZwFk
-----END PGP SIGNATURE-----