John Summerfield wrote:
> Troy Dawson wrote:
>> Hello,
>> We've been getting reports from several people about yum updates falling
>> due to  a file not being available.
>>
>> Please know that ftp.scientificlinux.org get's alot of traffic.
>> Especially after a new release, it get's an immense amount for about a
>> month or two.  This is why we moved rsync.scientificlinux.org to be a
>> different machine, hoping that it would help out.
>>
>> What kind of load and traffic are we talking about.  The current load is
>> 44, on a 2 cpu machine.  The current number of connections (http and
>> ftp) varies between 550 and 600 at any particular time this morning.
>>
>> The machine can handle it.  Our network can handle it.  That isn't
>> what's bothering me.
>>
>> There are some labs and universities that have their own mirror's, right
>> at the lab and/or university.  These same labs and universities also
>> have very large clusters (100's to 1000's of machines per cluster).
>> These same labs and universities are pointing their large clusters of
>> machines at ftp.scientificlinux.org instead of their own mirrors.
>> This is causing the quality of service to go down for all the other users.
>>
>> If you have a mirror.  Please use it, especially for your clusters.
>>
>> Fastestmirror is also good.
>>
>> We have the yum plugin fastestmirror in the repository, and we have
>> plenty of plublic mirrors around the world.  Please use them.
>>
>>   yum install yum-fastestmirror
>>
>> In the directory /etc/yum.repos.d/ edit the files sl.repo and
>> sl-security.repo, or sl-errata.repo.  Comment out the "baseurl="
>> line(s), and uncomment the "mirrorlist=" line.
>>
>> I don't want to have to do anything drastic like force people to use
>> mirrors, or force people to use fastestmirror.  I want mirrors to be an
>> option people choose, not something that is forced on them.
>>
>> So please, if you have a large cluster of machines, please use a mirror.
> 
> Troy, why not choose a number, and write a firewall rule that prevents
> anyone from getting that many connexions per hour/day, whichever seems
> good to you?
> 
> I do this to limit ssh connexions, so the ungodly have less chance to
> guess my passwords. My number is 2 and my interval is an hour, and I log
> both accepted and rejected connexions. I find I drop 90% of requests
> from outside my preferred area.
> 
> I figure that if I found myself needing to get in from outside my
> preferred area, that I should be able to get my password right half the
> time:-)
> 
> In your case, your number might be a little higher, and maybe when sites
> exceed that number you throttle them.
> 
> Depending on the numbers you choose, most people probably wouldn't
> notice anything.
> 
> Sites coming at you from public IP addresses might need some more
> thought, in my experience Linux _seems- to be counting for each
> individual IP address where you might want to control /24 or even /16
> addresses.
> 

Hi John,

This is exactly what I do *not* want to happen.  I do not want to limit people. 
  I want anyone in the world to be able to load scientific linux on their 
machine and have a good experience.

That being said, many people are abusing this system.  They customize their 
version of scientific linux.  They have alot of computers and resources, and 
yet they choose to point their clusters at the main distribution server.

This happened a couple of years ago, and I sent out a e-mail similar to this 
one, and things cleared up.  I am hopeful it happens again.

If not, personal e-mails will be sent out, and we'll deal with it one offender 
at a time.  But I really don't think it will come to that.  I think most 
offenders really didn't realize the impact they were having on others.

Troy
-- 
__________________________________________________
Troy Dawson  [log in to unmask]  (630)840-6468
Fermilab  ComputingDivision/LCSI/CSI DSS Group
__________________________________________________