On Thu, 30 Apr 2009, Jon Peatfield wrote:

> On Tue, 28 Apr 2009, Troy Dawson wrote:
>
>> Hello,
>> The distribution servers rsync.scientificlinux.org and 
>> ftp.scientificlinux.org will be going down tomorrow for software 
>> maintenance.
>> I will also be creating some hard links in the storage area's to save 
>> space. These hard links may or may not affect your syncing.  It might even 
>> save you space if you are using the H option when rsyncing.

> The other day I noticed that currently when a new sl5 (say) errata package is 
> released our mirroring ends up fetching it once each for the 50, 51, 52, 53 
> trees and in some cases the same rpm is also present in each of i386 and 
> x86_64 bits so we can be fetching the same package 8 times...
>
> We are doing rsync with -H so I'm assuming that the rpms were not hardlinked 
> at the server end (is that right or maybe my rsync options are breaking 
> things...)
>
> I did some checksums and from what I could tell the packages with identical 
> names do have the same contents so I almost spent the time hard-linking them 
> together at this end...

For precisely this purpose I have a script
 	http://www.dpmms.cam.ac.uk/~werdna/hardlinkify.pl
which takes a pair of directory trees and wanders down them.
Whenever it finds two files with the same relative path
that have the same permissions, size and ownership
it checks whether they are the same and if so hard links them together.

It doesn't touch files when both have link-counts greater than one.
This does mean that if the same file is in say i386, x86_64, fastbugs
and errata this script can't link all the copies together, however
many pairs of trees you run it on :-(

-- 
Dr. Andrew C. Aitchison		Computer Officer, DPMMS, Cambridge
[log in to unmask]	http://www.dpmms.cam.ac.uk/~werdna