On Wed, May 7, 2014 at 6:30 AM, Arnau Bria <[log in to unmask]> wrote:
> On Tue, 6 May 2014 10:23:27 -0400
> Nico Kadel-Garcia wrote:
>
>> Don't use repo sync: use rsync of an upstream server, which can
>> handle sym links and hard links •much• more efficiently.
>>
>> Yum and repo sync are grossly inefficient because of the size of the
>> repo data, compressed tables
>
> Sorry Nico, but I did not understand your answer.
> I use mrepo for replicating SL repos (that uses lftp).
>
> Cheers,
> Arnau

Ahh.  mrepo seems interesting. And *heh*: I remember when rewriting
the CD image merging tools for CentOS and fixing some ".discinfo"
bugs, and sending a copy to our favorite upstream vendor, Funny how
our favorite upstream vendor came out with DVD images very shortly
after that..... Looks like mrepo does a lot of that work, I'll have to
remember it. Thanks!

There is a tool called "reposync" for mirroring upstream yum
repositories. It also works with Red Hat's DRM controlled binary
repositories, and with other external repositories that don't allow
filesystem browseability, so I assumed that was what was being
discussed. It does not provide byte-for-byte syncing of the "repodata"
directory of the upstream repository. Instead, the yum client on your
local system is polling that data, and it's pretty inefficient because
*yum* is pretty inefficient.

lftp is better than reposync, I admit. But 'lftp', using HTTP based
access, can't replicate symlinks (which depend on viewability of the
repository). Even FTP based access won't automatically bring over the
hardlinks. And if you're mirroring both the i386 and x86_64
directories, a *lot* of the files are shared between them, and are
hardlinked in the Scientific Linux upstream repositories. They use
"rsync".

And rsync allows for some very useful options, such as using a very
sophisticated "excludes" file, or the ability to tie it to the
"rsnapshot" command which has lockfiles and provides a *staged*
transfer, one that can be activated locally in well defined time
increments rather than changing the contents of  your local mirror
while the transfer is going on.

Good tools, I've used all those others for repository mirroring. What
I've personally rejected, with a vengeance, is spacewalk. and the
upstream commercial version, Red Hat Network. It might have gotten
better, haven't had a chance to play with it in about 8 years.