On 01/20/2012 07:43 AM, Michael Tiernan wrote:
> On 1/19/12 6:17 PM, Douglas McClendon wrote:
>> It's noteworthy that I spent several months recently trying to more or
>> less recreate the SL6 build in a completely automated way for
>> Ascendos(.org) after Troy Dawson got the ball rolling on that project.
>> At the moment I've sort of thrown up my hands for awhile, as there
>> didn't seem to be sufficient interest in my particular brand of
>> solution and/or leadership.
>
> Oh boy. That doesn't sound good.
>
> I appreciated the work that was started on ascendos but I for one knew
> little of the overall picture and was/am unable to read hundreds of
> forum posts about what's going on. It's just that 24hrs/day problem.
>
> Maybe this discussion will help trigger a renewed push to the community.
>
>> And in general, is ScientificLinux capable of describing a
>> deterministic algorithm used to make those choices,
>
> I guess that's the summary of the problem right there. My friend has
> suggested that he's going to build a virtual box system that contains a
> virtual machine that he can pull out at any given moment, that's
> 'virgin' and say "build from scratch" and get an output that, other than
> the dates involved is identical to the existing distribution.

That is essentially how I designed and implemented 'el-build', except 
using kvm instead of virtualbox, as kvm/virsh is what is more of a 
stock(/RH-encouraged) solution found on e.g. an SL5 or F13 system.

Thus the idea (preaching to the choir here obviously) being that if you 
are on any libvirt/kvm capable system (e.g. centos5/debuntu/f13) you can 
easily do the pushbutton fully automated and deterministic build.  And 
that within the VM, the building environment OS (call it the MockHostOS) 
is a known/determin{ed,istic} quantity.  I actually chose to use SL60 as 
MockHostOS because I that is what Troy started Ascendos's initial koji 
with, and it happened to exist, and seemed likely to make things go 
easiest.  But the long term plan was once that was completely working, 
able to build all of EL6->currentupdates, I would then make the few 
tweaks necessary to use F13 instead as the MockHostOS, as my best guess 
was that was what SL used for SL6.  (actually maybe you folks used SL5, 
but even were that the case I'd still want to go with F13 because it 
feels like a less total bootstrap complexity to me, and in line with 
what I expected RH used.  Or rather maybe RH used EL5, in which case 
then I'm tilting between CentOS5 to match that but with added complexity 
surface*, vs F13).

(surface of code and builds making up the initial bootstrap binaries, 
i.e. whether or not security breaches within the centos/SL environments 
factor into the security of the bootstrap binaries.  Yes, I'm that paranoid)

Anyway, thats a bunch of rambling, but its also I think the root of the 
important 'deterministic genesis' conversation which I think SL should 
have dedicated more time and attention to by now.  (note my shameless 
naming of my own engineering price in the grandios email to ascendos-dev 
that I got lwn to post/reprint/reporton, with anti-Gitmo politicization 
intact).

Anyway, so el-build currently uses SL60 as the MockHostOS it first 
installs in its virtual build system.

Then within that, it's currently using the standard koji/mock toolset, 
entirely facilitated by the koji setup scripts from ScientificLinux 
(thanks, though after many months and decent introduction and 
understanding of koji, I'm not entirely sure its the right tool for this 
job.  Though no doubt entirely sufficient.).

It gets interesting of course then, as koji/mock is setting up the 
buildroots.

The first interesting question there (I think) which I've deferred thus 
far in code (I think, been a few weeks since I've been looking at the 
code), is whether or not you do an immediate dogfooding pass on the min 
buildroot.  I.e.

choice a) your mock/minbuildroot deps are satisified by your core 
bootstrap OS (call it FirstMinBuildRootOS).  Then further deps are 
satisified by your 6.0 output packages as they get built (* in what 
order? call that choice b, I went with reverse datesorted by srpm build 
date.  pkg by pkg, not weekly ala SL which I don't think is preferrable)

choice b) you just build the set of packages in that 
FirstMinBuildRootOS, then immediately cease using that first bootstrap 
minbuildroot and start exclusively using output, call that now 
SecondMinBuildRootOS.  You could even throw in another dogfooding pass 
(ThirdMinBuildRootOS) just for 
fun/pickiness/goodquickenoughsanitytest/etc..)

Clearly by my wording I like choice b, though as mentioned I may have 
deferred it.

Anyway, then the more fun choices comes when you start getting to past 
the first point release, and into updates, and again, certain options 
seem intuitively obvious and I either went with those, or took some 
easier route with a TODO in the code noting the long-term preferred 
solution.  (i.e. there is an option to switch your virt host to the new 
pointrelease, or not.  Though here the fact that things are done 
virt-wise allows that to be possible and have a single end-to-end build 
that doesn't require physically rebooting any system)

Another key aspect of my solution design was the complete seperation of 
the first online-source-and-bootstrap-materials fetching phase of 
things, and the second actual building phase of things, such that the 
entire build process can be done on a pure virgin system never needing 
to be defiled by connecting it to the internets.

Yet another core idea I've had and been quite unflinching about is the 
idea that the private part of the package signing key should never be 
accessible to a system that has ever or ever will be online.  I kindof 
doubt SL (or even centos or redhat) do this.

But all of this is a slice of a clearly larger discussion, that I'm a 
tad offended SL has seemed to ignore up until now (given my clear 
efforts, and even the Troy organizational connection).  (But yes, I 
know, no budget, expedience, people have lives outside work they want to 
live and not worry about nitpicking esoteric techno stuff like this, etc...)

But since you gave me an opening to step back up on my soapbox... :)

cheers,

-dmc



>
>> Anyway, for people interested in the landscape of this issue, this
>> recent lwn article (and my arguably inflammatory attached comments)
>> has some perspective on the issue.
>
> I'll read this later today, no way I can get it done soon.