I agree with your analysis, save for three comments.  Mine also is not a 
political comment, merely an analysis of fact.

Overwhelmingly throughout the world, HEP is funded by public funds 
(sometimes from totalitarian dictatorships if one can call such 
"public").  HEP addresses basic science, fundamental physics as do some 
aspects of astronomy and cosmology, which is of no interest to 
for-profits other than a bit of technology spin-off (sometimes useful in 
the "consumer" sector, more often in the weapons sector).  The only 
reasons a for-profit corporation in a "market" (including democratic 
representative government neo-liberal oligopolies) funds fundamental 
physics (I do not consider materials science fundamental, in that the 
Standard Model, with appropriate computational capability and methods of 
"solving" the underlying quantum field theory equations, including 
quantum statistical mechanics, seems to be in full agreement, and even 
has quantitative predictive power, for what has been discovered in 
materials science under terrestrial conditions) is for publicity, for 
proof of concept, and for tax writeoffs, often through vendor 
partnerships that offer "huge" discounts from the commercial prices. 
The equivalent areas of materials science, biology, etc., are of far 
greater interest, particularly if wealth transference can be used 
(public funding of the underlying research, but oligopoly profit from 
the actual deployed products).  This public funding stream is 
fundamental to carrying out experimental as well as computational 
theoretical physics -- instrumentality is required (science is not "pure 
mathematics", despite what some think looking at any modern hard science 
research paper).  To those in HEP, all of the above is known and 
"obvious", but to many in the wider SL community, it is not well understood.

There is adequate time to move servers past SL7 -- if IBM RH does not do 
for CentOS 7 what it did for CentOS 8.  Marketing promises from most 
for-profit entities are unenforceable and subject effectively to whim 
(typically perceived in the best interest of the entity, often only in 
terms of short-term stock value, not long-term, in the 
quarter-by-quarter financial market model).  However, if new hardware 
from vendors does not support the basic kernel gcc libraries of SL7, but 
requires features from later production releases (say, what currently is 
used by Ubuntu LTS), then the HEP community will have *LOTS* of 
backporting to do.

The issue of what to do past SL7 is the question.  Security compromises 
to SL7 probably will be minimal in so far as RHEL7 through CentOS 7 is 
supported, but once unsupported ("EOL"), it becomes increasingly 
hazardous to keep the OS in any "mission-critical" production 
environment.  My personal guess is that in order to allocate financial 
and personnel "resources" to other parts of the Fermilab/CERN 
activities, SL was dropped for CentOS.  This choice is not possible. 
Assuming that many of the servers actually are under a type 1 
hypervisor, it is relatively easy to start deployment of new supervisor 
environments and test these.  Otherwise, either machines must be 
de-allocated from production and put into test mode for a new supervisor 
environment, or additional platforms need to be procured.  Testing must 
be done near scale, as we all well know -- testing on a "high powered" 
workstation is not the same as testing on a clustered HPC machine, 
perhaps with a SAN, and other platform features.

As for the use of Mac OS X or MS Win (probably 10 right now) on the 
desktop, that is a matter of taste and funding.  As Apple again has 
changed the Mac platform, now from X86-64 to ARM, and as Apple strictly 
is a for-profit entity, there will need to be a massive re-investment in 
new Mac machines, unless the X86-64 platforms convert to BSD or Linux, 
and Linux is much easier to support for HEP built around SL.  A 
technical question:  for those HEP workstations that are using Mac OS X, 
is Fink or the equivalent installed so that "standard" applications 
easily can be ported?

On 12/11/20 8:09 AM, Brett Viren wrote:
> This is not a political reply.
> 
> Keith Lofstrom <[log in to unmask]> writes:
> 
>> The big physics labs that supported Scientific Linux get
>> much or all of their funding from the US government,
> 
> CERN is primarily funded by CERN nation states, of which US is not one.
> 
> FNAL, being a US DOE National Lab, is primarily funded by US DOE.
> 
>> I wonder how much IBM contributes to the politicians who
>> make the funding decisions for the labs, and I wonder if
>> there is subtle back-channel pressure on lab software
>> purchases and project funding decisions?
> 
> The subtle pressure theory is very doubtful to me.  Here is why:
> 
> 1. The various HEP/NP clusters are almost universally on SL7 so have
>     until ca 2024 to figure out wtf they will do next.  So, there's
>     simply nothing there to apply any subtle pressure against.  And, any
>     argument to move the clusters from SL7 before we get closer to 2024
>     would have to be very compelling.
> 
> 2. The cost of going "full RHEL" for the clusters is prohibitive.
>     Removing the previously expected "CentOS future" and applying subtle
>     pressure will not magically make funding appear to pay for full RHEL
>     licensing on the clusters.  IBM would have to offer a deep,
>     essentially exponential, price break as a function of $(nproc).  But,
>     even if they did, RHEL does not really offer anything novel and
>     useful for the clusters.  Clusters update their OS far less
>     frequently than the OS updates become available so something like
>     RHEN licensing is useless.  Cluster admins are DEEP experts so don't
>     need the paid hand holding.
> 
> 3. There is already some RHEL penetration in labs for various "servers"
>     so that market is partly saturated.  Even if it was still fully open,
>     it's a rather mid-level salesdroid "get" (in terms of profit).  So,
>     it does not seem to me to motivate any "subtle pressure" tactic.
>     Where special RHEL-only drivers can't be avoided, the few systems
>     that fall into this category can also use RHEL if there were no
>     alternative.
> 
> 4. Little real movement to CentOS 8 has occurred.  So, few have fallen
>     into that particular tar pit.  Some isolated suckers (and knowing
>     some of them well, I say that with fondness) have, and they'll need
>     to dig out but by and large, there is no ensnared market here to
>     leverage with subtle pressure.  I think the move to CentOS 8 was just
>     picking up steam.  Had IBM waited, say 1 year, they'd have ensnared
>     many more in the tar pit.  So, if any dodgy tactic was going on, they
>     blew it.
> 
> 5. At the physicist "personal" laptop/workstation level, Mac dominates,
>     with Windows next.  The small fraction of "personal" Linux I'd guess
>     SL is no a majority.  There's a lot of Ubuntu and Fedora.  Like in
>     general, most of those using SL are on SL7 so also fall into the "got
>     until 2024 to figure wtf they will do next".
> 
> So, the way I see it, labs are simply not in a position of being "subtly
> pressured" on this issue and have ample time to figure out a next step
> solution for the various demographics.  A few systems are in the tar pit
> but can (must) get out in some way.  A few others are already in on RHEL
> so don't care.
> 
> 
> My hope is they (we) take this current situation as a lesson and make a
> radical change that puts all of our computing on more sustainable
> footing as we go into the next decades.
> 
> -Brett.
>