On 2/28/20 5:00 AM, Paddy Doyle wrote:
> We're a university HPC centre.

[snip]

> Plus the stability of the longer RHEL life cycle has been a big
> plus for stable clusters (*).
> 
[snip]
> 
> (*) although more recently some people are asking more and more for the
> latest and greatest.. yes, we're looking at you ML and AI! :)

A combination of Singularity [1] and Spack [2] has kept my ML/AI users
quite happy, though there is a learning curve.

Singularity is pretty easy to grasp, it's just extra steps if the user
wants to build their own but it's been great because once they do they
jump on board the "reproducible science" aspect pretty quick. Greg
Kurtzer and his team are doing a great job with Singularity.

Spack is trivial if what they want is already there, a bit more
challenging if they have to add programs to it. Fortunately they've
offered full day training at Super Computing every year. The tutorial
section hasn't been posted yet for 2020, but here is their 2019 tutorial
and resources are online. [3]

[1] https://sylabs.io/singularity/
[2] https://spack.io/
[3]
https://sc19.supercomputing.org/?post_type=page&p=3479&id=tut164&sess=sess194

Hope this helps. I understand the sysadmin pain in trying to meet the
rapidly evolving ML/AI researcher demands. :-)

~Stack~