On Mon, Dec 04, 2023 at 03:03:46PM -0500, Edward Zuniga wrote:
> 
> We are upgrading our MRI Lab servers and workstations to AlmaLinux 8. We
> have used ext4 for the past 10 years, however we are considering using XFS
> for its better performance with larger files. Which file system do you use
> for your lab?
>

Historical background.

XFS filesystem with the companion XLV logical volume manager (aka "partitioning tool")
came to Linux from SGI IRIX, where it was developed circa late-1990-ies. XFS was copied
to Linux verbatim (initially with shims and kludges, later, fully integrated).
XLV was reimplemented as LVM.

The EXT series of filesystems were developed together with the linux kernel (first ext
filesystem may have originated with MINIX, look it up). As improvements were made,
journaling, no need to fsck after crash, online grow/shrink, etc, they were
renamed ext2/ext3/ext4 and they are still largely compatible between themselves.

For many purposes, both filesystems are obsoleted by ZFS, which added:

- added metadata and data checksums - to detect silent bit rot on current-generation HDDs and SSDs
- added online filesystem check - for broken data, gives you list of filenames instead of inode numbers
- added "built-in" mirroring - together with checksums, online fsck (zfs scrub) and monthly zfs scrub cron job, allows automatic healing of bit rot.
- added "built-in" raid-5 and raid-6 - again, together with checksums and online fsck, allows automatic healing and robust operation in presence of disk bad sectors, I/O errors, corruption and single-disk failure.
- other goodies like snapshots, large ram cache, dedup, online compression, etc are taken for granted for current generation filesystems.

On current generation HDDs and SSds, use of bare XFS and ext4 is dangerous, SSD failure or "HDD grows bad sectors" will destroy your data completely.

On current generation HDDs, use of mirrored XFS and ext4 is dangerous (using mdadm or LVM mirroring), (a) bit rot inevitably causes differences between data between the two disks. Lacking checksums, mdadm and LVM mirroring cannot decide which of the two copies is the correct one. (b) after a crash, mirror rebuild fill fail if both disks happen to have bad sectors (or throw random I/O errors).

Ditto for RAID5 and RAID6, probability of RAID rebuild failing because multiple disks have have sectors and I/O errors goes up with the number of disks.

ZFS was invented to resolve all these problems. (BTRFS was invented as a NIH erzatz ZFS, is still incomplete wrt RAID5/RAID6).

Bottom line, if you can, use ZFS. Current Ubuntu installer has a button "install on ZFS", use it!

-- 
Konstantin Olchanski
Data Acquisition Systems: The Bytes Must Flow!
Email: olchansk-at-triumf-dot-ca
Snail mail: 4004 Wesbrook Mall, TRIUMF, Vancouver, B.C., V6T 2A3, Canada