On 4/4/2017 6:59 PM, Konstantin Olchanski wrote: >>> Moving to ZFS... >>> ZFS is also scary... >> >> Heh - another soon to be victim of ZFS on linux :) >> > > No kidding. Former victim of XLV+XFS (remember XLV?), former > victim of LV+EFS, former victim of ext2, ext3, reiserfs, former > victim of LVM, current victim of mdadm/raid5/6/ext4/xfs. > >> >> You'll quickly realise that the majority of major features you'd expect >> to work - don't. > > I am not big on "features". For me the main features is open()/read()/write()/close(), > mkdir()/rmdir()/readdir() and those seem to work on all filesystems. Next features are: > a) non-scary raid rebuild after a crash or disk failure, > b) "online" fsck > >> >> You can't grow a ZFS 'raid'. You're stuck with the number of disks you first start with. (I know this is 2 quotes back) That's kind of unfair since here you're talking about a feature that was never offered, it's just an incorrect assumption. It took me a while to understand what ZFS does and does not offer as well - I missed many things from (ancient history) Digital's advfs - but ZFS does lots of things quite well. There really is no such thing as "a ZFS raid", that's probably most analogous to a zfs pool made of a single raidz vdev, but that's a very simple case. What other system lets you make large reliable storage pools from hundreds of drives on a single server? I built some with 200+ 4TB drives some years back. > We only have a few hardware configurations, all with fixed number of disks, so not a problem: > > a) single 120GB ssd for OS (/home on NFS) > b) single SSD for OS, dual 4-6-8 TB HDD for data, RAID1 configuration to protect against single disk failure > c) dual SSD for OS and /home, dual HDD for data, both RAID1 configuration to protect against single disk failure > d) single SSD for OS, multiple (usually 8) 6-8 TB HDDs for data, mdadm raid6+xfs and now raidz2 ZFS (protection against single disk failure + failure of second disk during raid rebuild). For case (b) for your data storage, you can expand a ZFS mirror reasonably easily. For case (c) I don't know how hard it is to use ZFS for the OS drive on linux; I only used it on BSD. But mdadm on linux is ok for that role. For case (d) it is true that you cannot expand a ZFS RAIDZ(2) vdev, but that's ok if you know that going in. > BTRFS is billed as "open source replacement for ZFS", but after testing it, > my impression is that it is only used by a couple of enthusiasts > in single-disk laptop configurations. In a single-disk system, it is not > clear how btrfs/zfs is better than old-plain ext4/xfs. I've never seen any good success stories for btrfs but to be fair I have not followed it closely. zfs can still give you some useful things on a single drive system: you get the data checksums, useful snapshots (as opposed to LVM), volume manager features, etc. By default the checksums would only warn you of problems with a single drive, but you can tell zfs to keep multiple copies of your data ("zfs set copies=n") so that might well also let it recover from bad blocks. G. -- Graham Allan Minnesota Supercomputing Institute - [log in to unmask]