On 4/4/2017 6:59 PM, Konstantin Olchanski wrote:
>>> Moving to ZFS...
>>> ZFS is also scary...
>>
>> Heh - another soon to be victim of ZFS on linux :)
>>
>
> No kidding. Former victim of XLV+XFS (remember XLV?), former
> victim of LV+EFS, former victim of ext2, ext3, reiserfs, former
> victim of LVM, current victim of mdadm/raid5/6/ext4/xfs.
>
>>
>> You'll quickly realise that the majority of major features you'd expect
>> to work - don't.
>
> I am not big on "features". For me the main features is open()/read()/write()/close(),
> mkdir()/rmdir()/readdir() and those seem to work on all filesystems. Next features are:
> a) non-scary raid rebuild after a crash or disk failure,
> b) "online" fsck
>
>>
>> You can't grow a ZFS 'raid'. You're stuck with the number of disks you first start with.

(I know this is 2 quotes back) That's kind of unfair since here you're 
talking about a feature that was never offered, it's just an incorrect 
assumption. It took me a while to understand what ZFS does and does not 
offer as well - I missed many things from (ancient history) Digital's 
advfs - but ZFS does lots of things quite well. There really is no such 
thing as "a ZFS raid", that's probably most analogous to a zfs pool made 
of a single raidz vdev, but that's a very simple case. What other system 
lets you make large reliable storage pools from hundreds of drives on a 
single server? I built some with 200+ 4TB drives some years back.

> We only have a few hardware configurations, all with fixed number of disks, so not a problem:
>
> a) single 120GB ssd for OS (/home on NFS)
> b) single SSD for OS, dual 4-6-8 TB HDD for data, RAID1 configuration to protect against single disk failure
> c) dual SSD for OS and /home, dual HDD for data, both RAID1 configuration to protect against single disk failure
> d) single SSD for OS, multiple (usually 8) 6-8 TB HDDs for data, mdadm raid6+xfs and now raidz2 ZFS (protection against single disk failure + failure of second disk during raid rebuild).

For case (b) for your data storage, you can expand a ZFS mirror 
reasonably easily.
For case (c) I don't know how hard it is to use ZFS for the OS drive on 
linux; I only used it on BSD. But mdadm on linux is ok for that role.
For case (d) it is true that you cannot expand a ZFS RAIDZ(2) vdev, but 
that's ok if you know that going in.

> BTRFS is billed as "open source replacement for ZFS", but after testing it,
> my impression is that it is only used by a couple of enthusiasts
> in single-disk laptop configurations. In a single-disk system, it is not
> clear how btrfs/zfs is better than old-plain ext4/xfs.

I've never seen any good success stories for btrfs but to be fair I have 
not followed it closely.

zfs can still give you some useful things on a single drive system: you 
get the data checksums, useful snapshots (as opposed to LVM), volume 
manager features, etc.

By default the checksums would only warn you of problems with a single 
drive, but you can tell zfs to keep multiple copies of your data ("zfs 
set copies=n") so that might well also let it recover from bad blocks.

G.
-- 
Graham Allan
Minnesota Supercomputing Institute - [log in to unmask]