On 10/13/2018 04:41 AM, Nico Kadel-Garcia wrote:
> On Fri, Oct 12, 2018 at 11:09 PM ~Stack~ <[log in to unmask]> wrote:
>>
>> On 10/12/2018 07:35 PM, Nico Kadel-Garcia wrote:
>> [snip]
>>> On SL 7? Why? Is there any reason not to use xfs? I've appreciated the
>>> ext filesystems, I've known its original author for decades. (He was
>>> my little brother in my fraternity!) But there's not a compelling
>>> reason to use it in recent SL releases.
>>
>>
>> Sure there is. Anyone who has to mange fluctuating disks in an LVM knows
>> precisely why you avoid XFS - Shrink an XFS formated LVM partition. Oh,
>> wait. You can't. ;-)
> 
> I gave up, roughly 10 years ago, on LVM and more partitions than I
> absolutely needed. I cope with it professionally, most recently with
> some tools to replicate a live OS to a new disk image with complex LVM
> layouts for the filesystem. LVM.... has usually involved complexity I
> do not need. In the modern day of virtualization and virtualization
> disk images, I just use disk images, not LVM, to create new
> filesystems of tuned sizes. Not so helpful for home desktops, I admit,
> but quite feasible in a "VirtualBox" or "Xen" or "VMWAre" set of Linux
> VMs.

Unfortunately, in the world I am in we have an Audit/Security
requirement that we *must* have separate partitions for /, swap, /tmp,
/home, and /var with a recommendation for /opt if it is heavily used.
I'm also in a world where researchers get to pick their layouts to have
the jr admins build the box to their specs. Then when they break
something, we few sr admins have to come in and fix it.


>> My server with EXT4 will be back on line with adjusted filesystem sizes
>> before the XFS partition has even finished backing up! It is a trivial,
>> well-documented, and quick process to adjust an ext4 file-system.
> 
> xfsresize is not working for you? Is that an LVM specific deficit?

Please provide more information. To the best of my knowledge, RH
official support still says shrinking an XFS partition can not be done.
Only growing. I am not familiar with a xfsresize command. Where do I
find it?

$ yum provides */xfsresize
No matches found
$ cat /etc/redhat-release
Scientific Linux release 7.5 (Nitrogen)

> 
>> Granted, I'm in a world where people can't seem to judge how they are
>> going to use the space on their server and frequently have to come to me
>> needing help because they did something silly like allocate 50G to /opt
>> and 1G to /var. *rolls eyes* (sadly that was a true event.) Adjusting
>> filesystems for others happens far too frequently for me. At least it is
>> easy for the EXT4 crowd.
> 
> That's a fairly compelling reason not to use the finely divided
> filesystems. The benefits in protecting a system from corrupting data
> when an application overflows a shared partition and interferes with
> another critical system has, typically, been overwhelmed by the
> wailing and gnashing of teeth when *one* partition overflows and
> screws up simple operations like logging, RPM updates, or SSH for
> anyone other than root.
> 
> If it's eating a lot of your time, there's a point where your time
> spent tuning systems is much more expensive than simply buying more
> storage and consistently overprovisioning. Not saying you should spend
> that money, just something I hope you keep in mind.

I don't disagree with you at all. But those partition regulations come
down from higher level than I. As for buying more disks, I'm quite glad
that the server class SSD's have fallen in price and they aren't buying
60GB disks anymore. Most of them are getting in the ~200GB range and it
is less of an issue.

> 
>> Also, I can't think of a single compelling reason to use XFS over EXT4.
>> Supposedly XFS is great for large files of 30+ Gb, but I can promise you
>> that most of the servers and desktops I support have easily 95% of their
>> files under 100M (and I would guess ~70% are under 1M). I know this,
>> because I help the backup team on occasion. I've seen the histograms of
>> file size distributions.
> 
> Personally, I found better performance for proxies, which wound up
> with many, many thousands of files in the same directory because the
> developers had never really thought about the cost of the kernel
> "stat" call to get an ordered list of the files in a directory. I....
> ran into that one a lot, especially as systems were scaled up, and
> some people got bit *really hard* when they found g that some things
> did not scale up linearly.y.
> 
> Also: if you're running proxies, email archives, or other tools likely
> to support many small files, ext4 only supports 4 Billion files. ext4
> supports 2^64.

(for others following along, typo. He ment XFS supports 2^64.)

We have had researchers who get pretty high up there, but thankfully no
one has hit that limit yet. For the foreseeable future I think I'm good
on that limit.


>> For all the arguments of performance, well I wouldn't use either XFS or
>> EXT4. I use ZFS and Ceph on the systems I want performance out of.
> 
> I've not personally needed those, but I've not been in the big
> filesystem world for.... a few years. Are they working well for you?

Yes. It took a while to tune our ZFS systems the way we wanted them but
they are crazy fast now and rebuilding a failed drive is much quicker
than any equivalent RAID system I've ever used before.
(Granted, we are doing pools of mirrored disks and we have caching SSD's
configured too.)

Some of the best things about ZFS are not necessarily "file system" but
the tools surrounding it. Trying to back up a many-TB-heavily-written-to
file system was a constant churn for the backup system. We never had a
complete backup. Using ZFS, we snapshot and zfs send - we can easily
prove a point-in-time backup. When I last tried doing this with XFS, it
was doable but only if I unmounted the XFS partition. If, that's changed
then I don't know about it.

Also, the ability to setup multiple partitions for different groups,
quotas, meta-data-tagging, ect in ZFS is fantastic for some of the
shared resource systems. The tool bundling and metrics are really nice
for ZFS.

As for Ceph, that is for our BIG couple hundred TB parallel performance
file system. It took us a while to tune that too. But the Ceph team has
gotten fantastic performance out of it vs a lot of commercial products
at a fraction of the cost of those commercial products. We are quite
happy with it (although I will be much happier when they finish working
out the issues with SELinux - not an issue in the current environment
but it is an issue in another where I need it enforcing).


[snip]
>> To me, it still boggles my mind why it is the default FS in the EL world.
> 
> I suspect its stability has improved. The "large filesystem" specs are
> pretty good, and the latency numbers do show some benefits.

This is actually a very solid point. If it got to the point where RH was
comfortable enough making it the default FS, then it shows a decent jump
in stability. By making it default, RH probably also got FAR more bug
reports and use-cases that they can use to further improve the stability.


~Stack~