On Wed, 31 Dec 2008, Miles O'Neal wrote:

> About a month ago, I said:
>
>> Our local vendor built us a Supermicro/Adaptec
>> system with 16x1TB SATA drives.  We have a 12TB
>> partition that they built as EXT2.  When I tried
>> to add journaling, it took forever, and then the
>> system locked up.  On reboot, the FS was still
>> EXT2, and takes hours (even empty) to fsck.  Based
>> on the messages flying by I am also not confident
>> fsck rally understands a filesystem this large.
>>
>> Is the XFS module stable on 5.1 and 5.2?  (The
>> vendor installed 5.1 because that's what they
>> have, but I ran "yum update"), so it's effectively
>> 5.2.
>
> I rebuilt 12TB partition as XFS.  But after about 11GB
> of data moved, the system locked up with "bus error".
> After reboot, the system looks fine.  The vendor always
> runs diags and burns the systems in, though it could
> still be hardware or driver issue.
>
> Is it likely to be the OS/XFS with the large partition,
> or would you just send it back for diagnostics again?

I'd send it back:

We have a (growing) number of SL5 systems with large XFS 
filesystems (8.9 or 12 TiB), and we haven't seen a single lockup.
One of those with an 8.9 TiB XFS has been up for nine months, and
moved .3 PB of data during that time, sometimes under considerable load. 
It's still running 2.6.18-53.1.14.el5.

> Supermicros have been very reliable for us, but between
> the Adaptec and 1TB SATAs, and the large partition, I'm
> not sure how reliable the current drivers are.

Maybe try an LSI based controller instead.

> I'd hoped to just have one mount point, but could make
> 2-3 smaller partitions if that seems to be the likely
> issue.

Probably not.

Hope this helps,
 	Stephan

> Thanks,
> Miles
>

-- 
Stephan Wiesand
   DESY - DV -
   Platanenallee 6
   15738 Zeuthen, Germany