On 07/17/2012 11:22 AM, Orion Poplawski wrote:
> Our SL6.2 KVM and nfs/backup server has been crashing frequently recently
> (starting around Fri 13th - yikes!) with Kernel panic - Out of memory and no
> killable processes.  The server has 48GB ram, 2GB swap, only about 15GB
> dedicated to VM guests.  I've tried bumping up vm.min_free_kbytes to 262144 to
> no avail.  Nothing strange is getting written to the logs before the crash.
>
> Happening with both 2.6.32-220.23.1 and 2.6.32-279.1.1.
>
> Anyone else seeing this?  Any other ideas?  I've set a serial console log to
> try to catch more information the next time it happens.
>

So I believe I've figured out what the issue was.  I was hitting a kernel BUG 
in the hidraw system (filed 
https://bugzilla.redhat.com/show_bug.cgi?id=846047) which was triggering the 
crashdump kernel and *that* was running out of memory.  I just wasn't looking 
back far enough in the console logs (starting from the bottom) to see the 
original issue.

-- 
Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA, Boulder Office                  FAX: 303-415-9702
3380 Mitchell Lane                       [log in to unmask]
Boulder, CO 80301                   http://www.nwra.com