If your atop service is on, you should be able to see something about 
what was happening shortly before the crash by viewing the appropriate 
/var/log/atop/<file> with the atop -r <file> command.
     You could just try increasing your swap space; you don't have very 
much compared with your ram.  Simple 'top' and 'atop' commands show, among 
other things, current swap usage.  I'd get nervous if most of it gets used 
up.

Steven Yellin

On Tue, 17 Jul 2012, Orion Poplawski wrote:

> Our SL6.2 KVM and nfs/backup server has been crashing frequently recently 
> (starting around Fri 13th - yikes!) with Kernel panic - Out of memory and no 
> killable processes.  The server has 48GB ram, 2GB swap, only about 15GB 
> dedicated to VM guests.  I've tried bumping up vm.min_free_kbytes to 262144 
> to no avail.  Nothing strange is getting written to the logs before the 
> crash.
>
> Happening with both 2.6.32-220.23.1 and 2.6.32-279.1.1.
>
> Anyone else seeing this?  Any other ideas?  I've set a serial console log to 
> try to catch more information the next time it happens.
>
> -- 
> Orion Poplawski
> Technical Manager                     303-415-9701 x222
> NWRA, Boulder Office                  FAX: 303-415-9702
> 3380 Mitchell Lane                       [log in to unmask]
> Boulder, CO 80301                   http://www.nwra.com
>