Hello everybody,

I have some puzzling issue. I see this in dmesg:

ksoftirqd/1: page allocation failure. order:1, mode:0x20
Pid: 9, comm: ksoftirqd/1 Not tainted 2.6.32-220.4.1.el6.x86_64 #1
Call Trace:
 <IRQ>  [<ffffffff81123daf>] ? __alloc_pages_nodemask+0x77f/0x940
 [<ffffffff81061f00>] ? enqueue_entity+0x390/0x420
 [<ffffffff8115dc62>] ? kmem_getpages+0x62/0x170
 [<ffffffff8115e87a>] ? fallback_alloc+0x1ba/0x270
 [<ffffffff8115e5f9>] ? ____cache_alloc_node+0x99/0x160
 [<ffffffff8115f3db>] ? kmem_cache_alloc+0x11b/0x190
 [<ffffffff8141f4b8>] ? sk_prot_alloc+0x48/0x1c0
 [<ffffffff8141f742>] ? sk_clone+0x22/0x2e0
 [<ffffffff8146c9b6>] ? inet_csk_clone+0x16/0xd0
 [<ffffffff81485883>] ? tcp_create_openreq_child+0x23/0x450
 [<ffffffff8148326d>] ? tcp_v4_syn_recv_sock+0x4d/0x2a0
 [<ffffffff81485641>] ? tcp_check_req+0x201/0x420
 [<ffffffffa0442762>] ? nf_ct_deliver_cached_events+0xa2/0xe2 [nf_conntrack]
 [<ffffffff81482c8b>] ? tcp_v4_do_rcv+0x35b/0x430
 [<ffffffffa04aa557>] ? ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
 [<ffffffff81484401>] ? tcp_v4_rcv+0x4e1/0x860
 [<ffffffff814620b0>] ? ip_local_deliver_finish+0x0/0x2d0
 [<ffffffff8146218d>] ? ip_local_deliver_finish+0xdd/0x2d0
 [<ffffffff81462418>] ? ip_local_deliver+0x98/0xa0
 [<ffffffff814618dd>] ? ip_rcv_finish+0x12d/0x440
 [<ffffffff81461e65>] ? ip_rcv+0x275/0x350
 [<ffffffffa057c15b>] ? ingress_enqueue+0x2b/0x94 [sch_ingress]
 [<ffffffff8142be6b>] ? __netif_receive_skb+0x49b/0x6e0
 [<ffffffff8108b789>] ? delayed_work_timer_fn+0x39/0x50
 [<ffffffff8142c14a>] ? process_backlog+0x9a/0x100
 [<ffffffff814307c3>] ? net_rx_action+0x103/0x2f0
 [<ffffffff81072001>] ? __do_softirq+0xc1/0x1d0
 [<ffffffff8100c24c>] ? call_softirq+0x1c/0x30
 [<ffffffff8100c24c>] ? call_softirq+0x1c/0x30
 <EOI>  [<ffffffff8100de85>] ? do_softirq+0x65/0xa0
 [<ffffffff81071b60>] ? ksoftirqd+0x80/0x110
 [<ffffffff81071ae0>] ? ksoftirqd+0x0/0x110
 [<ffffffff81090726>] ? kthread+0x96/0xa0
 [<ffffffff8100c14a>] ? child_rip+0xa/0x20
 [<ffffffff81090690>] ? kthread+0x0/0xa0
 [<ffffffff8100c140>] ? child_rip+0x0/0x20


I googled a bit for similar bugs, and, well, I just find it strange to
get these on my system. These can be typical on systems pressured for
kernel memory, but I find it hard to believe that the system in question
has it. It's SL6.2 running 2.6.32-220.4.1.el6.x86_64 kernel, with 8GB
RAM and
vm.min_free_kbytes = 131072
sysctl parameter, typically 150-250 MB are always free (and like 4 GB
cache+buffers). slabtop in the very same session (nothing bad seems to
happened after this message) shows

 Active / Total Objects (% used)    : 1533015 / 2020657 (75.9%)
 Active / Total Slabs (% used)      : 68451 / 68477 (100.0%)
 Active / Total Caches (% used)     : 120 / 202 (59.4%)
 Active / Total Size (% used)       : 212940.66K / 274324.79K (77.6%)
 Minimum / Average / Maximum Object : 0.02K / 0.14K / 4096.00K

I wonder if anyone knows what might be going on and can suggest any
tweak for solving this. While this isn't a problem per se, I fear that
it might be a symptom of something worse.

This is what /proc/meminfo looks like (having swapped stuff isn't a
problem, it's just something that wasn't used in ages that was thrown
there):
MemTotal:        7854796 kB
MemFree:          231400 kB
Buffers:          182564 kB
Cached:          3433284 kB
SwapCached:       197872 kB
Active:          4365836 kB
Inactive:        2800612 kB
Active(anon):    2686648 kB
Inactive(anon):   877976 kB
Active(file):    1679188 kB
Inactive(file):  1922636 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      10485744 kB
SwapFree:        9942660 kB
Dirty:               328 kB
Writeback:             0 kB
AnonPages:       3467456 kB
Mapped:            33580 kB
Shmem:             13880 kB
Slab:             298616 kB
SReclaimable:     129076 kB
SUnreclaim:       169540 kB
KernelStack:        5976 kB
PageTables:        30128 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    14413140 kB
Committed_AS:    3952728 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       50208 kB
VmallocChunk:   34359677904 kB
HardwareCorrupted:     0 kB
AnonHugePages:   2140160 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        6912 kB
DirectMap2M:     8380416 kB


-- 

Vladimir