On 3/29/13 10:12 PM, Paul Robert Marino wrote:
> well openmip is the app that executes it so thats where the limitation
> is probably coming from.
> With a little time on Google you will find plenty of posts on the
> subject of openmpi not being able to take advantage of all the
> resources available to it.
> The problem is Ive never seen an answer as to why, not that I looked
> all that long. Most of the suggestions talk about the ulimit setting
> which on the surface makes some sense but those numbers aren't right
> for an issue caused by a ulimit. the other  the most of the openmpi
> users who have asked the question and got told it was ulimits said
> latter that adjusting the ulimits didn't fix their issues. so again it
> sounds like a problem in the code for either openmpi or the code you
> are trying to execute with it.
> but the only other possibility is maybe SELinux is preventing
> something that capping the memory somehow as a side effect but i doubt
> it.
>

Very useful comments Paul. I am jumping to openmpi forum to ask if they 
are of help. Anyway, is there a way of testing the total memory of the 
system? Any simple bash program (no use of openmpi) that I can try for 
all the cores so that I can know that my system can take up to 8GB RAM?

Thanks,

D.

>
>
> On Thu, Mar 28, 2013 at 12:39 PM, Duke Nguyen <[log in to unmask]> wrote:
>> On 3/28/13 9:00 PM, Paul Robert Marino wrote:
>>> kernel.shmmax does nothing if you don't bump up kernel.shmall
>>> accordingly but I can tell you the cap is something wrong with your
>>> application not the OS.
>>> at one time I supported an application that in normal operation used
>>> 64BG Resident memory per instance.
>>> And currently my PostgreSQL servers often spike to as much as 2GB of
>>> ram per connection and would use more if i didn't cap it there in the
>>> configurations.
>>
>> Interesting, I never knew of any server process that takes that much of
>> memory. Anyway, it is good to know :).
>>
>>
>>> I don't think the kernel settings are your problem what language is
>>> the application written in?
>>> Is it executed by an other process like Apache or Tomcat for example?
>>
>> The app (a material simulation app) is just an input file which will calling
>> abinit (http://www.abinit.org/) using openmpi to run. So it is executed by
>> abinit. At the time the app runs, we make sure that no other process
>> (apache, tomcat etc...) is running, so basically the app should take all
>> available memory.
>>
>> Thanks,
>>
>> D.
>>
>>
>>>
>>> On Wed, Mar 27, 2013 at 11:09 PM, Duke Nguyen <[log in to unmask]> wrote:
>>>> On 3/27/13 11:52 PM, Attilio De Falco wrote:
>>>>> Just a stab in the dark, but did you check the Shared Memory kernel
>>>>> parameter (shmmax), type "cat /proc/sys/kernel/shmmax".  We have it set
>>>>> very
>>>>> high so that any process/thread can use as much memory as it needs.  You
>>>>> set
>>>>> the limit to 1 GB without rebooting by typing "echo 1073741824 >
>>>>> /proc/sys/kernel/shmmax"  or modify /etc/sysctl.conf and add the line
>>>>> "kernel.shmmax = 1073741824" so remains after a reboot.  I'm not sure
>>>>> about
>>>>> abinit but some fortran programs need shmmax limit to be set high…
>>>>
>>>> Hi Attilio, we already had it at very high value (not sure why, I never
>>>> changed/edited this value before)
>>>>
>>>> [root@biobos:~]# sysctl -p
>>>> net.ipv4.ip_forward = 1
>>>> net.ipv4.conf.default.rp_filter = 1
>>>> net.ipv4.conf.default.accept_source_route = 0
>>>> kernel.sysrq = 0
>>>> kernel.core_uses_pid = 1
>>>> net.ipv4.tcp_syncookies = 1
>>>> error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key
>>>> error: "net.bridge.bridge-nf-call-iptables" is an unknown key
>>>> error: "net.bridge.bridge-nf-call-arptables" is an unknown key
>>>> kernel.msgmnb = 65536
>>>> kernel.msgmax = 65536
>>>> kernel.shmmax = 68719476736
>>>> kernel.shmall = 4294967296
>>>> [root@biobos:~]# cat /proc/sys/kernel/shmmax
>>>> 68719476736
>>>>
>>>> Any other suggestions?
>>>>
>>>>
>>>>> On Mar 26, 2013, at 9:59 PM, Duke Nguyen <[log in to unmask]> wrote:
>>>>>
>>>>>> Hi folks,
>>>>>>
>>>>>> We have SL6.3 64bit installed on a box with two quad core and 8GB RAM.
>>>>>> We
>>>>>> installed openmpi, Intel Studio XE and abinit to run parallel (8
>>>>>> cores/processes) some of our applications. To our surprise, the system
>>>>>> usually takes only about half of available memory (about 500MB each
>>>>>> core)
>>>>>> and then the job/task was killed with the low-resource error.
>>>>>>
>>>>>> We dont really understand why there is a cap of "512MB" (I guess it
>>>>>> would
>>>>>> be 512MB instead of 500MB) for each of our cores whereas in theory,
>>>>>> each of
>>>>>> the core should be able to run up to 1GB. Any
>>>>>> suggestions/comments/experience about this issue?
>>>>>>
>>>>>> Thanks in advance,
>>>>>>
>>>>>> D.
>>>>>>