allocating shared memory fails beyond 8 GB
Hi, I am using managed_xsi_shared_memory to allocate memory as below: boost::interprocess::managed_xsi_shared_memory shmem(boost::interprocess::create_only, key, atol(argv[2])); For some reason, I cannot allocate more than 8 GB of memory. Both sysctl and ipcs have limits higher than that set (output below). ------ Shared Memory Limits -------- max number of segments = 1874919423 max seg size (kbytes) = 9765624999999 max total shared memory (kbytes) = 39999999999999996 min seg size (bytes) = 1 kernel.shmmax = 9999999999999999 kernel.shmall = 9999999999999999 kernel.shmmni = 1874919423 vm.hugetlb_shm_group = 0 Is there any reason why this allocation should fail ?
El 29/04/2014 3:56, vipin sachdeva escribió:
Hi,
I am using managed_xsi_shared_memory to allocate memory as below:
boost::interprocess::managed_xsi_shared_memory shmem(boost::interprocess::create_only, key, atol(argv[2]));
For some reason, I cannot allocate more than 8 GB of memory. Both sysctl and ipcs have limits higher than that set (output below).
Why type of exception throws? If an error is returned from the OS, the errno value will be stored in that exception. Best, Ion
It is error code 13; basically I would like to find out out beforehand how
much maximum shared memory (using MemFree from /proc/meminfo, sysctl info
etc) I can allocate on a machine and only make an allocation that big , and
only allocate a segment of that size. Are there any other system variables
I need to consider ? Thanks.
Error ID 13
terminate called after throwing an instance of
'boost::interprocess::interprocess_exception'
what(): Cannot allocate memory
Regards
vipin
On Tue, Apr 29, 2014 at 3:59 AM, Ion Gaztañaga
El 29/04/2014 3:56, vipin sachdeva escribió:
Hi,
I am using managed_xsi_shared_memory to allocate memory as below:
boost::interprocess::managed_xsi_shared_memory shmem(boost::interprocess::create_only, key, atol(argv[2]));
For some reason, I cannot allocate more than 8 GB of memory. Both sysctl and ipcs have limits higher than that set (output below).
Why type of exception throws? If an error is returned from the OS, the errno value will be stored in that exception.
Best,
Ion _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
El 29/04/2014 23:23, vipin sachdeva escribió:
It is error code 13; basically I would like to find out out beforehand how much maximum shared memory (using MemFree from /proc/meminfo, sysctl info etc) I can allocate on a machine and only make an allocation that big , and only allocate a segment of that size. Are there any other system variables I need to consider ? Thanks.
Is that value the errno value (get_native_error)? At least in linux (you haven't tell us which OS you are using), errno 13 is EACCES (Permission denied). ENOMEM is 12. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include... If you are referring to get_error_code(), then it's Interprocess' out_of_memory_error (it comes from ENOMEM Posix value). Maybe you don't have a contiguous page of a page of size, there are more processes consuming pages, old shared memory created by shmget has not been freed... According to POSIX ENOMEM can be returned if "the amount of available physical memory is not sufficient to fill the request". Do you have enough physical memory (not virtual) free? Perhaps you could debug the Interprocess code until the shmget() call to see if parameters are correctly passed to the OS. If those parameters are correct, then it's absolutely OS-dependent. Best, Ion
Thanks for your reply; I am using Linux on x86_64 and the get_native_error
was actually 12, get_error_code was 13. So I am running out of physical
memory; so it is not possible to limit the managed_xsi_shared_memory by the
amount of free virtual memory (your reply seems to imply that I am
constrained by the free physical memory ?
Regards
vipin
On Wed, Apr 30, 2014 at 12:59 AM, Ion Gaztañaga
El 29/04/2014 23:23, vipin sachdeva escribió:
It is error code 13; basically I would like to find out out beforehand
how much maximum shared memory (using MemFree from /proc/meminfo, sysctl info etc) I can allocate on a machine and only make an allocation that big , and only allocate a segment of that size. Are there any other system variables I need to consider ? Thanks.
Is that value the errno value (get_native_error)? At least in linux (you haven't tell us which OS you are using), errno 13 is EACCES (Permission denied). ENOMEM is 12.
https://git.kernel.org/cgit/linux/kernel/git/torvalds/ linux.git/tree/include/uapi/asm-generic/errno-base.h
If you are referring to get_error_code(), then it's Interprocess' out_of_memory_error (it comes from ENOMEM Posix value).
Maybe you don't have a contiguous page of a page of size, there are more processes consuming pages, old shared memory created by shmget has not been freed... According to POSIX ENOMEM can be returned if "the amount of available physical memory is not sufficient to fill the request". Do you have enough physical memory (not virtual) free?
Perhaps you could debug the Interprocess code until the shmget() call to see if parameters are correctly passed to the OS. If those parameters are correct, then it's absolutely OS-dependent.
Best,
Ion _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
El 30/04/2014 23:14, vipin sachdeva escribió:
Thanks for your reply; I am using Linux on x86_64 and the get_native_error was actually 12, get_error_code was 13. So I am running out of physical memory; so it is not possible to limit the managed_xsi_shared_memory by the amount of free virtual memory (your reply seems to imply that I am constrained by the free physical memory ?
I don't know what Linux does with this. In some OSs it might be limited by physical + swap space. In Linux, SHM_NORESERVE flag suggests that swap space is needed: http://linux.die.net/man/2/shmget SHM_NORESERVE (since Linux 2.6.15) This flag serves the same purpose as the mmap(2) MAP_NORESERVE flag. Do not reserve swap space for this segment. When swap space is reserved, one has the guarantee that it is possible to modify the segment. When swap space is not reserved one might get SIGSEGV upon a write if no physical memory is available. See also the discussion of the file /proc/sys/vm/overcommit_memory in proc(5) So maybe you could increase swap space. The OS needs to initialize (shared memory is zeroed) and maintain data somewhere. Best, Ion
participants (2)
-
Ion Gaztañaga -
vipin sachdeva