Geek asks and answers

How to manage large arrays

I have a C ++ program that uses several very large arrays of doubles, and I want to reduce the memory size of this particular part of the program. Currently, I allocate 100 of them, and they can be 100 MB each.

Now I have the advantage that eventually parts of these arrays become obsolete during subsequent parts of program execution, and in any case there is no need to ever have all one of them in memory.

My question is:

Is there a way to tell the OS after I created an array with a new one or malloc that some of it is no longer needed? I come to the conclusion that the only way to achieve this is to declare an array of pointers, each of which can point to a piece of, say, 1 MB of the desired array, so that old pieces that are no longer needed can be reused for new bits of the array. It seems to me that I am writing a user memory manager, which seems to be similar to a sledgehammer, which will also create a small performance hit

I cannot move data in an array because it will cause too many problems with thread conflict. arrays can be accessed by any of a large number of threads at any time, although only one thread ever writes to any given array.

+3
source share
5

. POSIX - Linux - madvise, . man:

madvise() , / , addr . , , ​​ . ( MADV_DONTNEED), . .

. madvise.

: -, . , , Linux.

mmap ( libc), . malloc . munmap - madvise:

void* data = ::mmap(nullptr, size, PROT_READ | PROT_WRITE,
    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
// ...
::munmap(data, size);

, madvise, :

madvise(static_cast<unsigned char*>(data) + 7 * page_size,
    3 * page_size, MADV_DONTNEED);

- , - , . , ​​ " " . , . , , . over-commit.

+4

, .

1 °) " , malloc, ?" " ". C ++ , .

2 °) ++, C, malloc.

3 °). , . std::vector.

4 °). , (std:: list), "" ( , ).

+1

A std::deque std::array<double,LARGE_NUMBER> , deque, , , , .

/, - .

+1

. , "", , , , , , . , , , ​​ . , () .

0

delete[] -ing new[] - . , . , .

0

More articles:


All Articles