目录
# 2. how to get how much has been allocated?
# 4. How much linux kernel alloc?
1. malloc introduction
Most implementations of C memory allocation functions will store accounting information for each block, either in-line or separately.
One typical way (in-line) is to actually allocate both a header and the memory you asked for, padded out to some minimum size. So for example, if you asked for 20 bytes, the system may allocate a 48-byte block:
- 16-byte header containing size, special marker, checksum, pointers to next/previous block and so on.
- 32 bytes data area (your 20 bytes padded out to a multiple of 16).
The address then given to you is the address of the data area. Then, when you free the block, free will simply take the address you give it and, assuming you haven't stuffed up that address or the memory around it, check the accounting information immediately before it. Graphically, that would be along the lines of:
____ The allocated block ____
/ \
+--------+--------------------+
| Header | Your data area ... |
+--------+--------------------+
^
|
+-- The address you are givenKeep in mind the size of the header and the padding are totally implementation defined (actually, the entire thing is implementation-defined (a) but the in-line accounting option is a common one).
The checksums and special markers that exist in the accounting information are often the cause of errors like "Memory arena corrupted" or "Double free" if you overwrite them or free them twice.
The padding (to make allocation more efficient) is why you can sometimes write a little bit beyond the end of your requested space without causing problems (still, don't do that, it's undefined behaviour and, just because it works sometimes, doesn't mean it's okay to do it).
-----------------------
2. how to get how much has been allocated?
There is no portable way and should not try to get it if it not for debug. check following stackoverflow answer. copied below and its link.
Everyone telling you it's impossible is technically correct (the best kind of correct).
For engineering reasons, it is a bad idea to rely on the malloc subsystem to tell you the size of an allocated block accurately. To convince yourself of this, imagine that you were writing a large application, with several different memory allocators — maybe you use raw libc malloc in one part, but C++ operator new in another part, and then some specific Windows API in yet another part. So you've got all kinds of void* flying around. Writing a function that can work on any of these void*s impossible, unless you can somehow tell from the pointer's value which of your heaps it came from.
So you might want to wrap up each pointer in your program with some convention that indicates where the pointer came from (and where it needs to be returned to). For example, in C++ we call that std::unique_ptr<void> (for pointers that need to be operator delete'd) or std::unique_ptr<void, D> (for pointers that need to be returned via some other mechanism D). You could do the same kind of thing in C if you wanted to. And once you're wrapping up pointers in bigger safer objects anyway, it's just a small step to struct SizedPtr { void *ptr; size_t size; } and then you never need to worry about the size of an allocation again.
However.
There are also good reasons why you might legitimately want to know the actual underlying size of an allocation. For example, maybe you're writing a profiling tool for your app that will report the actual amount of memory used by each subsystem, not just the amount of memory that the programmer thought he was using. If each of your 10-byte allocations is secretly using 16 bytes under the hood, that's good to know! (Of course there will be other overhead as well, which you're not measuring this way. But there are yet other tools for that job.) Or maybe you're just investigating the behavior of realloc on your platform. Or maybe you'd like to "round up" the capacity of a growing allocation to avoid premature reallocations in the future. Example:
SizedPtr round_up(void *p) {
size_t sz = portable_ish_malloced_size(p);
void *q = realloc(p, sz); // for sanitizer-cleanliness
assert(q != NULL && portable_ish_malloced_size(q) == sz);
return (SizedPtr){q, sz};
}
bool reserve(VectorOfChar *v, size_t newcap) {
if (v->sizedptr.size >= newcap) return true;
char *newdata = realloc(v->sizedptr.ptr, newcap);
if (newdata == NULL) return false;
v->sizedptr = round_up(newdata);
return true;
}To get the size of the allocation behind a non-null pointer which has been returned directly from libc malloc — not from a custom heap, and not pointing into the middle of an object — you can use the following OS-specific APIs, which I have bundled up into a "portable-ish" wrapper function for convenience. If you find a common system where this code doesn't work, please leave a comment and I'll try to fix it!
#if defined(__linux__)
// https://linux.die.net/man/3/malloc_usable_size
#include <malloc.h>
size_t portable_ish_malloced_size(const void *p) {
return malloc_usable_size((void*)p);
}
#elif defined(__APPLE__)
// https://www.unix.com/man-page/osx/3/malloc_size/
#include <malloc/malloc.h>
size_t portable_ish_malloced_size(const void *p) {
return malloc_size(p);
}
#elif defined(_WIN32)
// https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/msize
#include <malloc.h>
size_t portable_ish_malloced_size(const void *p) {
return _msize((void *)p);
}
#else
#error "oops, I don't know this system"
#endif
#include <stdio.h>
#include <stdlib.h> // for malloc itself
int main() {
void *p = malloc(42);
size_t true_length = portable_ish_malloced_size(p);
printf("%zu\n", true_length);
}Tested on:
- Visual Studio, Win64 —
_msize - GCC/Clang, glibc, Linux —
malloc_usable_size - Clang, libc, Mac OS X —
malloc_size - Clang, jemalloc, Mac OS X — works in practice but I wouldn't trust it (silently mixes jemalloc's
mallocand the native libc'smalloc_size) - Should work fine with jemalloc on Linux
- Should work fine with dlmalloc on Linux if compiled without
USE_DL_PREFIX - Should work fine with tcmalloc everywhere
3. Test
test on Ubuntu x86_64. From the test output, we can see the minimum allocated is 24 bytes, and aligned by 16 bytes (increase 16 bytes each time and the address's last 4 bits is always 0). The extra header used for counting and/or other purpose seems like 8 byte at least or (8+16n) bytes so that it is 16 bytes alligned.
#include <stdio.h>
#include <string.h>
#include <malloc.h>
int main()
{
char *p;
int n;
p = malloc(1);
n = malloc_usable_size(p);
free(p);
printf("n1: %d, p: %p\n", n, p);
p = malloc(24);
n = malloc_usable_size(p);
free(p);
printf("n24: %d, p: %p\n", n, p);
p = malloc(25);
n = malloc_usable_size(p);
free(p);
printf("n25: %d, p: %p\n", n, p);
p = malloc(41);
n = malloc_usable_size(p);
free(p);
printf("n41: %d, p: %p\n", n, p);
p = malloc(57);
n = malloc_usable_size(p);
free(p);
printf("n57: %d, p: %p\n", n, p);
}
hfyin@hfyin-VirtualBox:~/projects/c$ gcc t.c
hfyin@hfyin-VirtualBox:~/projects/c$ ./a.out
n1: 24, p: 0x55771fbc1260
n24: 24, p: 0x55771fbc1260
n25: 40, p: 0x55771fbc1690
n41: 56, p: 0x55771fbc16c0
n57: 72, p: 0x55771fbc1700
hfyin@hfyin-VirtualBox:~/projects/c$
4. How much linux kernel alloc?
malloc is c lib function, it uses brk()( or mmap?) to ask memory from kernel. So when malloc(1), how much kernel give?
Kernel manage process' memory include heap using memory area, memory area use page size as unit, which is usually 4K. So when request memory from kernel, the minimum size is page size. This can be find by checking process' memory map using pmap. Below is a example output for malloc(1), the 1st anon momery is for heap which is malloc(1), as it shows 4K bytes give to process.
It looks like very wasteful. So malloc() is here, malloc manage these 4k memory, so when process ask another mem like malloc(256), it will alloc memory from this 4k space.
a helpful link in stackoverflow.
$ sudo pmap -x 3137
3137: ./a.out
Address Kbytes RSS Dirty Mode Mapping
0000556fa9485000 4 4 4 r-x-- a.out
0000556fa9485000 0 0 0 r-x-- a.out
0000556fa9685000 4 4 4 r---- a.out
0000556fa9685000 0 0 0 r---- a.out
0000556fa9686000 4 4 4 rw--- a.out
0000556fa9686000 0 0 0 rw--- a.out
0000556fa9a0a000 132 4 4 rw--- [ anon ]
0000556fa9a0a000 0 0 0 rw--- [ anon ]