While working with glibc I tried to reduce the data segment using [sbrk](https://www.gnu.org/software/libc/manual/html_mono/libc.html#Resizing-the-Data-Segment) using a negative parameter, and found a most strange behaviour.

I first `malloc`, then `free` it, then reduce data segment with `sbrk`, and then `malloc` again with same size as the first one.

The issue is, if the `malloc` size (both `malloc`s with same size) is small enough (32k, or eight 4k pages) then everything works fine. But when I increase a little the `malloc`-`free`-`malloc` size (to nine 4k pages) then I get the core dump. What is even more strange is that when I raise the `malloc` size to cross the `mmap` threshold (128k) then I get the adjust-abort behaviour.

The C code:

```
#define _GNU_SOURCE 1
#include <stdio.h>
#include <malloc.h>
#include <unistd.h>

// set MMAP_ALLOC_SIZE to 8 4k-pages it will work, 
// set it to 9 4k-pages it raises a 'segmentation fault (core dumped)'
// set it to 33 4k-pages it raises a 'break adjusted to free malloc space' and 'abort (core dumped)'
#define MMAP_ALLOC_SIZE   (33 * 4096)
#define PRINT_MEM { \
    struct mallinfo mi; \
    mi = mallinfo(); \
    printf("ptr    %p\n", ptr); \
    printf("brk(0) %p\n", sbrk(0)); \
    printf("heap   %d bytes\n", mi.arena); \
    printf("mmap   %d bytes\n\n", mi.hblkhd); \
}

int main(int argc, char *argv[])
{
    void *ptr;
    ptr = NULL;                      PRINT_MEM
    printf("1) will malloc > MMAP_THRESHOLD (128 KiB) ...\n");
    ptr = malloc(MMAP_ALLOC_SIZE);   PRINT_MEM
    printf("2) will free malloc ...\n");
    free(ptr);                       PRINT_MEM
    printf("3) will reduce brk  ...\n");
    ptr = sbrk(-100000);             PRINT_MEM
    printf("4) will malloc > MMAP_THRESHOLD (128 KiB) ... \n");
    ptr = malloc(MMAP_ALLOC_SIZE);   PRINT_MEM
    printf("5) completion.\n"); // never happens if MMAP_ALLOC_SIZE is > 8 4k-pages
    return 0;
}

```

Compiled with:

```
gcc -Wall testbrk.c -o testbrk

```

Which gives the successful output for `MMAP_ALLOC_SIZE (8 * 4096)`:

```
ptr    (nil)
brk(0) 0xf46000
heap   0 bytes
mmap   0 bytes

1) will malloc > MMAP_THRESHOLD (128 KiB) ...
ptr    0xf25670
brk(0) 0xf46000
heap   135168 bytes
mmap   0 bytes

2) will free malloc ...
ptr    0xf25670
brk(0) 0xf46000
heap   135168 bytes
mmap   0 bytes

3) will reduce brk  ...
ptr    0xf46000
brk(0) 0xf2d960
heap   135168 bytes
mmap   0 bytes

4) will malloc > MMAP_THRESHOLD (128 KiB) ... 
ptr    0xf25670
brk(0) 0xf2d960
heap   135168 bytes
mmap   0 bytes

5) completion.

```

The following abort output for `MMAP_ALLOC_SIZE (9 * 4096)`:

```
ptr    (nil)
brk(0) 0x1b7f000
heap   0 bytes
mmap   0 bytes

1) will malloc > MMAP_THRESHOLD (128 KiB) ...
ptr    0x1b5e670
brk(0) 0x1b7f000
heap   135168 bytes
mmap   0 bytes

2) will free malloc ...
ptr    0x1b5e670
brk(0) 0x1b7f000
heap   135168 bytes
mmap   0 bytes

3) will reduce brk  ...
ptr    0x1b7f000
brk(0) 0x1b66960
heap   135168 bytes
mmap   0 bytes

4) will malloc > MMAP_THRESHOLD (128 KiB) ... 
Segmentation fault (core dumped)

```

And the following adjust-abort output for `MMAP_ALLOC_SIZE (33 * 4096)`:

```
ptr    (nil)
brk(0) 0x1093000
heap   0 bytes
mmap   0 bytes

1) will malloc > MMAP_THRESHOLD (128 KiB) ...
ptr    0x7fdd1c7f6010
brk(0) 0x1093000
heap   135168 bytes
mmap   139264 bytes

2) will free malloc ...
ptr    0x7fdd1c7f6010
brk(0) 0x1093000
heap   135168 bytes
mmap   0 bytes

3) will reduce brk  ...
ptr    0x1093000
brk(0) 0x107a960
heap   135168 bytes
mmap   0 bytes

4) will malloc > MMAP_THRESHOLD (128 KiB) ... 
break adjusted to free malloc space
Aborted (core dumped)

```

So the `sbrk` reduction call works without error, but the subsequent `malloc` raises a core dump even if enough memory was still available.

I'm I doing something wrong or is this a limitation in the data segment resize?

**EDIT**: Besides the code solution using `malloc_trim()` I've posted below, very welcome from the accepted answer, there are some important things to know about this issue, as recovered from [chat](https://chat.stackoverflow.com/rooms/240905/discussion-on-question-by-arc-abort-in-glibc-while-trying-to-use-sbrk-to-reduce):

First, the `man` page says avoid using `sbrk`, but the glibc manual does not.

`malloc.c` from glibc does contains comments on `sbrk` possibly being called with signed integer - and thus negative integer - values of memory parameter reduction, and does have provisions on the callings of `brk` and `sbrk` to be coherent with `malloc`. The workings of `malloc` are not unsensitive to `sbrk`, they are not from "different levels" of coding, they are supposed to work together in harmony, at least by the code comments. Also, my first test case does work well, which means that `sbrk` is not an issue itself to work with `malloc`, but is only not treated in some specific cases.

And, finally, it matters that someone can break glibc allocation, it might be a security loophole. For instance, a hacker could use some instance of the glibc being accessed by another layer of inderection in order to summon `sbrk` in order to cause library crash. I'm not a security expert, but given that glibc has so many different uses around the planet, it could be in principle that some malicious programmer use this `sbrk` crash to gain access to unprotected systems. Not sure, but sure it should be investigated by glibc developers.

And I'm sure this is not a frivolous question.