How can I make my Merge Sort Code In C more secure, effective and efficient
make the code more compact and secure


Separate sort code from test code

To well address security issues, OP needs to present a clear distinction between what is the sort code and what is not.  Put all sort code in a separate .c file.

int vs. size_t

C employs size_t as the type that is neither to narrow nor too wide for array indexing and sizing.

Although int is commonly sufficient, to make more secure and work with large data, use size_t.

Aside from changing int --> size_t and "%d" --> "%zu", a singular challenge is to remember size_t is some unsigned type and code needs to consider that.  (size_t)x &lt; 0 is never true.

Beware large VLAs

Security risk: int left[n1]; is subject to stack overflow.  Consider *alloc() instead.  Insufficient memory can be detected that way.

Avoid exposing helper functions

mergeSequence() is not meant to be used outside of mergesort().  To prevent other code from using this functions, make it static.  Further, put it and mergesort() in their own .c file.

This increases security as mergeSequence()  then not need to handle all sorts of arguments, just the ones mergesort() generates.

All warnings enabled?

Save time, enable all warnings.

I'd expect mergeSort(num,0,(n-1)); (upper case S) to warn about function usage before declaration.

void mergesort(int*,int,int); (lower case S) is declared but not used.

Avoid generic names when a specific type is required

mergeSort() sorts int.  With a name like mergeSort, I expected something like qsort() with its size-of-element parameter and compare function.

Of re-architect your merge sort code to take a generic object type.

void SS_mergeSort(void *base, size_t nmemb, size_t size, 
    int (*compar)(const void *, const void *));`


Compact by eliminating print out

It is unexpected that a sort routine would do any printing.  Remove printf() and displaySequence() from merge...() functions.

Compact left,right code

int left[n1]; int right[n2]; can be int left_right[n1+n2]; and then the copying facilitated with the fast lib function memcpy().

memcpy(left_right, num, sizeof left_right);
int *left = left_right;
int *right = left_right + n1;
...