Phone evicting inactive applications from ram, despite plenty of memory free
And now to answer myself. Looking at Android 4.4.2 (KOT39H) sources, in ProcessList.java we have defined a maximum number of cached apps, set to an arbitrary value of 24:

static final int MAX_CACHED_APPS = 24;


This value is further split as follows by computeEmptyProcessLimit function: 2/3 goes to empty process limit and 1/3 goes to cached process limit.

So now we have an arbitrary limit of 16 for empty and 8 for cached processes. (But go buy a 3GB ram phone, to see how much unused memory you'll have)

In stock android, the only way to change this limit is at runtime, via setProcessLimit(), which requires rooted phone, an application manually installed in /system/priv-app and the SET_PROCESS_LIMIT permission granted.

(In CM11 for example it can also be set in build.prop, but this code is not present in stock 4.4.2)

Luckily there already is such an application on github, called SetAndroidProcessLimit, which, installed correctly, works (my thanks to the author)

Now I have 20 cached apps in ram, that's about all apps I frequently use, and any of them will start instantly, zero delay.



I also figured out how to modify MAX_CACHED_APPS directly in /system/framework/services.jar


download the jar to your pc, unpack the jar
decompile classes.dex with baksmali (or via apktool)
modify in ProcessList.smali and in ActivityManagerService.smali - because this is bytecode, you have to replace not only declaration of MAX_CACHED_APPS but every occurence of it with new desired value
recompile classes.dex
repack the jar
download to phone, replacing original jar (you have to remount rw /system)
reboot the phone; mine automatically will regenerate dalvik cache