Creating a boolean for each value of each discrete variable explodes the number of features. I have an idea for solving it but I wasn't able to find anything on this on the internet, so perhaps someone could either point me to a resource or let me know whether they think this is possible. And if not, maybe point to another solution (all I could find was binarisation).So if we have training vectors $X_i$ and similarly labels $y_i$, I propose we partition the training set by some variable $j$, so that for each $X_i$ for which $j_i$ equals some value $q$, $X_i$ will be in $P(q)$.Then for each $q$ we calculate the mean $y_i$ for $P(q)$ and sort those averages, mapping each $q$ onto its index in the sorted list. This value is used as the feature for $j$. I suspect that this ought to be better than just giving each value a random index, as any remaining non-linearity will be hopefully fixed by the hidden layers. But it seems that these non-linear patterns will be easier to pick up if the function is somewhat quasi-monotonous.I think this could be improved, potentially instead of just finding the mean, another measure could work better and also for multiple discrete features, I am not sure if this process should be repeated for each variable separately or through a multi-dimensional partitioning.The main question is of course, would this process be helpful at all, and if not, is there anything similar that could solve this problem?
Instead of binarising discrete variables for NN algorithms, could I attempt to assign them a value by splitting into buckets and sorting?
machine learning;neural network