How to add more weight to certain features? I have extracted features from two types of signals. Prior to merging them to create one feature vector, I have computed an importance score of every feature within that type of signal.
I would like to weight the features according to those scores. Would the best way to do this be by multiplying every feature with its score and then concatenate the features of both signals, and should the data be normalized again after multiplication? Or, is there a different way to assign feature weights based on some precomputed score. (I have used multihead self attention to compute scores that I want to use as weights, for every feature within the modality). The model that I will use after merging the features, will be GRU.
The features should be first normalized before doing any kind of combination.
There are two ways to combine: concatenation and addition. You can try these both. Addition is only possible if the features sizes are same, though.