best activation function for ensemble? i have created some logistic regression model (different preprocessing) with softmax function. and i mix all model with an ensemble with a hierarchical method. so the output of all model (base) will be used as input for the final model (logistic regression too).

the default base model used a softmax function. i think transform a confident value into a probability will lose much information. so i have a plan to change the softmax into activation function.

what i learn in my CNN class, Relu is the best default activation function for an image. but my case is multi-class classification email. 

which activation function should i choose?
Relu, sigmoid or other?

thanks
I think you will have to experiment; there isn't generally a one-activation-fits-all for hierarchical models. Give the ones you mentioned a try, but perhaps starting with sigmoid and tanh and also a LeakyReLU.

The reason I'd perhaps leave out a normal ReLU at first is because that too would potential trim out information before the final model, as it simply removes negative weights to zero. A good final model should be able to correct for this, but I think its unnecessary to force it to.