Is there a way to set a different activation function for each hidden unit in one layer in keras?
If I get the point, you can use a similar code like the following:

from keras.layers import merge, Convolution2D, MaxPooling2D, Input

input = Input(shape=(256, 256, 3))

seq1 = Dense(1, activation = 'relu')(input)
seq2 = Dense(1, activation = 'sigmoid')(input)
seq3 = Dense(1, activation = 'tanh')(input)

acum = merge([seq1, seq2, seq3], mode='concat', concat_axis=1)


Depending on your task, specify concat_axis.