Is it possible to customize the activation function in scikit-learn's MLPRegressor? Similar to this question about MLPClassifier, I suspect the answer is 'no' but I will ask it anyway.

Is it possible to change the activation function of the output layer in an MLPRegressor neural network in scikit-learn?

I would like to use it for function approximation.  I.e.

y = f(x)

where x is a vector of no more than 10 variables and y is a single continuous variable.

So I would like to change the output activation to linear or tanh.  Right now it looks like sigmoid.

If not, I fail to see how you can use scikit-learn for anything other than classification which would be a shame.

Yes, I realise I could use tensorflow or PyTorch but my application is so basic I think scikit learn would be perfect fit (pardon the pun there).

Is it possible to build a more customized network with MultiLayerPerceptron or perhaps from individual layers (sknn.mlp)?

UPDATE:

In the documentation for MultiLayerPerceptron it does say:


  For output layers, you can use the following layer types: Linear or
  Softmax.


But then further down it says:


  When using the multi-layer perceptron, you should initialize a
  Regressor or a Classifier directly.


And there is no example of how to instantiate a MultiLayerPerceptron object.
I see in the code for the MLPRegressor, that the final activation comes from a general initialisation function in the parent class: BaseMultiLayerPerceptron, and the logic for what you want is shown around Line 271.

# Output for regression
if not is_classifier(self):
    self.out_activation_ = 'identity'
# Output for multi class
...


Then during a foward pass this self.out_activation_ is called (defined here):

# For the last layer
output_activation = ACTIVATIONS[self.out_activation_]
activations[i + 1] = output_activation(activations[i + 1])


That ominous looking variable ACTIVATIONS is simply a dictionary ,with the keywords being the descriptions you can choose as a parameter in your MLP, each mapping an actual function. Here is the dictionary:

ACTIVATIONS = {'identity': identity, 'tanh': tanh, 'logistic': logistic,
               'relu': relu, 'softmax': softmax}


With all of this information, you might be able to come up with a few ways of putting in your custom function. Off the top of my head, I can't see a quick way to simply provide a function. You could for example:


define your function where all the other activation functions are defined
add it to that ACTIVATIONS dictionary
make self.out_activation_ equal to your custom function (or even a new parameter in MLPRegressor
cross your fingers it doesn't break something somewhere else
run it and solve the inevitable small adaptations that will be necessary in a few places


I'm, afraid I have never looked at the source code of that library before, so cannot give more nuanced advice. Perhaps there is a beautifully elegant way to do it that we have both overlooked.