title: Best way to apply per neuron activation functions in pytorch
body: <p>I want to use a custom activation function that has a random component that gets applied to every neuron individually.</p>
  <p>If I use the standard method and call the activation function on a layer, it applies the same value to every neuron in that layer.</p>
  <p>I am looking for the most efficient way to have the activation function affect every neuron individually and would appreciate any advise on the topic.</p>
  <p>The only method I currently know about involves splitting the input tensor and applying the function each part and then concatenating the result, but I am hoping there is a more efficient way</p>