<h3 dir="auto">Describe the workflow you want to enable</h3>
<p dir="auto">Currently the GeneralizedLinearRegressor module only allows a scalar <code class="notranslate">alpha</code> regularization parameter to be used for all parameters. However, sometimes the modeler would like to input different amounts of regularization on certain parameters, either for interpretation or some other prior knowledge.</p>
<h3 dir="auto">Describe your proposed solution</h3>
<p dir="auto">The alpha parameter can take either a scalar value or an iterable of the same length as the number of parameters in the input matrix.</p>
<p dir="auto">The optimization algorithm will then flow as normal with the line below in glm.py naturally doing a vector to vector dot product instead of a scalar vector multiplication.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="coef_scaled = alpha * coef[offset:]"><pre class="notranslate"><code class="notranslate">coef_scaled = alpha * coef[offset:]
</code></pre></div>
<p dir="auto">I believe the alpha parameter passed to <code class="notranslate">scipy.optimize.minimize</code> in the <code class="notranslate">args</code> parameter isn't being used anywhere.</p>
<p dir="auto">For QA, I plan to output regression results in R or statsmodels for the same data and model specs</p>
<h3 dir="auto">Describe alternatives you've considered, if relevant</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Additional context</h3>
<p dir="auto">I modified this as part of my dayjob and also doing this as part of my programming work at my university. This is my first PR so please let me know if I'm missing anything.</p>
<h4 dir="auto">Description</h4>
<p dir="auto">This is a feature request in allow flexibility in elastic net, which allows the user to apply separate penalties to each coefficient of the L1 term. The default value of this penalty factor is 1, meaning the regular elastic net behavior. If the penalty factor of a feature is Zero, it means it's not penalized at all, and that the user wants this feature to be always there in the model.</p>
<p dir="auto">This feature is very useful in Bioinformatics and systems biology (that's why it is in Stanford's R package glmnet). With this feature, the user can run feature selection on a set of genes while making sure some genes stay in the system and not penalized (because of prior knowledge that they are involvid the system).</p>
<p dir="auto">Here is glmnet documentation explaining the penalty factor. Mainly, it's controlling the selection weight on the lasso term:</p>
<p dir="auto"><a href="https://web.stanford.edu/~hastie/glmnet/glmnet_alpha.html#lin" rel="nofollow">https://web.stanford.edu/~hastie/glmnet/glmnet_alpha.html#lin</a></p>
<p dir="auto">This feature is used in several papers, I've implemented in it in one of mine in scikit-learn. I've requests from other biologists to use this method in Python without having to recompile scikit-learn. I can do a pull request with this feature.</p>
<p dir="auto">Papers using this feature:</p>
<ul dir="auto">
<li>Altarawy, Doaa, Fatma-Elzahraa Eid, and Lenwood S. Heath. "PEAK: Integrating Curated and Noisy Prior Knowledge in Gene Regulatory Network Inference." Journal of Computational Biology 24.9 (2017): 863-873.</li>
<li>Greenfield, Alex, Christoph Hafemeister, and Richard Bonneau. "Robust data-driven incorporation of prior knowledge into the inference of dynamic regulatory networks." Bioinformatics 29.8 (2013): 1060-1067.</li>
<li>Friedman, Jerome, Trevor Hastie, and Rob Tibshirani. "Regularization paths for generalized linear models via coordinate descent." Journal of statistical software 33.1 (2010): 1.</li>
</ul>