Self-designed objective for linear regression learning A multiple linear regression is to use several predictor variables to predict the outcome of a response variable, like the following relationship:

$y_{i}=\beta_{1}x_{i1}+...+\beta_{p}x_{ip}+\epsilon_{i},   i=1,...,n$

I understand the typical objective to learn the $\beta$ paramters is least-squares, which means to minimize the sum of the sqaure of $\epsilon_{i}$. Now I want other kinds of objective, for example to maximize the Shannon entropy of the sequences of $\epsilon$ (or other self-specified objective). I googled towards this direction but no luck. I am wondering if there is any problem (and tool to solve it if possible) I can look into to do that?

Thank you for your help.
This question seems really old, it may be helpful to new visitors.

Yes, we can define any objective function and use gradient descent to minimize it. For that the objective function need to have some properties like


It should be convex function else we'll stuck at local minima or may never find the minimum value
It must be differentiable on every point and must have non zero derivative. Else gradient descent will not make progress and get stuck


If cost function have these properties then we can use tensorflow (or other library) to create graph to calculate the cost and then to calculate its gradient using auto-diff algorithm and use gradient descent to minimize it. 

Auto-diff basically breaks calculation of cost function into elementary arithmetic computation like addition, multiplication, division etc and use chain rule to find derivative.