title: cv.glmnet Ridge Regression lambda.min = lambda.1se?
body: <p>I'm currently running a ridge regression in R using the <code>glmnet</code> package, however, I recently ran into a new problem and was hoping for some help in interpreting my results. My data can be found here: <a href="https://www.dropbox.com/sh/hpxu3t0vqkrzfgf/AAB6F-yMYMfuI5E__gfDuW6sa?dl=0" rel="nofollow noreferrer">https://www.dropbox.com/sh/hpxu3t0vqkrzfgf/AAB6F-yMYMfuI5E__gfDuW6sa?dl=0</a></p>
  
  <p>My data consists of a 26531x428 observation matrix <code>x</code> and a 26531x1 response vector <code>y</code>. I am attempting to determine the optimal value of <code>lambda.min</code>, and when I run the code</p>
  
  <p><code>&gt; lambda=cv.glmnet(x=x,y=y,weights=weights,alpha=0,nfolds=10,standardize=FALSE)</code></p>
  
  <p>I get</p>
  
  <p><code>$lambda.min
  [1] 2.123479
  $lambda.1se
  [1] 619.0054</code></p>
  
  <p>which are results I would expect. However, I would like to add a slight tweak to this regression. I have prior knowledge of each of my 428 coefficients, and instead of shrinking each coefficient towards 0, as is the default with ridge regression, I would like to shrink each coefficient towards a specific value other than 0. After reaching out to Dr. Trevor Hastie, one of the creators of <code>glmnet</code>, he told me that this could be achieved by running the same code after substituting <code>y</code> with <code>y2</code>, where <code>y2 = y - x%*%d</code> and <code>d</code> is a 428x1 vector of coefficient priors. He said to then add <code>d</code> to my new coefficients, which would give me my prior-informed coefficients. After rerunning the code</p>
  
  <p><code>&gt; lambda=cv.glmnet(x=x,y=y2,weights=weights,alpha=0,nfolds=10,standardize=FALSE)</code></p>
  
  <p>I unfortunately get</p>
  
  <p><code>$lambda.min
  [1] 220.3026
  $lambda.1se
  [1] 220.3026</code></p>
  
  <p>The results of <code>plot(lambda)</code> look like this
  <a href="https://i.stack.imgur.com/ivP0b.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ivP0b.png" alt="lambda plot"></a></p>
  
  <p>Does anyone know why <code>glmnet</code> can't find a suitable <code>lambda.min</code>? Could it be because my vector of priors contains estimates that are too far off? Any help would be greatly appreciated!</p>