Posted by Jad Abou-Moussa on Wednesday 25 December 2013 at 14:21

# Ridge regression

(category : Théorie)

Hi,
I don't really understand how you compute the parameter gamma in the Ridge regression (set 7/8/9). I know that it's equivalent to solve the minimization problem, but how can you solve this problem?
Thanks,
Jad

Comments

One way of solving the ridge regression minimization problem is to rewrite it as an ordinary least-squares problem for which we know the solution. I wrote this down on a pdf which you can find at http://smat.epfl.ch/courses/Regression/ridge_regression.pdf

Posted by Mikael Kuusela on Sunday 29 December 2013 at 19:44

Thanks!

Posted by Jad Abou-Moussa on Wednesday 1 January 2014 at 19:45

Hi,

I have another question i don't really understand what does that mean to recentre and rescale the covariates in the ridge regression?

Thanks,

Jad

I have another question i don't really understand what does that mean to recentre and rescale the covariates in the ridge regression?

Thanks,

Jad

Posted by Jad Abou-Moussa on Thursday 2 January 2014 at 22:40

The scaling and centering of the covariates is by no means fundamental but can have important practical implications. By removing the mean and dividing by the standard deviation, all the covariates are centered around zero and have the same spread. As a result, the ridge penalty term, which is centered at zero and symmetric around the origin, has the same meaning for each dimension of the unknown. Without the scaling and centering, we would need to change the penalty term to account for the different locations / spreads of the covariates. The removal of the mean also has the added convenience that beta_0 is estimated by the mean of Y and gamma by ridge regression without the constant.

Posted by Mikael Kuusela on Saturday 4 January 2014 at 15:19

Thanks!

Posted by Jad Abou-Moussa on Saturday 4 January 2014 at 21:28