Home Forums Analytics / Predictive Analytics Penalties for adding variables

Viewing 3 posts - 1 through 3 (of 3 total)
• Author
Posts
• #3506
Porkins
Participant

I am confused on how to describe AIC and BIC. I think we can state that AIC requires a penalty to the loglikelihood of two for each parameter added and BIC requires a penalty of ln(n). This means for n larger than approx 7.4 BIC has a “larger penalty” than AIC for adding variables.

On one of the SOA solutions, where n is greater than 7.4, it states “For AIC, adding a variable requires an INCREASE in the loglikelihood of two per parameter” and furthermore “…BIC is a more conservative approach as there is a greater penalty for each parameter …”  isn’t a good fit indicated by MAXIMIZING the loglikelihood so wouldn’t that mean that BIC makes the loglikelihood BIGGER faster than AIC meaning it penalizes less and therefore AIC is the more stringent?

Someone please straighten me out here. Thanks.

#3596
greenlife
Participant

note that lower AIC / BIC means better model. when BIC increases log-likelihood faster than AIC, it means, adding one more predictor penalizes more.

#3943
P Singh
Participant

In case you’re still wondering…

Deviance = -2*loglikelihood

AIC = Deviance + 2p

BIC = Deviance + ln(n)*p

A *lower* AIC/BIC is better.

Viewing 3 posts - 1 through 3 (of 3 total)
• You must be logged in to reply to this topic.