Home Forums Analytics / Predictive Analytics Penalties for adding variables

Viewing 3 posts - 1 through 3 (of 3 total)
  • Author
    Posts
  • #3506
    Porkins
    Participant

      I am confused on how to describe AIC and BIC. I think we can state that AIC requires a penalty to the loglikelihood of two for each parameter added and BIC requires a penalty of ln(n). This means for n larger than approx 7.4 BIC has a “larger penalty” than AIC for adding variables.

      On one of the SOA solutions, where n is greater than 7.4, it states “For AIC, adding a variable requires an INCREASE in the loglikelihood of two per parameter” and furthermore “…BIC is a more conservative approach as there is a greater penalty for each parameter …”  isn’t a good fit indicated by MAXIMIZING the loglikelihood so wouldn’t that mean that BIC makes the loglikelihood BIGGER faster than AIC meaning it penalizes less and therefore AIC is the more stringent?

      Someone please straighten me out here. Thanks.

       

      #3596
      greenlife
      Participant

        note that lower AIC / BIC means better model. when BIC increases log-likelihood faster than AIC, it means, adding one more predictor penalizes more.

        #3943
        P Singh
        Participant

          In case you’re still wondering…

          Deviance = -2*loglikelihood

          AIC = Deviance + 2p

          BIC = Deviance + ln(n)*p

           

          A *lower* AIC/BIC is better.

        Viewing 3 posts - 1 through 3 (of 3 total)
        • You must be logged in to reply to this topic.