
#1




log likelihood
Hi all,
SAS provides both the Rsquare and adjusted RSquare when you build a linear regression model with proc glm or proc reg/hpreg. The regular RSquare is nondecreasing, which mean that it either stays the same or increase as you throw more variables into the model, regardless of the variable significance. I am trying to assess whether the log likelihood possess the nondecreasing property as regular RSquare. SAS Genmod provides the loglikelihood value, which is always negative. I want to know if this value is either the same or increase as we throw more variables into the model, regardless of the new added variables significance. Thanks in advance. 
#2




According CAS Monograph #5, page 61, adding more variables "always reduces deviance, whether the predictor has any relation to the target variable or not." Deviance being a function of the maximum theoretical loglikelihood and the loglikelihood of the model. Since the saturated model represents a perfect fit, I take this to mean that the loglikelihood of the model increases.
Therefore you'd want to use penalized measures like AIC or BIC when comparing models, or an Ftest if one model is a nested version of another.
__________________
Recommended Readings for the EL Actuary  Recommended Readings for the EB Actuary 
#3




No. The saturated model is the best fit possible, not a perfect fit. These two ideas can be different when there are replicants.
__________________
"What do you mean I don't have the prerequisites for this class? I've failed it twice before!" "I think that probably clarifies things pretty good by itself." "I understand health care now especially very well." 
#4




Quote:
Quote:
__________________
Recommended Readings for the EL Actuary  Recommended Readings for the EB Actuary 
#5




Observations that have identical predictors. For example, if you have two drivers that are MALE16DRIVERSEDSPORTSCAR one with 3 accidents and one with only 2, you'll never perfectly fit if you are trying to predict numberof accidents.
__________________
"What do you mean I don't have the prerequisites for this class? I've failed it twice before!" "I think that probably clarifies things pretty good by itself." "I understand health care now especially very well." 
#6




Quote:
__________________
"What do you mean I don't have the prerequisites for this class? I've failed it twice before!" "I think that probably clarifies things pretty good by itself." "I understand health care now especially very well." 
#7




In theory, it is impossible for loglikelihood to decrease with more variables added to the model, but in practice it can and often does happen. There are at least two situation where this happens, most commonly with Tweedie distribution in SAS.
The first situation where it can happen is when loglikelihood is estimated from scaled deviance. Technically it's not a real loglikelihood, but your software can present it as such. When you add more variables, both your scale estimate as well as your deviance can change, and the resulting scaled deviance is not guaranteed to decrease. The second situation occurs when your model fails to completely converge in a subtle way, without a warning. It can happen when the variable you add is strongly correlated with existing predictors, and the algorithm terminates prematurely because of overly lax default tolerances. It can also happen when there are numerical stability issues with the algorithm. 
#8




thanks all.

Thread Tools  Search this Thread 
Display Modes  

