viostorm | 9 May 01:50 2011
Picon

Hosmer-Lemeshow 'goodness of fit'

I'm trying to do a Hosmer-Lemeshow 'goodness of fit' test on my logistic
regression model.

I found some code here:

http://sas-and-r.blogspot.com/2010/09/example-87-hosmer-and-lemeshow-goodness.html

The R code is above is a little complicated for me but I'm having trouble
with my answer:

Hosmer-Lemeshow: p=0.6163585
le Cessie and Houwelingen test (Design library): p=0.2843620

The above link indicated they should be approximately equal which in my case
they are not, any suggestions or is there a package function people would
recommend in R for use with a logistic regression model?

Thanks in advance,

-Rob Schutt

--------------------------------
Robert Schutt, MD, MCS 
Resident - Department of Internal Medicine
University of Virginia, Charlottesville, Virginia 

------------------

########################################################
# Compute the Hosmer-Lemeshow 'goodness-of-fit' test
(Continue reading)

Frank Harrell | 9 May 02:38 2011
Picon

Re: Hosmer-Lemeshow 'goodness of fit'

Please read the documentation carefully, and replace the Design package with
the newer rms package.
The older Hosmer-Lemeshow test requires binning and has lower power.  It
also does not penalize for overfitting.  The newer goodness of fit test in
rms/Design should not agree with Hosmer-Lemeshow.

Frank

viostorm wrote:
> 
> I'm trying to do a Hosmer-Lemeshow 'goodness of fit' test on my logistic
> regression model.
> 
> I found some code here:
> 
> http://sas-and-r.blogspot.com/2010/09/example-87-hosmer-and-lemeshow-goodness.html
> 
> The R code is above is a little complicated for me but I'm having trouble
> with my answer:
> 
> Hosmer-Lemeshow: p=0.6163585
> le Cessie and Houwelingen test (Design library): p=0.2843620
> 
> The above link indicated they should be approximately equal which in my
> case they are not, any suggestions or is there a package function people
> would recommend in R for use with a logistic regression model?
> 
> Thanks in advance,
> 
> -Rob Schutt
(Continue reading)

viostorm | 9 May 03:21 2011
Picon

Re: Hosmer-Lemeshow 'goodness of fit'

Again, thanks so much for your help.

Is there a "for dummies" version for interpreting the le Cessie and
Houwelingen test.  I read the 1991 biometrics paper but honestly got lost in
the math.

Is it interpreted the same way the Hosmer-Lemeshow test is?  ie, a
non-significant result means model fits well.   

I guess what does, what would my p-value of p=0.284362 tell you about my
model?  

I would like to describe the goodness of fit of my model in a paper but I'm
worried the average medical reader would not know how to interpret the
result of this test whereas there are lots of references on interpreting
Hosmer-Lemeshow.

(my cross validated c-statistic was 0.69 and r^2 was 0.15)

Thanks again for your all help! (also with my k-fold crossvalidation
question!)

-Rob

-------------------------------- 
Robert Schutt, MD, MCS 
Resident - Department of Internal Medicine 
University of Virginia, Charlottesville, Virginia

--
(Continue reading)

Frank Harrell | 9 May 14:29 2011
Picon

Re: Hosmer-Lemeshow 'goodness of fit'

The test in the rms package's residuals.lrm function is the le Cessie - van
Houwelingen - Copas - Hosmer unweighted sum of squares test for global
goodness of fit.  Like all statistical tests, a large P-value has no
information other than there was not sufficient evidence to reject the null
hypothesis.  Here the null hypothesis is that the true probabilities are
those specified by the model.  Such an omnibus test, though having good
general power (better than Hosmer-Lemeshow) lacks power to detect specific
alternatives.  That's why I spend more time allowing for nonlinearity of
predictors.

Frank

viostorm wrote:
> 
> Again, thanks so much for your help.
> 
> Is there a "for dummies" version for interpreting the le Cessie and
> Houwelingen test.  I read the 1991 biometrics paper but honestly got lost
> in the math.
> 
> Is it interpreted the same way the Hosmer-Lemeshow test is?  ie, a
> non-significant result means model fits well.   
> 
> I guess what does, what would my p-value of p=0.284362 tell you about my
> model?  
> 
> I would like to describe the goodness of fit of my model in a paper but
> I'm worried the average medical reader would not know how to interpret the
> result of this test whereas there are lots of references on interpreting
> Hosmer-Lemeshow.
(Continue reading)

JimeBoHe | 29 Mar 20:34 2012
Picon

Re: Hosmer-Lemeshow 'goodness of fit'

I am a new user in R, so I am sorry if this is a basic question but I am kind
of lost here and I really would appreciatte your help.

I have behavioral information of sea lions and I've done a binomial
Generalized Linear model using Rcmdr, to understand what variables are
affecting the place where Male's fights are being made (water or land). One
of my possible models includes Agression Type, number of females around the
fight and temperature in the moment.

*******************
> GLM.2 <- glm(Place ~ AgType + Females + Temp, family=binomial(cloglog), 
+   data=June)

> summary(GLM.2)

Call:
glm(formula = Place ~ AgType + Females + Temp, family = binomial(cloglog), 
    data = June)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.4448  -0.7248   0.4220   0.6741   1.8798  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  3.43071    0.82164   4.175 2.97e-05 ***
AgType      -1.23463    0.37166  -3.322 0.000894 ***
Females      0.05812    0.01320   4.404 1.06e-05 ***
Temp        -0.08614    0.02453  -3.512 0.000444 ***
---
(Continue reading)

Michael Dewey | 30 Mar 14:23 2012
Picon

Re: Hosmer-Lemeshow 'goodness of fit'

At 19:34 29/03/2012, JimeBoHe wrote:
>I am a new user in R, so I am sorry if this is a basic question but I am kind
>of lost here and I really would appreciatte your help.

Dear Jimena

Comments in-line

>I have behavioral information of sea lions and I've done a binomial
>Generalized Linear model using Rcmdr, to understand what variables are
>affecting the place where Male's fights are being made (water or land). One
>of my possible models includes Agression Type, number of females around the
>fight and temperature in the moment.
>
>*******************
> > GLM.2 <- glm(Place ~ AgType + Females + Temp, family=binomial(cloglog),
>+   data=June)
>
> > summary(GLM.2)
>
>Call:
>glm(formula = Place ~ AgType + Females + Temp, family = binomial(cloglog),
>     data = June)
>
>Deviance Residuals:
>     Min       1Q   Median       3Q      Max
>-2.4448  -0.7248   0.4220   0.6741   1.8798
>
>Coefficients:
>             Estimate Std. Error z value Pr(>|z|)
(Continue reading)


Gmane