ROC curve comparison, software, statistics etc.

1. SPSS can draw line (also multiple lines for several predictors), can compute SE, 95%CI (nonparametric method), but cannot!!!! compare AUC ROC for two predictors with statistical test, (difference or lack of it can be deduced from overlap of 95%CI for the predictors beng compared, but no formal test, NO p-value!!!!).

2. methods for doing this - comparing AUC ROC of two curves (predictors), Delong 1988, Hanley 1982 ...., permutation/boostrapping CI ...

CF binormal distribution assumption.

SPSS - SE AUC ROC - two methods (options)
1. nonparametric,
2. - bi-negative exponential distribution????

*xxx* cf. Park et al 2004 Korean ..Journal - very nice intro, nonmath, but insightful what has to be done...., includes explanation on PARAMETRIC vs nonPARAMETRIC ROC estimation (smooth vs empirical), cf concept of partial AUC ROC
--- give nice and simple overview on assumptions, and when PARAMETRIC vs NONparametric methods should be used. (continuous, ordinal, skewed, sparse categories etc).


*xxx* more technical-but educational - SUGI .... tutorial on SAS macro that is able to compute p-value for difference between two AUC ROC, explaining different statistical approaches

more technical: SAS help on nonparametric comparison of two ROC curves.

the journal article reference and Rcode for three different methods of calculation of SE of AUC ROC. R forum -- (ref. Hajian-Tilaki, Hanley 2002?

Rcode for the DeLong method of AUC ROC SE estimation - nonparametric


ROCKIT by prof Metz from CHICAGO, free, downloaded, but IMHO will t ake some time to get around, to prepare input file, etc.

cf things with bootstraping, jacknifing etc, nonparametric tests.

- cf childhood predictors of young onset T2DM ... Franks et al. DIABETES 2007 = methods

CF comparison of 8 software packages for performing ROC analysis. (Stephan ClinChem 2003

medcalc by SCHOONJANS - commercial sw for cox hazard, ROC

Jane, Longton, Pepe 2009: Accommodating Covariates in ROC Analysis


ROC EXPLANATION

from Park 2004. cf Lyssenko, Meigs 2008 NEJM

One of the most popular measures is the area under
the ROC curve (AUC) (1, 2). AUC is a combined measure
of sensitivity and specificity. AUC is a measure of the
overall performance of a diagnostic test and is interpreted
as the average value of sensitivity for all possible values of
specificity (1, 2). It can take on any value between 0 and 1,
since both the x and y axes have values ranging from 0 to
1. The closer AUC is to 1, the better the overall diagnostic
performance of the test, and a test with an AUC value of 1
is one that is perfectly accurate (Fig. 2). The practical lower
limit for the AUC of a diagnostic test is 0.5. The line
segment from 0, 0 to 1, 1 has an area of 0.5 (Fig. 2). If we
were to rely on pure chance to distinguish those subjects
with versus those without a particular disease, the resulting
ROC curve would fall along this diagonal line, which is
referred to as the chance diagonal (Fig. 2) (1, 2). A diagnos-

cf values of test from 0.5 (random guess) to 1 (perfect predictor).