Development and validation of a composite score for excessive alcohol use screening
===================================================================================

* Wanzhu Tu
* Chenghao Chu
* Shanshan Li
* Suthat Liangpunsakul

## Abstract

This study was undertaken to develop a composite measure that combines the discriminant values of individual laboratory markers routinely used for excessive alcohol use (EAU) for an improved screening performance. The training sample consisted of 272 individuals with known history of EAU and 210 non-alcoholic individuals. The validation sample included 100 EAU and 75 controls. We used the estimated regression coefficients and the observed marker values to calculate the individual's composite screening score; this score was converted to a probability measure for excessive drinking in the given individual. A threshold value for the screening score based on an examination of the estimated sensitivity and specificity associated with different threshold values was proposed. Using regression coefficients estimated from the training sample, a composite score based on the levels of aspartate aminotransferase, alanine aminotransferase, per cent carbohydrate-deficient transferrin and mean corpuscular volume was calculated. The areas under the receiver operating characteristic curve (AUC) value of the selected model was 0.87, indicating a strong discriminating power and the AUC was better than that of each individual test. The score >0.23 corresponded to a sensitivity of 90% and a specificity of nearly 60%. The AUC value remained at a respectable level of 0.83 with the sensitivity and specificity at 91% and 49%, respectively, in the validation sample. We developed a novel composite score by using a combination of commonly used biomakers. However, the development of the mechanism-based biomarkers of EAU is needed to improve the screening and diagnosis of EAU in clinical practice.

*   Alcohol Drinking
*   Biological Markers

### Significance of this study

#### What is already known about this subject?

*   Drinking becomes excessive when it causes/elevates the risk for alcohol-related problems.

*   Failure to detect excessive alcohol use in a timely fashion could delay intervention, and leads to serious sequelae.

*   Reports from our research team and others clearly highlight the insufficient diagnostic accuracy of current non-invasive markers when they are used as a single marker for excessive alcohol use screening.

#### What are the new findings?

*   A composite measure that combines the discriminant values of commonly used individual laboratory markers improves screening performance for excessive alcohol use.

*   Our results need to be validated in a larger cohort.

#### How might these results change the focus of research?

*   Further research to identify mechanism-based biomarkers to screen for excessive alcohol use is needed.

*   The effects of ethanol on multiple organ systems are likely to reflect the changes in quantity or quality of constituents or novel serum proteins.

*   These changes in the serum protein may serve as the potential biomarkers for excessive alcohol use.

## Introduction

Excessive alcohol use is a major public health problem.1 Its incidence is on the rise with a parallel increase in the prevalence of alcohol-related diseases.2 ,3 Screening for excessive alcohol use (EAU) in clinical setting is of importance, as early detection could lead to timely intervention to prevent subsequent adverse health outcomes.4 ,5

Excessive alcohol drinking is usually screened in clinical practice through patient interviews or questionnaires, by using validated instruments such as AUDIT,6 CAGE,7 ,8 or reports from collateral family with direct interaction with patients.4 Laboratory measures are also used to aid alcoholism screening: Among the most commonly used markers, mean corpuscular volume (MCV) has only a modest level of diagnostic sensitivity (approximately 50%).4 Carbohydrate deficient transferrin (CDT) is generally more sensitive, as excessive alcohol consumption is known to lead to reduced number of carbohydrate residues attached to serum transferrin.9 ,10 The diagnostic accuracy of CDT, however, tends to vary greatly by clinical population, with sensitivity ranging from 53% to 80% in screening for subjects with chronic alcohol use.4 ,9 ,10 Similarly, serum levels of hepatic enzymes γ-glutaryl transferase (GGT), aspartate aminotransferase (AST), and alanine aminotransferase (ALT) are also potential markers because their activities are known to be altered by excessive and prolonged alcoholic exposure.11 Our research team recently reported that levels of diagnostic sensitivity of GGT, AST and ALT were 50%, 27%, and 27%, respectively, for excess drinking screening,4 which were generally close to the 35–60% range reported by previous studies.12–14 Reports from our research team and those from others clearly highlight the insufficient diagnostic accuracy of these markers when they are used single markers for EAU. The purpose of the current research was to explore the development of a composite measure that combines the discriminant values of these individual laboratory markers for an improved screening performance.

## Methods

### Study samples

Data from two separate groups of subjects were used to develop and to validate the composite screening measure. The training sample, that is, the group of participants whose data were used to develop the screener, consisted of 272 individuals with EAU (ie, cases), and 210 non-alcoholic individuals. These individuals were recruited from January 2012 to June 2014. The cases were patients admitted to Fairbanks Drug and Alcohol Treatment Center (Indianapolis, Indiana, USA) for alcohol rehabilitation. Enrolled cases met the criteria for ‘excessive drinking’; defined by NIH/NIAAA as men who drink more than four standard drinks in a day (or more than 14/week) and women who drink more than three standard drinks in a day (or more than 7/week). Patients reported the last use of alcohol within 0–72 h before enrollment. The non-excessive drinking participants were recruited from Richard L. Roudebush Veterans Administration Medical Center (RLR VAMC, Indianapolis, Indiana, USA). All study participants were aged at least 21 years. Individuals who had active and serious medical diseases (such as congestive heart failure, chronic obstructive pulmonary disease, cancer, uncontrolled diabetes, and chronic renal failure) at the time of screening, had a history of any systemic infection within 4 weeks prior to the study, or had a history of recent major surgeries within the past 3 months were excluded from study participation.

The validation sample, that is, the group of individuals whose data were used to validate the diagnostic measure, included 100 excessive drinkers and 75 non-excessive drinkers. Participants in the validation sample were enrolled, independently from June 2013 to December 2014, from the previously described recruitment sites. The inclusion and exclusion criteria remained the same as those in the training sample. Participants in the two study samples provided written informed consent prior to study enrollment. A local institutional review board approved the study design and the patient enrollment and assessment protocol.

### Data collection and clinical evaluation

All participants completed a self-administered questionnaire such as demographic data and AUDIT-C. The Time Line Follow-Back (TLFB) questionnaire was used to determine the amount of alcohol consumption over the 30-day period before the study date. It was administered in person by trained study coordinators who reviewed the instructions with the subjects prior to administering the questionnaire. The TLFB offers a retrospective report of daily alcohol consumption over the past 30 days; drinks per drinking occasion, and pattern of drinking can be computed.15 Blood samples were obtained for assay of commonly used markers to identify chronic alcohol use (such as GGT, CDT, AST, and ALT and MCV); samples were analyzed at the RLR VAMC.

### Statistical analyses

Demographic and clinical characteristics of the study participants in the training and validation samples were summarized and described separately. Characteristics were compared between the cases and non-cases. Continuous variables were compared using Student t tests, including % CDT, levels of GGT, AST, ALT and MCV. Categorical variables were compared using χ2 tests.

The study sample was divided into two subsets: data from 70% of the subjects were used for model development and the rest for model validation. Within the development sample, we used multiple logistic regression models to classify the cases and non-cases. Individual markers were first examined for their discriminant power in simple logistic regression models. We then performed a stepwise model selection, starting with all five markers in the model. We used Akaike Information Criterion (AIC) to determine the superiority of the competing models. AIC is a measure for the quality of statistical models that balances the goodness-of-fit with model complexity. It is widely used for model selection in analytical practice.16 We chose the model with the smallest AIC value. For the competing models, we also examined the p values of the individual markers, as well as the areas under the receiver operating characteristic curve (AUC).17 In particular, we compared the AUC of all single marker models against that of the selected model using a non-parametric test.18 The receiver operating characteristic curve (ROC) of the final model was presented graphically. To ensure the numerical stability of the selected model, we examined all pairwise correlations and calculated variance inflation factor (VIF) associated with the final model. A larger VIF value typically indicates the presence of multicolinearity.19

For a given individual, we used the estimated regression coefficients and the observed marker values to calculate the individual's composite screening score; this score was converted to a probability measure for excessive drinking in the given individual. We proposed a threshold value for the screening score based on an examination of the estimated sensitivity and specificity associated with different threshold values. Details of the score calculation and corresponding sensitivity and specificity were reported in the Results section.

To validate the proposed composite screening score, we calculated the AUC value of the model in the validation sample. Using the previously identified threshold value, we classified all subjects in the validation sample either as cases or as non-cases. We then calculated and reported the levels of sensitivity and specificity of the composite screener, as well as its positive and negative predictive values, in the validation sample. As expected, the AUC values were sample-specific. For a more accurate assessment of the predictive performance, we conduct a cross-validation study by resampling the original sample 20 times, for each resample, we divide the data into model development subset and validation subset. The mean AUC values based on the resampled data were obtained and reported.

## Results

### Demographic and clinical characteristics of training and verification cohorts

The detailed demographic and clinical characteristics for training and validation samples are presented in tables 1 and 2. For the training sample, excessive drinkers were older (39.7 vs 32.5 years, p<0.001), had higher percentage of divorce/separation (28% vs 18%, p=0.0001), and had lower BMI (27.8 vs 29.9 kg/m2, p=0.05) when compared to those of non-excessive drinkers. As expected, excessive drinkers had higher AUDIT scores (26.9 vs 4.7, p<0.0001), greater total standard drinks in the past 30 days (257 vs 13 drinks, p<0.0001), higher average drinks per drinking day (12.9 vs 2.5 drinks, p<0.0001), and a higher number of drinking days in the past month (20 vs 4.5 days, p<0.001). Excessive drinkers had significantly higher levels of serum AST (35.2 vs 26.0 U/L, p=0.0002), GGT (85.5 vs 35.6 U/L, p<0.001), MCV (93.4 vs 89.4 fL, p=0.002) and %CDT (2.55 vs 1.63%, p<0.001). Higher concentrations of ALT were observed in non-excessive drinkers (50.7 vs 36.5 U/L, p<0.001).

View this table:
[Table 1](/content/64/5/1006/T1)

Table 1 
Baseline demographic and alcohol drinking characteristics of the training cohort

View this table:
[Table 2](/content/64/5/1006/T2)

Table 2 
Baseline demographic and alcohol drinking characteristics of the validation cohort

Demographic and clinical characteristic of the validation sample resembled those of the training sample. Age, sex and race distributions of the study participants in the training and validation samples were similar. The cases in the two samples also had similar AUDIT-C scores as well as alcohol drinking patterns based on the TLFB.

### Development of a composite screening score

Simple logistic regression models (ie, models with one predictor) were used to determine the significance of individual markers (table 3). The areas under the ROC curves associated with the individual markers were obtained from simple logistic regression analysis and reported in table 3. The AUC values of the individual markers ranged from 0.63 to 0.77, with CDT having the highest value.

View this table:
[Table 3](/content/64/5/1006/T3)

Table 3 
Model estimates on training cohort

To develop the composite screening model, we used multivariate logistic regression. Different combinations of the individual markers were included in the separate logistic regression analysis, using data from the training sample.

The final model was selected based on the AIC. The model corresponding to the lowest AIC value included all markers except GGT. Adding GGT to the model resulted in an increase in AIC and the model became less stable numerically. Breslow-Lemeshow test indicated the selected model was not significantly different from the full model with all five markers.20

The regression coefficients and corresponding p values of the final model were tabulated in table 3. To ensure that the model does not suffer from multicollinearity, we examined the pairwise associations among the markers. ALT and AST had a moderate level of correlation (ρ=0.55); all other correlations were weaker. But these correlations did not cause model instability. VIF values were all below 2.0, indicating an absence of multicollinearity.

Using regression coefficients estimated from the training sample, we arrived at the following screening score: ![Formula][1]</img>

The derived screening score (P) could be interpreted as an individual's estimated probability of excessive drinking.

### Determination of threshold value

The AUC value of the selected model was 0.87 (see figure 1A), indicating a strong discriminating power. Compared to the single marker screening models, the composite screening model had superior classification accuracy, as evidenced by its greater value of AUC (see table 3). Formal comparison of the AUC values of the single marker models and that of the selected model confirmed that the latter had significantly greater AUC value (all p values <0.0001). We determined the threshold values for screening of excessive drinking by examining the levels of sensitivity and specificity corresponding to the different threshold values. We compared the sensitivity and specificity of various threshold values, as shown in table 3. In situations where sensitivity and specificity are of equal interest, the value that maximizes sensitivity+specificity−1 (known as the Youden's Index) is sometimes used as the cut-off.21 But due to the lack of universally accepted criteria for selecting optimal cut-off points, investigators often choose values that are most sensible to their applications. In the current study, we valued sensitivity more than specificity; so we chose a threshold value to ensure a good sensitivity. In particular, we noted that P score greater than 0.23 corresponded to a sensitivity of 90% and a specificity of nearly 60%. Lowering the threshold value would classify more patients as alcoholic, thus sensitizing the screener, at the expense of increased false positivity. For example, a threshold of 0.2 would increase the sensitivity to almost 94% while reducing specificity to 50% (table 4).

View this table:
[Table 4](/content/64/5/1006/T4)

Table 4 
Levels of sensitivity and specificity corresponding to different threshold values, estimated using data from the training sample

![Figure 1](/https://d3hme472k3gd2d.cloudfront.net/content/jim/64/5/1006/F1.medium.gif)

[Figure 1](/content/64/5/1006/F1)

Figure 1 
Receiver operating characteristic curves associated with the individual markers and the composite screener in the training and validation samples. (A) For the training dataset, the areas under the ROC curve (AUC) for ALT, AST, CDT, MCV, and the composite screener were 0.71, 0.63, 0.77, 0.66, and 0.87, respectively. (B) For the validation data, the AUC values for ALT, AST, CDT, MCV, and the composite screener were 0.79, 0.55, 0.53, 0.69 and 0.83, respectively. ALT, alanine aminotransferase; AST, aspartate aminotransferase; AUC, areas under the receiver operating characteristic curve; CDT, Carbohydrate-deficient transferrin; MCV, mean corpuscular volume; ROC, receiver operating characteristic curve.

### Validation of the screening score and the selected threshold value

To validate the proposed screening model, we presented the estimated ROC curve in the independent validation sample. As shown in figure 1B, the AUC value remained at a respectable level of 0.83. Using 0.23 as the screening threshold, we estimated the screener's sensitivity and specificity to be 91% and 49%, respectively, in the validation sample. The cross-validation study based on 20 resamples showed a mean AUC value of 0.88, thus confirming the predicative performance of the proposed screener.

## Discussion

Excessive alcohol use, if left undetected and untreated, could lead to significant health sequelae and devastating social consequences. Accurate screening for EAU, followed by appropriate counseling and patient abstinence, is essential for timely care of these patients.4 Thorough interview, good history taking, and the use of standardized questionnaires are important during clinic encounters to screen for EAU.6 ,7 ,22 ,23 However, a major weakness of the behavioral screening instruments is their suboptimal levels of sensitivity, which lead to significant portion of EAU undetected.24 It is therefore crucial to develop more sensitive and objective screening measures so that healthcare providers can properly assess the drinking status of their patients.

A number of laboratory tests have been routinely used as biomarkers indicative of a person's alcohol intake. Several reflect the activity of the hepatic enzymes, such as AST, ALT and GGT.4 ,22 Other non-hepatic enzyme markers have been used, for example, MCV and %CDT.22 MCV, the volume of red blood cells, has also been shown to be positively associated with heavy drinking.22 Transferrin molecules in the blood usually contain several carbohydrate components. In chronic heavy drinkers, however, the number of carbohydrate components in each transferrin molecule is reduced, resulting in the increase in %CDT.9

These markers, when used individually for screening purposes, are often not sufficiently sensitive. As we have reported previously, the diagnostic performance of these markers left much to be desired.4 In this research, we constructed a composite screening tool by combining the commonly used biomarkers, in hope for an improved diagnostic performance. The resultant score was expressed as a function of AST, ALT, CDT and MCV. We showed that using a threshold value of 0.23, the proposed screening tool was able to achieve a high level of sensitivity (>90%), without greatly sacrificing the specificity (∼60%). A validation study further confirmed the performance of the proposed screening method. The work has shown that it is possible to derive a much improved screening sensitivity by combining the markers into a composite measure, even when the individual markers are not sufficiently discriminant.

A few limitations deserve discussion. As previously mentioned, identifying the ‘true’ cases of EAU is challenging, given the fact that no ‘objective measurements’ exist. We thus used recruited subjects from the alcohol rehabilitation hospital as the true cases. As shown in tables 1 and 2, our cases had significantly higher AUDIT scores as well as reported quantity of alcohol consumption in the past 30 days using TLFB questionnaires. While the differences between the cases and non-cases are not unexpected, the separation could still influence the performance of the proposed screener. This said, we note that the diagnostic performance of individual markers in the same study sample has not been nearly as good, thus reassuring the advantages of the new screening tool. We also note that the excellent level of sensitivity of the proposed screener was achieved at the expense of lower specificity. For a screening tool we are mainly interested in maintaining a higher level of sensitivity, which allows us to more readily identify individuals at risk for EAU. Diagnostic decisions, however, will be made by care providers with additional clinical assessments, which help to compensate the lower specificity of the screener. Finally, considering the limited nature of our validation study, further investigations are needed to establish the generalisability of the proposed screener, and the appropriateness of the threshold values in different clinical populations.

Notwithstanding these limitations, we developed a novel composite score by using a combination of commonly used biomakers. However, the development of the mechanism-based biomarkers of EAU is needed to improve the screening and diagnosis of EAU in clinical practice.

## Footnotes

*   Contributors All authors have read and approved the manuscript for submission. All have made a substantial contribution to the conception, design, gathering, analysis and/or interpretation of data and a contribution to the writing and intellectual content of the article; and acknowledge that they have exercised due care in ensuring the integrity of the work.

*   Funding This study is supported by K08 AA016570 and U01AA021840 from the NIH/NIAAA, 1I01CX000361-01 from the Veterans Affairs Research and Administration, and W81XWH-12-1-0497 from USA Department of Defense (All to SL).

*   Competing interests None declared.

*   Provenance and peer review Not commissioned; externally peer reviewed.

## References

1.  Polednak AP. Surveillance of US Death Rates from Chronic Diseases Related to Excessive Alcohol Use. Alcohol Alcohol 2016;51:54–62. [doi:10.1093/alcalc/agv056](http://dx.doi.org/10.1093/alcalc/agv056)
    
    [Abstract/FREE Full Text](/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiYWxjYWxjIjtzOjU6InJlc2lkIjtzOjc6IjUxLzEvNTQiO3M6NDoiYXRvbSI7czoxOToiL2ppbS82NC81LzEwMDYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

2.  Trimble G, Zheng L, Mishra A, et al. Mortality associated with alcohol-related liver disease. Aliment Pharmacol Ther 2013;38:596–602. [doi:10.1111/apt.12432](http://dx.doi.org/10.1111/apt.12432)
    
    [PubMed](/lookup/external-ref?access_num=23889765&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 

3.  Cutright P, Fernquist RM. Predictors of per capita alcohol consumption and gender-specific liver cirrhosis mortality rates: thirteen European countries, circa 1970-1984 and 1995-2007. Omega (Westport) 2010;62:269–83. [doi:10.2190/OM.62.3.d](http://dx.doi.org/10.2190/OM.62.3.d)
    
    [PubMed](/lookup/external-ref?access_num=21495535&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 

4.  Gough G, Heathers L, Puckett D, et al. The Utility of Commonly Used Laboratory Tests to Screen for Excessive Alcohol Use in Clinical Practice. Alcohol Clin Exp Res 2015;39:1493–500. [doi:10.1111/acer.12780](http://dx.doi.org/10.1111/acer.12780)
    
    [PubMed](/lookup/external-ref?access_num=26110815&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 

5.  Liangpunsakul S, Lai X, Ross RA, et al. Novel serum biomarkers for detection of excessive alcohol use. Alcohol Clin Exp Res 2015;39:556–65. [doi:10.1111/acer.12654](http://dx.doi.org/10.1111/acer.12654)
    
    [PubMed](/lookup/external-ref?access_num=25704570&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 

6.  Aalto M, Alho H, Halme JT, et al. The Alcohol Use Disorders Identification Test (AUDIT) and its derivatives in screening for heavy drinking among the elderly. Int J Geriatr Psychiatry 2011;26:881–5. [doi:10.1002/gps.2498](http://dx.doi.org/10.1002/gps.2498)
    
    [CrossRef](/lookup/external-ref?access_num=10.1002/gps.2498&link_type=DOI) 
    
    [PubMed](/lookup/external-ref?access_num=20661878&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 

7.  Knight JR, Sherritt L, Harris SK, et al. Validity of brief alcohol screening tests among adolescents: a comparison of the AUDIT, POSIT, CAGE, and CRAFFT. Alcohol Clin Exp Res 2003;27:67–73. [doi:10.1097/01.ALC.0000046598.59317.3A](http://dx.doi.org/10.1097/01.ALC.0000046598.59317.3A)
    
    [CrossRef](/lookup/external-ref?access_num=10.1111/j.1530-0277.2003.tb02723.x&link_type=DOI) 
    
    [PubMed](/lookup/external-ref?access_num=12544008&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 
    
    [Web of Science](/lookup/external-ref?access_num=000180709800011&link_type=ISI) 

8.  Ewing JA. Detecting alcoholism. The CAGE questionnaire. JAMA 1984;252:1905–7. [doi:10.1001/jama.1984.03350140051025](http://dx.doi.org/10.1001/jama.1984.03350140051025)
    
    [CrossRef](/lookup/external-ref?access_num=10.1001/jama.1984.03350140051025&link_type=DOI) 
    
    [PubMed](/lookup/external-ref?access_num=6471323&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 
    
    [Web of Science](/lookup/external-ref?access_num=A1984TN44700023&link_type=ISI) 

9.  Golka K, Sondermann R, Reich SE, et al. Carbohydrate-deficient transferrin (CDT) as a biomarker in persons suspected of alcohol abuse. Toxicol Lett 2004;151:235–41. [doi:10.1016/j.toxlet.2004.01.023](http://dx.doi.org/10.1016/j.toxlet.2004.01.023)
    
    [CrossRef](/lookup/external-ref?access_num=10.1016/j.toxlet.2004.01.023&link_type=DOI) 
    
    [PubMed](/lookup/external-ref?access_num=15177658&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 
    
    [Web of Science](/lookup/external-ref?access_num=000222226100026&link_type=ISI) 

10. Golka K, Wiese A. Carbohydrate-deficient transferrin (CDT)—a biomarker for long-term alcohol consumption. J Toxicol Environ Health B Crit Rev 2004;7:319–37. [doi:10.1080/10937400490432400](http://dx.doi.org/10.1080/10937400490432400)
    
    [CrossRef](/lookup/external-ref?access_num=10.1080/10937400490432400&link_type=DOI) 
    
    [PubMed](/lookup/external-ref?access_num=15205047&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 
    
    [Web of Science](/lookup/external-ref?access_num=000221219700004&link_type=ISI) 

11. Liangpunsakul S, Qi R, Crabb DW, et al. Relationship between alcohol drinking and aspartate aminotransferase:alanine aminotransferase (AST:ALT) ratio, mean corpuscular volume (MCV), gamma-glutamyl transpeptidase (GGT), and apolipoprotein A1 and B in the U.S. population. J Stud Alcohol Drugs 2010;71:249–52. [doi:10.15288/jsad.2010.71.249](http://dx.doi.org/10.15288/jsad.2010.71.249)
    
    [PubMed](/lookup/external-ref?access_num=20230722&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 
    
    [Web of Science](/lookup/external-ref?access_num=000276278700011&link_type=ISI) 

12. Bilban M, Vrhovec S, Karlovsek MZ. Blood biomarkers of alcohol abuse. Arh Hig Rada Toksikol 2003;54:253–9.
    
    [PubMed](/lookup/external-ref?access_num=14994647&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 

13. Montalto NJ, Bean P. Use of contemporary biomarkers in the detection of chronic alcohol use. Med Sci Monit 2003;9:RA285–90.
    
    [PubMed](/lookup/external-ref?access_num=14646983&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 

14. Neumann T, Spies C. Use of biomarkers for alcohol use disorders in clinical practice. Addiction 2003;98(Suppl 2):81–91. [doi:10.1046/j.1359-6357.2003.00587.x](http://dx.doi.org/10.1046/j.1359-6357.2003.00587.x)
    
    [CrossRef](/lookup/external-ref?access_num=10.1046/j.1359-6357.2003.00587.x&link_type=DOI) 
    
    [PubMed](/lookup/external-ref?access_num=14984245&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 
    
    [Web of Science](/lookup/external-ref?access_num=000187515600010&link_type=ISI) 

15. Hoeppner BB, Stout RL, Jackson KM, et al. How good is fine-grained Timeline Follow-back data? Comparing 30-day TLFB and repeated 7-day TLFB alcohol consumption reports on the person and daily level. Addict Behav 2010;35:1138–43. [doi:10.1016/j.addbeh.2010.08.013](http://dx.doi.org/10.1016/j.addbeh.2010.08.013)
    
    [CrossRef](/lookup/external-ref?access_num=10.1016/j.addbeh.2010.08.013&link_type=DOI) 
    
    [PubMed](/lookup/external-ref?access_num=20822852&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 

16. Burnham KP, Anderson DR. Model selection and multimodel inference: a practical information-theoretic approach. 2nd edn. Springer-Verlag, 2002; ISBN: 0-387-95364-7.
    
    

17. Zhou XH, McClish DK, Obuchowski NA. Statistical methods in diagnostic medicine. John Wiley&Sons, 2009.
    
    

18. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–45. [doi:10.2307/2531595](http://dx.doi.org/10.2307/2531595)
    
    [CrossRef](/lookup/external-ref?access_num=10.2307/2531595&link_type=DOI) 
    
    [PubMed](/lookup/external-ref?access_num=3203132&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 
    
    [Web of Science](/lookup/external-ref?access_num=A1988Q069100016&link_type=ISI) 

19. Kutner MH, Nachtsheim CJ, Neter J. Applied linear regression models. 4th edn. McGraw-Hill Irwin, 2004.
    
    

20. Hosmer DW, Lemeshow S. Applied logistic regression. John Wiley & Sons, 1989.
    
    

21. Schisterman EF, Perkins NJ, Liu A, et al. Optimal cut-point and its corresponding Youden Index to discriminate individuals using pooled blood samples. Epidemiology 2005;16:73–81. [doi:10.1097/01.ede.0000147512.81966.ba](http://dx.doi.org/10.1097/01.ede.0000147512.81966.ba)
    
    [CrossRef](/lookup/external-ref?access_num=10.1097/01.ede.0000147512.81966.ba&link_type=DOI) 
    
    [PubMed](/lookup/external-ref?access_num=15613948&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 
    
    [Web of Science](/lookup/external-ref?access_num=000226079600011&link_type=ISI) 

22. Peterson K. Biomarkers for alcohol use and abuse—a summary. Alcohol Res Health 2004;28:30–7.
    
    [PubMed](/lookup/external-ref?access_num=19006989&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 
    
    [Web of Science](/lookup/external-ref?access_num=000233606500004&link_type=ISI) 

23. Frank D, DeBenedetti AF, Volk RJ, et al. Effectiveness of the AUDIT-C as a screening test for alcohol misuse in three race/ethnic groups. J Gen Intern Med 2008;23:781–7. [doi:10.1007/s11606-008-0594-0](http://dx.doi.org/10.1007/s11606-008-0594-0)
    
    [CrossRef](/lookup/external-ref?access_num=10.1007/s11606-008-0594-0&link_type=DOI) 
    
    [PubMed](/lookup/external-ref?access_num=18421511&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom) 
    
    [Web of Science](/lookup/external-ref?access_num=000256027500012&link_type=ISI) 

24. Bearer CF, Bailey SM, Hoek JB. Advancing alcohol biomarkers research. Alcohol Clin Exp Res 2010;34:941–5. [doi:10.1111/j.1530-0277.2010.01168.x](http://dx.doi.org/10.1111/j.1530-0277.2010.01168.x)
    
    [PubMed](/lookup/external-ref?access_num=20374221&link_type=MED&atom=%2Fjim%2F64%2F5%2F1006.atom)

 [1]: /embed/graphic-1.gif