- Open Access
- Open Peer Review
Pilot study of a repeated random sampling method for surveys focusing on date-specific differences in alcohol consumption among university students
Pilot and Feasibility Studies volume 5, Article number: 26 (2019)
This paper proposes and pilots a repeated random sampling method to promote the likelihood of collecting drinking data equally representative of the behavior of university students at all times through the academic year.
From October, 2016, to May, 2017, random samples of 1350 students were selected from the 39,155 undergraduate students enrolled in the fall semester at University of Houston. These students were sent an email inviting them to complete an online survey (entered into a weekly draw for a $50 gift certificate if responded).
The response rate was low (6%). Among participants who reported drinking in the last week, there was a variation as expected in the amount of drinking observed depending on the time of year (e.g., during exams).
While the sampling methods show promise, procedures would need to be implemented to substantially increase response rates before the proposed methods could be seen as an advantage over existing survey sampling procedures.
Epidemiological surveys containing measures of alcohol use [1, 2], in both general population and university student samples, have largely focused on collecting estimates of levels of alcohol consumption as well as their associated consequences [3,4,5]. Such epidemiological surveys have also been used to describe weekday versus weekend variations in consumption [6, 7]. While some surveys employ sampling methodology that allow for the assessment of variations in consumption at different times during the year (e.g., the CAMH Monitor) , the majority of surveys have not focused on time-specific issues relevant to alcohol consumption, such as season, or specific calendar-related heavy drinking events. When epidemiological survey data has been used to track calendar-specific variations in drinking (e.g., Christmas or New Year) , the results, while indicative of variations in consumption, suffer from the limitation that the survey sampling methodology does not allow for confidence regarding whether the drinking data produced is equally representative on each calendar date. This is because the survey sampling frame was generated at one (or, at most, several time-points) and then attempts were made to contact participants over an extended period of time. The further away in time from when the sample frame was generated, the greater the likelihood that the participant was hard to reach (displaying systematic differences in demographic characteristics compared to participants who were contacted after one or two attempts) [10, 11]. However, it is important to note that, as calendar-related heavy drinking is not the purpose of these surveys, the lack of information on this topic in these surveys is not a weakness—just a fact relating to the differing purpose associated with their data collection.
One example of a survey tradition that has paid attention to recent drinking events was developed as part of the recurring surveys conducted in Finland . A component of these surveys was a series of questions that asked detailed questions about the participant’s last drinking events. This focus makes particular sense in the Finnish context, at least that context several decades ago, where drinking in Finnish culture was structured around occasional heavy drinking rather than daily, or almost daily alcohol consumption. Another example of a survey focusing on recent heavy drinking events is the “big night out” research conducted by Dietz and colleagues .
In the general population, the exploration of event-specific drinking patterns has largely relied on relating temporal trends in alcohol sales data to alcohol-related hospital admissions, ambulance attendances, or deaths [14, 15]. In addition, studies have been conducted that link survey data to other registers, such as those recording cause of death . In university settings, research has more often employed convenience samples, relating increases in alcohol consumption to specific social events (holidays, birthdays, and sporting events) [17,18,19,20] or to follow a convenience sample longitudinally and note the variation in their consumption over time [21,22,23,24]. While both of these approaches have their strengths and limitations, there is merit in considering other methods of collecting drinking data that are calendar-specific and which pay attention to the representativeness of the sampling frame across time. This would allow for increased confidence that the detailed information collected through convenience samples is generalizable if the same patterns of results are observed when epidemiological sampling techniques are employed.
Aims and objectives
This paper proposes and pilots a survey methodology of repeated random sampling. The goal was to target the academic year of the entire undergraduate population of a university. Student drinking was chosen as an example because of the variable nature of their drinking over the academic year . Further, studying drinking patterns during the university period is important because it is a time when drinking patterns are established that can continue through extended periods of the person’s lifespan . Many students drink in an episodic fashion, with heavy drinking incidents occurring around specific events in the university academic year. In addition, some of the stressors of student life (e.g., exams, assignments) also occur in a systematic pattern across the school year. This means that there may be times of the academic year where similar quantities of alcohol consumption could cause greater or lesser harm as well as variable levels of mental distress. Accurate estimates of student drinking over the academic year could prove particularly useful for targeted interventions, such as event-specific normative feedback . The proposed survey sampling method should allow for the creation of a day-by-day summary, tracking the incidence and co-occurrence of drinking, mental distress, and negative consequences.
This pilot study was conducted for the period from mid-October to the end of April. We accessed the email list of the 39,155 undergraduate students registered in the fall of 2016 at the University of Houston. From this population, random samples (without replacement) of 1305 potential participants were selected weekly and were sent a link asking them to participate in a student life survey (email invitation sent on Fridays; all undergraduate students were sent an email invitation at some point during the pilot study period; the randomization for this pilot employed the randomization feature provided in the software program housing the email list; see Additional file 1: Appendix 1 for a copy of the email invitation). The email contained a link to a brief description of the study, an electronic consent form, and the survey itself. Potential participants were informed that those completing the survey each week would be entered into a draw for a $50 Amazon.com coupon.
Why not randomize with replacement?
One alternate sampling design would be to randomize with replacement (i.e., the emails that are selected each week are “replaced” back onto the full list of emails so that they have an equal chance of being selected again when the list of emails is selected the next week and so on). While this is a “purer” form of randomization, it was judged to be more feasible to employ random sampling without replacement (i.e., the emails, once selected, are not added back onto the list of emails) in order to ensure that all undergraduate students would have the opportunity to participate and to reduce response burden (and the concomitant likelihood of reduced response rates) resulting from some students being asked to fill out a survey on multiple occasions throughout the academic year.
The primary content of the online survey was:
Demographics characteristics: age, sex, year of undergraduate study, live on or off campus, and ethnic background using NIH categories.
Measurement of health-related quality of life (HRQOL) using the EUROHIS-QoL8. .
Smoking variables: whether smoked cigarettes daily, occasionally, or not at all in the past 12 months; number of cigarettes usually smoked each day; and time upon waking to smoking first cigarette.
Drinking variables: frequency of alcohol consumption in the last year (including no alcohol use option), number of drinks on each day of the previous week, the AUDIT-C three-item alcohol consumption scale that estimates severity of alcohol consumption [28, 29].
Those who consumed alcohol in the last week were asked about the experience of any alcohol-related problems using the 18-item version of the Rutgers Alcohol Problem Index (RAPI) .
Measurement of psychological distress using the Kessler 10 (K10) .
Of the 39,155 emails sent, only 2432 (6%) of students replied to the email, consented to the study, and provided responses on the survey. While the number of email invitations sent each week was the same (n = 1305), the number of participants responding each week varied, ranging from 18 to 133. Further, while all email invitations were sent out on the Friday of each week, the number of participants who replied on each day of the week clustered in a surprising fashion (particularly as there was no mention of a time limit within which participants needed to respond that had been provided with the email invitation). The largest proportion of participants replied on the Friday (28.7%), 23.8% replied on Tuesday, and 22.9% replied on Thursday.
Table 1 provides a summary of the demographic characteristics of the participants who responded and of the entire 39,155 students registered for the 2016 fall semester. It appeared that, while the average age of respondents who responded to the survey was roughly similar to the student undergraduate population, males may have been less likely to respond as well as those in the senior academic year.
A total of 1461 participants reported drinking alcohol in the last year and 685 reported drinking something on at least one of the days in the last week. Figure 1 displays the weekly variation in average weekly consumption, highest number of drinks consumed on one occasion during the last week, and the number of drinks consumed on the Saturday of each week (weeks 42–52 presented, with week 42 starting on Friday, October 14th, 2016). As can be observed, there was some weekly variation in alcohol consumption, with the lowest quantities reported being in weeks 49 and 50 (the point in the year where major assignments were due and exams were conducted). The largest quantities reported during the period presented on Fig. 1 appeared to be participants who replied to their surveys in week 51, which started on Friday, December 16th. See Additional file 2: Appendix 2 for a copy of drinking patterns across the full period of data collection.
The goal of this project was to pilot a sampling method that promoted the likelihood of the collection of survey data that was equally representative of the behavior of students at all times through the academic year. In the worst case, one might argue that this was a failed pilot project because the response rate was so low (6%) that discussions of representativeness are not meaningful . The pilot could also be seen as having some successful elements because it demonstrated that the proposed technique was feasible to implement. In addition, even with the low response rate, the expected variations in alcohol consumption could be observed (e.g., reduced alcohol consumption around the time of exams and final assignments).
There are a number of limitations associated with the proposed sampling method and this pilot trial. As with other population surveys, there is the likelihood of a poor response rate. This could be offset to a certain extent by employing techniques to promote participation in online surveys of this type in undergraduate populations . These methods include advertising widely in the study population that the survey was occurring, sending out a paper letter to participants prior to the email invitation to alert them that they will be receiving an email invitation, more aggressive follow-up of nonresponders, and reimbursing participants for completing the survey. It would also have been valuable to collect some qualitative data from students who did not respond to get an idea as to why they were choosing not to participate. Also of concern is the possibility that response rates would vary across the academic year, with low response rates during exam times and holidays. This would lead to challenges with interpreting differences in drinking patterns observed across the calendar year. Further, for the current pilot trial, we did not attempt to collect data during the very start of the academic year. The orientation period both would be important to monitor because of the heavy use of alcohol during this period , but also might be a more challenging period to conduct the survey (e.g., getting access to a finalized student email list). Finally, while there were multiple random samples generated throughout the school year, there is some potential for a differential level of recall bias on some days of the week versus others (e.g., recall of the previous Monday compared to the previous Sunday). This is because the time when a link to the survey was sent to participants varied in a systematic fashion across the week (sent on Fridays). Still, the recall bias for past week drinking is likely to be considerably lower than assessments that ask students to report drinking over the past month or past 3 months.
An alternate design would have been to select random samples on every day of the academic year as opposed to just once a week. In this alternate version, the survey would ask about the occurrence of the variables of interest over each day of the previous week up to and including the previous day. The analysis process would, as its first step, compile the responses of participants by matching up participants’ responses specific to each calendar date that they are asked about. Thus, data collected for each specific date would consist of up to X participants discussing their activities from the previous day (number of participants depends on size of the university student population), an additional potential X participants discussing their activities from the day 2 days earlier, etc. One complexity with implementing this design, as opposed to the weekly sampling method tested here, is that it would be more important to be sure of the dates for each of the days the participants responded to (i.e., so the data could be compiled for analysis). Further, there would be challenges to implementation and interpretation if participants did not respond to the survey request on the day it was sent but instead wanted to complete it at a later date.
Examples of research questions benefitting from data collected in this manner
The specific variables chosen to include in the survey could be modified to optimize the collection of data addressing the particular research question under study. There are a number of research questions that would benefit from access to data collected in this manner. These include the possibility of mapping date-specific drinking and drug use to social events occurring at the university and to control initiatives implemented by university staff. It would also be possible to match drinking patterns to disturbances recorded by campus security or local police. Data collected using a repeated random sampling method could also be employed to track patterns of alcohol use and level of mental distress over the academic year (note: while mental health distress was assessed in this pilot survey, these analyses were not conducted as the response rate was too low for any results to be meaningful). Similar analyses could be conducted to track the co-occurrence of drinking with other health behaviors (e.g., smoking) or with health-related quality of life. This aggregated, date-specific data would complement longitudinal research on this topic which employed convenience samples of participants . Further, means of multiple random samples of students reports of past week drinking represents a sampling distribution. In comparison to single annual or biannual assessments of campus drinking (assuming similar participation rates), estimates derived from a multiple samples provides a more complete and accurate representation of typical drinking on campus. This approach also allows for the direct calculation of standard errors whereas single assessments only provide estimated standard errors. Finally, accurate data of patterns of alcohol consumption during date-specific drinking events could be used to generate population norms for personalized feedback interventions targeting people who drink in a hazardous fashion.
While the proposed repeated randomization method does not address all limitations, it could provide a survey sample that is more representative of drinking on each calendar day compared to other population sampling methods. The method could also work in a general population setting if the researcher had good access to contact data for residents of the country that would allow for a similar repeated sampling strategy (e.g., voter registration files) and a method for delivering the surveys on a daily basis.
Alcohol Use Disorders Identification Test Consumption scale
Health-related quality of life
Rutgers Alcohol Problem Index
Room R. Measuring drinking patterns: the experience of the last half century. J Subst Abus. 2000;12(1–2):23–31.
Dawson DA, Room R. Towards agreement on ways to measure and report drinking patterns and alcohol-related problems in adult general population surveys: the Skarpo Conference overview. J Subst Abus. 2000;12(1–2):1–21.
Grant BF, Moore TC, Shepard J, Kaplan K. Source and accuracy statement. Wave 1. National Epidemiologic Survey on Alcohol and Related Conditions (NESARC). Bethesda: National Institute on Alcohol Abuse and Alcoholism; 2003.
Hallett J, Howat PM, Maycock BR, McManus A, Kypri K, Dhaliwal SS. Undergraduate student drinking and related harms at an Australian university: web-based survey of a large random sample. BMC Public Health. 2012;12:37.
Gliksman L, Adlaf EM, Demers A, Newton-Taylor B. Heavy drinking on Canadian campuses. Canadian J Public Health. 2003;94(1):17–21.
Kypri K, Langley J, Stephenson S. Episode-centred analysis of drinking to intoxication in university students. Alcohol Alcohol. 2005;40(5):447–52.
Room R, Makela P, Benegal V, Greenfield TK, Hettige S, Tumwesigye NM, Wilsnack R. Times to drink: cross-cultural variations in drinking in the rhythm of the week. Int J Public Health. 2011;57(1):107–17.
Ialomiteanu AR, Hamilton HA, Adlaf EM, Mann RE. CAMH Monitor eReport: substance use, mental health and well-being among Ontario adults, 1977–2013. In: CAMH research document series, no 40; 2014.
Kushnir V, Cunningham JA. Event-specific drinking in the general population. J Stud Alcohol Drugs. 2014;75(6):968–72.
Fitzgerald R, Fuller L. I hear you knocking but you can’t come in: the effects of reluctant respondents and refusers on sample survey estimates. Sociol Methods Res. 1982;11(1):3–32.
Dunkelberg W, Day G. Nonresponse bias and callbacks in sample surveys. J Mark Res. 1973;10:160–8.
Simpura J, Paakkanen P, Mustonen H. New beverages, new drinking contexts? Signs of modernization in Finnish drinking habits from 1984 to 1992, compared with trends in the European community. Addiction. 1995;90:673–83.
Dietze PM, Livingston M, Callinan S, Room R. The big night out: what happens on the most recent heavy drinking occasion among young Victorian risky drinkers? Drug Alcohol Rev. 2014;33(4):346–53.
Lloyd B, Matthews S, Livingston M, Jayasekara H, Smith K. Alcohol intoxication in the context of major public holidays, sporting and social events: a time-series analysis in Melbourne, Australia, 2000-2009. Addiction. 2013;108(4):701–9.
Poikolainen K, Leppanen K, Vuori E. Alcohol sales and fatal alcohol poisonings: a time-series analysis. Addiction. 2002;97(8):1037–40.
Makela P, Martikainen P, Nihtila E. Temporal variation in deaths related to alcohol intoxication and drinking. Int J Epidemiol. 2005;34(4):765–71.
Lee CM, Maggs JL, Rankin LA. Spring break trips as a risk factor for heavy alcohol use among first-year college students. J Stud Alcohol. 2006;67(6):911–6.
Neighbors C, Atkins DC, Lewis MA, Lee CM, Kaysen D, Mittmann A, Fossos N, Rodriguez LM. Event-specific drinking among college students. Psychol Addict Behav. 2011;25(4):702–7.
Neighbors C, Spieker CJ, Oster-Aaland L, Lewis MA, Bergstrom RL. Celebration intoxication: an evaluation of 21st birthday alcohol consumption. J Am Coll Heal. 2005;54(2):76–80.
Nelson TF, Wechsler H. School spirits: alcohol and collegiate sports fans. Addict Behav. 2003;28(1):1–11.
Tremblay PF, Graham K, Wells S, Harris R, Pulford R, Roberts SE. When do first-year college students drink most during the academic year? An internet-based study of daily and weekly drinking. J Am Coll Heal. 2010;58(5):401–11.
Greenbaum PE, Del Boca FK, Darkes J, Wang CP, Goldman MS. Variation in the drinking trajectories of freshmen college students. J Consult Clin Psychol. 2005;73(2):229–38.
Del Boca FK, Darkes J, Greenbaum PE, Goldman MS. Up close and personal: temporal variability in the drinking of individual college students during their first year. J Consult Clin Psychol. 2004;72(2):155–64.
Beets MW, Flay BR, Vuchinich S, Li KK, Acock A, Snyder FJ. Longitudinal patterns of binge drinking among first year college students with a history of tobacco use. Drug Alcohol Depend. 2009;103(1–2):1–8.
Riordan BC, Conner TS, Thrul J, Flett JAM, Carey KB, Scarf D. Just a first-year thing? The relations between drinking during orientation week and subsequent academic year drinking across class years. Subst Use Misuse. 2018;53(9):1–10.
Neighbors C, Walters ST, Lee CM, Vader AM, Vehige T, Szigethy T, DeJong W. Event-specific prevention: addressing college student drinking during known windows of risk. Addict Behav. 2007;32(11):2667–80.
Schmidt S, Muhlan H, Power M. The EUROHIS-QOL 8-item index: psychometric results of a cross-cultural field study. Eur J Pub Health. 2006;16(4):420–8.
Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch Intern Med. 1998;158(16):1789–95.
Dawson DA, Grant BF, Stinson FS, Zhou Y. Effectiveness of the derived Alcohol Use Disorders Identification Test (AUDIT-C) in screening for alcohol use disorders and risk drinking in the US general population. Alcohol Clin Exp Res. 2005;29(5):844–54.
White HR, Labouvie EW. Towards the assessment of adolescent problem drinking. J Stud Alcohol. 1989;50(1):30–7.
Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand SLT, Walters EE, Zaslavsky AM. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med. 2002;32(6):959–76.
Fosnacht K, Sarraf S, Howe E, Peck LK. How important are high response rates for college surveys? Rev High Educ. 2017;40(2):245–65.
Kypri K, Gallagher SJ. Incentives to increase participation in an Internet survey of alcohol use: a controlled experiment. Alcohol Alcohol. 2003;38(5):437–41.
We would like to thank Dr. Jason Ferris at the University of Queensland for providing the idea for the weekly sampling method design (the daily sampling method was the original proposed design). A draft version of this paper (without pilot trial data) was presented at the 42nd Annual Alcohol Epidemiology Symposium of the Kettil Bruun Society, Stockholm, Sweden. This research was undertaken in part thanks to funding from the Canada Research Chairs program for support of Dr. Cunningham, the Canada Research Chair in Addictions. Support to CAMH for salary of scientists and infrastructure has been provided by the Ontario Ministry of Health and Long Term Care. The views expressed in this article do not necessarily reflect those of the Ministry of Health and Long Term Care.
This research was undertaken thanks to funding from the Canada Research Chairs program.
Availability of data and materials
Data available upon request to the corresponding author.
Ethics approval and consent to participate
This research was approved by the Ethics Review Board at the Centre for Addiction and Mental Health (No. 066-2016). All participants provided online electronic consent.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.