Pass/Fail Grading Your AOD Program
It's a truism: you can't manage what you can't measure. It's true in manufacturing, and it's true for alcohol and other drug abuse prevention.
Evaluation is part of an effective program at every step. Ongoing evaluation helps you focus your efforts, make your program more effective, and demonstrate the program's value at budget time. Looking at what can be measured is also a good tactic for breaking big, amorphous issues into workable, concrete pieces.
It's never too early in the process to start thinking about setting measurable goals and figuring out how you're going to measure them. Tools such as MyStudentBody's surveys and assessments, the National College Health Assessment (NCHA) survey, and others that you have on your campus generate a number of measurements that can be useful in guiding your efforts. Follow-up assessments are also a great way to track change over time within the same student population.
Not all data comes from surveys. You can also learn from one-on-one conversations, focus groups, campus police, health services, academic and other records and statistics.
Here are things that can be measured that may help indicate the success of a drug, alcohol, or sexual violence prevention program:
- Perceptions—Survey students, faculty, campus employees, or neighbors to learn what they think about campus drinking, drug use, or safety.
- Knowledge—Survey students to determine what they know about the legal, health, and academic risks of drug and alcohol use and sexual violence, local and campus policies and penalties, risk reduction techniques, and options for assistance or treatment.
- Experience—Survey students' past and present drug and alcohol usage and/or involvement in sexual violence.
- Legal consequences—Campus and local police can provide data about such things as noise complaints, property damage, DUI stops, reported crimes, and arrests.
- Health consequences—University health services or local hospitals can give statistics on injuries, emergency room visits, hospital admissions, deaths.
- Counseling interventions—Campus counseling services can provide statistics on the rates of substance abuse and mental health counseling among students.
- Academic consequences—The college's own records can provide graduation rates, frequency and severity of academic sanctions, and other data about student success.
- Satisfaction—you can also survey students' and other participants' satisfaction with any program or activity.
- Process—You can survey how well your message got out or how well a policy was implemented. This can be a useful marker in itself, and is also a useful step in fine-tuning your assessment of a message or intervention.
If you haven't built evaluation into your prevention efforts, it's never too late to begin. Many of the measurements listed above can be used to set a baseline condition at the start of the program after the fact—for instance, you should be able to find the number noise complaints from neighbors of Greek Row during pledge week for previous years. While you won't be able to establish that your program is a direct cause of any change, you will be able to see if the factor you're measuring has improved while the program has been in effect. So, if those noise complaints decreased after you established a no-alcohol policy for pledge week, you have an indication that the policy may be effective. You'll want to evaluate further, but it gives you a place to begin.
Center for the Advancement of Public Health. (2007). Impact Evaluation Resource. George Mason University. Retrieved November 30, 2011, from http://www.caph.gmu.edu/files/NCAA/IMPACT/IMPACT_Evaluation%20Resource.pdf
Community Anti-Drug Coalitions of America. National Community Anti-Drug Coalition Institute. (2008). Evaluation primer: Setting the context for a drug-free community coalition evaluation. Community Anti-Drug Coalitions of America. Retrieved November 30, 2011, from http://www.cadca.org/resources/detail/evaluation-primer
DeJong W, Langford LM. (2006) Evaluating Environmental Management Approaches to Alcohol and Other Drug Abuse Prevention. The Higher Education Center for Alcohol and Other Drug Prevention. Retrieved November 30, 2011, from http://searchpubs.higheredcenter.org/files/product/evaluating.pdf
Muraskin LD. (1993). Understanding Evaluation: The Way to Better Prevention Programs. U.S. Department of Education. Retrieved on November 30, 2011, from http://www.ed.gov/PDFDocs/handbook.pdf
Saltz R, DeJong W (2002). Reducing alcohol problems on campus: A guide to planning and evaluation. U.S. Department of Health and Human Services, National Institute on Alcohol Abuse and Alcoholism, Retrieved November 30, 2011, from http://www.collegedrinkingprevention.gov/NIAAACollegeMaterials/planEvalHandbook.aspxBack