Retention continues to be a challenge for institutions of higher education around the world. Over the last few decades, many universities have turned to early alert programs to address retention issues. These programs involve identifying and notifying at-risk students and offering various intervention options in an attempt to reduce failure and attrition. However, research on these programs has only recently started to emerge. Studies thus far often fail to provide a clear picture of the effects of early alert programs, and there is a significant deficit of information on student and faculty perspectives. In addition, the focus on potential retention gains overshadows questions about suitability of early alert programs in meeting other higher education goals. This study provides an analysis of existing studies, as well as a comprehensive, mixed-method evaluation of the early alert program at a small public university in the southwestern region of the United States. Although the classroom study and student perspectives offered some support for early alert programs, faculty responses and program data in general did not provide compelling evidence that early alerts are effective for reducing course failure or attrition. The overall design of this study was intended to serve as a template for early alert program evaluation in higher education, including how to study the effects of early alerts and how to appropriately use and interpret program data. Keywords: early alerts, higher education, retention, college students, faculty, data analytics