Andrew K. Koch, Executive Vice President and Chief Academic Leadership & Innovation Officer, John N. Gardner Institute for Excellence in Undergraduate Education
Matthew D. Pistilli, Director, Assessment and Planning for the Division of Student Affairs, Indiana University Purdue University Indianapolis
Robert Rodier, Director of Technology & Informatics, John N. Gardner Institute for Excellence in Undergraduate Education
The term “analytics” refers to a broad range of statistical techniques and predictive modeling approaches that have garnered great interest across U.S. higher education over the past decade (Campbell, DeBlois & Oblinger, 2007). But, as Hampson notes, “analytics in higher education is relatively new and descriptions are often imprecise. Different types of analytics, with little in common, are regularly lumped together” (2014). Analytics can be used for operational purposes in universities or colleges – such as maximizing classroom and course space usage, or for packaging financial aid to boost student recruitment yield. In fact, van Barneveld et al. list seven separate kinds of analytics, with upwards of two-dozen definitions for the various types. It is no wonder, then, that this term has a great deal of confusion surrounding it (van Barneveld, Campbell & Arnold, 2012).
Analytics tools offered by many for-profit vendors often cost tens, if not hundreds, of thousands of dollars per year to use. But, as a recent EDUCAUSE report makes clear, these expensive tools often are purchased only to “satisfy credentialing or reporting requirements rather than to address strategic questions, and much of the data collected are not used at all” (Bichsel, 2012, p. 3). As a result, at best, a small number of faculty make use of analytics for the benefit of a small subset of students. In addition, the high cost of commercially available analytics tools make them virtually inaccessible to smaller-enrollment institutions—the very institutions that enroll the largest levels of first-generation and low-income students (Rine & Eliason, 2015, p. 9-12). Thus, the college completion promise of analytics is far greater than the actual benefits realized for students on campuses across the United States.
The JNGI Analytics in Pedagogy and Curriculum aims to address the fuzzy definitions, high cost and low application issues associated with analytics in higher education by: 1) placing a specific framework around the use of analytics within higher education; 2) providing that framework at an accessible, non-profit fee; and 3) providing support for the adoption and continuous application of analytics.
For starters, we do not believe that analytics is simply a tool, nor a singular “thing” that can simply be applied as a panacea for all challenges facing an institution. Rather, analytics are processes that combine large data sets in an effort to create actionable intelligence – information that can be used to directly and positively affect outcomes associated with students and institutions.
The assertion that analytics is a process is an important one. Many institutions believe that they “have analytics” once they have come upon or purchased a model or tool of some form. In reality, all they actually have is a mathematical formula or product that predicts, models, or represents an outcome – they have a “thing.” They do not have a process or processes to apply that data, to a high level, to effect positive change. Envisioning analytics as a process moves from simply having a model to being able to do something with the information fed into and obtained by an algorithm. In short, our definition of analytics as a process means that analytics is actually about using data to continuously shape and improve institutional and/or faculty actions surrounding teaching, learning and student success. Analytics is a tool that shapes process – not an answer in-and-of itself. To derive the full benefits of analytics, you need to focus on the process of applying the data output.
The JNGI Analytics in Pedagogy and Curriculum is firmly focused on a form of analytics called “learning analytics.” As defined by the International Conference for Learning Analytics and Knowledge, learning analytics is, “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (2011).
Specifically, the JNGI Analytics in Pedagogy and Curriculum focused on the application of learning analytics to improve the way faculty teach and students learn in gateway courses. This highly defined focus allows for deep exploration into and work on the process of applying analytics outcomes into actions with gateway courses – thereby maximizing the benefits of analytics. In other words, the focus helps increase the likelihood that faculty and staff will actually do something with analytics-related reports and dashboards to affect positive changes in teaching and learning.
The JNGI Analytics in Pedagogy and Curriculum includes five key components. These include:
Our experience has led us to believe that when it comes to learning analytics, many institutions, on some level, struggle with similar issues, encounter comparable roadblocks, and need to address concerns regarding scale and scope. We believe that there is a real void in the educational space on how to even think about analytics, much less having capacity to build or scale an effort. The JNGI Analytics in Pedagogy and Curriculum seeks to fill this space for those institutions willing to embark on implementing an analytics process in a cooperative, mutually beneficial manner.
Arnold, K. E., Lonn, S., & Pistilli, M. D. (2014). An exercise in institutional reflection: The Learning Analytics Readiness Instrument (LARI). In K. E. Arnold, A. Pardo, & S. Teasley (Eds.), Proceedings from the 4th International Conference on Learning Analytics and Knowledge (pp. 163-167). New York: ACM. doi: 10.1145/2567574.2567621
Bichsel, J. (2012). Analytics in higher education: Benefits, barriers, progress, and recommendations. EDUCAUSE Center for Applied Research.
Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007). Academic analytics: A new tool for a new era. EDUCAUSE review, 42(4), 40.
First Annual Conference on Learning Analytics and Knowledge. (2011). Call for proposals. https://tekri.athabascau.ca/analytics/about.
Hampson, K. (April 25, 2014). Analytics in online higher education: Three categories. http://acrobatiq.com/analytics-in-online-higher-education-three-categories/
Rine, J. P. & Eliason, J. (2015). Expanding access and opportunity: How small and mid-sized colleges serve first-generation and low-income students. Washington, DC: The Council of Independent Colleges.
van Barneveld, A., Arnold, K. E., & Campbell, J. P. (2012). Analytics in higher education: Establishing a common language. ELI Paper 1. Littleton, CO: EDUCAUSE.
Watson, H. J. (2011). Business analytics insight: Hype or here to stay? Business Intelligence Journal 16(1), 4-8.