Higher education: why metrics matter

It is easy to underestimate the impact that digital technology is having on society, changing it in ways that no-one anticipated. Education is no exception, and pedagogy is evolving rapidly.

Chalk and talk is a thing of the past as educators move learning out of the classroom to make it more experience based. To ensure its relevance and longevity in supporting the careers of future graduates, formal education is moving towards a greater emphasis on work-integrated learning, with an explicit intent to embed broader career and life skills.

While knowledge and skills will always be important, educators are now also asked to foster the attitudes and behaviours that empower students to manage their own wellness, to work creatively, and to work effectively in teams. At the same time, assessment is shifting away from the traditional test where students are asked to demonstrate their knowledge of a subject, to creating environments where they can display their ability to apply that knowledge in a context that better reflects the world outside the classroom.

While these shifts are discernible in higher education, we don’t all agree on the merits of particular educational interventions, nor do we have a clear, empirical basis for measuring the outcomes or understanding the impact they have on the education process. Engineers will tell you that you don’t really understand something until you can measure it.

When it comes to measuring the outcomes and impact of our evolving education landscape, we are sorely lacking. This is not to say that we don’t currently measure. For example, student attrition and graduate employment outcomes are well established metrics of student outcomes and institutional performance—but we know these measures are not without their limitations and we know they are not sufficient. “Not all attrition is bad attrition” is a well-worn phrase employed by higher education commentators. Some students will know only after enrolment whether their degree or university is right for them.

Metrics which encourage institutions to maximise enrolments and retention risk unintended consequences by discouraging students from exercising their best option. Our metrics should encourage institutional behaviour that is good for students, not just for league tables. But designing the right retention incentives can be difficult and often requires a level of information and nuance which is not readily attainable, at least in a timely fashion, with today’s data.

If we know that the metrics of today won’t be appropriate for the graduates of tomorrow, how do we make the necessary transition while minimising adverse effects on institutional incentives? How can we provide incentives for our institutions to adapt to the needs of the graduates of the future by creating the most appropriate measures of value, without creating perverse incentives and without over-prescribing what the future value of higher education should look like?

We know that the best predictors of educational gain are measures of educational process that assess what institutions do with their resources to help students make the most of their abilities and interests. The metrics that best predict gains are not those associated with the educational resources themselves, or with student satisfaction with their use.  Rather, they are the metrics associated with practices that engender student engagement in learning. However, this is the area where we have the least evidence and insight upon which to base metrics of value.

New approaches to measurement need to identify where we need to intervene, to educate the educators and the education system, and to help us determine what works, and what doesn’t. We also need a longitudinal component in our metrics against which we can determine if the “good teaching” we were measuring actually results in the long-term “good education” that we were after and that we were promising our students.

Metrics are powerful tools that can help us build the education system that will guide us through an uncertain future. The metrics that we use must be robust to the diversity of the higher education sector, in terms of the characteristics of students and the missions of our institutions. They must represent our values and aspirations and be comprehensible to society. Finally, they must align, as closely as possible, with the factors that are most critical to students’ success in the workforce and society of the future.

Shifts towards performance and outcome based funding arrangements combined with budgetary pressures heightens the need to instigate approaches that will result in the development of the metrics that matter now, and for the future.

The future, we are told, is an undiscovered country: the people are different there. The metrics that matter will be different too.

Want to stay up-to-date?

Stay on trend and in the know when you sign up for our latest content