Giselle's List

(List #80) Rethinking Rankings and Ratings as Tools for Reaching the SDGs (Part 2)

What should we rate and is it even possible?

Giselle Weybrecht

Feb 20, 2023 • 3 min read

Last week I shared some summaries of resource documents produced by HESI’s subgroup focused on rethinking the rankings and assessments to ensure that they are incentivising the SDGs. These were introduced during a recent HESI webinar exploring rankings and ratings.

This week, I thought I’d continue this theme and share some more specific points brought up during the webinar, as well as my own thoughts. Lots of questions looking for answers...

1. COMPETITION VS COLLABORATION

Evaluations for rankings, ratings and assessments are usually competitive rather than cooperative. Individual faculty may be collaborating internally, across the university, or across universities, and increasingly are in relation to SDG relevant projects. But business schools aren’t celebrated, or ranked based on those collaborations but instead based on their own performance. How do we balance moving forward together and moving forward individually? How can we share more?

2. FULL VS DETAILED

Evaluations don’t show the full scope of the activities that a business school is undertaking. Sometimes it can be very difficult to bring everything a school is doing together. By summarizing to a limited number of measures, an assessment may not be picking up what is important to you, or even the bigger picture of how it all comes together. Some parts of the institution may be doing interesting things while others may be resisting change outright. Same goes for the SDGs. Does an assessment cover all of the SDGs? Only certain SDGs? How do you decide which SDGs are most important, relevant, and material to a/your business school?

3. OUTCOMES VS OUTPUTS

Things that are measured are usually those things that are easy to measure. However, these might not be what are most important. It is easy to measure outputs, but what we really want to measure is outcomes. Is the work that the university is doing resulting in positive change? That may not always be the case (and is important to consider). Obviously, this is easier said than done. You may be putting lots of resources into faculty training, but unless it results in changes in curriculum and research, for example, has it had a positive outcome? (Interestingly, the International Development Research Centre uses outcome mapping as an alternative form of assessment for example).

4. WHAT ARE YOU TRYING TO ACHIEVE?

Kathleen Ng from McGill University, one of the facilitators of the webinar, shared a cautionary tale about the need to be clear about what you are trying to achieve. The university dorm rooms organised a competition to reduce energy. The dorm with the lowest energy consumption that month would win a pizza party. Rather than lead to behaviour change, as was the original intention, it led to students visiting the competing dorms to sabotage them by turning on all of their lights. Reports don’t often include these stories, but these stories will help other schools prevent the same thing from happening to them.

5. IS IT POSSIBLE?

Rankings and assessments aim for standardization. However, a concern is whether that standardization can limit progress. Every business school is different. Each is a different size, has a different budget, different focus areas and different relevant issues. Assessments should have equity built in. Selected measurements need to allow for space for a diverse set of views and that is equally relevant to institutions, and students, around the world. A university in one country may approach the SDGs in a completely different way than another university, but both can be equally effective. How can we measure “progress” in a way that is not too prescriptive and that understands that different routes can lead to the same outcomes?