What do university rankings actually measure?
Whatever their compilers decided to weight, which differs enormously between them — and the resulting tables measure institutional characteristics far more than they measure what a student would experience.
What goes into them, with weightings varying by publisher:
Research output and citations, which in international rankings frequently dominate — and which have almost no relationship to undergraduate teaching quality. A university can rank highly on research while teaching large classes badly.
Reputation surveys, sent to academics and employers. These are heavily lagged and self-reinforcing — reputation reflects past reputation, which is why the top of the table barely moves.
Staff to student ratio, a proxy for contact and attention.
Entry standards, which measure who the university admits rather than what it does with them.
Graduate outcomes, which are strongly influenced by intake and subject mix.
Spending per student, and facilities.
Student satisfaction, in some domestic tables.
Why the rankings disagree with each other. The same university appears in very different positions across publishers, purely because of weighting choices — which demonstrates that the underlying quantity being measured is not well defined.
The known problems:
Gaming. Institutions optimise the measured inputs, including how staff and student numbers are reported.
Subject mix distorts everything. A university strong in sciences will rank differently from one strong in arts, regardless of quality.
Value added is not measured. A university that takes students with modest entry grades to good outcomes is scored worse than one that admits the highest achievers and adds little.
International rankings favour large, old, research-intensive, English-speaking institutions structurally.
What a prospective student should actually look at: the subject table rather than the institutional one, since departmental quality varies enormously within a university; teaching contact hours and assessment methods; the specific modules; graduate outcomes for that course; student satisfaction data for that department; placement and year-abroad options; location, cost and accommodation; and — above all — visiting.
Rankings are most useful for eliminating obviously poor fits, and least useful for choosing between comparable options.