Universities
The CWTS Leiden Ranking 2024 includes 1506 universities worldwide. These universities have been selected based on their number of Web of Science indexed publications in the period 2018–2021. As discussed below, a sophisticated data collection methodology is employed to assign publications to universities.
Identification of universities
Identifying universities is challenging due to the lack of clear internationally accepted criteria that define universities. Typically, a university is characterized by a combination of education and research tasks in conjunction with a doctorate-granting authority. However, these characteristics do not mean that universities are particularly homogeneous entities that allow for international comparison on every aspect. As a result of its focus on scientific research, the Leiden Ranking presents a list of institutions that have a high degree of research intensity in common. Nevertheless, the ranking scores for each institution should be evaluated in the context of its particular mission and responsibilities, which are strongly linked to national and regional academic systems. Academic systems - and the role of universities therein - differ substantially between countries and are constantly changing. Inevitably, the outcomes of the Leiden Ranking reflect these differences and changes.
The international variety in the organization of academic systems also poses difficulties in terms of identifying the proper unit of analysis. In many countries, there are collegiate universities, university systems, or federal universities. Instead of applying formal criteria, whenever possible we follow common practice based on the way these institutions are perceived locally. Consequently, we treat the University of Cambridge and the University of Oxford as entities, whereas in the case of the University of London we distinguish between the constituent colleges. For the United States, university systems (e.g. the University of California) are split up into separate universities. The higher education sector in France, like in many other countries, has gone through several reorganizations in recent years. Many French institutions of higher education have been grouped together in Communautés d'Universités et Etablissements (COMUEs), succeeding the earlier Pôles de Recherche et d'Enseignement Supérieur (PRES). Except in the case of full mergers, the Leiden Ranking still distinguishes between the different constituent institutions. The Leiden Ranking 2024 includes French organisations that are designated as "établissements publics expérimentaux (EPE)". This is a new type of HEI in France created by the law of 12 December 2018 in which different research and higher education institutions work together in order to eventually form a single HEI. Research and educational organisations that are part of a EPE as "établissements-composantes" will no longer be included as separate organisations in the Leiden Ranking 2024.
Publications are assigned to universities based on their recent configuration. Changes in the organizational structures of universities up to 2023 have been taken into account.
Affiliated institutions
A key challenge in the compilation of a university ranking is the handling of publications originating from research institutes and hospitals affiliated with universities. Among academic systems, a wide variety exists in the types of relations maintained by universities with these affiliated institutions. Usually, these relationships are shaped by local regulations and practices affecting the comparability of universities on a global scale. As there is no easy solution for this issue, it is important that producers of university rankings employ a transparent methodology in their treatment of affiliated institutions.
CWTS distinguishes three different types of affiliated institutions:
- Component
- Joint research facility or organization
- Associated organization
In the case of a component, the affiliated institution is actually part of or controlled by the university. Universitaire Ziekenhuizen Leuven is an example of a component, since it is part of the legal entity of Katholieke Universiteit Leuven.
A joint research facility or organization is identical to a component except that it is administered by more than one organization. The Brighton & Sussex Medical School (the joint medical faculty of the University of Brighton and the University of Sussex) and Charité (the medical school of both the Humboldt University and the Freie Universität Berlin) are examples of this type of affiliated institution.
The third type of affiliated institution is the associated organization, which is more loosely connected to a university. This organization is an autonomous institution that collaborates with one or more universities based on a joint purpose but at the same time has separate missions and tasks. In many countries, hospitals that operate as teaching or university hospitals fall into this category. The Massachusetts General Hospital, one of the teaching hospitals of the Harvard Medical School, is an example of an associated organization.
The Leiden Ranking 2024 counts a publication as output of a university if at least one of the affiliations in the publication explicitly mentions either the university or one of its components or joint research facilities. In a limited number of cases, affiliations with institutions that are not controlled or owned by the university are also treated as if they were mentioning the university itself. The rationale for this is that in some cases institutions – although formally being distinct legal entities – are so tightly integrated with the university that they are commonly perceived as being a component or extension of that university. Examples of this situation include the university medical centers in the Netherlands and some of the academic health science systems in the United States and other countries. In these cases, universities have actually delegated their medical research and teaching activities to the academic hospitals and universities may even no longer act as the formal employer of the medical researchers involved. In other cases, tight integration between a university and an academic hospital may manifest itself by an extensive overlap in staff. In this situation, researchers may not always mention explicitly their affiliation with the university. An example of this tight integration is the relation between the University Hospital Zurich and the University of Zurich.
The list of affiliated institutions for the 2024 edition is available here. Our approach is discussed in more detail in a blog post and in this paper on academic hospitals.
Affiliated institutions that are not classified as a component or a joint research facility or treated as such are labeled as associated institutions. In the case of publications with affiliations from associated organizations, a distinction is made between publications from associated organizations that also mention the university and publications from associated organizations that do not include a university affiliation. In the latter case, a publication is not considered to originate from the university. On the other hand, if a publication includes an affiliation from a particular university as well as an affiliation from an associated organization, both affiliations are considered to represent that particular university. The effect of this procedure depends on the counting method that is used in the calculation of bibliometric indicators. The procedure influences results obtained using the fractional counting method, but it has no effect on results obtained using the full counting method.
Selection of universities
The Leiden Ranking 2024 includes 1506 universities from 72 different countries. These are all universities worldwide that have produced at least 800 Web of Science indexed publications in the period 2018–2021. Only so-called core publications are counted, which are publications in international scientific journals. Also, only research articles and review articles are taken into account. Other types of publications are not considered. Furthermore, collaborative publications are counted fractionally. For instance, if a publication includes five authors of which two belong to a particular university, the publication is counted with a weight of 2 / 5 = 0.4 for that university.
It is important to note that universities do not need to apply to be included in the Leiden Ranking. The universities included in the Leiden Ranking are selected by CWTS according to the procedure described above. Universities do not need to provide any input themselves.
Data quality
The assignment of publications to universities is not free of errors, and it is important to emphasize that in general universities do not verify and approve the results of the Leiden Ranking data collection methodology. Two types of errors are possible. On the one hand, there may be false positives, which are publications that have been assigned to a university when in fact they do not belong to the university. On the other hand, there may be false negatives, which are publications that have not been assigned to a university when in fact they do belong to the university. The data collection methodology of the Leiden Ranking can be expected to yield substantially more false negatives than false positives. In practice, it turns out to be infeasible to manually check all addresses occurring in Web of Science. Because of this, many of the 5% least frequently occurring addresses in Web of Science have not been manually checked. This can be considered a reasonable upper bound for errors, since most likely many of these addresses do not belong to universities.
