Introduction
The scientific performance of universities is often evaluated through the number of scientific publications in top-tier journals and their citations (Koler-Povh et al., 2014). The ‘publish or perish’ threat currently seems to be a driving force behind scientists’ performance (Bukowska and Lopaciuk-Gonczaryk, 2018). The Chilean University System (CUS) is not immune to this belief. The number of documents published by Chilean universities in top-tier journals, mainly in the Web of Science (WoS) and Scopus, is an important indicator when competing for research funds. Furthermore, the Chilean National Accreditation Agency (CNA) uses the number of papers published in top-tier journals as an essential indicator for certifying Chilean universities in the area of “Research”. Chilean research-oriented universities are better positioned in national and international rankings (Ganga-Contreras et al., 2018). Therefore, the formulation and implementation of public policies that enhance high-quality research is a challenge faced by all universities. Higher education governance is more challenging in emerging economies such as Chile.
The CUS shows a sustained increase in scientific production over the past forty years. While in 1980, Chile produced 1 072 documents in journals in the Social Science Citation Index, Science Citation Index-Expanded, and the Arts & Humanities Citation Index; in 2019, productivity grew twelvefold (13 194 documents), showing exponential increment. This increase in scientific production facilitates the bibliometric assessment of the citation-based impact of the knowledge produced by Chilean universities.
Previous bibliometric studies on the Chilean University System’s scientific production mainly focused on exploring the patterns of its scientific output. Quezada-Hofflinger and Vallejos-Romero (2018); Muñoz-García et al., (2019); Muñoz (2016); Krauskopf and Pessot (1980) analyzed scientific production at the macro-level. This way, previous studies explored the productivity of specific research fields such as chemistry (Rivas and Palacio, 2020), sport science (Pérez-Gutiérrez et al., 2016), ecology (Molina-Montenegro and Gianoli, 2010), fisheries science (Hidalgo et al., 2015; Elgueta, 1999), and medicine (Díaz, 2011), or analyzed the productivity of a particular university (Broekhoff, 2019) or a specific faculty (Krauskopf et al., 1995). Recently, Koch and Vanderstraeten (2019) analyzed the internationalization of the Chilean scientific community.
A few studies have examined the citation-based impact of Chilean higher education publications, such as that by Meza and Ortega (2019), which studied the patterns of self-citations of documents published in the Scielo Chile database. Urbizagastegui and Cortés (1998) studied the citation patterns of publications in the Revista Geológica de Chile. Rivas and Palacio (2020) assessed the citation-based impact of Chilean chemistry publications, and Molina-Montenegro and Gianoli (2010) assessed the citation impact of ecology publications. There is a lack of studies that assess the growth rate of scientific production and the citation-based impact of the Chilean University System in the literature. This exploration would provide information for research policymakers, Chilean research funding units, and research administration groups in the Chilean University System and in higher education institutions. This study aims to analyze the growth rate of scientific production and the citation-based impact of the Chilean University System.
Background
The Chilean University System. A brief overview
There are 56 universities in the Chilean University System. In Chile, they universities have existed since the 17th century. The first higher education institutions emerged in colonial times, when the country was under Spanish rule. In 1622, the Pontificia University of Santo Tomás de Aquino was founded, from which students graduated through 1747. That university offered only higher education degrees in theology. A year later, the Convento San Francisco Javier emerged. This institution functioned from 1623 through 1767 under Jesuit supervision, educating only the devoutly religious. Finally, the Real Universidad de San Felipe operated between 1747 and 1843. It belonged to the Spanish Kingdom and offered degrees in theology, medicine, law, philosophy, and mathematics.
The creation and development of the Chilean University System was influenced by profound change processes (Bravo Lira, 1992). In 1843, the Chilean government and specific sectors of Chilean society began to create universities, seeking to contribute to the country’s development in various fields. That led to the creation of the Universidad de Chile, which is the leading university in the country since it is ranked among the top ten Universities in Latin America and the Caribbean.
Before 1980, there were 10 universities in Chile. After 1980, new universities emerged. At the same time, a group of private universities opened their doors, and initiated the competitive market of Chilean higher education. At the beginning of the 21st century, Chilean universities covered more than 50% of the country’s tertiary demand for higher education (Escobar et al., 2020).
The introduction of the General Law on Universities in 1981 allowed the Universidad de Chile (UCh) and the Universidad Técnica del Estado (UTE) to create new public universities in the country’s regions through the transformation of their regional campuses. After the change made by the law, the number grew to 22 universities: 16 public universities, and 6 private universities. In March 1990, the Constitutional Organic Education Law 18962 (LOCE) was enacted, which created the Higher Council of Education (CSE) to supervise and accredit the new private universities. Therefore, another 29 were created (Cruz-Coke, 2004). For more information about this stage, read the study by Espinoza (2008) that contains a detailed description of the privatization process of Chilean universities.
The primary mission of the Chilean University System is teaching, research, and outreach. That is why teaching universities are presently shifting towards both teaching and research. In this context, the enhancement of research visibility through high-quality research publications in top-tier journals is a matter of primary concern. In recent years, the scientific production and citation-based impact of the Chilean research-oriented university system has increased exponentially (Figure 1).
Note: The citation impact was calculated using the SCI-Expanded, SSCI, AHCI, and ESCI. Document types: articles and reviews.
Source: The information was obtained from the Core Collection of the Web of Science.
The research questions of the study are:
Is there a scaling correlation between citation-based impact and the scientific production of Chilean research-oriented universities?
Which university type shows a higher growth rate in scientific production and citation-based impact: public, public-private, or private universities?
Unit of analysis
The unit of analysis is the 26 universities in the Chilean University System that are accredited by the CNA in the area of Research. This selection guarantees that the universities included in the study are higher education institutions that, in addition to pre-and post-graduate teaching, are centered on the generation and dissemination of new scientific knowledge. The data for the study were obtained from the WoS database using the query OG = (university names separated by the Boolean OR), and DT= (article or review), and PY = (year of publication). The following citation indexes were used: Science Citation Index Expanded SCIE, Social Science Citation Index SSCI, Arts & Humanities Citation Index A&HCI, and Emerging Sources Citation Index ESCI.
Chilean universities (Appendix) are classified into three categories: Public universities that make up the Consortium of Public Universities of Chile (CUECh), Public-private universities belonging to the Council of Rectors, and Private universities that do not belong to any national consortium (Law number 21091 on Higher Education, 2018).
Public universities: This group includes 15 universities; among them is the oldest university in the CUS, namely the Universidad de Chile. These universities were created between 1842 and 1993. The average age in this group is 65. These universities are financed by the Chilean government.
Public-private universities: This class embraces 9 universities; 67% are Catholic. The universities in this group were created between 1918 and 1991. The average age in this group is 72. One of the indicators for receiving funds from the government is the number of papers published in top-tier journals (JCR and Scopus).
Private universities: 35 universities form this group. At an average age of 30, they are the newest in the CUS as they appeared after Chile returned to democracy in 1990. They do not receive financial support from the Chilean government.
Timeframe
The timeframe used for scientific productivity is from 2006 to 2018. This segment of time was divided into 7 points in time (years): 2006, 2008, 2010, 2012, 2014, 2016, and 2018. The impact covers from 2006 to 2020.
The variables
Scientific production
The present study delimits scientific production as the number of articles and reviews published by a Chilean research-oriented university (public, public-private, and private) in the citation indexes Science Citation Index Expanded, Social Science Citation Index, Arts & Humanities Citation Index, and Emerging Sources Citation Index of the Core Collection of the Clarivate Analytics Web of Science (WoS). These document types are rigorously peer-reviewed and they are considered the primary route for disseminating new scientific knowledge (Adams and Gurney, 2018).
Citation-based impact
Citations are the currency of scholarship (Thomson Reuters, 2014). The number of citations a paper receives is a measure of its impact on the papers citing it. The citation-based impact of a Chilean university (public, public-private, or private) is the sum of the citations to the papers published by researchers from that university. The citation impact of papers at each point in time considers three years citation window. For example, that, applied to the Chilean University System in 2018, is the sum of the citations to those papers in 2018, 2019, and 2020. This method prevents the drawback of the papers’ age, allowing all documents to have the same probability of receiving citations over time. It also reduces the noise of citation fluctuations (Katz, 2000) caused by journal impact factor and citation differences across research areas.
The allometric model
The analysis used the allometric model (Huxley, 1923) to explore the growth rate of scientific production and the citation impact of the 26 Chilean research-oriented universities, according to their types. Equation 1 shows the model. α is the allometric exponent (slope of the log-log regression line). The allometric exponent was estimated in equation 1 using the Marquardt-Levenberg algorithm (Marquardt, 1963). The statistical assumptions of the test are: (1) the normal distribution of the source population around the regression, (2) the variance of the dependent variable in the source population is constant irrespective of the value of the predictor variable, and (3) the residuals are independent of each other.
The following reasoning is used to interpret the results of the allometric equation. There are three possibilities given by that exponent:
-
1) When the exponent is equal to one, the result suggests an isometric relationship; that is, both variables y and x in equation 1 grow at the same rate.
-
2) When the exponent is greater than one, the result indicates that it grows at a rate higher than x.
-
3) When the exponent is less than one, the result indicates that y grows at a rate lower than x.
Results
Table 1 shows the scientific output and citation impact of Chilean research-oriented universities included in the study according to their types. Public universities account for 44% of scientific productivity, and 41% of the citation impact. Public-private universities account for 46% of scientific production, and 48% of the citation impact. Private universities account for 10% of scientific productivity and 11% of the citation impact.
Public | Public-private | Private | ||||
---|---|---|---|---|---|---|
Year | P | C | P | C | P | C |
2006 | 1544 | 6921 | 1498 | 7417 | 95 | 342 |
2008 | 1961 | 8381 | 2039 | 10 325 | 156 | 667 |
2010 | 2350 | 10 959 | 2424 | 13 228 | 289 | 1227 |
2012 | 2863 | 15 819 | 3240 | 24 893 | 496 | 2363 |
2014 | 3338 | 19 995 | 3520 | 24 912 | 874 | 4636 |
2016 | 4502 | 30 245 | 4747 | 35 264 | 1233 | 11 545 |
2018 | 5153 | 46 394 | 5573 | 47 285 | 1676 | 17 716 |
Source: The information was obtained from the Core Collection of the Web of Science.
Note: The citation impact was calculated using the SCI-Expanded, SSCI, AHCI, and ESCI. P = scientific production, C = Citation impact considering a fixed citation window t o+2
The allometric growth of scientific production and citation impact
Figure 2 shows the results of the allometric correlation. This way, the allometric exponent is greater than one, suggesting that the scientific production of Chilean private universities grow at a faster rate than the scientific production of the public-private A, and public universities B. The growth rate of the scientific production of Chilean private universities is 2.25 or 21.17 times with respect to public-private universities, and 2.35 or 21.23 times with respect to public universities.
Note: A Private universities’ scientific production ~ Public universities’ scientific production. B Private universities’ scientific production ~ Public-private universities’ scientific production. C Private universities’ citation impact ~ Public universities’ citation impact. D Private universities’ citation impact ~ Public-private universities’ citation impact.
Source: Results of the power-law regression. A: Durbin-Watson Statistic 2.24 Passed. Normality Test (Shapiro-Wilk) Passed P = 0.06. W Statistic= 0.76 Significance Level = 0.05. Constant Variance Test Passed P = 0.66. Power of performed test with alpha = 0.05: 0.99. B: Durbin-Watson Statistic 2.27 Passed. Normality Test (Shapiro-Wilk) Passed P = 0.20. W Statistic= 0.87 Significance Level = 0.05. Constant Variance Test Passed P = 0.29. Power of performed test with alpha = 0.05: 0.99. C: Durbin-Watson Statistic 1.42 Passed. Normality Test (Shapiro-Wilk) Passed P = 0.16. W Statistic= 0.86 Significance Level = 0.05. Constant Variance Test Passed P = 0.60. Power of performed test with alpha = 0.05: 0.99. D: Durbin-Watson Statistic 2.20 Passed. Normality Test (Shapiro-Wilk) Passed P = 0.07. W Statistic= 0.82 Significance Level = 0.05. Constant Variance Test Passed P = 0.90. Power of performed test with alpha = 0.05: 0.99.
Moreover, the allometric exponent in Figure 3 is greater than one, suggesting that the citation impact of Chilean private universities grow at a faster rate than the citation impact of public-private C, and public universities D. The growth rate of the citation impact of Chilean private universities is 6.59 or 22.72 times with respect to the citation impact of public-private universities and 5.78 or 22.53 times with respect to the citation impact of public universities.
The scientific production of private universities is 4.5 times less than public universities and 4.8 times less than public-private universities. However, the growth rate of the scientific production of private universities between 2006 and 2018 is significantly higher than that of public and public-private universities. The scientific production of private universities in 2018 grew 17.6 times with respect to 2006. Also, its impact grew 51.8 times with respect to the same period. The scientific productivity of public universities grew 3.3 times in 2018 and its impact grew 6.7 times with respect to 2006. The scientific production of public-private universities in 2018 increased 3.7 times, and its impact grew 6.4 times in relation to 2006. This result confirms the Katz and Ronda-Pupo (2019) conclusions that the effect of size matters when comparing entities of vastly different sizes in a complex innovation system.
Private universities, with 3.6 times less citation impact than public universities and 4.2 times less than public-private universities, show a much higher scientific productivity growth rate and impact than these universities.
To confirm the results obtained, an allometric analysis was performed, assuming the impact of a type of university in one year to be the quotient of the number of citations and its scientific production. For example, the impact of private universities in 2018 (Table 1) is . The results of the allometric analysis confirm that the growth rate of the impact of private universities is higher than that of public universities and of public-private universities.
Discussion and conclusions
The results show that the growth rate of the scientific production and citation impact of Chilean private universities is higher than that of public and public-private universities. This result does not support Casani’s conclusion on the Spanish university system. Casani et al. (2013) reported that Spanish private universities conduct research less intensively than public institutions.
Chilean private universities are making significant investments in infrastructure. Some private universities are also developing research in areas such as astronomy and biomedical sciences, which are highly productive scientific areas. Those strategies provide competitive advantages to private universities that contribute to enhancing their citation impact. Studies show that most higher education institutions are efficient in only one activity (Moncayo-Martínez et al., 2020). Chilean private universities began to shift from only teaching to both teaching and research-oriented. This policy contributes to enhancing their productivity and citation impact.
Private universities have policies to attract academics with high scientific performance through the allocation of better remunerations and incentives, driving academic exchange to develop international collaboration networks with universities with high scientific productivity.
The results open new research questions: Should Chilean higher education institutions shift from publication-centered to reward and strategic resource management strategies? Gómez-Mejía and Balkin (1992) reported a positive correlation between academics’ salaries and their number of publications in top-tier journals. Universities seeking to achieve and sustain high research performance should increase the allocation of research funds to prevent their academics from migrating to universities that offer better salaries or that have a more attractive publishing reward system. Public universities should pay special attention to the situation mentioned above to avoid intellectual decapitalization. Based on the information available, the Universidad Católica del Norte (public-private university) pays 52% more in monetary incentives to researchers that publish papers in JCR journals ranked in the first quartile (UCN, 2017) (≈ USD 2429, December, 2020) than the Universidad de la Frontera (public university) (UFRO, 2020).
The practical implication of the allometric model used is that it confirms the Chilean University System is characterized by scale-invariant emergent properties. The exponent of the allometric equation can be used for informing public policy about the scale-invariant emerging properties of this complex innovation system. Furthermore, this model gives decision makers novel insights unobtainable using conventional measures as number of citations.
The limitation of the study is that the use of a three-year citation window could punish universities that are specialized in humanities and/or social sciences, since these areas take more time to attract citations.