Ranking system of computer science departments

Research question I: What is the importance of different subjective decisions for department ranking, such as handling multi-author publications, inclusion criteria for publications and publication venues, accounting for the quality of publication venues, and accounting for the sub-areas of computer science?

An ablation study is performed to evaluate the importance of different decisions for department ranking (Our paper). The correlation between the resulting rankings and the peer assessment of computer science departments provided by the U.S. News was measured to evaluate the importance of different decisions. The results show that the selection of publication venues has the highest impact on the ranking. In contrast, decisions related to publication recency, multi-author publications, and clustering publications into subareas have less impact. Overall, Pearson’s correlation coefficient between the publication-based scores and the U.S. News ranking is above 0.90 for a large range of decisions, indicating a strong agreement between the objective measure and the subjective opinion of peers.

Research question II: Can we build a ranking system based on publication records of faculty? What are the benefits compared to other popular ranking systems?

Faculty citation measures are highly correlated with peer assessment of computer science doctoral programs (Our paper. We built and maintained a new CS ranking website based on the updated facultu citation data and incorporate citation information in our ranking metrics (Our ranking website).

Research question III: Is there a significant correlation between research quality and teaching quality for both faculty-level and department-level analysis?

We leveraged data gathered from RateMyProfessor, a widely used online platform for student evaluations. To conduct this analysis, a dataset comprising teaching ratings of faculty members was collected from RateMyProfessor, encompassing a diverse range of courses in computer science. Another dataset of faculty citations was used to evaluate the research quality of a department. Statistical methods were employed to quantify the strength and direction of the relationship between teaching and research quality indicators. The implications of the findings extend beyond individual faculty evaluations, potentially influencing institutional strategies for faculty development, curriculum design, and resource allocation.