This article originally appeared in Inside Higher Ed.
For the last 15 years, I have been involved in the study and reform of academic reward systems. Academic reward systems are fascinating to study because they reflect assumptions, values, goals, and aspirations held by institutions and fields.
I have studied academic reward system change in such areas as redefining scholarship, post-tenure review, stopping the tenure clock, and efforts to include ways to appraise new and diverse approaches to scholarly dissemination in the tenure process. My work has caused me to reflect on the current state of dominant academic reward systems, the assumptions that guide them, and the specific things I would like to see colleges and universities NOT do anymore, and start changing.
As a preamble to what I want us not to do anymore, I set forth the following principles. Most colleges and universities are charged with the goal of advancing knowledge within and through a diverse, inclusive community. By inclusive, I mean inclusive of both diverse individuals and diverse contributions to knowledge. Second, academic reward systems are about the valuing of professional lives and contributions. They are symbolic and concrete artifacts of what an institution values and aspires to become. Third, academic reward systems should ensure that faculty making excellent contributions to scholarship, teaching, and service should be retained and advanced. Yet, what excellence looks like in 2013 may differ from what it looked like in 1960 and 50 years from now.
Now, to the attitudes and policies that need to change:
First, the assumption that the process is unbiased, objective, and without partiality is naive at best, and at worst, harmful to professional lives.
Across the world, quality social science has demonstrated the pervasive nature of implicit and explicit bias in every aspect of a faculty career that is evaluated. When professors write letters of recommendation for male and female candidates for academic positions, there is bias in how candidates of equal qualifications are presented. When academics are sent two job applications with equal qualifications but different names for a laboratory manager position, they choose the male candidate and offer him a higher salary. Across research and doctoral universities, more women and faculty of color resign from their institution before going up for promotion or are advised to withdraw from the process. White researchers applying for grants from the National Institutes of Health are nearly twice as likely to win them as African-American scientists. There is bias embedded in scholarly publishing and the order of authors, as well as in service activities, and years to advancement in many fields. The Matthew effect, wherein certain senior scholars benefit from cumulative advantage in the numbers of scholars who cite them, and other less known scholars are not cited for equally meritorious work, have well-documented negative influences on women’s citations. Bias is more than a possibility; it is probable and real.
Second, the assumption that we know a scholar’s work is excellent if it has been recognized by a very narrow set of legitimacy markers, adds bias to the process and works against recognition of newer form of scholarship.
On May 16, 2013, a group of 150 scientists and 75 science organizations prepared a joint statement called the San Francisco Declaration on Research Assessment, which noted that metrics such as the Journal Impact Factor are used as quick and dirty assessments of academic performance and should not be. Thus, even scientists engaged in work most likely to be regarded through these indexes are increasingly negative about their use.
Typically, candidates for tenure and promotion submit a personal narrative describing their research, a description of the circulation, acceptance rate and impact factors of the journals or press where they published, a count and list of their citations, and material on external grants. This model of demonstration of impact favors certain disciplines over others, disciplinary as opposed to interdisciplinary work, and scholarship whose main purpose is to add to academic knowledge.
In my view, the problem is not that using citation counts and journal impact factors is a way to document the quantity and quality of one’s scholarship. The problem is that it has been normalized as the only way. All other efforts to document scholarship and contributions—whether they be for interdisciplinary work, work using critical race theory or feminist theory, qualitative analysis, digital media or policy analysis are then suspect and marginalized.
Using the prestige of academic book presses, citation counts, and federal research awards to judge the quality of scholarship whose purpose is to directly engage with communities and public problems misses the point. Interdisciplinary and engaged work on health equity should be measured by its ability to affect how doctors act and think. Research on affirmative action in college admissions should begin to shape admissions policies. One may find key theoretical and research pieces in these areas published in top tier journals and cited in the Web of Science, but they should also find them in policy reports cited at NIH, or used by a local hospital board to reform doctor training. We should not be afraid to look for impact of scholarship there, or give that evidence credibility.
Work that is addressing contemporary social problems deserves to be evaluated by criteria better suited to its purposes and not relegated to the back seat behind basic or traditional scholarship.
Third, the assumption that a scholar is excellent if others vouch for him or her in prestigious positions adds bias to the process and works against recognition of newer entrants and forms of scholarship.
One of the ironies of the current promotion and tenure system in many universities is that we make many attempts to authenticate the process as impartial, objective, and fair. For example, we make sure external evaluators have not published with the candidates. Yet the norms supporting choice of external reviewers make the assumption that only scholars at institutions of equal or greater prestige should evaluate the candidate’s work. In other words, we then add bias to the process by assuming the prestige of the external reviewer’s institution makes her a better reviewer than someone at a lower level in the academic hierarchy who might know the work better.
Much research has shown women faculty and faculty of color often do not to have the same “sponsorship” and career trajectory that lands them in full view of those in the top-ranked programs. Given this, they can be at a distinct disadvantage when external reviewers are chosen for their location and rank, as opposed to ability to judge the merits of the work before them.
So, how should administrators look to reform their institution’s promotion and tenure guidelines? First, don’t pretend bias doesn’t exist.
This starts with identifying the most likely places bias might appear (e.g., sponsorship/mentoring, citation counts, teaching evaluations, choice of writing venues). Promotion and tenure committee chairs might receive training to reduce the bias that infiltrates the process.
Don’t try to legitimize institutions and programs through the selection of external reviewers.
Instead, choose reviewers on and off-campus based on knowledge of faculty contributions. Promotion and tenure guidelines should say that if a faculty member is involved in work that is arguably distinct from the norms of the department, an on-campus faculty member who does similar work (e.g. interdisciplinary, qualitative and constructivist, community engaged, using feminist or critical race theory, digital media) should be added to the committee. Promotion and tenure guidelines should say that the choice of external reviewers for peer review should be based on the content of scholarship, including those using similar methodologies and frameworks.
There are good reasons a committee may want to choose external reviewers more advanced in career than the candidate. Yet they should not allow the fact that the potential reviewer is at a lower-ranked institution to prevent their selection. Some of the best scholars in a field select less prestigious institutions at which to work for quality of life reasons. I also think promotion and tenure committees should invite nonacademic reviewers if relevant to the case the faculty member is trying to make for the impact of their work.
Don’t judge candidates for promotion and tenure by one static, monolithic view of scholarship.
Redefine scholarship to include newer forms of knowledge making, whether they be with partners in communities, via digital media, or in efforts to eradicate injustice in laws, school systems, health care access, or the environment.
Furthermore, it should be considered an act of academic freedom to pursue academic work linked centrally to community engagement, just as it is to work in ways that are interdisciplinary, engaging cutting-edge technologies, and contributing the most basic next-step science to a cure for cancer. Neither promotion and tenure criteria or merit or post-tenure review criteria should constrain such actions. They should only require that this work be high quality, innovative and impactful, as in the cases of other forms of scholarship.
Don’t make faculty demonstrate the quality and impact of their work with a very narrow set of legitimacy markers.
Add language to promotion and tenure guidelines to identify a set of criteria that might be used to assess the products of all scholarship, and that there are multiple ways the faculty member can document they have achieved those criteria. Faculty could be encouraged to present impact statements, allowing them to present a case for the impact of scholarship in ways that capture the intent of their knowledge-making. Products showing impact could include funding from multiple sources, policy reports, downloadable curriculum, diagnostic instruments, broadcasts, discussion of research in legal cases and policy reports.
Scholars might still use citations, journal impact factors, peer-reviewed journal articles in top journals or books with top academic presses, and federal grants to establish the quality and reception of their work. However, it should be stated directly in the guidelines that faculty are invited and encouraged to make arguments for their work in other ways if those other ways are more appropriate to the form and purposes of their scholarship, and these alternative pathways are not second class strategies, and will be judged on their own merit.
It is important to note that these biases, along with the tendency to use prestige as a crutch for quality, also affect faculty outside the promotion and tenure process. Universities are losing talent because of bias in academic reward systems and work environments. Having engaged in exit interviews and retention studies of faculty leaving the academy, it is clear universities pay a major price by not acknowledging bias and expanding their definitions of scholarship in terms of the diversity of people and contributions they attract, retain, and advance.
It’s essential for those involved in promotion and tenure reform recognize that excellence is a socially constructed notion. As human beings in social systems within universities, we are flawed. Efforts to become a more diverse, inclusive community are intimately tied to the kinds of work our academic reward systems value, how we evaluate it, and how conscious we can be about the biases we bring to the table. If we start making these kinds of reforms, we will get closer to fulfilling the promise of advancing knowledge in old and new ways, through a more diverse, inclusive community.
