Bloomington Faculty Council adopts policy on responsible metrics
May 4, 2016
Metrics guide the hand of contemporary research evaluation. Global and national ranking systems have long included quantitative indicators, some of which have been derided by the institutions which they evaluate.
However, it is the move from institutional- to individual-level analysis that has heightened concerns about the accuracy, transparency and relevance of indicators. Metrics have crept into grant applications, promotion and tenure dossiers, and academic job applications.
As the use of metrics has expanded, so too has the pool of available data and range of evaluative systems. Administrators in higher education have quickly adopted these systems -- justifying the expense as a matter of competitiveness: peer comparison with other Association of American Universities institutions, for example, is an easy selling point for any new product on the metric market.
Many of the systems that measure faculty scholarly activity can be used strategically by administrators to support the research mission of an institution. Yet, there can also be several unintended consequences. One has only to consider the history of the Journal Impact Factor to understand how easily an indicator can be misapplied and abused.
Given the proliferation of new data sources and tools, it is imperative that institutions develop policies that promote responsible use of metrics. On April 27, the Bloomington Faculty Council approved such a policy. This policy governs all systems that monitor and aggregate information on the research and creative activities of IU Bloomington faculty members where data are made available at the individual level.
There are five foundational principles, informed by the Leiden Manifesto:
- Systems used by faculty and administrators should acknowledge and take into account the heterogeneity of disciplines by making coverage transparent and including field normalized indicators.
- Quantitative indicators generated within these systems should be used to supplement rather than supplant other forms of review, such as peer review.
- The structure, data, and use of the system should align with the values of the institution and not incentivize behavior incompatible with these values.
- Systems should provide data that are accurate and can be made available for validation.
- Data about faculty members should be made available to those faculty members.
The implementation section of the new policy ensures faculty members are involved in the selection systems and have an opportunity to communicate, through formal representative faculty governance, recommendations to discontinue the use of systems should they fail to meet these principles.
The policy is not a reactive policy that will restrict use of contemporary data, but rather a positive and proactive policy to promote principled use of metrics.
This policy continues IU Bloomington’s longstanding commitment to shared governance and tradition of excellence and transparency in scholarship. It is hoped that this will serve as a model for other institutions seeking to adopt policies on responsible use of metrics.