Academic Productivity Index
John M. Malouff, PhD, JD
Abstract
This article describes the need for assessing productivity in academics and then presents the Academic Productivity Index (API), which provides a single number reflecting the annual revenue-producing accomplishments of an academic. The API includes (1) number of students taught, (2) number of students supervised who earned a higher research degree, (3) number of books and peer-reviewed journal articles published, and (4) dollar amount of research grants received. Each amount is weighted by the amount of money it provides to the university that year. The article describes strengths and weaknesses of the index and presents possible uses, including evaluating and motivating individual academics and, when used as an average across a group of academics, evaluating university departments.Introduction
Productivity can be defined as the level of economic output per unit of input, for instance, per worker (Dolman, Parham, & Zheng, 2007). Productivity is important to the wealth and well-being of a nation, and efforts to improve productivity occur at all levels of society (Diewert & Lawrence, 2006). The productivity of nations is positively related to the average education level of the populace (Dolman et al., 2007). Hence, universities play an important role in national productivity by providing education. Universities also have internal needs for productivity (Snell, 1982). Government funding of public universities around the world has in recent years been more and more closely tied to evidence of productivity (Orr, Jaeger, & Schwarzenberger, 2007).
In Australia, the federal government has recently viewed universities more and more as business competitors in a global education market (Pick, 2006). Developing measures of performance for use in evaluating universities is now in vogue (see e.g., Coates, 2007). Australian universities receive government funding for (1) number of students taught, (2) number of doctorates awarded, (3) dollar amount of government research grants received, and (4) number of publications (see Australian Government Department of Education, Science and Training, 2007; Universities Australia, 2007).
At universities, productivity issues sometimes take the form of workload requirements (see Houston, Meyer, & Paewai, 2006; Layzell, 1996). For example, at the University of New England in Australia, most academics are expected to meet a minimum level of an index called TSI (Teaching and Supervision Index). The TSI adds Effective Full-Time Student Load (EFTSL) for an academic (here generally amounting to number of students taught in a unit divided by 8) and the number of higher education students supervised (as main supervisor). The minimum TSI for individuals in disciplines such as psychology is 23 (UNE Workplace Agreement 2006-2008).
The TSI has potential value in that it is objective. However, the TSI has a weakness in that it measures number of research higher degree students supervised, when Australian universities ultimately get funded for the number who graduate, not number enrolled. What precisely a university asks of its academics can be important; if the university goal is set at number of students supervised, that rate may be high while the rate of graduations is not. Other indices exist specifically for research productivity: number of publications per year, number of citations of published works (Marinova & Newman, undated), and the new h index, produced by an algorithm involving number of publications and number of citations (Hirsch, 2005).
The present article presents a new index potentially suitable for use in setting work loads for academics and for examining productivity levels. The Academic Productivity Index (API) quantifies how much revenue is attributable to the work of individual academics. The four annual components relevant to Australian universities include: (1) number of students taught (or EFTSL), (2) number of students supervised (as primary supervisor) who earned a higher research degree, (3) number of DEST publication points earned (these relate to number of books and peer-reviewed journal publications; see Australian Government Department of Education, Science and Training, 2007), and (4) amount of research grant money received. Joint outputs, such as co-authored publications, are divided equally among the academics involved in producing the university’s revenue related to the work. Each of the four components is weighted by the amount of money it provides to the university that year. Hence, the API is a dollar amount. Because the actual amount of revenue produced by each of the four API components might vary slightly from year to year with variations in the amount of fees students are charged and variations in government funding, universities might need to re-weight the components annually. However, universities, for the sake of simplicity, could peg the API component weights to amounts of the initial year of use and alter the weightings only once every few years.
Some examples will make apparent how a university could calculate APIs. The following API component weightings for the University of New England come from information provided by university administrators Professors Peter Flood and Ken Watson.
Suppose that we have two academics, A and B, in a psychology department at the University of New England. Let us imagine that for 2007 A had 12 journal or book chapter publications. Each of these would bring $1,600 to the university, for a total of $19,200. However, for 10 of the publications, there is one other academic at the university who is a co-author, so A receives only half credit ($800) for these 10 publications. The publications thus amount to 10 x $800, plus 2 x $1,600, for a total of $11,200 attributable to A. The academic also teaches units with 150 and 11 students. Together, these two units amount to 21.125 EFTSL (161/8). A also supervises five 4th year research students (each student representing 0.5 EFTSL), amounting to a total EFTSL of 2.5. A also co-teaches a unit with 140 students. Giving A credit for half of the 140 students leads to an EFTSL of 8.75 (70/8). The grand total EFTSL for A is 21.125 + 2.5 + 8.75 = 32.375. Because these are behavioural sciences students, the university receives $11,253 per EFTSL (there are other amounts for students in different courses of study), for a total teaching revenue attributable to A of $364,316. The academic has no grants that bring any extra government funding to the university, and has no research-doctorate students graduating. His total amount for revenue produced (his API) is $375,518, with the following components: Teaching: 364,316, supervision of graduating PhD students: 0, publications: 11,200, grants: 0.
Academic B teaches only one 11-student unit, producing 1.375 EFTSL (11/8) for a revenue of 1.375 times $11,253 = $15,473. She has one PhD student graduate; that high-cost psychology PhD completion brings $74,000 to the university. Also, B has three journal article publications, producing a total of $4,800 in revenue. B and a colleague at another Australian university share equally $200,000 in Australian Research Council research funding for the year. The $100,000 in research funds received by B this year from the grant produces approximately $47,000 in extra funding to the university (including $15,000 under the Research Training Scheme, $12,000 under the Institutional Grants Scheme, and $20,000 under the Research Infrastructure Block Grants Scheme). Note that the $100,000 for the grant itself is not added to B’s API because this money goes to the researcher for the study, not to the university to use. The API components for B thus include teaching: $15,473; supervising a PhD graduate: $74,000, publications: $4,800, research grant funding (for the use of the university): $47,000. B’s API is the sum of these amounts: $141,273.
Both the API totals for academics A and B and their API component totals convey important information. For this year, Academic A is substantially more productive in producing revenue than B. One might conclude that A has a larger API because he teaches far more students. His high number of publications makes no significant difference, because little revenue comes from publications under the current government funding schemes. One obvious way for A to improve his API is to have a student earn a PhD; another way is to obtain a federal research grant. An obvious way for B to produce a higher API is to teach more students. However, any improvement in component productivity will lead to some increase in API.
Universities other than the University of New England might apply different weights from those described above. That would create no problem as long as there is a logical financial basis for the weights.
The API has several positive attributes. First, it focuses on education and research outcomes universities tend to seek – outcomes closely related to the mission of universities (see Snell, 1982). Second, it is objective and transparent. Third, it provides information in a convenient single number, much like a grade point average for students. Fourth, it is relatively easy to calculate once the weightings are chosen. Fifth, its component amounts provide important information about the best avenues for improved productivity.
With the API, university administrators could evaluate individual academics for financial contributions to the university. Administrators could use average API to evaluate departments (and the department heads), other administrative units (e.g., faculties or programs), and ranks of academics (e.g., professors). The API might also prove valuable for the states and the federal government to compare universities using average API if weightings could be made uniform across universities for this purpose.
To improve productivity with the API, university administrators would have to do more than calculate API scores. The administrators would have to set and announce API goals and help motivate academics to pursue the goals (see Taylor, 2001). One simple way to do this might be to provide academics with their API and the average for their department or other relevant groups. Of course, recognition and other rewards for high API’s might also help increase productivity (see Gilmore & To, 1992; Hales, Shahrokh, & Servis, 2005).
The API has several limitations. First, it does not consider administrative work or public service. Although teaching and research are usually thought of as the most important tasks of academics (Houston, Meyer, & Paewai, 2006), administrative work is needed to keep a university operating, and public service can lead to important benefits. Further, doing administrative work and providing public service can reduce time available to teach and do research (see Taylor, Fender, & Burke, 2006). Hence, for some purposes, one would want to supplement the API with information about administrative performance and public service.
Second, the API does not assess the quality or impact of teaching or research contributions, and quality is important (Burke, 1993; Layzell, 1996). Development of useful measures of research quality and impact is now a driving force in Australia, with the emerging Research Quality Framework initiative (Australian Government Department of Education, Science and Training, undated). But whatever the theoretical value of assessing quality of research or teaching, the practical difficulties of doing so are substantial (Layzell, 1996). At any rate, qualitative assessments of teaching and research or non-financial quantitative assessments such as student evaluations of teaching and number of times an article is cited could be used to supplement the API. It is imprudent to use any index, whether the API or an IQ score, in isolation to make important decisions.
Third, the API does not assess certain admirable types of behaviour, such as being agreeable or helpful. Instead, the API focuses exclusively on outputs of revenue-related behaviour. A humanistic approach to administration also values these other types of behaviour. Hence, it would be prudent to supplement the API with other types of information in evaluating academics and subunits of the university. The problem with these other types of behaviour is, of course, that they are notoriously hard to quantify. The API, by contrast, is as objective as dollars and cents.
Fourth, universities in some nations may not have available the sort of information needed for the four components of the API or may not be able to assign appropriate weights to the four components. However, the components of the API can be varied to suit the local situation. The essential core of the API involves quantifying research and teaching activities in some way and then weighting them in a manner that reflects revenue-related outputs.
In conclusion, the proposed new measure of academic productivity, the API, like all performance measures, has both strengths and weaknesses. Its main value is that it can provide universities with a single objective number showing the financial contribution of an academic to the university. It is also versatile enough to assess subunits of the university when used as an average across a group of academics.
References
Australian Government Department of Education, Science and Training (undated). Research quality framework. Retrieved May 9, 2007, from http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issues/research_quality_framework/.
Australian Government Department of Education, Science and Training (2007). Specifications for the collection of 2006 data. Retrieved May 15, 2007, from http://www.dest.gov.au/sectors/research_sector/online_forms_services/higher_education_research_data_collection.htm.
Burke, J. C. (1993). Preserving quality while increasing productivity. Studies in Higher Public Education, Number 4, 1-44.
Coates, H. (2007). Excellent measures precede measures of excellence. Journal of Higher Education Policy and Management, 29, 87-94.
Diewert, W. E., & Lawrence, D. (2006). Measuring the contributions of productivity and terms of trade to Australia’s economic welfare. Retrieved May 7, 2007, from http://www.pc.gov.au/commission/work/productivity/publications/reports.html.
Dolman, B., Parham, D., & Zheng, S. (2007). Can Australia match US productivity performance? Retrieved May 7, 2007, from http://www.pc.gov.au/commission/work/productivity/publications/reports.html.
Gilmore, J. L., & To, D. (1992). Evaluating academic productivity and quality. New Directions for Institutional Research, 75, 35- 47.
Hales, R. E., Shahrokh, N. C., & Servis, M. 2005). A department of psychiatry faculty practice plan designed to reward educational and research productivity. Academic Psychiatry, 29, 244-248.
Hirsch, J. E. (2005). An index to quantify an individual’s research output. Proceedings of the National Academy of Sciences, 102, 15,569-572.
Houston, D., Meyer, L. H., & Paewai, S. (2006). Academic workloads and job satisfaction: expectations and values in academe. Journal of Higher Education Policy and Management, 28, 17-30.
Layzell, D. T. (1996). Faculty workload and productivity: Recurrent issues with new imperatives. Review of Higher Education, 19, 267-281.
Marinova, D., & Newman, P. (undated). Academic productivity and the changing research funding models in Australia: What is the true picture? Retrieved May 16, 2007, from http://72.14.209.104/search?q=cache:NS6yRFHeXsQJ:www.mssanz.org.au/modsim05/papers/marinova_2.pdf+academic+productivity&hl=en&ct=clnk&cd=3&gl=au.
Orr, D., Jaeger, M., & Schwarzenberger, A. (2007). Performance-based funding as an instrument of competition in German higher education. Journal of Higher Education Policy and Management, 29, 3-23.
Pick, D. (2006). The re-framing of Australian higher education. Higher Education Quarterly, 60, 229-241.
Snell, W. (1982). Efficiency and productivity in education systems. Journal of Higher Education Policy and Management, 4, 177-184.
Taylor, J. (2001). The impact of performance indicators on the work of university academics: Evidence from Australian universities. Higher Education Quarterly, 55, 42-61.
Taylor, S. W., Fender, B. F., & Burke, K. G. (2006). Unraveling the academic productivity of economists: the opportunity costs of teach and service. Southern Economic Journal, 72, 846-859. UNE Workplace Agreement 2006-2008. Retrieved May 9, 2007, from http://www.une.edu.au/hrs/.
Universities Australia (2007). Australia’s universities create, preserve, transmit knowledge. Retrieved May 29, 2007, from http://www.universitiesaustralia.edu.au/content.asp?page=/universities/overview.htm.