Evaluating Research by Publication
- Overview: Research Excellence Evaluation, Colin Steele, Emeritus Fellow, ANU.
- The Australian Research Council -ERA National Perspectives, Andrew Calder, Director, Research Performance and Analysis, ARC.
- The ANU Experience, John Wellard, Director ANU Research Office.
- Perverse and Beneficial Outcomes From Research Assessment, Professor Andrew Cockburn, Director, College of Medicine, Biology and Environment, ANU.
- Scholarly Publishing Economics: an International Research Perspective, Professor John Houghton, Victoria University.
- Future Publishing Metrics – The Way Forward?, Dr Danny Kingsley, Manager Scholarly Communications, ANU.
- Research Impact – the Wider Dimension, Dr Claire Donovan, Lecturer in Sociology, ANU.
Most of the "transcripts" are simply text, but Andrew Calder's "The Australian Research Council -ERA National Perspectives" is a set of slides. Here is a text version of the slides. Note that the FoR codes referred to are the Australian and New Zealand Standard Research Classification (ANZSRC) and "Scopus" is a list of journal articles and citations:
Excellence in Research for Australia
Director – Research Performance and Analysis
Australian Research Council
General ERA Principles
- Unit of Evaluation is the four‐digit ANZSRC Field of Research code (ie. 157 possible Units of Evaluation); evaluation occurs at the two‐digit level as well
- Evaluation by Research Evaluation Committees in discipline clusters; eight clusters in total
- There is a minimum level of output for a discipline to be considered ‘research active’ for evaluation in ERA
- Evaluations informed by a ‘dashboard ‘ of discipline‐specific indicators
- Some peer review of outputs accessed through institutional repositories in some clusters
ERA Process Overview
|Metrics Profile 1||Metrics Profile 2|
|Metrics Profile 3||Metrics Profile 4|
|Metrics Profile 5||Metrics Profile 6|
|Peer Review (if included)|
Research Evaluation Committee
Note - There are no weightings!
Mythbusting ‐ Ranked Outlets
- Only one of a number of unweighted
- indicators on the “Dashboard”
- Ranked conferences are essential for ICT,
- Engineering & Built Environment
- Note discipline‐specific practices
Role of the ranked outlets list in ERA
- Ranked journals
- Ranked conferences
- FoR assignments
- Ranks used for ranked journal/conference profile
- FoR assignments used to derive discipline specific benchmarks for citation analysis
Developing the ranked journal list
Initial development by Learned Academies and Peak bodies
Public consultation (June‐Aug 2008)
Expert Review of the public feedback
Omitted journal public feedback (Aug‐Nov 2009)
Final expert review of consolidated list (Dec 2009 – Jan 2010)
Parameters for inclusion in the lists
- publishes original research
- peer reviewed or equivalent process
- active during the ERA reference period (2003‐2008)
- has an ISSN
- able to withstand international scrutiny
The 2010 Ranked Outlet Lists
- The 2010 lists are fixed for this ERA evaluation
- ERA 2010 Ranked Journal List
- 20,712 journals included
- Over 88% agreement on ranks and FoR
- ERA 2010 Ranked Conference List
- 1,952 conferences included
|Volume and Activity|