Learning University Teaching: Lesson 4

Last week I attended day three of an introductory coruse in university teaching. The fourth and last day this week is about "Designing and Marking Assessment Tasks". Here is my thoughts in preparation:

Designing and Marking Assessment Tasks

It is useful to see what the public perception of university assessment is, or at least the media's perception. So here are the recent five top stories from Google News which mention "University Assessment":

1. Two Mumbai varsity staffers caught stealing answer papers, Hindustan Times,23 Jun 2011‎: The University of Mumbai on Wednesday caught two of its temporary staff trying to steal engineering answer papers from the university's central assessment centre at Kalina. The duo had tied eight answer papers to their legs and was walking out when the ...
2. Assessment and learning in the digital age, Media Newswire (press release): The symposium, Assessment and learning in the digital age, will take place at the University of Bristol's Graduate School of Education ( GSOE) on Friday 17 June from 1 to 5 pm in Room 4.10, GSOE, 35 Berkeley Square, Bristol. ...
3. KCPE and KCSE face scrapping, The Standard, Augustine Oduor, ‎Jun 21, 2011‎: The exams whose hallmark has been cutthroat competition among schools, and which hold the key to good secondary places and lucrative university courses, might be replaced with a list of assessment tests spread across the learning system. ...
4. Teaching quality under pressure as unis chase money, The Australian, Julie Hare, ‎Jun 20, 2011‎: SPIRALLING class sizes, overcrowding, tutorials replaced by seminars, few avenues for feedback and interaction, a shift to online and peer-assessment as a cost saving measure -- the dire state of teaching in Australian universities emerges from just a ...

These indicate that assessment issues are of concern globally, including: fairness, adaption to on-line delivery, reduction in the use of large end of semester examinations and peer assessment.

A search of Google Scholar, shows six documents featuring the words "university assessment" in the title for 2011:

1. Talking the talk: oracy demands in first year university assessment tasks: C Doherty, M Kettle, L May… - Assessment in Education: …, 2011 - informaworld.com... 18, No. 1, February 2011, 27–39 ISSN 0969-594X print/ISSN 1465-329X
2. Towards Fairer University Assessment: Recognising the
   
Concerns of Students N Flint… - 2011 - books.google.com: After all the hours of studying, reading and preparation, the nights spent revising and the writing and re-writing of assignments, 'success' for university students can often be represented with a single grade or digit, ...
3. 'Worldmarks': Web guidelines for socially and culturally responsive assessment in university classrooms, CE Manathunga, D MacKinnon - … Conference 2002: The …, 2011 - espace.library.uq.edu.au ... understanding. Yet university assessment in Australia is often based on a western template of knowledge, which automatically places International, Indigenous, as well as certain groups of local students at a study disadvantage. ...
4. 'In Press' Measuring up? Assessment and students with disabilities in the modern university, J Bessant - International Journal of Inclusive Education, 2011 - researchbank.rmit.edu.au, International Journal of Inclusive Education, vol. TBA, no. TBA, pp. ...
5. Teaching in the Corporate University: Assessment as a Labor Issue, J Champagne - 2011 - academicfreedomjournal.org ... Following the self-study, our provost established the Coordinating Committee on University Assessment. In March 2006, that ...
6. Towards fairer university assessment: recognizing the concerns of students,: A Iredale - 2011 - eprints.hud.ac.uk: This book is aimed at higher education academics, administrators and managers, researchers, and to some extent undergraduate and postgraduate students. It explores assessment as a determiner of student satisfaction, and is based upon Nerilee Flint's PhD thesis. A ...

These articles address fairness, cultural responsiveness and dealing with disability in assessment. Most interesting is that two of the six refer to the book: "Towards Fairer University Assessment: Recognizing the Concerns of Students" by Nerilee Flint (Routledge, 2011). Dr Nerilee Flint is Education Advisor, Student Equity, ANU. In the paper "Unfairness in educational assessment: Modifiers that influence the response students have to a perception of unfair" (2007) and later in her book, she suggests assessment is important to universities and is a powerful way to influence student behavior.

Integrating Assessment with Course Design

While education theory and items in the media suggest assessment is important, in practice it tends to be left to later, both in design and delivery of courses. Design of assessment is generally left until after course content is decided. Also much of the assessment of a university course is by way of an end of course examination, where the results of that examination cannot be used to help the student with learning in that course (as the course is over). If assessment is important, then it should be designed alongside the content and delivered before the end of the course.

My approach is to provide all assessment items at the beginning of the course (or preferably before the student enrolls). As an example, all assessment items for my two e-leaning courses "Green Technology Strategies" and "Electronic Document and Records Management" are available before the student starts the course. The assessment items are based on real world tasks the student will be expected to be able to carry out after the course. This avoids the philosophical conundrum of attempting to assess what the student "knows", instead assessing what they can do. It also appeals to students looking to do the course in order to get a better job.

To promote a sense of fairness and to avoid unnecessary requests for remarking, when marking assignments I first make detailed comments, giving the students examples of what is good, what could be improved, how and why. Rather than add up some marking scheme to give an arbitrary total, I instead form an assessment of the grade of the work (fail, pass, credit, ...) and then a mark within that grade. I provide the student with the detailed comments, the grade and the mark. This is a way to clearly tell the student my assessment of their work (this "credit" level work). It avoids the time wasting scramble for marks, where the student is tempted to ask for a few more marks to push them up into the next grade.

What is assessment for?

The accepted wisdom in educational theory is that there is formative and summative assessment. Formative assessment helps the guide the student with their learning while it is in progress, whereas summative assessment is for an external report on the results at the end. While, Harlen & M. James ("Assessment and Learning: differences and relationships between formative and summative assessment", 1997) argue that things are not as clear cut in the real world and there is a creep to wards using assessment originally developed for formative use as summative, this still seems a useful distinction to make. However, I like to provide marks for formative work, as a way to motivate students to do it (24% of the total marks seems to be sufficient).

When deciding how much assessment and in what form to use in an ANU course, I did a quick survey of Australian university assesment. For the usual 13 week course (of 9 to 10 hours work per week), universities typically require 40 to 60 words per percent. That is 4,000 to 6,000 words of assignments written by students for a complete course, with a set number of words corresponding to a length of examination or oral presentation. As an example, University of Melbourne equates one hour of examination or ten minutes of individual oral presentation to 1000 words of assignment.

Automated Assessment

Learning Management Systems, such as Moodle, have provision for automated quizzes built in. This would seem to be ideal for formative assessed, particularly when assessing what the student already knows at the start of a course, so they can be guided to concentrate on what they don't know. Professor Geoffrey Crisp, Director of the Centre for Learning and Professional Development, at University of Adelaide and author of the "e-Assessment Handbook" has given seminars in Canberra on how to automate assessment. This could be useful, but it takes considerable work to set up in advance and so far none of the organizations who commission me to design cruses have been willing to fund the work needed for this.

In designing automated assessment, it needs to be kept in mind that the system must, if possible, be designed to be usable by a wide range of students. As an example, the student may be remote from the campus, on a slow telecommunications link, have a low powered computer or have a disability. This needs to be taken into account when designing the assessment. As an example, if the assessment is in the form of a Flash animation, the student may not be able to use it, thus be disadvantaged and leaving the institution open to charges of unlawful discrimination.

In addition assessment should not be designed to make up for inappropriate course design. As an example, it is not possible to provide individual feedback in a live lecture to hundreds of students. "Clickers" can be used to conduct a quick quiz, students can submit questions via pieces of paper or Twitter. But one lecturer can't deal with all the questions from hundreds of students in a live environment and should not pretend they can.