Interactive e-Assesment

Professor Geoffrey Crisp, Director of the Centre for Learning and Professional Development and Professional Development and Director of Online Education at the University of Adelaide, presented a seminar on "Assessment: more than just multiple-choice" at ANU in Canberra today. He showed a simple way to implement computer assisted assessment using existing tools, supported by research. I have been doing some of this with Moodle for commercial courses, but did not consider to be academically respectable. Now I know it is okay, I can use it for university courses. ;-)

Geoff started with an image from a medieval painting of a lecture in a monastery. He pointed out that apart from the lack of women, much of the way a lecture was presented then is much the same as now. The lecturer was at a podium at the front of the room, the students were in rows, some listening, some reading, some talking.

Geoff went through the types and purposes of assessment (it is not just because you need to pass). One use is to see what the students already know. "Clickers" can be used to get a quick assessment of the students knowledge in class. But more complex electronically collected assessment need not be marked automatically, it can be used just to collect the answers for human marking. However, before setting assessment the teacher needs to decide what scheme is appropriate and what is being assessed.

e-Assessment can have advantages for immersion and interactivity, social interactions. The results can look like computer and role playing games.

There are some examples of research on interactive, some beter than others, such as Literature Review of E-assessment (Futurelabs) and "Effective Practice with e-Assessment" (JSC).

Geoff showed an approach where the interaction is separated from the assessment. In this model the question invokes the interactive component. As an example the question is asked in a web page in a learning system, such as Moodle, this then invokes an interactive application provided via a web browser plugin and when the plugin is finished the student is returned to the web page where they can enter their answer.

What I found confusing was that this did not seem to be "interactive assessment", in that there was no interaction between the system asking the question and the application the student uses to research the answer. The computer presents a tradition text question, launches the interactive plugin for the student to interact with and then asks them for a traditional answer. There no assessment in the interactive environment. Geoff pointed out that this makes it a lot easier to create and is more flexible, but it took me some minutes to work out that this could be considered a legitimate "interactive" assessment.

Geoff pointed out the advantages of this approach: the academic can use pre-prepared modules and ask different questions about it for different levels of students. What also appears an advantage is that this only a small step beyond the computer based tutorials I prepare for ANU courses and is essentially the same as the system I used for a commercial training course.

Interactive Tutorial Example

With the ANU tutorials, a web page is prepared which looks much like a traditional set of notes for a tutorial/lab. However, the student is provided with a hypertext link which takes them to an online tool which they use to carry out part of the work. They copy some parameters from the tutorial sheet into the tool and copy some results back as their answer. An example is questions one to three of the metadata tutorial for "IT in E-Commerce" (COMP3410/COMP6341). In this the student is first asked to hand code some metadata, then use an online tool and compare the results and reflect on the differences. It would not have occurred to me to describe this approach as "interactive".

Interactive Assessment Example

With "Writing for the Web", a one day course prepared for a local government agency, there are a series of modules consisting of a set of slides used by the presenter live in the classroom, with accompanying notes for the students to read, followed by an exercise for them to do. The slides, notes and exercises are implemented using Moodle. The assignment module of Moodle was used to implement the exercises, but with no expectation that the results the students, just as a way for the student to type in the answers.

As an example in "Search the web for useful words", the student is asked to conduct search using two web tools and compare the results. They type their answers into the Moodle assignment web form (only visible to students enrolled in the course).

Interactive Assessment for Public Service Course

The Electronic Document Management module I am preparing for Systems Approach to the Management of Government Information was going to use the same style of online exercises as the writing fore the web course. However, I had assumed that I would need to provide a paper based test as well. But Professor Crisp's seminar seems to show that the interactive assessment could be used at least for part of the assessment. What would also make this easier would be purpose built flexible learning rooms for this form of teaching.

Postgraduate Online Professional Education for ICT

The ACS provides online courses in IT management using a Moodle based system for its Computer Professional Education Program. ACS also provide a Diploma of Information Technology for budding IT professionals in the Asia-Pacific region. e-Assessment could be of value for these.

More Interactive Assessment

However, to me having a web page launch a web application and then ask the student to copy result from it into a web form is not really "interactive assessment". Geoff commented that the assessment can be built into the interactive application, but this then reduces the flexibility. However, a level of interactive could be created by the assessment system passing parameters to the application and the application passing back some results at the end. This could use Web Services. It could be as simple as the assessment system adding some parameters to the end of the URL used to invoke the application. Results could be returned using XML encoded data, as is done for electronic tax returns, or even just a web page with the results included with Microformats. This would need some work, as my one attempt at including microformats in Moodle, the Moodle web editor modified the content breaking the Microformat.

Next Gen Not Wired

Geoff ended the seminar by cautioning that "Next Gen" students will not necessary benefit from, or want to use, social networking tools for education, even thought they use them socially. So educators should not assume the students will want to do everything via a computer, mobile phone and iPod.

See Also

• Centre for Learning and Professional Development's moodle system.
• Books on e-Learning
• "The e-Assessment Handbook" by Geoffrey Crisp, Continuum International Publishing Group, 2007 (available from Amazon.Com):

  Assessing learning in an online environment is being used by teachers and institutions at an increasing rate. Learners are demanding a more flexible approach to assessment activities just as they have done with learning. This demand will increase as online practice becomes embedded into all schools, further and higher educational courses and corporate training programs. As students are engaging with content in an online environment, they will also need to be assessed using the same tools that are used for the learning experience. This book provides practical guidance to various aspects of online assessment including:

  • types of assessment
  • choosing the right software
  • examples of e-assessment over a wide-variety of disciplines
  • making e-assessment interactive

  
  
  Table Of Contents Preface
  Section 1 How does assessment affect learning?
  1. Overview

  • Why do we assess?
  • Relationship between learning and assessment
  • Assessment for learning
  • Assessment and grading

  
  
  2. Guidelines for assessment practices
  - Professional bodies publishing guidelines on assessment
  - Assessment models
  - e-Assessment practices and their relationship to learning
  3. Types of e-Assessment items
  - types of e-assessment
  4. Cosequences of using e-assessment
  - Risks involved in using e-assessment
  - How is e-assessment currently being used?
  - What are the limitations of e-assessments?
  5. Choosing the right e-assessment software
  - choosing an appropriate assessment format
  - how do you want to administer your e-assessments?
  - what do you want to administer your e-assessments?
  - what type of questions do you want to ask?
  - choices in e-assessment software
  - other types in online assessment software
  - adaptive assessment software
  
  Section 3 What does e-assessment look like?
  6. e-assessment examples by discipline
  - General sources of e-assessment questions
  - Accounting, Commerce and Economics
  - Arts and Humanities
  - Biological and Health Sciences
  - Chemical Sciences
  - Earth and Environmental Sciences
  - Physics and Engineering
  - Mathematics
  
  Section 4. What are the practical issues when doing e-assessment
  7. Item and Test Bank use
  - constructing Ttem and Test Banks
  - using Item and Test Banks
  - non-adaptive test delivery methods
  - adaptive test delivery methods
  8. Grading and marking Items from the Test Bank
  - general grading and marking issues
  - negative and confidence level marking of objective items
  - marking selective response items
  9. Validity and reliability
  - overview of validity and reliability
  - classical test theory
  - Item response theory
  - Rasch modelling
  - estimating reliability
  - evaluation of test banks
  

  10. Standards, specifications and guidelines
  - assessment standards
  - national standards
  - professional association standards and guidelines
  - identifying standards
  - content standards, specifications and guidelines
  - assessment content metadata
  - interoperability
  11. Security issues
  - authentication of appropriate access
  - secure assessment environments and software
  - standards and guidelines
  - what to do when things go wrong
  12. Accessibility issues
  - statutory requirements for accessibility
  - good practice guidelines

  - testing accessibility

  - improving accessibility

  - assistive technologies

  Section 5 How Do I Make e-Assessment Interactive

  13. Assessing discussion groups and collaborative tasks

  - communicating and collaborating online

  - overview

  - examples of software for e-collaboration

  - design principles

  - collaborative problem solving and group work

  - online discussion groups

  14. Interactive assessment and Java applets

  - learning designs

  - simulations

  - Java applets

  15. Assessing role playing and games

  - characteristics of online role-plays

  - senario-based

  - game-based

  Section 6 What about the future?

  16. The future of e-assessment

  - current perceptions of where e-assessment is heading

  - drivers for change

  - what will it be like in the future

  Appendices

  Glossary