Making Assessment Count Evaluation project

The Making Assessment Count (MAC) project ran from November 2008 to October 2010, funded by JISC as part of their Transforming Curriculum Delivery through Technology programme and led by the University of Westminster.  Focused on the desire to engage students with assessment feedback provided to them, it explored processes for encouraging and guiding student reflection on feedback and developing a dialogue between learners and teachers around feedback.  Participants at a joint Making Assessment Count/JISC CETIS event on Assessment and Feedback back in February 2011 heard not only from project staff but also from students who were actively using the system and whose enthusiasm for it and recognition of its impact on their development as learners was genuinely inspiring.

Although the project team developed the eReflect tool to support the Making Assessment Count process, the primary driver of the project was the conceptual model underlying the tool and the development of eReflect was a pragmatic decision based on the lack of suitable alternative technologies.  The team also contributed a service usage model (SUM) on their feedback process to the eFramework.

The Making Assessment Count Evaluation (MACE) project will see the Westminster team overseeing the implementation of the MAC model in an number of partner institutions, and will see the eReflect tool in use in the Universities of Bedfordshire and Greenwich, and Cardiff Metropolitan University (formerly the University of Wales Institute Cardiff).  By contrast, City University London and the University of Reading will be focusing on implementing the MAC model within Moodle (City) and BlackBoard (Reading), and exploring how the components already provided within those VLEs can be used to support this very different process and workflow.

It is perhaps the experiences of City and Reading that will be of most interest.  It’s becoming increasingly evident that there are very strong drivers for the use of existing technologies, particularly extensive systems such as VLEs, for new activities: there is already clear institutional buy-in and commitment to these technologies, institutional processes will (or at least, should!) have been adapted to reflect their integration in teaching practice, and embedding innovative practice within these technologies increases the sustainability of them beyond a limited funding or honeymoon period.  The challenges around getting MAC into Moodle and BlackBoard are those that led to the need for the eReflect tool in the first place: traditional assessment and survey technology simply isn’t designed to accommodate the process of dialogue and the engagement of a student with feedback provided that the MAC process drives.  Of course, Moodle and BlackBoard, representing community-driven open source software and proprietary commercial systems respectively, present very different factors in integrating new processes, and the project teams’ experiences and findings should be of great interest for future developers more generally.

Deployment of the MAC process in such a wide range of institutions, subject areas and technologies will provide a very thorough test of the model’s suitability and relevance for learners, as well as providing a range of implementer experiences and guidance.  I’m looking forward to following the progress of what should be a fascinating implementation and evaluation project, and hope to see learners in other institutions engage as enthusiastically and with such good outcomes as the participants in the original MAC project.

Under development: SWANI

Anyone who’s ever worked on a European funded project or programme will be all too familiar with the volume of paperwork and time spent on administration and auditing to meet European funding and reporting requirements.  Digital signatures, although highly time and cost efficient, are not acceptable for auditing purposes with only hand signed documentation being permitted.

As part of a consortium providing a significant amount of European funded work based learning in Wales, Coleg Sir Gâr were keen to find a solution that would meet both European and Welsh Assembly Government requirements for hand written signatures as well as providing the elegance and efficiency of the online learner management and learner support systems colleges and tutors wished for.

The Secure Work-Based Learning Administration through Networked Infrastructure (SWANI) project, funded under the JISC Learning and Teaching Innovation Grants SWaNI FE programme, therefore set out to identify ways of addressing this tension and establish a pilot project as a proof of concept to form the basis of a long term solution.

After some research the project team settled on the Fastdox digital document system as offering exactly the combination of hand signed originals and timestamped digital copies necessary to meet the needs of all parties.

The documents to be signed are created in a MySQL database supported by a very user friendly and remotely accessible web interface.  These are then printed using the Fastdox software which applies a unique pattern of microscopic dots to the physical document to communicate with the digital pen.  The pen functions just like an ordinary pen, allowing trainers to sign the documents normally and therefore produce the required hand signed physical document, but the pen also stores all the written information, time stamped, for later downloading into the online learner management and auditing system: an excellent overview of the entire process is available from the product site itself and an exploration of how it was put into practice can be found on the project’s blog.  At between £4-500 for each pen and software package it represents a one-time investment that fulfils a long term requirement, requires little training for tutors to use and meets all the requirements the project set out to address – indeed, the biggest problem the project team ran into was the lack of standardisation in documents across WBL providers and changes to the document design part way through the project which required some revision.

With the pilot now coming to a close, the project team will be adding further information to the project website and undertaking a series of dissemination activities.  Their solution should be useful not only to FE colleges with similar funding and auditing requirements but for anyone looking for efficient and effective digital document management and tracking.

Under development: xGames

xgames-logoThe xGames project, a collaboration between Reid Kerr and Anniesland colleges, has been running for nearly a year and is currently in the final stages of piloting its innovative use of wireless xBox360 controllers for classroom quizzes.  Funded as part of the JISC Learning and Teaching Innovation Grants: SWaNI (Scotland, Wales and Northern Ireland) FE programme, the project has produced a highly user friendly question editor to allow complete novices to quiz and game design to easily author questions.  These questions can then be played in one of several games designed by the project on a large screen linked to a standard Windows PC fitted with USB receivers for up to four wireless xBox controllers.  Using wireless controllers is crucial as the range of the sensors allows a great deal of flexibility in classroom set up, permitting the use of breakout groups to discuss topics and feedback, for example.  Additionally, xBox controllers are familiar to many learners who are more confident using them than PC gaming.

The video below demonstrates the system’s use in a primary school classroom, and the engagement and enthusiasm of the children is immediately obvious, with lively discussions about the quiz questions and clear enjoyment of the session, the immediate indication of correct and incorrect answers providing instant feedback to the pupils.  The use of the large screen allows the teacher to constantly maintain a clear overview of the progress of the entire class, allowing her to identify topics that are generally not understood and which require whole class revision or struggling individuals within the group.  Discussion amongst the older group of college students is more muted, but their focus on the game mechanics and subject matter is evident.

[youtube]http://www.youtube.com/watch?v=ZRZZj1u9KQ0[/youtube]

The games, screenshots for which can be found under the games menu on the project site (software will be available from this site in due course), are designed using industry standard software such as XNA Game Studio, 3D Studio Max, Fireworks and Illustrator, with the question editor using a Visual Basic form for generating plain text files containing the question stem, distractors and correct response.  Unlike a commercial system such as Quia, questions are stored in a shared public folder so they can easily be shared and reused by teachers in different institutions.  XGames has generated interest from FE and, particularly, schools, and may well see further uptake as an affordable and easily adopted way of bringing game based learning into more classrooms.

First CFP for CAA 2012 now out

The first Call for Papers for the 2012 International CAA Conference: Research into E-Assessment is now available on the conference website, which also includes links to previous proceedings and a wide range of other important information about the event.

The conference will take place on 10-11 July in Southampton, and is jointly organised by Electronics and Computer Science at the University of Southampton and the Institute of Educational Technology at The Open University.

Thanks to @drdjwalker for the heads up :)