Archive for June, 2008
Congratulations to all at QuestionMark, whose flagship product Perception was named ‘Best Assessment Tool’ at the LearnX Asia Pacific 2008 eLearning and Training Awards held in Melbourne, Australia a couple of weeks ago. Described by the judges as ‘the best online testing and reporting solution available - stable and actually works’, the award reflects QMP’s maturity, significant share of the eassessment market and ongoing popularity both in the UK and beyond.
Those planning to attend the 2008 QuestionMark European Users Conference on 14-16 September in London still have a few days to register for a £50 reduction - early bird registrations close on 30th June.
John’s post on his experiences with the FireFox 3 del.icio.us plugin provided me with one of those OMG moments that happen every so often, mainly when I realise that I’m still a fundamentally web 1.0 person in an increasingly 2.0 world.
My problem with the whole social sharing aspect of delicious is that I actually find delicious rather useful just for me, and began using it as a personal repository of links long before I ever really considered the knowledge sharing aspect. I regularly switch between three different computers, so having an online set of bookmarks seemed like a very good idea. It does run a bit too slowly to use it for links that I can remember myself or find with a little effort, but as a place to store links to ’interesting stuff’ it seemed ideal. It never occurred to me that anyone would actually look at what I’d been linking, so when someone casually mentioned that they’d read a link I’d tagged I felt rather as though someone had been rummaging through my drawers, raising a sardonic eyebrow here and there and sneering at my much loved Bagpuss socks. Reading John’s comment that ‘I spend a few minutes each morning looking at what my network has been bookmarking’ reinspired that uncomfortable feeling and created an overwhelming desire to tag loads of (possibly NSFW/offensive) Spore porn to discourage further reading (is it actually possible to troll one’s own delicious page?).
In all honesty, my delicious page isn’t all that useful, even to me, mainly because I could really have put a lot more effort into tagging things in a more meaningful way. The tags are, in their own way, impressive: they’re so random, generic and inconsistent that they’re actually effectively useless for finding anything - if I want to find something I’m certain I’ve added I’ve resorted to just scrolling through the entire list clicking on possible candidates rather than try to work out which of a screed of undescriptive tags I’ve used. Although they’re not quite as bad as ‘important‘ or ‘me‘, they’re really not too far off it; combined with creative use of synonyms and avoidance of the ‘description’ and ‘notes’ fields in the tag form, I’ve managed to create a set of bookmarks in which it’s virtually impossible to find anything and which becomes less and less useful and useable the more I add to it. Go me.
If I’d thought about it in advance, of course, I’d have created separate accounts for work links and personal interest links - except that they’re frequently the same thing, so perhaps I should have two accounts and just duplicate the vast majority of entries? Perhaps I should have a separate account for each topic I’m interested in? - but then, that completely undermines the point of tagging entries in the first place. I’ve always felt fairly sheltered from the clashing of different areas of my life as I’m not on FaceBook, but my cunning use of the same ‘anonymous’ handle on delicious, Skype, Twitter, PMOG, Digg, Flickr, FriendFeed (which is a sad and lonely experience when no one you know is on it) and just about everywhere I went has proven to be not the best idea if I’m going to get touchy about people coming across my collection of links on how to play a mage well in World of Warcraft, or that hilarious Craigslist sex baiting prank.
Although I realise it doesn’t sound like it, I do think that the social aspects of tools such as delicious are incredibly useful. I’ve added links to delicious pages tagged QTI and eassessment (but should it have been e-assessment?) to our assessment domain page, and have found some invaluable resources because of other people’s tagging (similarly, I had more responses to posting details about SURF’s book on Twitter than I did from my blog post on it). I could make my non-work delicious tags private, but that would mean that they weren’t available to non-work people who would find them useful. For me, John’s post highlights the increasingly pressing need to be able to define the communities with which we engage rather than being defined by them, the need to respect these different personae, and to reconceptualise the walled garden as user-centric and user-defined rather than something that is imposed on us by disinterested parties for the sake of technological and commercial convenience.
A new white paper from the SQA, Innovating Assessment in Scotland, is now available for download. Reporting on work undertaken by the Scottish Online Assessment Resources (SOLAR) project using BTL’s ContentProducer, the white paper discusses their methodology for developing and delivering eassessments, analyses benefits, barriers and drivers for the use of eassessment and explores future approaches to the use of large scale eassessment and substantial item banks within and beyond Higher National qualifications.
A recent post on Hitwise reveals that, in the UK, we’re spending more of our browsing time visiting blogs and personal websites than ever before. These sites accounted for 1.19% of all UK internet traffic in the week to 7 June - that’s a remarkable 1 in 84 of all internet visits. Hitwise’s statistics show that this short team leap is reflected in the longer term view, with blog visits having risen from just over 0.3% of traffic in May 2005. Curiously, there’s a marked difference between UK and US trends, with blogs having a significantly smaller part of the market in the US than they do in the UK; despite this, the number of British blogs in the top 20 is surprisingly low, particularly once blog aggregators such as Guardian blogs are discounted (see Robin Goad’s comment for the breakdown).
A significant new publication from the SURF project ‘Active Learning, Transparent Assessment’ is now available as a free pdf downloadand is recommended as an excellent study of eassessment design and development methodology.
Chapters include a detailed taxonomy of item types based on response structure, mapped to other taxonomies such as QTI v2.x and QuestionMark Perception; an in-depth study of multiple response items; an examination of the inadequacies of existing guidelines for question authors with proposals for an improved set of guidelines with reports on their use in their fifteen case studies; guidance on how to select and enact an appropriate subset of the new guidelines in particular contexts; methodology and task allocation for item and assessment design and development; and a detailed discussion on how the QTI specification can support the design process and innovation in assessment. Many of the author names will be familiar as leaders in the domain, including amongst others Rob Hartog of Wageningen University (who also edits the book),Ignace Latour of Cito and Pierre Gorissen of Fontys University of Applied Science.
This book is an invaluable addition to the literature on eassessment, and will be of interest to everyone from subject matter experts and educational researchers to educational technologists, systems developers and even managers and administrators.
Found all by myself this time :-) was a series of tweets detailing Pierre Gorissen and Steve Lay’s successful integration of the ASDEL QTI Playr with Moodle, meaning that Pierre is now able to run QTI 2.1 assessments within a Moodle course. This is a signficant step forward for making the specification more attractive to users, given the huge popularity and dynamic community that surrounds Moodle. It might also make QTI a more attractive alternative to OpenMark within the Moodle community - there are some interesting comments to be found on QTI in the Moodle forums.
Thanks also go to Adam for pointing out RM’s Test Authoring System which claims to be fully compliant with IMS QTI 2.1, making it one of the earliest commercial products to implement the revised specification. I couldn’t find a demo to try out, but it is good to see the specification finally being implemented in this type of system and market sector. Also on the website are also a couple of research reports on the impact of ICT in the classroom which are well worth reading.
Things may be quiet at the moment on the release of the final version of IMS QTI 2.1, but there’s been quite a bit of activity in the background around implementations and integration with other systems.
Keris, the Korea Education and Research Information Service (similar to the UK’s JISC), have been quite active within IMS and in February signed a formal memorandum of understanding with IMS to launch IMS Korea. Amongst their activities is involvement with Teaching Mate, a commercial product which aims to support QTI export by the end of the year. Also from Keris is a QTI 2.1 player that does import and export QTI 2.1 - it’s in Korean, but Adam cunningly pointed out that you can use the browser status bar to work out what each button actually does.
It’s worth having a look around the Keris site to see the extent to which they support ICT in education. Of particular interest is EDUNET, the National Teaching and Learning Center established in 1996 which, amongst a range of other services, provides a large range of school-level teaching materials and ‘an online testing service to evaluate students’ achievements’. Would teachers in the UK welcome a centrally provided eassessment service to support the government’s eassessment targets?
Now this is just cool. CamSoft is a software product that allows you to control your computer with, well, anything. It uses a basic webcam to identify and lock onto your chosen object - examples from the video include empty soft drink bottles, maracas, even fingertips - and relate them to controls and actions within your programme. The precision and sensitivity of control suggested by the video even in its current pre-release state is seriously impressive, particularly in the first person shooter, and frankly, it’s impossible to stop imagining the potential of this. This could have some really exciting applications in educational simulations, could offer some creative accessibility options, and just looks like being thoroughly good fun, and best of all, they’ve promised it will always be free. You can sign up for the beta here.
A recent joint meeting of the JISC CETIS Assessment, Enterprise and Portfolio SIGs drew a wide range of participants to discuss topics of interest to all three SIGs. The morning sessions covered a range of topics that touched on all three domains, while the afternoon was given over to a special session on student retention.
John Winkley of AlphaPlus Consultancy, who has been working with JISC as an expert consultant in the area of assessment, opened the meeting by introducing delegates to a number of funding opportunities in the domain that JISC will be releasing in the next few weeks. These opportunities include at least two and up to four demonstrator projects, funded to build on and further develop outputs from earlier JISC toolkit activities, and two Invitations to Tender for desktop research studies. These studies will look at advanced eassessment techniques, and at quality concerns around eassessment. The demonstrator projects must be led by a HEFCE-funded institution, while the ITTs will be open to all bidders including Scottish insitutions, FE colleges with less than 400 HE students, and the private sector. All work is due to be completed by March 2009, and will add considerably to JISC’s portfolio of work in this area.
One project which has benefited from JISC funding for part of its lifetime is the WebPA project based at Loughborough University. Nic Wilkinson presented the successful peer assessment system to delegates, illustrating some of the reasons for its success at the recent IMS Learning Impact Awards in Austin, Texas. One of the most signficant factors in the system’s ongoing success is the effort the project team have put into attracting and supporting a signficiant number of participating organisations that have now integrated the system into their own teaching practice. It was also extremely interesting to learn how positively the students themselves have responded to the system, and their attitudes towards the anonymity of peer marks: the system awards each member of a group an aggregated mark derived from the individual scores awarded by their peers, and students are reported to not want to receive individual marks in order to avoid potential clashes outside the classroom.
After the break, Karim Derrick of TAGLearning discussed a proposed British Standard for managing the transmission of coursemarks and portfolios of digital evidence of coursework between schools and awarding bodies. Based on TAG’s extensive experience in this area, the proposed standard includes ‘an XML schema for describing the relationship between components, options and exam specifications’ and a ‘universal translator’ API to support data exchange between the various systems used by exam centres and awarding bodies. Although the current focus for this work is firmly on the schools sector, if adopted it’s not hard to see how it could be extended to support the universities admissions process and external marking at all levels, particularly in vocational courses where a single accrediting body has to deal with substantial amounts of data.
Alan Paull of APS Ltd closed the morning with a lively journey round the admissions domain landscape and the DELIA project. DELIA enables the sharing of enhanced learner information as part of the admissions process, enabling admissions officers to make more informed decisions when evaluating borderline applications. This not only improves the quality of the admissions process, enabling a closer matching between applicants and course requirements, but can have a positive impact on subsequent retetention of such students.
The afternoon featured a special session on student retention, looking at a range of issues around the topic and attempting to capture requirements for work in the domain. Simon Grant of JISC CETIS and the Centre for Recording Achievement led an interactive session that asked participants to consider self-assessment of suitability for courses and the different personas we adopt as our contexts change. Simon also touched on some of the problems that arise when our different personas come into conflict, a situation which can be exacerbated by the widespread use of social networking services and individuals’ lack of awareness of the potential implications of forgoing privacy when using them.
Helen Richardson, also of JISC CETIS and the Centre for Recording Achievement, closed the day by discussing some of the findings of the STAR project and the National Audit Office’s report on student retention. The STAR project produced a detailed series of guidelines to help support students both before and during their university careers, including the use of technologies such as SMS messaging to aid this.
We’re grateful to all our presenters for sharing their work with us and for being so willing to respond to questions and comments from the audience, and to all those who attended on the day and helped to make it a success.