Archive for January, 2008
Registrations are now open for our next Assessment SIG meeting, and you’re warmly invited to book your place for this event hosted by the University of Cambridge. It’s a joint meeting with the CETIS Educational Content SIG, something we’ve been planning to do for some time, looking in particular at two standards of interest to assessment: IMS Common Cartridge and IMS Tools Interoperability.
Common Cartridge hasn’t even been released yet, but has already generated significant interest amongst content vendors and publishers and has been heavily promoted by IMS. It combines profiles of a number of different standards, including IMS Content Packaging v1.1.4, IMS Question and Test Interoperability v1.2.1, IMS Tools Interoperability Guidelines v1.0, IEEE LOM v1.0 and SCORM v1.2 and 2004. The resultant learning object package or ‘cartridge’ is intended to be fully interoperable across any compliant system allowing content to be delivered to any authorised individual.
The appeal of Common Cartridge coupled with authentication and digital rights management systems to content publishers is clear, and the specification is particularly suited to the American educational system where there is a closer relationship between content vendor and courses than in UK Higher Education; in the UK, its primary impact may be in the schools and Further Education sectors where there is more of a history of buying content from publishers than HE. The inclination of many UK HE lecturers to produce their own content and the bespoke nature of many higher level courses are issues we’ve already encountered when looking at topics such as open content and item banking, but there is some interest within UK education, in particular from the Open University. As a major content producer whose resources are used far beyond their own courses, Common Cartridge has clear potential, and Ross McKenzie and Sarah Wood of OU OpenLearn will offer an insight into their experiences of implementing the specification and developing cartridges. There has been very little work to date on the delivery of assessment material through Common Cartridge, a topic which will be addressed by Niall Barr of NB Software. Our own Wilbert Kraan and Adam Cooper will update delegates on the current position of Common Cartridge.
IMS Tools Interoperability has received rather less fanfare, but is a valuable specification which takes a web services approach to seamlessly integrating different tools. It allows specialist tools to be ‘plugged in’ to a learning management system, such as integrating a sophisticated assessment management system with a VLE which only provides limited native support for assessment, or discipline-specific tools such as simulators. It also supports accessibility requirements through the (optional) incorporation of the user’s IMS Accessibility Learner Information Package profile to allow silent interface configuration. Warwick Bailey of Icodeon will be discussing his experiences with the specification.
In the morning, Steve Lay of CARET, University of Cambridge, will be providing an update on the current state of IMS QTI v2.1. Steve is co-chair of the IMS QTI working group (with Pierre Gorissen of SURF and Fontys University).
The afternoon will feature a presentation by Linn van der Zanden of the Scottish Qualifications Authority on the use of wikis and blogs in education and assessment, picking up on an increasing interest in the use and potential of Web 2.0 technologies in this domain.
The meeting is colocated with a workshop by the three JISC Capital Programme projects focusing on assessment to which you are also invited…
The three JISC Capital Programme projects working on assessment will be hosting a dissemination workshop at the University of Cambridge the day before the joint CETIS Assessment and Educational Content SIGs meeting at the same venue. The workshop will feature demonstrations of the tools, discussion on future directions for the programme and explore ways of building an open source development community to support it.
In the morning, participants will have a chance to see the tools demonstrated and the role of web services in delivering an end-to-end assessment process. The afternoon session will split into two tracks. Track I, Building an Open Source Community to support QTI-based tools, will have a technical focus, incorporating discussion on implementation issues and introducing participants to the projects’ open source development support. Track II, Innovation and Interoperability in Assessment, will look at some of the issues around assessment and evaluation software within the community together with more innovative and imaginative uses of QTI.
The projects on display will be of considerable interest. Offering the first implementations of IMS Question and Test Interoperability v2.1 freely available to the community, they provide functionality to support assessment from authoring to delivery. AQuRate, based at Kingston University, supports item authoring, with one particularly notable feature being its attractive and friendly user interface. Minibix, based at the University of Cambridge, provides item banking functionality suitable for both high stakes private item banks for summative assessment and low stakes item banks for resource sharing and formative assessment. The trio is completed by AsDel, based at the University of Southampton, which provides a range of small web-based tools for test delivery, test validation, test management and basic test construction.
As with the SIG meeting, registration is free and open to all.
Many thanks to Pierre Gorissen for forwarding this link discussing the United States Patents and Trademarks Office’s final rejection of Test.com’s controvertial patent, which claimed intellectual property rights over many aspects of online assessment. The breadth of the claimed patent is clear from the original patent text.
The third Economist debate launched yesterday, debating the proposition that ’social networking technologies will bring large [positive] changes to educational methods, in and out of the classroom’. Opening arguments from Ewan McIntosh (Learning and Teaching Scotland) speaking in favour of the motion, and Michael Bugeja (Iowa State University of Science and Technology) in opposition have already been posted, as have an impressively large number of comments from the virtual floor. As Bugeja wryly observes, his chances of winning an online debate (held under a version of the Oxford Union rules that The Economist rather quaintly refers to as Oxford 2.0) on this topic are slim, and voting so far is as one-sided as might be expected.
Rebuttals will be posted on Friday 18th, followed by closing arguments on Wednesday 23rd; the debate itself closes with the final count of votes on Friday 25th. There’s still plenty of time to get involved, but are the books already closed on the outcome?
Update: owing to a gloriously ironic technical fault with the website, the dates above have all been moved forward by a day. As moderator Robert Cottrell observes, ‘you might say that this hiccup has lent support to Dr Bugeja’s argument that applied technology is dangerously fallible.’ Could Web 2.0 be it’s own worst enemy?
The third Future of the Internet survey sponsored by Pew Internet will be available online for the next few weeks and is well worth participating in. Participants are encouraged to express their views on topics such as digital inclusion, DRM, privacy and digital identity, and virtual and mirror worlds in the year 2020, and can remain anonymous or identify themselves as they wish. It’s a stimulating and thought-provoking exercise, as well as an opportunity to contribute to a significant study on perceptions of our online futures.
You can also check out the results from previous surveys and a range of other internet-related resources; particularly fun are the predictions from the early 90s. One that stood out for me was Eric Hughes’s 1992 comment, ‘In the world of the future, people will use low-cost Radio Shack equipment to spy on themselves to find out who they are’: in the world of FOAF and Facebook fakes, we need to spy on ourselves to find out who we’ve been constructed as. No comments about tinfoil hats, thank you very much.
Gales are howling, trains in chaos, so it must be January and time to look ahead to what 2008 has in store…
The final release of QTI v2.1 should be out this spring, and it’ll be interesting to see what uptake is like. This will be the most stable and mature version of the specification to date, supported by a long public draft stage and a number of implementations. Angel Learning are a significant commercial early adopter, and other vendors are bound to be looking at their experiences and whether Angel’s embracing of the specification has an impact on their own customer demand for QTI 2.1.
Other significant implementors of 2.1 are the JISC Capital Programme projects which will be concluding around March. AQuRate offers an item authoring tool, Minibix provides support for a range of item banking functions while ASDEL is an assessment delivery engine which supports both standalone use and integration with a VLE. These projects should deliver quality resources to the community which will provide a firm foundation for use of the specification. There was a sneak preview of these projects at our last SIG meeting.
Talking of SIG meetings, dates for the next two meetings can now be confirmed.
On 19 February there will be a joint meeting with the CETIS Educational Content SIG in Cambridge. This meeting will cover a range of shared concerns such as new content related specifications such as Common Cartridge and Tools Interoperability, and innovative approaches to educational material and assessment. Information about this meeting and online registration will be available very soon. This will be preceded by a workshop hosted by the Capital Programme projects discussed above.
The focus shifts from assessment as content to assessment as process with another joint meeting on 1 May in Glasgow. This meeting will be a joint meeting with the CETIS Portfolio and Enterprise SIGs and will offer an opportunity to explore some of the shared issues in these domains. Again, information on the event will be available on the mailing lists, on this blog and on the website in due course.
Another event of note is the annual International Computer Assisted Assessment Conference on 8 and 9 July at Loughborough. The call for papers is already out, with submissions due by 29 February. As always, this should be a lively and important event in the CAA calendar. Alt-C 2008, Rethinking the Digital Divide, will be held in Leeds on 9 - 11 September; again, the closing date for submissions is 29 February. There’s also a regularly updated list of non-CETIS assessment related events on the wiki.
And what about the trends for eassessment in 2008? The results of Sheila’s poll, with a strong emphasis on Web 2.0 technologies and possibilities, do seem to reflect to some extent the comments on the last meeting’s evaluation forms which suggested increasing interest in innovative technologies, signficant concern with transforming and enhancing the assessment experience and direct engagement with teaching and learning rather than the more abstract issues of standards and specifications for their own sake. It will be interesting to see how the more ‘traditional’ XML-based QTI v2.1 fares in the light of the increasing popularity of mashups and web services in 2008.