Wilbert’s work blog

I wanted to demo my meshup of a triplised version of CETIS’ PROD database with the impressive Linked Data Research Funding Explorer on the Linked Data meetup yesterday. I couldn’t find a good slot, and make my train home as well, so here’s a broad outline:

The data

The Department for Business Innovation and Skills (BIS) asked Talis if they could use the Linked Data Principles and practice demonstrated in their work with data.gov.uk to produce an application that would visualise some grant data. What popped out was a nice app with visuals by Iconomical, based on a couple of newly available data sets that sit on Talis’ own store for now.

The data concerns research investment in three disciplines, which are illustrated per project, by grant level and number of patents, as they changed over time and plotted on a map.

CETIS have PROD; a database of JISC projects, with a varying amount of information about the technologies they use, the programmes they were part of, and any cross links between them.

The goal

Simple: it just ought to be possible to plot the JISC projects alongside the advanced tech of the Research Funding Explorer. If not, than at least the data in PROD should be augmentable with the data that drives the Research Funding Explorer.

Tools

Anything I could get my hands on, chiefly:

• The D2R toolkit
• OpenLink’s Virtuoso platform and associated kit like their RDF browser
• Talis’ Platform

The recipe

For one, though PROD pushes out Description Of A Project (DOAP, an RDF vocabulary) files per project, it doesn’t quite make all of its contents available as linked data right now. The D2R toolkit was used to map (part of) the contents to known vocabs, and then make the contents of a copy of PROD available through a SPARQL interface. Bang, we’re on the linked data web. That was easy.

Since I don’t have access to the slick visualisation of the Research Funding Explorer, I’d have to settle for augmenting PROD’s data. This is useful for two reasons: 1) PROD has rather, erm, variable institutional names. Synching these with canonical names from a set that will go into data.gov.uk is very handy. 2) PROD doesn’t know much about geography, but Talis’ data set does.

To make this work, I made a SPARQL query that grabs basic project data from PROD, and institutional names and locations from the Talis data set, and visualises the results.

Results

An excerpt of PROD project data, augmented with proper institutional names and geographic positions from Talis’ Research Grant Explorer, visualised in OpenLink RDF browser.

Zooming in on a project, this time to show the attributes of a single project. Still in OpenLink RDF browser.

A project in D2R’s web interface; not shiny, but very useful.

From blagging a copy of the SQL tables from the live PROD database to the screen shots above took about two days. Opening up the live server straight to the web would have cut that time by more than half. If I’d have waited for the Research Grant Explorer data to be published at data.gov.uk, it’d have been a matter of about 45 minutes.

Lessons learned

Opening up any old database as linked data is incredibly easy.

Cross-searching multiple independent linked data stores can be surprisingly difficult. This is why a single SPARQL endpoint across them all, such as the one presented by uberblic’s Georgi Kobilarov yesterday, is interesting. There are many other good ways to tackle the problem too, but whichever approach you use, making your linked data available as simple big graphs per major class of thing (entity) in your dataset helps a lot. I was stymied somewhat by the fact that I wanted to make use of data that either wasn’t published properly yet (Talis’ research grant set), or wasn’t published at all (our own PROD triples).

A bit of judicious SPARQLing can alleviate a lot of inconsistent data problems. This is salient to a recent discussion on twitter around Brian Kelly’s Linked Data challenge. One conclusion was that it was difficult, because the data was ‘bad’. IMHO, this is the web, so data isn’t really bad, just permanently inconsistent and incomplete. If you’re willing to put in some effort when querying, a lot can be rectified. We, however, clearly need to clean up PROD’s data to make it easier on everyone.

SPARQL-panning for gold in multiple datastores (or even feeds or webpages) is way too much fun to seem like work. To me, anyway.

What’s next

What needs to happen is to make all the contents of PROD and related JISC project information available as proper linked data. I can see three stages for this:

1. We clean up the PROD data a little more at source, and load it into the Data Incubator to polish and debate the database to triple mapping. Other meshups would also be much easier at that point.
2. We properly publish PROD as linked data either on a cloud platform such as Talis’, or else directly from our own server via D2R or OpenLink Virtuoso. Simal would be another great possibility for an outright replacement of PROD, if it’s far enough along at that point.
3. JISC publishes the public part of its project information as Linked Data, and PROD just augments (rather than replicates) it.

Ray Henderson, President of Blackboard’s Teaching and Learning division, formerly of Angel learning, made a very public commitment to supporting standards yesterday.

Although even Ray admits that the final proof will be in the software if and when it arrives, the public statement alone is something that I genuinely thought would never happen. From its inception, Blackboard, and most of the rest of the closed source educational technology community, have followed a predictable US technology market path: to be the last competitor standing was the goal, and everyone would betray every stakeholder they had before they’d be betrayed by them. As with other applications of Game theory in the wild, though, there seems to be at least a suggestion that people are willing to cooperate, and break the logic of naked self-interest.

What’s on offer from Ray is, first and foremost, implementation of IMS’ Common Cartridge, followed by other IMS specifications such as Learning Tool Interoperability (LTI) and Learner Information Services (LIS). SCORM and the Schools Interoperability Framework (SIF) also get a mention.

On the CC front, the most interesting aspect by far is a pledge to support not just import of cartridges, but also export. In a letter to customers, Ray explicitly mentions content authored by faculty on the system, which suggests that it wouldn’t just mean re-export of canned content. You’d almost think that this could be the end of the content “Blackhole”

Catches?

The one immediate catch is this:

creators of learning content and tools will of course still need to have formal partnerships (for example in our case participating in the Blackboard Building Blocks program or the Blackboard Content Provider network) with platform providers like us in order to connect their standards-compliant tool or content to eLearning platforms through supported interfaces.

This doesn’t strike me as at all obvious, and the given reasons - to ensure stability and accountability - not entirely convincing. That customers are on their own if they wish to connect a random tool that claims to exercise most of IMS LTI 2.0, I can understand. But I don’t quite understand why a formal relationship is required to upload some content, nor how that would work for content authors who don’t normally enter into such formal relations with vendors. It’s also not easy to see how such a business requirement would be enforced without breaking the standard.

The other potential catch is that Blackboard’s political heft, combined with its platform’s technical heft, means that the standards that it wants to lead on end up with high barriers to entry. That is, interfaces that are easy to add to Blackboard, may not be so easy to add to anything else. And given Blackboard’s market position, it’s their preferences that might well trump others.

Still, the very public commitment is to the open standards, and the promise is that the code will vindicate that commitment. Even a partial return on that promise will make a big difference to interoperability in the classic VLE area.

The announcement of Google’s new Wave technology seems to be causing equal parts excitement and bafflement. For education, it’s worth getting through the bafflement, because the potential is quite exciting.

What is Google Wave?

There’s many aspects, and the combination of features is rather innovative, so a degree of blind-people-describing-an-elephant will probably persist. For me, though, Google Wave exists on two levels: one is as a particular social networking tool, not unlike facebook, twitter etc. The other is as a whole new technology, on the same level as email, instant messaging or the web itself.

As a social networking tool, Wave’s brain, erm, ‘wave’ is that it focusses on the conversation as the most important organising principle. Unlike most existing social software, communication is not between everyone on your friends/buddies/followers list, but between everyone invited to a particular conversation. That sounds like good old email, but unlike email, a wave is a constantly updated, living document. You can invite new people to it, watch them add stuff as they type, and replay the whole conversation from the beginning.

As a new technology, then, Google Wave turns every conversation (or ‘wave’ in Google speak) into a live object on the internet, that you can invite people and other machine services (’robots’) to. The wave need not be textual, you can also collaborate on resources or interact with simple tools (’gadgets’). Between them, gadgets and robots allow developers to bring in all kinds of information and functionality into the conversation.

The fact that waves are live objects on the internet points to the potential depth of the new technology. Where email is all about stored messages, and the web about linked resources, Wave is about collaborative events. As such, it builds on the shift to a ‘realtime stream’ approach to social interaction that is being brought about by twitter in particular.

The really exciting bit about Wave, though, is the promise that - like email and the web, and unlike most social network tools - anyone can play. It doesn’t rely on a single organisation; anyone with access to a server should be able to set up a wave instance, and communicate with other wave instances. The wave interoperability specifications look open, and the code that Google uses will be open sourced too.

Why does Wave matter for teaching and learning?

A lot of educational technology centres around activity and resource management. If you take a social constructivist approach to learning, the activity type that’s most interesting is likely to be group collaboration, and the most interesting resources are those that can be constructed, annotated or modified collaboratively.

A technology like Google Wave has the potential to impact this area significantly, because it is built around the idea of real time document collaboration as the fundamental organising concept. More than that, it allows the participants to determine who is involved with any particular learning activity; it’s not limited to those that have been signed up for a whole course, or even to those who where involved in earlier stages of the collaboration. In that sense, Google Wave strongly resembles pioneering collaborative, participant-run, activity focussed VLEs such as Colloquia (disclosure: my colleagues built Colloquia).

In order to allow learning activities to become independent of a given VLE or web application, and in order to bring new functionality to such web applications, Widgets have become a strong trend in educational technology. Unlike all these educational widget platforms (bar one: wookieserver), however, Wave’s widgets are realtime, multi-user and therefore collaborative (disclosure: my colleagues are building wookieserver).

That also points to the learning design aspect of Wave. Like IMS Learning Design tools (or LD inspired tools such as LAMS), Wave takes the collaborative activity as the central concept. Some concepts, therefore, map straight across: a Unit of Learning is a Wave, an Act a Wavelet, there are resources, services and more. The main thing that Wave seems to be missing natively is the concept of role, though it looks like you can define them specifically for a wave and any gadgets and robots running on them.

In short, with a couple of extensions to integrate learning specific gadgets, and interact with institutional systems, Waves could be a powerful pedagogic tool.

But isn’t Google evil?

Well, like other big corporations, Google has done some less than friendly acts. Particularly in markets where it dominates. Social networking, though, isn’t one of those markets, and therefore, like all companies that need to catch up, it needs to play nice and open.

There might still be some devils in the details, and there’s an awful lot that’s still not clear. But it does seem that Google is treating this as a rising platform/wave that will float all boats. Much as they do with the general web.

Will Wave roll?

I don’t think anyone knows. But the signs look promising: it synthesises a number of things that are happening anyway, particularly the trend towards the realtime stream. As with new technology platforms such as BBSs and the web in the past, we seem to be heading towards the end of a phase of rapid innovation and fragmentation in the social software field. Something like Wave could standardise it, and provide a stable platform for other cool stuff to happen on top of it.

It could well be that Google Wave will not be that catalyst. It certainly seems announced very early in the game, with lots of loose ends, and a user interface that looks fairly unattractive. The concept behind it is also a big conceptual leap that could be too far ahead of its time. But I’m sure something very much like Wave will take hold eventually.

Resources:

Google’s Wave site

Wave developer API guide. This is easily the clearest introduction to Wave’s concepts- short and not especially technical

Very comprehensive article on the ins and outs at Techcrunch

In the twelve years of its existence, an awful lot has been learned about interoperability by IMS staff and members. This is nowhere more apparent than in the most quintessentially educational of interoperability standards: question and test items (QTI). A recent public spat about the IMS QTI specification provides an interesting contrast to two emerging views of how to achieve interoperability. Fortunately for QTI, they’re not incompatible with each other.

Under the old regime, the way interoperability was achieved was by establishing consensus among the largest number of stakeholders possible, create a spec, publish it and wait for the implementations to follow. With the benefit of hindsight, it’s fair to say that the results have been mixed.

Some IMS specs got almost no implementation at all, some galvanised a lot of development but didn’t reach production use, and some were made to work for particular communities by their particular communities. On the whole, many proved remarkably flexible in use, and of sound technical design.

Trouble was, more often than not, two implementations of the same IMS spec were not able to exchange data. To understand why, the QTI spec is illustrative, but not unique.

For a question and test spec to be useful to most communities, and for several of these communities to be able to share data or tools, a reasonably wide range of types needs to be supported. QuestionMark (probably the market leader in the sector) uses the wide range of question types that its product supports as a key differentiator. Likewise, though IMS QTI 2.1 is very expressive, a lot of practitioners in the CETIS Assessment SIG frequently discuss extensions to ensure that the specification meets their needs.

The upshot is that QTI 2.1 is implementable, as a fair old list of tools on wikipedia demonstrates, but implementing all of it isn’t trivial. This could be argued to be one reason why it is not in wider use, though the other reason might well be that QTI 2.1 was never released as a final specification, and now is no longer accessible to non-IMS members.

To see how to get out of this status quo, the economics of standard implementation need to be considered. From a vendor’s point of view - open or closed source - , implementing any interoperability spec represents a cost. The more complex and flexible the specification, the higher that cost is. This is not necessarily a problem, as long as the benefit is commensurate. If either the market is large enough, or else the perceived value of the spec high enough for the intended customers to be willing to pay more, the specification will be economically viable.

Broadly two models of interoperability can be used to figure out a way to make a spec economically viable, and which you go for largely depends on your assumptions about the technical architecture of the solution.

One model assumes that all implementations of a spec like QTI are symmetrical and relatively numerous. Numerous as in certainly more than two or three, and possibly double digits or more, and systems as in VLEs. With that assumption, the QTI situation needs clear adjustment. The VLE market is not that large to begin with, and is fairly commoditised. There is little room for investment, and there has not been a demonstrated willingness to pay for extended interoperable question and test features.

From the symmetrical perspective, then, the only way forward is to simplify the spec down to a level that the market will bear, which is to say, very simple indeed. Since, as we’re already seeing with the QTI 1.2 profile in Common Cartridge, it is not possible to satisfy all communities with the same small set of question and test items, there will almost certainly need to be multiple small profiles.

There are several problems with such an approach. For one, reducing the feature set to the lowest cost has a linear relation to the value of the feature set to the end user. Beyond a minimum it might be almost useless. Balkanising the spec’s space to several incompatible subsets is likely to exacerbate this; not just for end-users, but also tool and content vendors.

What’s worse, though, is that the underlying assumption is wrong. Symmetrical interoperability doesn’t work. To my knowledge, and I’d love to be corrected, there are no significant examples of an interoperability spec that has significant numbers of independent implementations that happily export and import each others’ data. The task of coordinating the crucial details of the interpretation of data is just too onerous once the number of data sources and targets that a piece of software has to deal with gets into the double digits.

Symetrical, many-to-many interoperability; 8 systems, 56 connections that need to work

Within the e-learning world, SCORM 1.2 (and compatible IMS Content Packages) came closest to the symmetric, many-to-many ideal, but only because the spec was very simple, the volume of the market large, compliance often mandated and calculated into Requests For Proposals (RFPs), and vendors were prepared to coordinate their implementations in numerous plugfests and codebashes as a consequence. Also, ADL invested a lot of money in continuous implementation support. Even then, plenty of issues remained, and, crucially, most implementations were not symmetrical: they imported only. Once the complexity of the SCORM increased significantly with the adoption of Simple Sequencing in SCORM 2004, the many-to-many interoperability model broke down.

Instead, the emergence of solutions like Icodeon’s SCORM 2004 plug-in for VLEs brought the spec back to the norm: asymmetrical interoperability. Under this assumption, there will only ever be a handful of importing systems at most, but a limitless number of data sources. It’s how HTML works on the web: uncountable sources that need to target only about four codebases (Internet Explorer, Mozilla, WebKit, Opera), one of which dominates to such an extent that the others need to emulate its behaviour. Same with JPEG picture rendering libraries, BIND implementations and more. In educational technology it is how Simple Sequencing and SCORM 2004 got traction, and it is starting to look as if it will be the way most people will see IMS Common Cartridge too.

Under this assumption, implementing a rich QTI profile in two or three plug-ins or web services becomes economically much more viable. Not only is the amount of required testing much reduced, the effective cost of implementation is spread out over many more systems. VLE vendors can offer the feature for much less, because the total market has effectively paid for just two or three best-of-breed implementations rather than tens of mediocre ones.

Asymetrical, many-to-many interoperability; 8 source systems, 2 consuming systems, 16 connections that need to work

This is not a theoretic example. Existing rich QTI 2.1 implementations make the asymmetric interoperability assumption. In Korea, KERIS (Korea Education and Research Information Service) is coordinating the development of three commercial implementations of the rendering and test side of QTI, but many specialised authoring tools are envisaged. Likewise, in the UK, two full implementations of the rendering and test management side of QTI exist, but many subject specific authoring tools are envisaged. All existing renderers can be used as a web application, and QTIEngine is also explicitly designed to work as a local plug-in or web service that can be embedded in various VLEs.

That also points to various business models that asymmetric interoperability enables. VLE vendors can focus on the social networking core, and leave the activity specific tools to the specialists with the right expertise. Alternatively, vendors can band together and jointly develop or adopt an open source code library, like the Japanese companies that implemented Simple Sequencing under ALIC auspices, back in the day.

Even if people still want to persist with symmetrical interoperability, designing the specification to accommodate both assumptions is not a problem. All that’s required is one rich profile for the many-to-few, asymmetric assumption, and a very small one for the many-to-many, symmetric assumption. Let’s hope we get both.

Resources

A brief overview of the current QTI 2.1 discussion

Wikipedia’s QTI page, which contains a list of implementations

More on the KERIS QTI 2.1 tools

The QTIEngine demo site

An interview with Kyoshi Nakabayashi, formerly of ALIC, about joint Simple Sequencing implementation work

Initially, it’s hard to get past the eye-popping numbers (1876 universities, 17 million students and so on) but once you do, you’ll see that the higher education sector in China is facing remarkably familiar challenges with some interesting solutions.

We were very fortunate here at IEC that Prof Zhu Zhiting and colleagues from Eastern China Normal University and the China e-Learning Technology Standardization Committee agreed to visit our department after attending the JISC CETIS conference yesterday. He kindly agreed to let us publish his slides, which are linked below.

The two most noticeable aspects of prof. Zhu’s presentation are the nature of planning e-education in China, and the breadth of interests in Prof. Zhu’s Distance Education College & e-Educational System Engineering Research Center.

Because the scale of education in China is so vast, any development has to be based on multiple layers of initiatives. The risks involved mean that the national ministery of education needs to plan at very high, strategic levels, that set out parameters for regional and local governments to follow. This is not new per se, but it leads to a thoroughness and predictability in infrastructure that others could learn from.

The department in Shanghai, though, is another matter. Their projects range from international standardisation right down to the development of theories that integrate short term and long term individual memory with group memory. Combined with concrete projects such as the roll-out of a lifelong learning platform for the citizens of Shanghai, that leads to some serious synergies.

Learn more from Prof. Zhu’s slides

More about IEC and what it does.

Control of a technology may seem a vendor business: Microsoft and Windows, IBM and mainframes. But by understanding how technology moves from prototype to ubiquity, the people who foot the bill can get to play too.

Though each successful technological innovation follows its own route to becoming an established part of the infrastructure, there are couple of regularities. Simplified somewhat, the process can be conceived as looking like this:

The interventions (open specifications, custom integration et al) are not mutually exclusive, nor do they necessarily come in a fixed chronological order. Nonetheless, people do often try to establish a specification before the technology is entirely ‘baked’, in order to forestall expensive dead-ends (the GSM mobile phone standard, for example). Likewise, a fully fledged open source alternative can take some time to emerge (e.g. mozilla / firefox in the web browser space).

The interventions themselves can be done by any stakeholder; be they vendors, buyers, sector representatives such as JISC or anyone else. Who benefits from which intervention is a highly complex and contextualised issue, however. Take, for example, the case of VLEs:

When VLEs were heading from innovation projects to the mainstream, stakeholders of all kinds got together to agree open standards in the IMS consortium. No-one controlled the whole space, and no-one wanted to develop an expensive system that didn’t work with third party tools. Predictably, implementations of the agreed standards varied, and didn’t interoperate straight off, which created a niche for tools such as Reload and course genie which allowed people to do some custom integration; a glossing over the peculiarities of different content standard implementations. Good for them, but not optimal for buyers or the big vle vendors.

In the mean time, some VLE vendors smelled an opportunity to control the technology, and get rich off the (license) rent. Plenty of individual buyers as well as buyer consortia took a conscious and informed decision to go along with such a near monopoly. Predictability and stability (i.e. fast commodification) were weighed against the danger of vendor lock-in in the future, and stability won.

When the inevitable vendor lock-in started to bite, another intervention became interesting for smaller tool vendors and individual buyers: reverse engineering. This intervention is clearer in other technologies such Windows file and print server protocols (the Samba project) and the PC hardware platform (the early ‘IBM clones’). Nonetheless, a tool vendor such as HarvestRoad made a business of freeing content that was caught in proprietary formats in closed VLEs. Good for them, not optimal for big platform vendors.

Lastly, when the control of a technology by a platform becomes too oppressive, the obvious answer these days is to construct an open source competitor. Like Linux in operating systems, Firefox in browsers or Apache in webservers, Moodle (and Sakai, atutor, .LRN and a host of others) have achieved a more even balance of interests in the VLE market.

The same broad pattern can be seen in many other technologies, but often with very particular differences. In blogging software, for example, open source packages have been pretty dominant from the start, and one package even went effectively proprietary later on (Movable Type).

Likewise, the shifting interests of stakeholders can be very interesting in technologies that have not been fully commodified yet. Yahoo for example, has not been an especially strong proponent of open specifications and APIs in the web search domain until it found itself the underdog to Google’s dominance. Google clearly doesn’t feel the need to espouse open specifications there, since it owns the search space.

But it doesn’t own mobile phone operating systems or social networking, so it is busy throwing its weight behind open specification initiatives there. Just as they are busy reverse engineering some of Microsoft’s technologies in domains that have already commodified such as office file formats.

From the perspective of technology users and sector representatives, it pays to consider each technology in a particular context before choosing the means to commodify it as advantageously but above all as quickly as possible. In the end, why spend time fighting over technology that is predictable, boring and ubiquitous when you can build brand new cool stuff on top of it?

Rather than the computer science foundation, the annual semantic technology conference in San Jose focusses on the practical applications of the approach. Visitor numbers are growing at a ‘double digit’ clip, and vendors are starting to include big names such as Oracle. We take a look.

It seems that we’re through the trough of disillusionment about the fact that the semantic web as outlined by the Tim Berners Lee in 1999 has not exactly materialised (yet). It’s not news that we do not all have intelligent agents that can seek out all kinds of data on the ‘net, and integrate it to satisfy our specific queries and desires. What we do have is a couple of interesting and productive approaches to the mixing and matching of disparate information that hint at a slope of enlightenment, heading to a plateau of productivity.

Easily the clearest example from the conference is the case of Blue Shield of California, a sizeable health care insurer in the US. They faced the familiar issue of a pile of legacy applications with custom connections, that were required to do things they were never designed to do. As a matter of fact, customer and policy data (core data, clearly) were spread over two systems of different vintage, making a unified view very difficult.

In contrast to what you might expect, the solution they built leaves the data in the existing databases- nothing is replicated in a separate store. Instead, the integration takes place in a ’semantic layer’. In essence, that means that users can ask pretty complex and detailed questions of any information that is held in any system, in terms of a set of models of the business. These queries end up at the same old systems, where they get mapped from semantic web query form into good old SQL database queries.

This kind of approach doesn’t look all that different from the Enterprise Service Bus (ESB) in principle, but takes takes the insulation of service consumers from the details of service providers rather further. Service consumers in a semantic layer have just one API for any kind of data (the W3C’s SPARQL query language) and one datamodel (RDF, though output in XML or JSON is common). Most importantly, the meaning of the data is modelled in a set of ontologies, not in the documentation of the service providers, or the heads of their developers.

While the Blue Shield of California case was done by BBN, other vendors that exhibited in San Jose have similar approaches, often built on open source components. The most eye catching of those components (and also used by BBN) is netkernel: the overachieving offspring of the web and unix. It’s not necessarily semantic tech per se, but more of a language agnostic application environment that competes with J2EE.

Away from the enterprise, and more towards the webby (and educational!) end of things, the state of semantic technology becomes less clear. There are big web apps such as the Twine social network where the triples are working very much in the background, or powerset, where it is much more in your face, but to little apparent effect.

Much less polished, but much, much more interesting is dbpedia.org- an attempt to put all public domain knowledge in a triple store. Yes, that includes wikipedia, but also the US census and much more. DBpedia is accessible via a number of interfaces, including SPARQL. The result is the closest thing yet to a live instance of the semantic web as originally conceived, where it really is possible to ask questions like “give me all football players with number 11 shirts from clubs with stadiums with more than 40000 seats, who were born in a country with more than 10M inhabitants“. Because of the inherent flexibility of a triple store and with the power of the SPARQL interface, dbpedia could end up powering all kinds of other web applications and user interfaces.

Nearer to a real semantic web, though, is Yahoo’s well publicised move to start supporting relevant standards. While the effect isn’t yet so obvious as semantic integration in the enterprise or dbpedia. it could end up being significant, for the simple reason that it focusses on the organisational model. It does that by processing data in various ’semantic web light’ formats that are embedded in webpages in the structuring and presentation of search results. If you’d want to present a nice set of handles on your site’s content in a yahoo search results page -links to maps, contact info, schedules etc- it’s time to start using RDFa or microformats.

Beyond the specifics of semantic standards or technologies of this point in time, though, lies the increasing demands for such tools. The volume and heterogeneity of data is increasing rapidly, not least because means of fishing structured data out of unstructured data are improving. At the same time, the format of structured data (its syntax) is much less of an issue than it once was, as is the means of shifting that data around. What remains is making sense of that data, and that requires semantic technology.

Resources

The semantic conference site gives an overview, but not any presentations, alas.

The California Blue Shield architecture was built with BBN’s Asio tool suite

More about the netkernel resource oriented computing platform can be found on the 1060 website

Twine is still in private beta, but will open up in overhauled form in the autumn.

Powerset is wikipedia with added semantic sauce.

DBpedia is the community effort to gather all public domain knowledge in a triple store. There’s a page that outlines all ways of accessing it over the web.

Yahoo’s SearchMonkey is the project to utilise semweb light specs in search results.

Lots of places and even people have a pile of potentially useful content sitting in a retired CMS or VLE. Or have content that needs to work on a site as much as a pdf or a booklet. Or want to use that great open stuff from the OU, but with a tweak in that paragraph and in the college’s colours, please.

The problem is as old as the hills, of course, and the traditional answer in e-learning land has been to use one of the flavours of IMS Content Packaging. Which works well enough, but only at a level above the actual content itself. That is, it’ll happily zip up webcontent, provide a navigation structure to it and allow the content to be exchanged between one VLE and another. But it won’t say anything about what the webcontent itself looks like. Nor does packaging really help with systems that were never designed to be compliant with IMS Content Packaging (or METS, or MPEG 21 DID, or IETF Atom etc, etc.).

In other sectors and some learning content vendors, another answer has been the use of single source authoring. The big idea behind that one is to separate content from presentation: if every system knows what all parts of a document mean, than the form could be varied at will. Compare the use of styles in MS Word. If you religiously mark everything as either one of three heading levels or one type of text, changing the appearance of even a book length document is a matter of seconds. In single source content systems that can be scaled up to include not just appearance, but complete information types such as brochures, online help, e-learning courses etc.

The problem with the approach is that you need to agree on the meaning of parts. Beyond a simple core of a handful of elements such as ‘paragraph’ and ‘title’, that quickly leads to heaps of elements with no obvious relevance to what you want to do, but still lacking the two or three elements that you really need. What people think are meaningful content parts simply differs per purpose and community. Hence the fact that a single source mark-up language such as the Text Encoding Initiative (TEI) currently has 125 classes with 487 elements.

The spec

The Darwin Information Typing Architecture (DITA) specification comes out of the same tradition and has a similar application area, but with a twist: it uses specialisation. That means that it starts with a very simple core element set, but stops there. If you need to have any more elements, you can define your own specialisations of existing elements. So if the ‘task’ that you associate with a ‘topic’ is of a particular kind, you can define the particularity relative to the existing ‘task’ and incorporate it into your content.

Normally, just adding an element of your own devising is only useful for your own applications. Anyone else’s applications will at best ignore such an element, or, more likely, reject your document. Not so in DITA land. Even if my application has never heard of your specialised ‘task’, it at least knows about the more general ‘task’, and will happily treat your ‘task’ in those more general terms.

Though DITA is an open OASIS specification, it was developed in IBM as a solution for their pretty vast software documentation needs. They’ve also contributed the useful open source toolkit for processing content into and out of DITA (Sourceforge), with comprehensive documentation, of course.

That toolkit demonstrates the immediate advantage of specialisation: it saves an awful lot of time, because you can re-use as much code as possible. This works both in the input and output stage. For example, a number of transforms already exist in the toolkit to take docbook, html or other input, and transform it into DITA. Tweaking those to accept the html from any random content management system is not very difficult, and once that’s done, all the myriad existing output formats immediately become available. What’s more, any future output formats (e.g. for a new Wiki or VLE format) will be immediately useable once someone, somewhere makes a DITA to new format transform available.

Moreover, later changes and tweaks to your own element specialisations don’t necessarily require re-engineering all tools or transforms. Hence that Darwin moniker. You can evolve datamodels, rather than set them in stone and pray they won’t change.

The catch

All of this means that it quickly becomes more attractive to use DITA than make a set of custom transforms from scratch. But DITA isn’t magic, and there are some catches. One is simply that some assembly is required. Whatever legacy content you have lying around, some tweakery is needed in order to get it into DITA, and out again without losing to much of the original structural meaning.

Also, the spec itself was designed for software documentation. Though several people are taking a long, hard look at specialising it for educational applications (ADL, Edutech Wiki and OASIS itself), that’s not proven yet. Longer, non-screenful types of information have been done, but might not offer enough for those with, say, an existing docbook workflow.

The technology for the toolkit is of a robust, if pedestrian variety. All the elements and specialisations are in Document Type Definitions (DTDs) –a decidly retro XML technology– though you can use the hipper XMLSchema or RelaxNG as well. The toolkit itself is also rather dependent on extensive path hackery. High volume. real time content transformation is therefore probably best done with a new tool set.

Those tool issues are independent of the architecture itself, though. The one tool that would be difficult to remove is XSL Transforms, and that is pretty fundamental. Though ‘proper’ semantic web technology might have offered a far more powerful means to manipulate content meaning in theory, the more limited, but directly implementable XSLTs give it a distinct practical edge.

Finally, direct content authoring and editing in DITA XML poses the same problem that all structural content systems suffer from. Authors want to use MS Office, and couldn’t care less about consistent meaningful document structuring, while the Word format is a bit of a pig to transform and it is very difficult to extract a meaningful structure from something that was randomly styled.

Three types of solution exist for this issue: one is to use a dedicated XML editor meant for the non-angle bracket crowd. Something like XMLMind’s editor is pretty impressive and free to boot, but may only work for dedicated content authors simply because it is not MS Word. You can use MS Word with templates either directly with an plug-in, or with some post-processing via OpenOffice (much like ICE does). Those templates makes Word behave differently from normal, though, which authors may not appreciate.

Perhaps it is, therefore, best to go with the web oriented simplicity, and transform orientation of DITA, and use a Wiki. Wiki formats are so simple that mapping a style to a content structure is pretty safe and robust, and not too alien and complex for most users.

Not a new idea, but one that took a few decisive steps this week in a series of meetings in London: the moving of the SCORM educational content format out of the US Department of Defense’s ADL initiative, and into something that reflects the current SCORM user community. The parents will continue to cough up some fees, and provide a home if necessary, but would rather SCORM found its own way now.

The ’something that reflects the current SCORM user community’ is, of course, the crucial bit. A prospectus for such a something (LETSI- Learning, Education & Training Systems Interoperability) was introduced to coalesce thinking about the issue at the meetings. That prospectus tentatively assumed that LETSI would be a new, stand-alone organisation. Another acronym to put on the pile, if you like.

This is not universally popular, for the obvious reason that the education technology interoperability sphere is plenty fragmented as it is. Australia’s DEST (Department for Education, Science and Training) makes that point eloquently. In the London event, offers from existing organisations such as IMS to look after a LETSI like set-up where dismissed by the observation that not one of the existing organisations is truly representative of the community that now uses SCORM.

The other point of controversy is whether the organisation is to do more than just SCORM. As the DEST paper indicates, there is no concrete indication of what such work might look like in the prospectus, and the meetings didn’t clarify that further either.

After the meetings, both issues are still relatively open. What was decided was that a strategic and operational charter is to be drawn up by June, for ratification come September, in a meeting parallel to the ISO JTSC 1 SC36 (the educational technology arm of the most formal standards body) quarterly in Toronto. Contributions of time and/or money are solicited- there’ll be more on the adl community site about that.

It’ll be in that process that the heft of LETSI (or some other, better name) and its scope will be decided. It could be an open annual meeting and forum to tweak SCORM and nothing more, or it could be a full bells-and-whistles, big dollar membership organisation that looks at everything to do with education and interoperability, worldwide, etc, etc, etc.

It seems unlikely that we’ll fully escape another acronym, regrettably, but, on the bright side, you could argue that it is in effect a direct swap out of ADL for not-LETSI. That is, from the point of view of the education community, the total acronym population remains constant, but the ecosystem shifted slightly.