As planned the datasets have been archived. Ed Summers has also backed these up to the Internet Archive. Thank you Ed.

If you’re wondering what to do with your datasets, then you have several options:

• datahub.io is a free community directory of datasets run by the Open Knowledge Foundation. While it doesn’t offer RDF storage or APIs, its a great place to list your dataset so that others can find it. You can link to data dumps, APIs, and provide useful metadata for developers
• If you want cloud hosting for RDF then the closest service to Kasabi is Dydra. Dydra offers RDF storage in the cloud with a simple RESTful API layer. You could use that to host your own data using a commercial solution
• If you want to host the data yourself then we recommend looking at Apache Jena. We’ve been using the TDB triple store to host data in Kasabi and the Talis team contributed to its development. Epimorphics offer commercial suppport for Jena. Of course there are other commercial stores out there, including Stardog, OWLIM, etc.

All the best from the team.

There is a list of download links now available. The spreadsheet lists each of the datasets, their license and a download link for the data. The spreadsheet contains nearly 200 datasets that were publicly available in Kasabi.

The list doesn’t contain any unpublished (private) datasets. It also doesn’t include a few datasets that Talis was hosting, but which are still available elsewhere, e.g. those from data.gov.uk or those that were straight mirrors of other sources. VoiD descriptions of each of the datasets — including their title, etc — is harvestable from data.kasabi.com.

If you have a private dataset that you need to have exported, then please get in touch.

However its time to admit that Kasabi is not getting the traction we thought it would. For many reasons, organisations are not yet ready to fully embrace data marketplaces. We are still too early in the open data revolution for third-party data marketplaces to really have a clear niche. Many of the other startups exploring similar ideas have already begun to pivot and explore different aspects of their vision.

We have therefore made the difficult decision to shutdown Kasabi. We want to give people time to stop using the system and take away their data, so we are giving everyone three weeks notice, from today, that the service will no longer be available. This means the website and APIs will be shutdown by 30th July. We will stop accepting submissions of new data from Wednesday 11th July. We have already disabled new registrations to the site.

To ensure everyone can get their data, we’ll also be exporting the datasets and publishing them as an archive to allow anyone to take a copy. For all currently published datasets we will publish a spreadsheet containing links to allow anyone to download the data. If you have unpublished datasets, then please contact us for a copy.

If you have further questions or any concerns then please contact us:

Following MySociety’s relaunch of FixMyStreet (previous post here), Myfanwy Nixon catches us up on their progress.

FixMyStreet.com relaunched on March 12th. The most immediately obvious change was its smart new look, but there are several changes to the site’s functionality and underlying processes, too – particularly as regards access via mobile devices.

The new version makes the most of one of FixMyStreet’s major strengths – maps. These now expand to fill the entire available browser space, while remaining as navigable, draggable and zoomable as before. The resulting interface capitalises on the inherent beauty of maps, but also enhances usability and accessibility.

This flexibility is a feature of FixMyStreet’s new responsive design. Resize your browser window, and you will see the site elegantly adapt to any dimensions, providing a mobile-specific interface when it is reduced to iPad size and below.

Those accessing the site from mobile devices are guided down a slightly different path than desktop users – additional steps show them a fullscreen map for pinpoint location, and a confirmation step to prevent reports being sent in error – all too easy on a small screen.

Thanks to these features, the mobile web browser version of FixMyStreet should cater for our users’ needs while we now turn our attention to upgrading the standalone app.

Other features of the relaunch include:

• Much higher resolution for uploaded images, and auto-rotation for those that need it
• Report data is pushed to Kasabi’s data store, with access for anyone who wishes to analyse it. As an organisation which promotes open data, we are particularly pleased with this development.
• We’ve implemented Javascript to load in the background, so that the maps on the subsequent page will display more quickly.

You can read more about Talis’ App Fund project here, and if you would like to talk with Talis and learn more about working with data-rich applications, drop us a line.

Everyone knows the legend that is John Peel—and everyone who knows John Peel knows the Peel Sessions.

For 37 years, from 1967 to 2004, a vast range of bands recorded live in the BBC studio – normally laying down 4 tracks; some of them rough and ready, others polished and neat – but all of them resonating with energy and passion.

Peel’s particular specialism was actually his lack of specialism – the music he promoted and featured on the Sessions ranged from reggae to dance to pop to death metal. The bands he featured were often experiencing national coverage for the first time via his show – and – possibly as a consequence of this – often went on to follow a trajectory of enormous fame. The Sessions therefore map out a large and important chunk of music history – a bit like following a band from their first appearance at your local club to their appearance at Wembley five years down the line.

When you go to the Wikipedia page which features the entire list of the Peel Sessions, the first thing you notice is that there is lot of scrolling to do..

The size of this dataset sets the backdrop to the challenge that we’re trying to solve with our Kasabi-driven app, The John Peel Time Machine. We wanted to try and deliver an interface which delivers the richness of the Sessions – and all the potential associated data as well – without overwhelming the user.

Our basic user interface device is a draggable timeline, built using jQuery – but with a series of filters included so that users can either browse or dig in if they know what they’re looking for.

Clicking on a session pops up an overlay box with further information about the session, including an artist biography, track listing, links to Spotify or iTunes where relevant and artist photos:

The data is mostly derived from several different Kasabi datasets: the sessions come from the John Peel Archive (which is itself derived from the awesome John Peel Wiki), while the artist information is pulled from Discogs and the also awesome Dbpedia—a linked-data representation of some of Wikipedia.

One of the things we spend a lot of time doing at Storm Consultancy is understanding and using new technologies which couple cross-browser consistency with deep interactive capability. Once upon a time, this kind of functionality would have been built in Flash. Nowadays, the options provided by jQuery and html5 are becoming rich enough to support this kind of faceted, drag-drop approach – and also do it cross-browser, without the need for any proprietary plugins or downloads.

Some of the particular challenges have included…

• Dealing with such a large dataset. There are 3,662 session, by 1,895 artists and groups, and 10,396 tracks, all with connected meta-data. Wrangling that much data on the server is easy, representing it in a browser is rather more of a challenge.
• The incompleteness of the data, especially from the early days. Many of the bands were not around for long so we don’t know much about them, and record-keeping was often pretty incomplete.

This has been a fascinating project for us to undertake – we’re obsessed with the Sessions and the Kasabi data offering has really given us a unique opportunity to blend user experience design with a rich dataset to provide what we hope will be an interesting way of delving into a piece of music history.

The John Peel Timemachine is an ongoing experiment, and we’ll be adding more data, and more ways to navigate around the sessions over the next few months.

If you would like to get in touch with Talis learn more about working with data-rich applications, drop us a line.

We want to work with developers who share our interests in history, government and culture to devise and build systems using our records as well as datasets from across the web. Or to use a more technical term, build cool stuff.

We hold millions of documents covering almost a millennium of world history: everything from the Domesday Book and Cabinet Papers to the UK Government Web Archive and our Legislation API.

Ha, perfect! It seems to have gone down well, because there are no spaces left on the Eventbrite site, though I’d suggest adding your name to the waitlist in case a space opens up. Tim Hodson will be at Hack on the Record to hack and to appear on a judging panel, so make sure to say hello.

For swotting up on relavent data before hacking, the National Archives are using Kasabi to publish the ARCHON dataset—possibly the coolest name for a set yet. The ARCHON is a directory of archive-holding institutions in the UK, providing simple yet useful locations that hold some sort of records. More importantly—to developers and information architects—the ARCHON provides identifiers useful for linking, or, as their description has it:

It provides archives with a unique repository code which separates them from other services and can be used when undertaking ISAD (G) compliant cataloguing and when contributing to networks.

We are also working on an extract of a subset of the National Register of Archives, with a particular focus on women in the archives. This dataset will link back to ARCHON and will be released by Saturday.

At mySociety, our goal is to build websites that improve access to civic and democratic rights. FixMyStreet.com is a typical example of that: it’s a simple site which encourages people to take action within their own communities.

Got a problem such as a pothole or broken streetlight? Nip onto FixMyStreet, input the postcode or street name, and off it goes to the local council. The whole process takes a couple of minutes, and – hopefully – encourages a sense of ownership within the community.

We launched back in February 2007. We wanted to make the site accessible to all, with a particular aim of reaching people who had never before reported problems to the council. The result was a pared-down site with a single function.

That’s not to say there isn’t complex data behind it. All a user need do is input the postcode of the problem they wish to report. Behind the scenes, complex databases route the report to the correct contact in the relevant local authority, using data such as postcodes, council boundaries, and contact email addresses for range of departments across hundred of councils.

In 2008, we produced FixMyStreet as an iPhone app; volunteers also developed Android and Nokia apps using our open-source code. But the digital world moves quickly, and an update is long overdue – which is where Kasabi come in.

Kasabi’s funding is allowing us to take down the existing apps and replace them with a responsive, mobile web version of FixMyStreet. With the advent of HTML5, we can re-architect the entire HTML front end of FixMyStreet.com: the same page will radically reformat depending on the device which is being used to view the site. The result will be a faster, clearer, more powerful way for everyone to report street problems on the fly.

Things are moving. We’ve collaborated with the design agency Supercool to create an updated look and feel for the new mobile version. Development is also well underway, with the first half of the work completed at the test stage.

FixMyStreet is based, like all mySociety sites, on transparency. Much of its success lies in the fact that reports are published online as well as being sent to the council. It relies on open data, for its maps and for its postcode database.

Thanks to this collaboration, we see an opportunity to open up our own report data more fully, living up to our own standards of transparency and accessibility. We’ll be pushing our reports data live to Kasabi. The goal is that individuals looking to process or analyse the state of fault reports in the UK will have access to a regular repository.

We are very excited – both in anticipation of launching a more responsive app, and at being able to make our report data available via another platform. By doing so, we feel that we’re serving two core mySociety communities: FixMyStreet’s users, and the open data movement.

5* Data

Making data available online is something many different organisations have been exploring, for quite diverse reasons, but one question seems to surface during any discussion about publishing data: How useful will this data be? Developers discuss the schemas and intricacies, project managers ask about the measured benefits, and civil servants try to work out how to meet their remit by opening up public data that they’re looking after. But every topic comes back to the reuse of the information.

One way to model this problem is to use Sir Tim Berners-Lee’s star-rating system for good-quality practices in publishing data. Sir Tim discussed this idea at the Gov 2.0 Expo in 2010, and it’s worth watching the video:

The Talis Consulting blog introduced the star-ratings last spring, and I’ve virtually stolen their summary to give you a quick-glance overview of the 5*’s of publishing data:

1. On the web, open licensed: get your data out there, in any form, for others to use under an open license, such as the Open Government License for Public Sector Information—clear and unambiguous. For many, this is one of the significant steps, because it often includes the convincing of others that this might be a good idea.
2. Machine-readable data: make the data you have just published readable by software. If it was a spreadsheet that you previously published as a nicely formatted pdf—make the Excel file available in addition.
3. Non-proprietary format: publish a csv file as well, then it can be used in software and applications different from those Microsoft ones.
4. Linked Data: start using URIs as identifiers—and publishing in RDF format. This step is another that needs a bit more thought as to how you are going to describe your data
5. Do the Linking: link, or use, identifiers in your data to identifiers published out on the wider web of data. For example, if you are using a UK post code, why not use the Ordnance Survey URI (http://data.ordnancesurvey.co.uk/id/postcodeunit/WR112RE)?

Public-Sector data

The star-rating system seems to apply very readily to the public sector, where there is a responsibility on a body to make its information available online, and accessible. This is an area we’d love to spend some more time exploring, because Kasabi has been built with the intention of making data as usable as possible, and the system is designed to publish 5* data.

A good example of this is Ordnance Survey, which has been a pioneer of publishing its information as Linked Data. Other colleagues of mine worked with the OS on their project (more information here), and there are also some OS sets available in Kasabi. They’ve been highly linkable, and usable as John Goodwin demonstrates with his “linksets,” that match up OS data with other information to provide a reusable hub of data about a specific area:

• Education to OS Postcode Linkset
• Hampshire
• Southampton

Have Data? Drop Us a Line

My Kasabi-colleagues Tim, Alison and Rob have all worked on public-sector data projects, and we are looking for more projects to get our teeth into. We’re looking for more data to work with, and to talk with original source of data to discuss their own efforts in publishing public information. In particular, if you are wondering how to get your data up to Sir Tim’s 5*s, drop Alison an email at alison.kershaw@kasabi.com, or leave a comment below!

Image “2 – Piece Stars (Traditional)” by Origamiancy via flickr, CC by

This year’s challenge asks: ‘How can we transform neighbourhoods in Britain together?’

As with previous GeoVation Challenges we are looking for great ideas that address the identified problems using geography, technology and design. Ordnance Survey will be offering a slice of £115,000 in development funding for best use of our data, including OS OpenData and OS OpenSpace.

The challenge runs to 28 March and the best ideas will be invited to a weekend GeoVation Camp in Southampton from18 – 20 May 2012. The finalists from this camp will be invited to a GeoVation Showcase on 20 June 2012 to pitch for a share of the funding.

It sounds like a solid, wide-ranging challenge with plenty of room for different perspectives on improving neighbourhoods, and I’m looking forward to reading up on the challengers later this spring. Previous winners—full list here—have tackled problems ranging from exploring history and sourcing ethical food, through to planning cycling journeys and fixing the streets.

These projects resonate strongly with Kasabi, because so many applications feed on geographic data. As an example, John Goodwin has used our platform for a range of datasets from Ordnance Survey (interviewed here on the blog), and has created a series of data mash-ups for specific locations.

If you have answers to this year’s question, you really should consider entering the GeoVation challenge, and also coming along to chat with us and the OS in April. Ordnance Survey’s Ian Holt will also be speaking at the conference. If you’d like to share your ideas, or get some help using Kasabi, email me or leave a comment below.

Kasabi is sponsoring the Open Data Cities Conference, which will be in Brighton on 20 April, and tickets cost £149—or £100 if you order before 1 March.

Image: “Compass” by Roland Urbanek via flickr CC: by 2,0

So what to do?

You could go down the code obfuscation route. And while it doesn’t protect you from a determined hacker, it does mean that the casual browser of code (if is there such a person!) will not immediately know he is looking at your key. But really it can’t be a long term solution.

Proxy

One alternative option is to use your web host to proxy all requests to Kasabi. If you are running an apache instance, then here is a config file that should work for you. You will still need to authenticate with your proxy, as you don’t want someone finding it and using it willy nilly.

This example proxy configuration is matching all requests to a certain directory (which doesn’t need to exist on disk) and passing those requests on to the kasabi api servers. The key thing here (pun intended) is to use the RequestHeader directive to set the X-KASABI-APIKEY in the forwarded request. The beauty of this is that the additional header will only be sent in the request and not passed back to the original caller.

NameVirtualHost *
<VirtualHost *>
  ProxyRequests Off
   <Proxy http://127.0.0.1/*>
      Allow from all
   </Proxy>
   <LocationMatch "/kasabi">
      ProxyPass http://api.kasabi.com/
      ProxyPassReverse http://api.kasabi.com/
      RequestHeader set X-KASABI-APIKEY "MySecretAPIKey"
   </LocationMatch>
</VirtualHost>

This approach can also be used (as in our example) to run a proxy on your development machine.

Environment

Another approach (assuming you have some code running on your server) is to have the key in your user environment.
Add the following to your user’s .profile:

KASABI_API_KEY=123YOURKEY123 ; export KASABI_API_KEY

The following code would get the key in PHP:

$kasabi_api_key = getenv('KASABI_API_KEY');

I would be interested in whether these approaches work for you. If you have another approach, then maybe we can showcase that too.