DESIGN BRIEF
SUMMARY
SPARKD is intended to increase inputs from the social environment into newsrooms or any other nexus that seeks to sort, analyse and synthesise raw information into news. The service is premised on five key elements that we label, as a whole, “Flit”. Each element is designed to be run as a distributed service with a Web frontend:
- Launch – an uploader.
- Hatch – a transcoder & hasher (hashes identify pieces of news information within the ecosystem)
- Wire – a distributed indexing system
- BirdSeed – a seeder, allowing anyone to contribute their bandwidth to the greater good.
- Swift – specifically the developmental Swift player, which allows P2P-powered, web-embedded HTML5 video.
The system is entirely atomic. Anyone can run any part of the infrastructure, anywhere.
HOW IT WORKS
1. LAUNCH
After a video is captured a user can upload it to any Hatch server to be transcoded and indexed. This can be done using a normal browser page, a Firefox add-on or an app if on a mobile device. Metadata such as author, location and tags can be entered or is collected automatically by the device.
Uploading is handled via HTML5 / HTTP upload and can happen from a mobile device, laptop or desktop, within a widget or through a web browser: it’s just an extension, or a small javascript widget you can add to a blog, post to a social network. Anywhere can become a point of upload. Keep costs distributed and lower the chance of outright failure.
Processing uploaded files is onerous : why not let people run their own transcoding services on their own computers/servers? It’s pretty simple: transcode the file, create a hash, and add the hash to an index (as specified by the uploader.)
2. HATCH
After the file has been uploaded, the Hatch server will analyze the video and transcode it into the necessary formats to be ready for web streaming (WebM, Theora and H.264). A hash is created and the file is submitted to one or several indexes and seeders depending on the information supplied by the user and the Hatch server settings.
Once the transcode is completed, the Hatch server pushes the hash back to the Launch server and the end-user who can then add it to his watchlist inside the Firefox add-on or app.
The Wire collects hashes and additional metadata for video files and provides this information to end-users, seeders and any web service that utilizes the content. A list of videos can be subscribed to by any RSS service or from inside the Flit Firefox add-on.
3. BIRDSEED
Any user or organisation who wants to support a specific wire can download and install Birdseed, which will automatically download and seed all videos added to the Wire.
4. SPARKD — BRINGING IT ALL TOGETHER
While any of the Flit services can be run independently most of the time, a single server can provide all of the Launch, Hatch, Wire and Birdseed functionality to enable a video to be uploaded and seeded immediately. Sparkd is a web service that in addition to the bare-bone indexing and seeding service also provides a web front end and community where users can watch and discuss all the videos submitted to Sparkd. Sparkd will run as a stand-alone web service buy any content can easily be embedded on other websites and established news channels could incorporate a Sparkd channel to complement their reporting on a specific and current topic.
FIREFOX ADD-ON
The Flit Firefox add-on will enable users to upload and index content straight from their web browser, either by using a public Hatch service or by transcoding and submitting their videos straight to a Wire. After a video has been submitted they can keep track on the status of the video and see how many people are streaming and sharing it.
Core benefits of this approach:
- Cost of distribution is offloaded onto the audience itself. – No central capital expenditure makes censorship and other such problems less likely.
- Atomised infrastructure allows value to be added “down the chain” in unpredictable ways. SPARKD is designed as a ‘minimum viable product’, facilitating communities in distributing media through their social graph. Both the original publiser and consequent viewers are able to mark up SPARKD videos in such a way as to create an emergent narrative trail between videos, which is readable both from within the SPARKD/flit ecosystem and within social network such as Twitter and Identica utilising the hashtag system. This allows people to identify videos in which they’re interested, but also can pull conversation, for example from the Twittersphere, into the HTML5/video playback environment.
- SPARKD videos are watched wherever they are embedded, and constantly recontextualised with this extrinsic social data, according to display motifs to be explored here. Urgent visual information can be drawn into ongoing conversation, displayed via or within social media clients, or responded to from tablet or phone devices.
USE-CASE
A citizen in Libya captures a sensitive video on her phone which she wishes to share. She selects the level of anonymity and privacy she needs and uses the free flit app for her Android phone to upload the information anonymously to the SPARKD server. The video is transcoded, indexed and made available for streaming.
As events develop in Libya, those monitoring SPARKD, its RSS or social media outputs have seen the ‘Libya’ index appear. They now note a new anonymous video has appeared, which they are immediately able to watch as it is seeded by SPARKD. As they watch, they too become seeders of the video, and as they begin to spread it through their social graphs, so to do their connections.
Some users choose to start seeding the entire Libya index, while others support only this particular video. They can spread the film through their social networks via a shortlink, or embed it as they might a YouTube video.
The video turns out to have political importance and a State security service objects. SPARKD is compromised either by legal attacks or hacking and its entry is removed from the maintained index. However, because knowledge of the hash is all that is needed to extract the video from the P2P swarm, the anonymous video is still able to be accessed. Moreover, the indexing system used by SPARKD means that the index itself has already been replicated and recombined many times, making the attack insufficient to prevent access to the video.
BUSINESS CASE
The problems Sparkd solves: insufficient inputs from the grassroots / location / street-level; insufficient opportunities for users to sort this data before it hits the newsroom’
How Sparkd facilitates telling a story: with information moving so fast, often outpacing the newsroom’s capacity to keep up, it’s important to facilitate increased speed of inputs into the newsroom. Stories should be directly connected to immediate experiences and wherever possible to citizen media. The disconnect between media and citizen should be broken down. The organisation that does this will become full of win.
Who will benefit: citizen journalists will feel increasingly connected to the news gathering process, seeing their work filter into network news. Journalists will benefit by being exposed to the real concerns of local people even if those concerns are not ready for primetime.
Does your product leverage off other tools? Yes, this project is premised on exploring the groundbreaking Swift embedded, P2P streaming video protocol in conjunction with P2P Next, a partnership we’ve already established.
Why does Sparkd make sense to build from a news organization’s perspective? A newsroom needs to connect with the populace. Sparkd gives it means to do that. Being at the center of an infrastructure that improves people’s capacity to communicate with one another audiovisually, without trying to dominate or own that infrastructure, will be of immense benefit to positioning an organisation for the future of news gathering.