Articles / Kenosis and the World Free Web

Kenosis and the World Free Web

Four years ago, I wrote an article for freshmeat called "The World Free Web" in which I described a way to make Web content available in a distributed and anonymous way via Freenet. Back then, I expected, as did many others, that Freenet was on the verge of completion, and all that remained was to think of interesting new applications to write on this new platform.

Now, for the record, I still have high hopes for Freenet and am still a contributor to the Freenet Foundation. But as it stands, Freenet simply does not work, and it is not a suitable platform for the development of new applications.

Two years ago, Malcolm Handley and I started the Dasein Software Partnership in order to create new peer-to-peer tools and applications for the Free Software world. We started writing applications for Freenet, but grew frustrated with Freenet's lack of stability. Next, we switched to The Circle, a distributed hashtable based on Chord. Despite its maturity, it too is not stable or reliable enough to form a suitable platform.

So we decided that we would need to create a new system, designed from the ground up for simplicity, stability, and scalability. We call that system Kenosis.

Kenosis is a fully-distributed peer-to-peer RPC system built on top of XMLRPC. Nodes are automatically connected to each other via a Kademlia-style network and can route RPC requests efficiently to any online node. Kenosis does not rely on a central server; any Kenosis node can effectively join the network ("bootstrap") from any connected node.

Some background on Kademlia

Kademlia is an excellent distributed hashtable algorithm developed at NYU. It provides a mechanism for nodes in a network to automatically organize themselves into a "circle" based on their node ID. This is a strategy followed by many distributed hashtable systems. Nodes try to keep in close contact with other nodes that are "close" to them. Then, to route to any node N in the network, you just need to find a node that is close enough to N that it keeps in close contact with N. The key to the success of this kind of scheme is the particular way nodes figure out their distance from each other. This scheme is called a "distance metric."

Unlike algorithms like Chord, Kademlia's distance metric, which is used to identify nodes that are close to each other in the circle, is symmetric. This means that every time a node A is contacted by another node B, A can correctly place B in its routing table.

Back to Kenosis

Kenosis is also "zero-defect software". Every line of Kenosis has been subjected to extensive unit testing, simulation testing, and scalability testing. We've tried to adopt as many XP practices as we could: unit testing, pair programming, and aggressive refactoring, just to name a few.

We believe strongly in the need for peer-to-peer software that Just Works. Too many other projects have demonstrated early success, only to flounder indefinitely as bugs and increasing complexity make it impossible to use the software reliably. Kenosis is designed from the ground up for simplicity, stability, and scalability. It does one thing and does it well. It is our hope that others will use Kenosis as a primitive to create new and exciting peer-to-peer applications.

To demonstrate Kenosis's suitability for these new applications, we have used it to improve upon another peer-to-peer filesharing application that Just Works: BitTorrent. BitTorrent does one thing incredibly well. Using a centralized "tracker", BitTorrent manages efficient distribution of data that is in high demand. We have extended BitTorrent, using Kenosis, to eliminate this dependence on a centralized tracker.

Our modified BitTorrent software is 100% backwards-compatible with existing BitTorrent clients via a DNS-based Kenosis bridge. People currently running BitTorrent trackers can safely upgrade to a Kenosis-enabled tracker, taking advantage of the built-in failover that Kenosis provides, without fear of breaking compatibility with any existing clients. Since the Kenosis-DNS bridge represents a potential central point of failure, those running BitTorrent clients may wish to upgrade to a Kenosis-enabled client. A complete fork of the latest BitTorrent codebase that includes a Kenosis-enabled tracker and client is available as part of the Kenosis distribution.

How does Kenosis-enabled BitTorrent work?

There are two cases to consider: legacy BitTorrent clients and Kenosis-enabled BitTorrent clients. In either case, imagine we have two Kenosis-enabled BitTorrent trackers being run on machines TA and TB (both on port 1234). Machines TA and TB automatically organize themselves into a Kademlia-style network via Kenosis. Let's further imagine a file being shared by the operator of TA, called FA. FA's .torrent file is constructed to talk to this tracker URL: http://foo.bt.kenosisp2p.org:1234/announce.

For Kenosis-enabled BitTorrent clients:

  1. The client is asked to download the .torrent file for FA.
  2. The client discovers the embedded URL http://foo.bt.kenosisp2p.org:1234/announce. Based on this URL, the client calculates a node address (NA) in the Kenosis network based on the sha1 hash of the key "foo".
  3. The client does a Kenosis lookup for NA. Assume that FA is the closest node in the Kenosis network for node address NA. The client connects to the tracker on TA (port 1234) and downloads FA as normal.

For legacy BitTorrent clients:

  1. The client is asked to download the .torrent file for FA.
  2. The client does a DNS lookup on the URL http://foo.bt.kenosisp2p.org:1234/announce. The Kenosis-DNS bridge returns the IP address of FA.
  3. The client proceeds to download FA as normal.

So far, this is fairly straightforward. The situation gets more interesting when we consider what happens if the machine TA is down at the time that the client wants to download FA. In this case, both the legacy and Kenosis-enabled clients automatically switch to using the tracker for the nearest Kenosis node that is not down. In our hypothetical example above, this would be TB. Clients in the middle of downloading from TA would simply have to retry (prompting a new node lookup) in order to discover the new tracker node to use. Because Kenosis only returns nodes that are currently reachable in response to a node lookup request, and because Kenosis node lookups are stable, all clients will use the same tracker IP address for the same Kenosis node address.

Kenosis also supports an alternate URL syntax for tracker operators who would like to have greater control over which tracker is used for requests for their files. In this case, the operator of TA in our example above could specify a tracker URL like this:

http://NA.node.bt.kenosisp2p.org

Where NA is the hex address of the Kenosis node running on TA. In this case, both legacy and Kenosis-enabled BitTorrent clients will prefer to use TA as the tracker for FA as long as TA is reachable. However, if TA goes down, all clients will immediately failover to the nearest reachable node. When TA comes back up, clients will immediately resume using TA.

Beyond failover: distributed BitTorrent trackers

The current Kenosis-enabled BitTorrent software provides automatic tracker failover. This prevents any single tracker from becoming the central point of failure for any given file. As soon as a tracker goes down or becomes unreachable, clients automatically switch over to another tracker. However, future versions of the Kenosis-enabled BitTorrent could do better. In particular, several attempts have been made (including trackerbt and Exeem) to create distributed trackers. These are BitTorrent trackers that share information about peers so a client can connect to any of these trackers to get metadata related to a given file. Using Kenosis to create a distributed tracker is not a difficult extension of the modifications we've made to BitTorrent. Each tracker simply needs to query Kenosis to find the nearest N nodes in the network that are also running trackers, and share peer information with them. Then, a client may connect to any of these N trackers to download a given file.

Distributed trackers are essential for situations in which a given file is so popular that the swarm of clients trying to download it exceeds the capacity of any individual tracker. However, in many cases, failover is sufficient to ensure that information continues to be highly available.

Future Applications

We hope that Kenosis-enabled BitTorrent is both a useful improvement to BitTorrent and a demonstration of the usefulness of Kenosis. We are also actively developing additional technologies, built on top of Kenosis. Calvin is a distributed-trust primitive based on Kenosis (but portable to other filesharing systems). It allows clients on a p2p filesharing system to come to consensus about the quality of data based on a simple distributed-trust metric. We're also at work on a music sharing system that takes advantage of Calvin to provide high-quality filtered results.

Future Directions

Version 1.0 of Kenosis aims to provide a simple distributed XMLRPC primitive suitable for creating reliable and scalable peer-to-peer applications. It does not address problems of anonymity, privacy, or distributed data retention, although we hope to address these issues in future versions. For example, anonymity could be provided by proxying requests through intermediate nodes, so that neither the originator nor the provider of the results for any request can be determined. Although Kenosis currently works via simple HTTP, future versions may use SSL for link-based encryption and public key encryption for end-to-end verification of the authenticity of requests and node metadata. Using the ideas contained in the original Kademlia paper, Kenosis could also easily be extended to provide distributed hashtable semantics, providing distributed data retention services. More information, including opportunities to get involved with the development of Kenosis, can be found at http://kenosis.sf.net/.

Compatibility

One of our goals with Kenosis was to provide a solid basis for creating a wide range of p2p applications. It is therefore important that Kenosis work on a wide range of platforms. Kenosis is built in 100% pure Python and has been tested on Windows, MacOS X, and Linux. Furthermore, because Kenosis uses XMLRPC for its network communications, it works in almost any networking environment, including restrictive corporate firewalls. It can even work with an HTTP proxy.

Adding a custom service to Kenosis is as simple as writing a few lines of Python. Kenosis handles all the details of routing requests through the network, as well as the conversion of data and methods to and from the XMLRPC standard.

The World Free Web, revisited

Although the World Free Web hasn't materialized in the past four years, I haven't given up on it. Kenosis is a first step towards that vision. Back then, I wrote:

The WFW is a first step towards accomplishing a more intelligent mainstream net architecture which recognizes that information cannot and should not be controlled by an elite few. But this is just an outline, a sketch of what's coming.

Freedom of information is under attack on the Internet today. Four years ago, I didn't know just how many there were in the "elite few" camp that want to limit what information you can access and share. We can have few illusions about it now.

In the coming years, we need to develop the technologies that will allow anyone to publish information without fear of retribution and without needing the backing of a giant corporation to provide hosting and protection. Today, most of us can't serve information from our desktop computers, because they are increasingly walled behind NAT, firewalls, proxies, and ISPs that limit access to the full potential of the Internet. Some of us are lucky enough to have control of servers with a dedicated Internet connection, but these are only suitable for serving content that is not too controversial and not too popular. Controversial content is increasingly being litigated off the net, and popular content is becoming increasingly expensive to host. Consider the number of small sites that go down every week from being featured on Slashdot.

But there is hope. As I pointed out in the WFW article, information that is off the air because of Slashdot is suffering from being too widely distributed. A good distributed data sharing and replication mechanism can solve that problem. When that mechanism is combined with anonymity, it will be possible for anyone on the planet to distribute information as widely as its audience merits. The last step is to ensure that everyone with a Net connection can participate in that audience.

Kenosis is a step in this direction. At Dasein, we've taken it about as far as two people working alone can. Next, we need to figure out how Kenosis scales in a real world environment. There is no substitute for this crucial step, and for that, we need your help. Please join us as a beta tester. If you already run a BitTorrent tracker, Kenosis-BitTorrent can provide automatic backup services for your tracker. If you've always wanted to distribute content using BitTorrent but don't have your own tracker, consider using Kenosis-BitTorrent to make your .torrent file. Your clients will automatically use the appropriate Kenosis-BitTorrent tracker; you don't even need to know which one in advance. If you're a programmer, consider helping us hack on Kenosis or embedding it into your own applications. And for everyone else, consider just running a Kenosis node and letting us know what happens. We've designed Kenosis so that, in its default configuration, it won't take too many resources on your machine. By running a node, you'll be helping the worldwide network of nodes find each other and route requests efficiently. No matter how you decide to get involved, please share your experiences with us. The kenosis-discuss mailing list is a good place to start.

RSS Recent comments

11 Jan 2005 12:02 tlarolle

trusted source
There's still the issue of trusted source. When you d/l a torrent file from SN you know that it's from a trusted and reliable source.

With Kenosis, everyone become a torrent distributor. Who can you trust? My suggestion is to have a trusted encrypter. Someone like SN encrypt the torrent file with their private key and release the public key. Then release it, encrypted torrent file, back to the public. Kenosis can auto d/l the pub key from key trusted source and check all torrent to make sure that it's comming from a trusted source. This rule out SN as being a central distributor.

Not sure if something like this has already being done or thought of. Just an idea I guess.

11 Jan 2005 12:52 dJCL

Re: trusted source
Actually, it has nothing to do with the trusted source. SN did not run a tracker, they just listed the torrent files for download.

With this app, you still have to get your hands on the torrent file, just like at SN. The difference this app makes is that it allows the person running the tracker(usually the one vulnerable to being wiped out by the **AA) can use a distributed system for it, and have some failover too.

There is still the need to find .torrent files, and that is something that needs a killer app. If you could figure out how to distribute SuprNova, you would have a killer app for the net(not just torrents). SuprNova rocked because it was moderated, they verified the files were good and then posted them. Trust was aquired by some and used to post more...

Anyway...

11 Jan 2005 13:23 tlarolle

Re: trusted source

> SuprNova rocked because it

> was moderated, they verified the files

> were good and then posted them.

Right, so what I propose is to have someone like SN do the verified process before releasing them to a distributed network.

11 Jan 2005 14:12 cameron

Re: trusted source

>

> % SuprNova rocked because it

> % was moderated, they verified the files

> % were good and then posted them.

>

> Right, so what I propose is to have

> someone like SN do the verified process

> before releasing them to a distributed

> network.

>

No.

Post first.

Then your moderator pool (consisting of interested people who

will fetch a file for experimental purposes) post detached signatures indicating "I fetched this and it was good".

You need a a way to link signatures to the posted files, but that's

easy because the signature can sign a text files that mentions

to source object.

Then you just decide whom you trust and limit (or better, rank) your searches according to who signed what.

11 Jan 2005 15:25 eries

Re: trusted source
I couldn't agree more. Kenosis doesn't solve the distributed trust problem, but it's our hope that Calvin will. Check out the link the original article. If you're interested in helping out please get in touch and/or join the mailing list.

13 Jan 2005 00:11 juahonen

My 2 cents
I think it's a good strategy to be compatible with the existing applications. Without that advantage it would hardly be surprising if no-one started using your application. At the moment BT is the most popular (according to some) P2P network. But it's not enough just to be compatible with it. There has are already been talks about decentralizing BT, and maybe someone has already taken steps towards that goal. You should be in touch with the BT team, maybe they want to implement your decentralization?

The Circle has most genious source authentication method I've seen on a P2P application. It is based solely on the user community -- If you don't know anyone, you can't distrust any source more than some other one. But if you take part in the community, for example in the chat, you'll get to know good people who you can trust more than the rest. This will create a social network of trust. It also discourages illegal activities because doing such would tarnish your own digital identity in the Circle community.

Extendability is good, as it will allow for creation of legal applications for the peer network. If such applications were impossible, the network is easy to attack, in the legal point of view.

29 Jan 2005 18:05 ashore

A "serious" application
Guys, I'm working around the fringes of the emergency response community, where they're trying hard to be able to respond to disasters of you-name-it varieties.

The bulk of the approaches are http-based, which builds in a pretty obvious vulnerability at the server.

I'm just beginning to get my head around the suitability of p2p to end-run that vulnerability and to provide a number of other advantages, and so far it's looking very, very good.

Authentication is mandatory, as is the ability to handle a variety of media types.

I'll appreciate any thoughts here - mine are still jello - esp re any work already done or written about this in this particular context.

Screenshot

Project Spotlight

Webalizer Xtended

A Web server log analysis program, forked from Webalizer.

Screenshot

Project Spotlight

W3Perl

A server logfile statistic analysis program.