Problem
What is wrong with Twitter's current architecture? How would you fix it? What parts must be scaled out and how?
Alex Payne, one of the Twitter core developers, has finally blogged on this topic and described the real problem:
Twitter is, fundamentally, a messaging system. Twitter was not architected as a messaging system, however. For expediency's sake, Twitter was built with technologies and practices that are more appropriate to a content management system.
Many of the best practices in scalability are inapplicable to the peculiar problem space of social messaging. Many off-the-shelf technologies that seem like intuitive fits do not, on closer inspection, meet our needs.
A good explanation of the problems was published on TechCrunch:
The issue is that group messaging is very difficult to achieve at a grand scale. Other large sites such as Wordpress and Digg are mostly dealing with known problems, such as how to serve a large number of pages or a large number of images. Twitter is unique in that it needs to parse a large number of messages and deliver them to multiple recipients, with each user having unique connections to other users.
Idea
Many ideas have been published on what and how to fix it.
Russel Beattie was talking about a decentralized implementation in the form of Jabber bot in his posts Decentralized Twitter thoughts and has even created a demo Peep - an Open Twitter Server.
Eran Hammer-Lahav who created Nouncer, a developer platform for building microblogs, explains in Scaling a Microblogging Service - Part I his experience on dealing with such architectures.
Implementation
This is a 'discussion' I've caught last night:
Tim Bray: How many geeks, I wonder, are scribbling Solutions To The Twitter Problem on their internal mental whiteboards at this very moment?
Alex Miller: Seriously, though, I really wonder how many twitter 2.0s are *launching* right now
asteris: Too many, but not enough to produce a solution, apparently.
Currently there are a few alternative similar systems, like Pownce or Jaiku, but considering that the feature set and the traffic and the additional environment is not the same, it is difficult to say if they were able to address all the problems Twitter is facing.
Details
The above mentioned alternatives and most of the suggested solutions have been discussed over and over again. Alex Payne is assuring the Twitter users that:
We'd like people to know that we're motivated by the community discussion around our architecture. We're immersed in ideas about improving our system, and we have a clear direction forward that takes into account many of the bright suggestions that have emerged from the community.
If you have ideas you'd like to share you can reach me on Twitter. Or you can try applying your ideas by working on Twitter.
What we see at Twitter today is a very useful and popular service, but one with very complex underlying technical challenges to overcome. Twitter will require not only a new architecture approach and a big injection of the best minds they can find [...], but will also need a little patience from users and those of us observing.
Post a Comment