While archiving the documents used to plan and implement Nouncer, I came across the original document written a little over two years ago (with some minor more recent revisions). It details my service plan, ideas for patents and trademarks, the original name for the service ‘JabAbout’, and a long list of features. It also includes the initial thoughts on the architecture and technical challenges. If you love microblogging or am working on a startup, this is a good read. One funny thing to note, it was written around the time Twitter was launched and before I knew about it – hence Twitter isn’t even mentioned as an example.
The document one line explanation of the service is: “Creating an internet service to enable companies and individuals to send messages in real-time to large audiences of interested subscribers via Instant Messaging and other technologies, and providing subscribers with tools to find and filter content to fit their needs and ability to handle the content load.”
You download the document here.
There used to be a big difference between API access and regular human-oriented HTML access: the speed in which requests are made. When a request is made via a browser, there is inherit delay from human interaction, browser response, page rendering, and fetching of images. Most of this is gone once a machine makes the call. However, with recent improvement in browser technology and the wide use of AJAX techniques on the client side, even the human-readable pages can make API calls to render pages.
Scalability plays a central role when designing the ways in which data can be requested from a service, be it via an API call or HTML page request. Both types fetch raw data, process it, and then format it into a presentation format such as HTML, XML, JSON, etc. In the case of a server-rendered HTML page, all the different requests are made internally, hidden from the user, and a single page is returned. If the page uses AJAX scripts, the browser makes multiple API calls to fetch individual data sets, but the server still has to fetch the raw data, process it, and format it. It is the size of the batch that makes the difference.
When discussing microblogging scalability, the conversation includes scaling each individual service, but also scaling the network and relationship between services. Part I discussed the challenges of scaling a single microblogging site with focus on dealing with a large and constantly changing content database. In that post I mentioned that the proposal by some critics to build a distributed or federated microblogging service as a scaling solution will actually make thing worse. This second part will elaborate on that claim.
When discussing a distributed microblogging service, the conversation touches the long debate on the future of social networks and linking communities across individual walled gardens. After all, microblogging is one aspect of the social web, and status updates lives side by side sharing photos, videos, and other personal information and experiences. Being able to choose a social network and make friends from another without having to sign up for multiple accounts is one of the visions being offered. Another is the approach being advocated by the Data Portability group, which focuses on being able to move an entire experience off to another network instead, creating multiple identities.
When it comes to Twitter, everyone’s a critic.
The irony is, the majority of the technical criticism written about Twitter reveals more about the lack of understanding of the author than anything about Twitter. Creating Nouncer – a developer platform for building microblogs and similar services – provides a firsthand understanding of the inner-working and challenges of microblogging sites. This post is not specifically about Twitter but as the leading microblogging service, it is the best working example of the challenges of scaling in the space.
People don’t seem to get what is so hard about scaling the Twitter service. Some think it has to do with Rails, while others look at the hosting company or other operating factors. Then there are those who incorrectly think a distributed or federated service is the solution (which will only make things worse). The idea that building a large scale web application is trivial or a solved problem is simply ridiculous. In a way it is surprising that there are so few companies out there dealing with commoditizing web developing scaling.
Building a business out of a microblogging service is not a trivial task. There has been a lot of chatter regarding Twitter’s eventual monetization, but so far the only people who made money off microblogging are the Jaiku founders when they sold the business to Google. I have written before about the difficulty of monetizing microblogging, but difficulty is something to overcome, not a brick wall. This model is an attempt to figure out the different options of running such a business.
You can download the Excel file or try the Google Docs version (if you want to play with it make a copy first). If you know your way around financial models, jump right into the model. Continue reading for some background and points of interest.
Being critical of Twitter is really a compliment. It comes with the territory of being the market leader in a new space. My recent negative rants about the Twitter are really about how it is being used than its qualities. Like many others, I have a vested interest to see Twitter succeed. While Nouncer does not compete with Twitter, it builds upon the usefulness and experience of microblogs users. Most of my points about Twitter apply equally to other microblogs like Jaiku, Pownce and others. And there are many others.
The best features Twitter has to offer are their powerful platform and open API. It is also the reason they are more successful and why others are coming out with their own API almost as fast as their website. I am excited about the soon to be released Pownce API and have been playing around with the Jaiku API. These three sites and the many who try to improve the space (using their lower load as an advantage to build new functionality), all serve an important purpose of getting microblogging into the mainstream. We are still in the imagination phase, trying to figure out what to do with this powerful tool we stumble upon.