6 Months with Node.js

For the past 6 months I’ve been spending most of my time building a new service called Sled. I’ll write more about Sled in the coming weeks, but for now all you need to know is that Sled is a collaborative list making tool for small groups.

Sled is built entirely in JavaScript, both on the back-end and front-end. The back-end is build using Node.js (aka Node), the relatively new server-side JavaScript platform. There is some published experience available about building web applications and services using Node, but the experience available is focused on using Node for real-time applications like instant messaging, chat rooms, and games. It is unusual to hear someone recommends Node for building a new web site if it is not focused on real-time.

I expect this to change very fast.

While Sled has a bit of real-time functionality in the form of live list updates from other participants, the core experience is still closer to a traditional web site. We have two main servers, one serving mostly static pages and scripts for rendering by the browser, and an API server. On the client, we use HTML5, CSS, and lot of JavaScript to talk to the API server and display data.

The next few posts are a random collection of things I’ve learned after 6 intense months of working with Node.

To Node or not to Node

To be honest, I jumped at the opportunity to use JavaScript on the back-end. I’m a C++ guy at heart, coming from a long background of building real-time trading systems on Wall Street. I like C++, and I wish I could still be using it, but last time I tried building a web service in C++, it didn’t work out so well… JavaScript has similar esthetics (no meaningful-white-space for me, thank-you-very-much), it is trivial to read by C++ developers, and is very easy to pick up.

Late 2010, I was introduced to Node by Peter Griess who has been an early participant in the Node effort. At the time, Peter was leading an effort to integrate Node into the Yahoo! Mail back-end environment. Once I figured the overall architecture for Sled, I met with Peter to discuss platform options. I knew Peter was working with Node but was reluctant to use it given its early state.

After an hour of talking, it was clear that I should be giving Node a closer look. I envisioned the API server as a completely statelessness machine with a a simple document-based data store requirements, serving REST calls using as much of HTTP as possible. Node sounded like a perfect fit. For the first time in over a year, I got excited about doing web development on a new platform. Don’t underestimate being excited about your technology.

Ironically, when I asked Peter if I could count on him to help out he told me he just gave notice… Great! But he assured me that the Node community is pretty awesome and that getting help should not be a problem. He was right.

My main reasons for choosing Node 6 months ago were:

  • JavaScript is a natural choice for developers with C/C++ background
  • JavaScript-all-around makes code management and reusability easier
  • Team members can easily move between front-end and back-end
  • Node is the hottest new platform, making it a good fit for a small project looking to attract talent

Given the early stage of the project, I wasn’t too concerned about performance (which proved to be very high) or stability (which hasn’t been a blocking issue, but see below). I was mostly concerned about libraries and shared experience.

My reasons for choosing Node proved to be right. One exception is code reusability between the back-end and front-end which didn’t really proved to be all that useful. We do a little bit of code reusability, mostly for OAuth 2.0 MAC tokens, but even there, the availability of native cryptographic facilities in Node make the reusable code narrower when moving to the browser.

What I didn’t know 6 months ago, but made me stick with Node:

  • JavaScript is an extremely productive language, especially for back-end development
  • The Node community is young which leaves plenty of room to impact core components and library development, and the leads are very open to new ideas and suggestions
  • It’s fun. Super fun.

Is it Ready for Prime Time?

Last year, when I told people I am going to use Node for sled.com, I was greeted with a lot of suspicion, warnings, and skepticism. Six months later I can say with confidence that it was the best decision I’ve made on the engineering side of the project. It got us off to such a great start and that despite the early age of the platform.

As a developer, Node has been amazingly stable. Our server experienced one week of instability in over 6 months due to a race condition bug in the Node HTTP client layer. This was easy to patch and was fixed within a week. I was very surprised at just how stable the API server have been. I’ve started with version 0.2.6 and now running on 0.4.9, but experienced little to no disruptions during the transition from 0.2 to 0.4. The Node core team has done a great job at keeping the project stable, even in this very early stage.

What isn’t so great is library stability.

There are days when I can’t believe some of these libraries were ever executed, and this extends to some of the more popular modules. This is very much a work in progress and it requires a high degree of direct involvement in the library development. If you are going to use a module, you should join its mailing list and keep an eye on what is going on. Being part of the community is absolutely required if you are going to use Node.

The Community to the Rescue

Node has one of the best communities I had the pleasure of working with over the past 20 years. Everyone is accessible, friendly, and eager to help. It doesn’t matter if you are an expert or a novice, you are going to get attention and the help you need. This is especially true for the major module leads. Getting a library patch within hours is a pretty common thing and it helps if you work with them to pinpoint the problem.

All things considered, whatever instability Node introduces, the community counters with plenty to spare.

More Coming

This will be the first in a series of posts about my on-going experience with Node. I hope people new to Node will find it useful. The next post will focus on Node’s non-blocking nature and how it impacts coding style.

Got an interesting Node project going? I’d love to hear about it.

6 thoughts on “6 Months with Node.js

    • I did. It is an OAuth 2.0 provider but it is tightly integrated with the API server and I’m not sure if there is real value in open sourcing it. OAuth 2.0 is pretty simple on its own. The complexity is in how you decide to deploy it.

  1. It is very nice you made a decision and are happy with it. I’am sure node.js is fun and that the community is nice.

    Through, I don’t buy the hotest platform and talent part. For me this is simply not true and maybe even offencive for other communities.

    For other arguments like JS productivity, reutilisability or management in particular for backend developement I’am a bit concerned to what you compare to? It seems you are a C++ developper mainly. This language is not particulary considered as highly productive but more highly efficient.

    How would it compare to python? How would it compare to clojure? How would it compare to scala?

  2. Hey Eran,
    thanks for the article. Coming from a Enterprise Java background I just started in a relatively new Node.js project at a client. One of the first things I was a bit weary about is the number of 0.x libraries that are being developed at a breakneck speed. Who know which ones are going to be around even 3 months from now. That’s a bit different than Enterprise Java, where things have settled down to a few well-known frameworks that are supported by a good number of commercial committers (Hibernate, Spring, etc.). That worries me a bit.
    On the other hand the guys on our team who have been working with Node.js for the past 3 months they are pretty happy with the choice. From my understanding they are also saying that Node.js is pretty stable even though it is only 0.4x. In Enterprise Java I would never have considered a framework that is not at least at version 1.1. I wonder why the Node.js guys are not committing themselves to the current API and calling it a 1.0 version. As far as I have seen the upcoming stuff revolves around stuff like worker threads, which I would consider an add-on to the main API. This doesn’t have to be part of a 1.0 release.
    I would like to see Ryan and the gang move towards a 1.0 release quickly, because that gives companies the confidence to use Node.js in a production environment. Considering that Node.js has been quite reliable from early on – we have a sister website online with a 0.2x version of Node.js – it just gives a signal to corporations that Node.js is stable. On top of that it gives people building libraries around Node.js no more excuse to call their libraries 0.x, a thing that scares enterprises to bet on this new technologies.
    Just my 2 cents worth

    Cheers

    Marc

    • I don’t really see how a 1.0 label helps stability any more than actually making it stable. Also, with the upcoming Windows support in 0.6, things are going to change at least a bit. The key with new libraries is to establish a relationship with the authors. Yes, it is very time consuming, but that’s the nature of this technology. It also depends how much library support you need. We use only a handful for Sled.com.

Comments are closed.