Running NowJS natively on Windows

Node.js v0.6.0 marked the first stable release of a native Windows build of Node.js, so for all the Windows users out there, Node in Cygwin should be a thing of the past!

NowJS on Windows

  1. Install node.exe and NPM for Windows and make sure all your paths are correct
  2. Install Microsoft Visual C++ Runtime (4.8 MB)
  3. Download the NowJS Windows branch zip or `git clone git://github.com/Flotype/now.git` and `git checkout windows`
  4. Rename `Flotype-now-XXXXXX` to `now` and put that in your node_modules folder

Congrats! You should now be able to `require('now')` from your app as long as the node_modules folder is in the path.

What's different in the NowJS Windows branch?

The only difference is in proxy.js, insteading of `require('node-proxy')` we `require('../bin/proxy.node')`

proxy.node is the compiled Windows binary of the node-proxy module and included as part of the Windows branch of NowJS.

Why has it taken so long to get NowJS working on Windows native?

In order to provide the `now` namespace functionality, we use the node-proxy plugin. Unfortunately node-proxy is a C++ V8 extension and thus requires compilation. With the release of Node v0.6.0, the toolchain for native module compilation for Windows was undocumented. This meant that we weren't able to compile the node-proxy dependency for NowJS. Several workarounds were proposed including using native harmony proxies found in current V8 builds (when run with --harmony_proxies flag). Unfortunately the current V8 implementation seems to contain a bug preventing correctly triggering setters when dynamic properties names are given (See http://code.google.com/p/v8/issues/detail?id=1543#c10). Eventually we settled on compiling node-proxy in Visual C++ and distributing that. 

Happy hacking!

Why We Threw out All Our Code (And Why You Should Too)

This past week, we proudly released NowJS v0.7. This version is the most stable and performant version of NowJS to date. It's managed to survive our vicious benchmarking tools and our ridiculously comprehensive test cases. But our greatest accomplishment with this release is that every single line of code is brand new.

In a matter of minutes, we decided to scrap 3 months of work and rewrite our architecture. No file, function or concept was left unquestioned. What inspired this iconoclastic frenzy? Here's why we threw out all our code:

1. Optimizing for real-world usage

When we started working on NowJS, we knew what we wanted the software to do. But there was no way of telling how people would use it and how it would perform in these scenarios. With v0.7, we optimized and made tradeoffs based on the data we collected from real-world usage.  It turns out that the vast majority of usage was for RPC and the occasional syncing of very simple variables. We reduced code complexity by not worrying too much about syncing complex nested/circular structures, and optimized the remote procedure calls process to be as close to constant time as possible.

2. Perfect opportunity to future-proof

Over time, we became much better at anticipating the type of features we were going to add to the library. This meant that we weren't shoehorning functionality onto a codebase that was never designed for it in the first place.  No longer were we wrestling with absurdly complex data structures and the equivalently insane functions needed to manipulate them. We eliminated the need for cryptic utility functions like "mapAndMergeFunctionsInScopesHelper". After the complete rewrite, features that we indefinitely put off took mere minutes to implement. 

3. Sunk costs and egos get in the way of good software

Our job as developers is to provide the best library so that you can write high-performance realtime web applications easily. We could have defended our code just because we worked hard on it for months. But accepting the large upfront cost of a rewrite meant that we would save on development time in the future and also provide a better product to our users.

So take a good look at your own codebase. Does it frustrate you when you try to add new features? Are you smarter now than when you first wrote it? If so, then perhaps it's time to dump it.

P.S. Check out NowJS v0.7 . It's rock solid and super-optimized for the best real-time RPC experience.

 

NowJS and Security

Security is a large issue when dealing with realtime applications, especially when there is bidirectional server-client communication and even more so with client-server-client interaction. It becomes an even larger concern in a project such as NowJS that synchronizes state between the server and clients.

A common misconception is that NowJS actually sends functions over the wire as part of the scope-synchronization process. This does not happen. Here's a high-level overview of how we handle remote procedure calling (RPC): when the client sends his scope to the server (and vice versa), instead of actually transmitting the body of any functions, he sends a serialized form of the function indicating only where the function is located within his now namespace. On the server, this serialized form is translated into a wrapper function whose sole function is to ask the client to execute the original function with the arguments passed in to this wrapper function.

What this means for the developer is that functions are only executed where they were created. Any given user can't do anything to other users that the server hasn't explicitly specified he can do. Furthermore, since the bodies of functions are not actually transmitted over the wire, the clients never see the actual serverside code that's running -- we provide, as one particular user in the IRC channel put it, "javascript IP protection".

The next concern individuals have regarding the shared namespace problem is that a malicious user might attempt to alter other users' now namespaces. The only way a user can do this is if a server-side function is provided to allow him to do so -- the server actually keeps track of all users' individual now namespaces, and as with Dropbox, a user's changes to his own now namespace will trigger an update in the server-side copy, and vice versa. For developers who want absolute control, the client's power can be restricted further, as changes to the client-side now namespace will be ignored if the clientWrite flag is set as false in the configuration options passed when initializing NowJS.

NowJS 0.7 supports passing in an HTTPS server upon initialization. No additional configuration is required.

Note that NowJS does not deal with preventing cross-site scripting (XSS) attacks; that is beyond the scope of this framework. If you're really worried about XSS, just add a thin layer of protection in your client code with

msg = $('<div/>').text(msg).html()

before inserting any arbitrary text into the document as HTML.

As always, let us know what you think in the comments below, and in our IRC channel #nowjs on irc.freenode.net

NowJS and Reconnects

This particular feature has been highly anticipated by the NowJS community for some time. With the switch to Socket.IO 0.6.18, which now has built-in support for socket reconnecting, implementation of this feature was actually quite doable. We've tested with both simulated server downtime and client network issues, and it's worked satisfactorily. Several minor changes were necessary to support this feature in Now -- nothing that actually affects the API for developers using this tool; just a few changes to how things work in the backend.

Several disclaimers to make about Now and the general state of reconnects: first of all, it's not possible to preserve the client's session ID without forking Socket.IO. I have it from one of the Socket.IO folks that 0.7 will make everything public, including generateSessionId(), so when we bridge over to that version (we'll have to make sure that nothing breaks -- we've been developing a test suite, so that should hopefully ease the transition), this may change.

Additionally, client groups are not preserved across reconnects. One way to handle this would be to have the client know which groups he's in, and then send this information when the socket's reconnect event is triggered.

Note that client scopes, however, are preserved, so in the multiroom chat example, for instance, a reconnecting user originally in room 3 will be treated as if he's in room 1, but he'll still have the same name.

And now, the code.

The first block of code, pulled from handleDisconnection(), is as follows:

// y-combinator trick
(function (y) { 
  y(now, y, [now]); 
})(function (obj, fn, seen) { 
  for (var i in obj) { 
    if (obj[i] && seen.indexOf(obj[i]) === -1 && 
        typeof obj[i] === 'object' && obj[i] != document) { 
      seen[seen.length] = obj[i]; 
      fn(obj[i], fn, seen); 
    } 
    else if (typeof obj[i] === 'function' && obj[i].remote) { 
      delete obj[i]; 
    } 
  } 
});

I've omitted the tiny portion from constructRemoteFunction() where I tagged all remote functions as such. Essentially, this snippet runs through the now object and deletes all remote functions -- if this is not done, then upon reconnecting, the user will find himself running into infinite recursion -- when re-establishing the connection, since the client transmits his scope first, the server-side function will be redefined to call the client-side function, which simply calls the server-side function.

About the actual code: I figured it would be amusing to use the y-combinator trick instead of conventional methods which involve binding values to variables, especially since Flotype (the company behind NowJS) is a YC-funded company. Plus, it's just an awesome concept.

The other snippet, pulled from handleNewConnection(), is both straightforward and standard:

if (client.handled) return;
client.handled = true;

Since the rest of the function involves attaching function listeners, this ensures that these listeners are only attached once.

Well, that's that. Let us know what you think in the comments below, and in our IRC channel #nowjs on irc.freenode.net

Weekends are for hacking. Here's some unCRUDdy inspiration

It's Friday, Friday ... it's time to bust out the weekend hack!

Here are some awesome pieces of web technology that might help you pick a hack:

On the client

RaphaelJS  Github

If you're looking to build a visualization, RaphaelJS is really the go-to vector graphics library. The graphics are based on SVG and VML so it's compatible with just about every browser. As a bonus, this means every graphic element is a DOM element so the normal DOM manipulation you do still works. Yep even animations are supported, beautifully.

Three.js Github

Three.js is an API for WebGL. Ok it's actually much more than that. It's a JavaScript 3d engine that can render to <canvas>, <svg> or WebGL. 
They've got many jaw-dropping demos on the Github page, and this WebGL one is my favorite (view on a WebGL browser like Chrome). Here's another awesome demo that actually uses NowJS.

Backbone Github

Want UI bindings in JavaScript? Yep that's what Backbone does. Your data is organized in `models` with custom events that trigger changes in `views`. Definitely a useful tool for your hack.

JSMidi Github

Who DOESN'T want to play midi's via JavaScript in the browser. Go build some awesome synths!

Popcorn.js Github

Popcorn enables rich interaction with the HTML5 video player. I believe interactivity during video watching is one of those things that people have become jaded against, thanks to video annotation spammers on YouTube. There are several cool demos showing what you can do by interaction with video context, but I think there is definitely room for some creative hacking with this library.

jPlayer Github

This is probably the easiest way to use HTML5 audio in your application. Seriously, that spec is so messy right now. jPlayer detects your audio file type and uses a flash fallback as necessary. Protip: iOS devices require audio playing to be triggered by a user event so don't try to make a game in the browser if you value sound.

nude.js Github

Yes, you can test for nudity with JavaScript scanning a canvas. This implementation is no joke. WebWorkers (with an appropriate fallback) is used with the algorithm to detect boobies, so it won't block your UI thread. Someone make a Chrome/Firefox browser extension!

Guacamole

Yep this is VNC in the browser (without Java applet or Flash). We've all thought about it before but these guys made a robust and performant implementation. The server setup is a bit involved but the client side is a sweet HTML5 canvas. None of that Java applet business that had become the default for web VNC viewers.

On the server

node-bittorrent Github

This library is a work in progress. It hasn't been updated in a while but give it a try. This is far from feature complete but the basis is there for you. There's no documentation either. Yep this is a bonus hack for those seeking a challenge.

node-webworker Github or `npm install webworker`

An implementation of WebWorkers API in Node. Time to make your current Node hacks more efficient. 

node-usb Github or `npm install node-usb`

libusb bindings for node? I might have to stop writing this post to try this out.

node-ssh Github or `npm install ssh`

This is a pretty young library but substack has made an excellent step into SSH server awesomeness in Node. Unlike most Node modules this is actually a C++ Node extension. Give it a try! 

node-qrcode Github or `npm install qrcode`

A really simple interface for generating QR codes in node. One particular awesome features is the ability to spit out a data URI representation of the output. First person to make a QR code that contains its own data URI as the value wins a pony.

say.js Github or `npm install say`

Dead simple Node text-to-speech. 24 voices available and the simplest API with no config. Added bonus: it's async. Awesome.

Jerk Github or `npm install jerk`

Face it, we've all wanted to make an IRC bot at some point in our lives. Be initimidated no more, as Jerk makes it dumb easy, for better or for worse.

run.js Github or `npm install run`

Run.js automatically restarts your Node server on file change. No more `ctrl+c, up arrow, enter`. An essential tool for any Node developer.

 

Make a realtime hack!

That's what we do here at NowJS. We make realtime web applications really easy by creating a shared namespace that is accessible by both server and client. In fact, I would also like to take this time to announce that NowJS just released v0.5!

Here's the big awesome new feature:
Groups, aka channels, aka rooms, are now available so you can address a subset of `everyone`!

Yep, this new feature retains the ease-of-use of NowJS.

Here are some other changes in this release

  • Fixes for bugs and performance issues in IE6/7/8
  • Ability to pass on socket.io options to customize its operation
  • Security switch `clientWrite` option to prevent the client from writing into server memory (fixed a potential memory-based DoS!)
  • Proper events implementations for `ready`, `connect`, and `disconnect` on the client side and `connect` and `disconnect` for each group on the server-side

Check out the docs to learn the api for groups or read the changelog

No more excuses: Try Node.js for $0.02 or less in just a few minutes

In the last few months Node.js has been picking up in popularity. You see it on Twitter, you see in on Hacker News, so why haven't you tried it yet?

For me it was because Cygwin on Windows was a bit nebulous. Even if you're on OSX, the process might have been too much of a hassle.

Well be excused no more because I made an Amazon EC2 AMI
(community AMI ami-d7451792, US West region) loaded with the Node basics. Node.js, npm, express, NowJS all come pre-installed, pre-configured, and ready to use!

 

Why EC2

For new customers Amazon is giving away a whole year of dedicated hosting on their micro instance. Even if you're not a new customer, it's only $0.02 an hour. So let's get started javascripting!

If you're already familiar with EC2, read the TLDR

 

If you're new to EC2 or want extra guidance getting your instance setup, continue reading

Step-by-step: Setting up a FREE or $0.02/hr EC2 Micro instance with Node.js

Step 1: Get a EC2 account

Step 2: Enter the AWS Management Console. Choose the "US West region" and click "Launch Instance"

Step1

 

Step 3: In the `Request Instance Wizard` click `Community AMIs` and search for `ami-d7451792`. Click the Select button

Step3


Step 4: Choose the `Micro` instance size to take advantage of the free tier. Click continue

Step6

 

Step 5: Leave the two `Instance Details` page as their defaults and click Continue

(download)

 

Step 6: Now you need two create an SSH keypair

If you're not familiar with SSH, this might sound intimidating but it really isnt. In order to connect to your new server, you will be using SSH to communicate. SSH with EC2 requires that the server have your `public key` and you have the `private key` with these two pieces, you can authenticate and login to your server. We're going to generate a public/private keypair now.

Step6

This will spit out a .pem (a security certificate file) file for you. This is your private key! Keep it in a safe place

(Important: If you're planning on using PuTTY for your SSH client, you're going to have to convert that .pem to a PuTTY compatible .ppk! )

Then click continue

 

Step 7: Configure the Firewall

Just for testing purposes, lets open all the ports. Don't tase me bro.

Step7

 

Step 8: Review settings & Click Launch!

Step8
After you click launch, many magical things will happen at the Amazon datacenter. Jigahertz are traveling through tubes and somehow an instance matching exactly what you've told the wizard is booting up!

 

Step 9: Find out the server's hostname

After the server is done loading, the `public dns` will show up, thats the address to your shiny server!

Step9

 

Step 10: Connect to your server!

If you're on an unix system, go to your terminal and do something like this:

`ssh -i mykeypair.pem ec2-user@myec2serveraddress.compute.amazonaws.com`

If you're on PuTTY

Step10

 

Step 10: Lets run the example

Since Node, express, Now.js are already installed, lets run the NowJS chat example:

When you have established the SSH connection,

`cd examples`
`node helloworld_server.js`

That's it! Now you have a Node.js server running with the NowJS example chat server (on port 8080).

Navigate with your browser to `http://myec2serveraddress.compute.amazonaws.com:8080` to see it.
If the page won't load, you probably forgot to open the right ports in the firewall ("security group") step of launching the instance. 

 

Now you're ready to hack on Node.js!!!! If you have any difficulty,
Try the #node.js irc channel or #nowjs irc channel, both on Freenode.

 

 

TLDR (If you're familiar with EC2);

Simply launch an EC2 instance for community AMI 
ami-d7451792, US West region and you'll be up and running

Try cd'ing into the `examples` folder, and running `node helloworld_server.js`. Then go to `http://myec2serveraddress.compute.amazonaws.com:8080` to try out the chat example.

 

More about NowJS

NowJS makes real-time applications easier to develop by synchronising functions and variables in a namespace between clients and a server.

http://nowjs.com