DIY node.js server on Amazon EC2

I’m involved with a project where our ruby/rails developer dropped out, so I decided to take on the job using node.js (rather than learn rails). We initially were using services from dotCloud, but it was too flakey from day to day and our demo was coming up. For hosting, Amazon’s EC2 was the obvious candidate, but I’d have to setup and provision the entire server from scratch. This is that story :)

Here’s what we’ll do

  • choose a Linux image
  • create a HelloWorld node.js server
  • use git to push code changes to the server
  • automatically restart node after pushing with git
  • set up node to run long term using supervisor

Setup a New EC2 Instance

launch a new Ubuntu instance

First things first, login to your AWS console and launch a new Ubuntu Linux image for your new EC2 server. Select the Community AMIs tab and search for this one:

099720109477/ebs/ubuntu-images/ubuntu-maverick-10.10-i386-server-20101225

I choose Ubuntu over other Linux distributions because more of what I needed was already available via the standard package manager (redis, couchdb, etc…). At this point, I usually assign an elastic IP address to my new instances before proceeding with ssh.

update your new system

Once your new instance is up and running, login and update the system. The upgrade might take some time.

$ sudo apt-get update
$ sudo apt-get -y upgrade

install the rcconf service utility

This will make it easy to manage services

$ sudo apt-get install rcconf

install some build tools including git

$ sudo apt-get install build-essential
$ sudo apt-get install libssl-dev
$ sudo apt-get install git-core

libssl-dev is needed to use the crypt node.js package

build node

You can view the [node.js installation instructions](https://github.com/joyent/node/wiki/Installation “Node.js installation”) or just follow what I did.

$ wget http://nodejs.org/dist/node-latest.tar.gz
$ tar xzf node-latest.tar.gz
$ cd node-v0.4.7
$ ./configure --prefix=/usr
$ make
$ sudo make install

I used –prefix=/usr to install node on the existing PATH. make install can take quite a while… go brew some espresso.

install the node package manager npm:

Get the latest npm from github and install.

$ cd ~
$ git clone http://github.com/isaacs/npm.git
$ cd npm
$ sudo make install

get some really good node packages :)

$ cd ~
$ npm install connect redis connect-redis jade express express-resource futures emailjs

install a web server

$ sudo apt-get install nginx

Edit the nginx default configuration file replacing the root (/) location section

$ sudo vi /etc/nginx/sites-enabled/default

Use a proxy passengry entry. This will forward your requests to your node server

location / {
    proxy_pass          http://127.0.0.1:8124/;
}

Restart ngnix

$ sudo /etc/init.d/nginx restart

Hello New Server!

create a directory for your node server

$ mkdir ~/www
$ cd ~/www

create a HelloWorld server

$ cat > server.js

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(8124, "127.0.0.1");
console.log('Server running at http://127.0.0.1:8124/');

Start the server

$ node server.js

test your new server

Test in a browser by navigating to http://your.static.ip/

Shutdown your new server before continuing…

Bring in Source Control!

Now we’ll get to the good part where we can leverage git to deploy new server code

create a remote repository for our node project

Create a bare repository outside the www folder

$ mkdir ~/repo
$ cd ~/repo
$ git init --bare

create a git hook

Create a post-recieve hook that will copy over new code after it’s been pushed to the repository

$ cat > hooks/post-receive

#!/bin/sh
GIT_WORK_TREE=/home/ubuntu/www
export GIT_WORK_TREE
git checkout -f

$ chmod +x hooks/post-receive

add this remote repository to your LOCAL repository:

On your local development machine, setup a repository for your new server

$ mkdir helloworld
$ cd helloworld
$ git init
$ git remote add ec2 ssh://ubuntu@your.static.ip/home/ubuntu/repo

create a local HelloWorld node server

$ cat > server.js

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(8124, "127.0.0.1");
console.log('Server running at http://127.0.0.1:8124/');

add and commit it to your local repository

$ git add server.js
$ git commit -m 'first'

push our local code changes to the remote repository:

$ git push ec2 +master:refs/heads/master

you only need to specify +master:refs/heads/mast the first time

test your new server

Test in a browser by navigating to http://your.static.ip/

Have Your Node Server Run Forever

dotCloud uses supervisor to keep node servers up after reboot and crashing. We’ll do the same. So, back on your EC2 instance:

$ sudo apt-get install python-setuptools
$ sudo easy_install supervisor

install it as a service

$ curl https://raw.github.com/gist/176149/88d0d68c4af22a7474ad1d011659ea2d27e35b8d/supervisord.sh > supervisord
$ chmod +x supervisord
$ sudo mv supervisord /etc/init.d/supervisord

set the service to start on boot

check the supervisord service in the services list using rcconf

$ sudo rcconf

create a configuration file

$ sudo echo_supervisord_conf > supervisord.conf
$ sudo mv supervisord.conf /etc/supervisord.conf

edit the new configuration file

$ sudo vi /etc/supervisord.conf

set the user that supervisord runs as to be ‘ubuntu’

This is under the [supervisord] section

user=ubuntu

add a section that describes your node server

Note that we can set the NODE_ENV variable here.

[program:node]
command=node server.js
directory=/home/ubuntu/www
environment=NODE_ENV=production

reload supervisord

$ supervisorctl reload

You can also restart the service

$ /etc/init.d/supervisord restart

restart supervisord when changes are pushed

In the post-recieve git hook we previously created, append a final command that restarts supervisord

$ vi ~/repo/hooks/post-receive

#!/bin/sh
GIT_WORK_TREE=/home/ubuntu/www
export GIT_WORK_TREE
git checkout -f
sudo supervisorctl restart node

That’s It :)

Now you can work on a local development machine, push your changes to the server using git, sit back and relax while the post-recieve hook tells supervisor to restart your node server. Refresh the browser and throw the confetti!

  • Anonymous

    Did you configure iptables?

  • http://cuppster.com Jason

    I didn’t… just relying on EC2′s security groups, only letting web and ssh in. Should I apply both on a production server?

  • http://twitter.com/fstrnet FSTR

    That’s the bit that stood out for me. It looks like the NGNIX is acting as a proxy forwarder. Requests to port 80 seem to be forwarded to localhost:8120 (someone correct me if I’m wrong)

    I’ve recently built an amazon micro node.js server in a similar fashion, but have used this article: http://www.debian-administration.org/articles/386 to configure IPTABLES. My express app runs on port 3000 and all requests to 80 are autoforwarded to 3000.

    Still there are some steps in this article that have taught me something. I like the install as service bit and the setting of an environment variable to production.

    With respect to running as a service though… what happens if the app crashes? Does it autorestart or should I still be using a nodemon / spark2 style option?

  • http://cuppster.com Jason

    NGNIX is definitely overkill, I’m only using it as a reverse proxy. I’m working on a haproxy version now and will post it when i’m done. Also I heard that ngnix doesn’t handle websockets well…

    Using supervisor is the best part. This is what dotCloud is using. When your server boots or your node server crashes, supervisor will start it back up… I only use nodemon for local development.

  • http://ceejayoz.com/ ceejayoz

    EC2′s security groups are great but they don’t regulate outgoing traffic, just incoming. Even for incoming traffic it’s probably wise to have iptables as a backup, in case there’s ever a hole discovered in EC2′s security groups.

  • http://cuppster.com Jason

    Ahh, yes… that makes sense… thanks for the tip!

  • http://twitter.com/fstrnet FSTR

    I was watching Pedro Teixeira’s nondetuts where he tells us about different sorts of node tools. The up-to-the-minute recommendation* seems to be that we use Cluster** for allowing node to leverage the power of multiple cores.

    I wonder if there’s a way to set Cluster to work with Supervisor… or maybe it already works like supervisor. I’ve not fully researched either side of the equation yet, but there might be something there.

    * http://nodetuts.com/tutorials/14-some-nodejs-tools.html#video (Pedro answered the question about 4 days ago, but I only saw it just now).
    ** http://learnboost.github.com/cluster/

  • http://cuppster.com Jason

    Cluster and Supervisor, yes… that’s a good research project. If cluster’s ‘resuscitation’ works on crashed node workers, then supervisor won’t see it. But supervisor can still be configured to kick-off your cluster on reboot…

  • http://twitter.com/webguy Joel Haasnoot

    Cluster is nice, but left over the weekend or overnigth occasionally produced hundreds or thousands of reboots. Time for another research project to fix that one…

  • http://www.doorty.com Brent Daugherty

    I’ve been waiting for something as detailed as this for Node on EC2. I only had 2 snafus:

    1) git push ec2 +master:refs/heads/master ….. Permission denied (publickey). fatal: The remote end hung up unexpectedly
    2) supervisorctl reload …… error: , [Errno 2] No such file or directory: file: line: 1

  • http://cuppster.com Jason

    I remember getting that myself at some point, did you have a file at /etc/supervisord.conf?

  • http://www.doorty.com Brent Daugherty

    Yes. The file is there now, and I still get that error.

  • http://cuppster.com Jason

    Who’s the owner of and is /etc/init.d/supervisord executable?

  • http://www.doorty.com Brent Daugherty

    -rwxr-xr-x 1 ubuntu ubuntu 185 2011-05-18 03:11 supervisord

  • http://cuppster.com Jason

    Looks like you found a bug! The supervisord script is just an HTML page. The link should use ‘https’ instead of ‘http’. Take a look at yours and re-download it if it’s just HTML. I’ve updated the steps above.

    Also, /etc/init.d/supervisord restart may not work (I think the script is buggy). If you have to kill it, use ‘sudo pkill supervisord’, then issue ‘/etc/init.d/supervisord start’

    If all else fails, delete the .conf file, ‘apt-get’ remove supervisor and start again… :)

    Also, supervisord is not owned by root, I think this is ok (works for me), but maybe something more to this…

  • http://www.doorty.com Brent Daugherty

    Ahh. okay. I have the script now (not an html page), but now I get: -bash: /etc/init.d/supervisord: /bin/sh^M: bad interpreter: No such file or directory. Guess I’ll need to start over in the morning when I’m thinking clearly. Thanks for the help.

  • http://cuppster.com Jason

    should probably look into using ‘upstart’ instead of init… looks a lot simpler… found this: http://lincolnloop.com/blog/2010/jun/24/automatically-running-supervisord-startup/ will have to try it out…

  • Stefan Richter

    Hi, node noob here. Why are you using nginx again? Could you quickly explain why it is needed/not needed in thsi setup? All I’m interested in is running node (but the git part is of course also very neat!). Cheers.

  • http://cuppster.com Jason

    You’ll likely want a front-end in production for caching, load-balancing, serving static files (so node.js doesn’t have to be configured to do it)… or all three. Instead of nginx, you could have substituted varnish or haproxy… I just used nginx as one example of a front-end reverse proxy.

  • http://cuppster.com Jason

    I’ve got a new recipe up for using ‘upstart’ instead of classic init scripts to manage ‘supervisord’ http://cuppster.com/2011/05/18/using-supervisor-with-upstart

  • http://cuppster.com Jason
  • http://www.doorty.com Brent Daugherty

    Worked like a charm. Thanks. Each git push now causes a restart. Now, will this restart Node if it crashes? How can I test that?

  • Ryan

    Upstart would be a much nicer solution to babysitting the node process. Bear in mind however that you will probably have to change over to systemd at some point.

  • Ryan

    Calling nginx “overkill” is an engineering fallacy. Exactly *how* do you think it is overkill? You are wrong anyway but I would just like to know.

  • http://cuppster.com Jason

    Thanks for that… didn’t know about systemd…

  • http://cuppster.com Jason

    just a guess, but nginx (or apache) probably has a lot more features than haproxy — a reverse proxy is all I need for the purposes of the most simple (but useful) example for this recipe… I’m still trying to get my head around the best ways to use varnish, haproxy, nginx/apache and node servers together for a larger deployment. If you have any tips, I’d love to know!

  • Michael Heuberger

    And what about databases? Possible to install the CouchDB for free there?

  • http://cuppster.com Jason

    try: $ sudo apt-get install couchdb

    I use it with my micro-instance. couchdb won’t cost you anything. You’ll just pay for amazon’s hosting…

  • Michael Heuberger

    Thanks for that. I assume “sudo” alone isn’t enough and some more changes on the server are required as well to make the couchdb work?

  • http://cuppster.com Jason

    I think it starts itself. You can check with rcconf, or just run: sudo /etc/init.d/couchdb restart It will be running on the default port without user/password
    Try the util interface in a browse: http://127.0.0.1:5984/_utils/

  • http://andrewseddon.com Andrew Seddon

    Yeah I had some trouble with NowJS (websockets) and Nginx so I’m just listening on port 80 now. Will let you know if I find a solution. Otherwise great recipe, very tasty!

  • http://cuppster.com Jason

    I’m working on a recipe that uses haproxy. Nearly there. Issues with keeping session cookies…

  • http://twitter.com/kvz Kevin van Zonneveld

    Ubuntu can also keep your node process running itself, through Upstart. Did a post about it here http://kevin.vanzonneveld.net/techblog/article/run_nodejs_as_a_service_on_ubuntu_karmic/

  • http://cuppster.com Jason

    Thanks for that! supervisor can be flaky at times… this looks like a better way to do it on distros that have upstart…

  • http://andrewseddon.com Andrew Seddon

    FYI just setup a proxy with https://github.com/nodejitsu/node-http-proxy it’s super simple and websockets work out of the box.

  • http://cuppster.com Jason

    Checking that out, thanks!

  • Stuart

    Great post! Thanks for the trailblazing. I ran in to some confusing problems… I built from the Ubuntu Maverick image you recommended. I installed what I needed plus some, but *not* nginix. When I ran the HelloWorld server it started, but I could not connect to it — I was able to ssh in to the instance no problem (using both the assigned external domain name, and an attached elastic IP). I battled this for hours thinking it was the port firewalling on ec2 of port 80… finally started poking around with netstat and noticed that ssh was listening at IP 0.0.0.0, not 127.0.0.1. I had two entries showing with ‘route -n’, one destination was the instances internal ec2 IP, the other was 0.0.0.0. So I edited the server.js file and changed the listen() to use “0.0.0.0″ IP — et voilla, I was then able to connect via http from the browser. And finally it strikes me as I go back and re-read your recipe that the 127.0.0.1 IP is where you have nginix forwarding to!! Let this be a lesson to anyone who skips part of an installation instruction without reading the part that’s being skipped :-)

  • http://cuppster.com Jason

    wow… sorry to hear all that! It *is* tricky… I myself had to do it a few times and again to verify for this blog post. I’ve done it to support other domains by adding ‘servers’ to my nginx conf forwarding them to other node servers on different ports… =D

  • http://twitter.com/clintandrewhall Clint Andrew Hall

    Just a heads up, as I used the latest micro ubuntu instance…

    1. my ubuntu instance only had yum installed, and I was unable to install apt-get, so I had to hunt down the alternate package names. No big deal, but it may confuse some folks. I may blog the differences at some point.
    2. the user for that instance was ec2-user; just a substitution wherever I saw “ubuntu”.
    3. In order to allow the git push to the remote repo, on my mac, I executed “ssh-add -K /path/to/pem.pem”… this added my creds and allowed a seamless push.
    4. I edited nginx.conf directly to add the proxy location.

  • http://www.facebook.com/RobLourens Rob Lourens

    Thanks for the post! If you’re forwarding all requests to the node server, how do you set up nginx to serve the static files? I’m trying to do something like, serve a static file and have the client connect to the socket.io node server, but couldn’t get it to work, so I guess I’ll just skip nginx for the moment…

  • Jason Cupp

    Thanks for that! Yes… I took the easy route with Ubuntu. The packages I needed (beyond the ones mentioned here) were already available.

  • Jason Cupp

    I didn’t have to serve static files from nginx yet, but it seems this would be a very common scenario — should be docs about this. Probably a better idea than having node/express do it…

  • Mike Newell

    Thank you soooooo much! this tutorial is amazingly detailed. I am a beginning linux user on centOS and I was able to follow along…

    Also, if anyone runs into a “file not in directory” error its because git redirects you on:

    curl https://gist.github.com/raw/176149/88d0d68c4af22a7474ad1d011659ea2d27e35b8d/supervisord.sh > supervisord

    so just go to that url, copy all the text and paste it into the file if the curl turns up empty or with html in it…

    also, you can check boxes in rcconf by hitting the space bar.

    Thanks again!
    Mike

  • http://cuppster.com Jason

    Thanks for that! I’ve updated the link in the instructions… Cheers!

  • http://www.xicom.biz/ web development services

    yeah..nice one

  • http://iwearshorts.com/blog/nodejs-websockets-and-ec2/ NodeJS, Websockets and EC2 | Mike Newell

    [...] References: Installing FTP on EC2, Socket.io getting started, inspiration for this article. [...]

  • Anonymous

    Great article!!! I didn’t know about git hooks like that, until now, thanks!

  • Anonymous

    Hi there! Just wondering how you got around the permission denied issue when pushing the master to ec2… I have the same problem and can’t seem to overcome the hurdle.

    Thanks,
    Banx

  • http://cuppster.com Jason

    No problem! =D I admit I’m using rsync more often than git to deploy, but… there’s a time/place for everything!

  • http://www.kensingtondental.com/cosmetic_smile_smiledesign.shtml cosmetic dentistry

    loved it..

  • http://www.gofleet.ca/gps-tracking/vehicle-tracking-system.html gps vehicle tracking

    woaaah

  • http://twitter.com/pablocantero Pablo Cantero

    I had the same problem.

    It answer worked for me.
    http://stackoverflow.com/a/5654728/464685

    cat ~/.ssh/id_dsa.pub | ssh -i amazon-generated-key.pem ec2-user@amazon-instance-public-dns “cat>> ~/.ssh/authorized_keys”

    :)

  • http://twitter.com/pablocantero Pablo Cantero

    To serve static files I use

    /etc/nginx/sites-enabled/default

    location ^(/images|/javascripts|/stylesheets)$ {
    root /home/ubuntu/www/public;
    autoindex on;
    }

  • http://twitter.com/chrisdeely Chris Deely

    Thanks for adding your notes, Clint. #3 was key for me, I was getting “Permission denied (publickey).” errors until I added the SSH key.

  • http://twitter.com/vyumix vyumix

    Cool stuff.. thanks a lot for this write up

  • Anonymous

    for me supervisor didn’t work with error:

    $ sudo supervisorctl start node
    node: ERROR (no such file)

    until I changed “command node server.js” to “command /usr/local/bin/node server.js”

    i.e. full path to node.js

  • Danislav Kostov

    Or gwan ;)