DIY node.js server on Amazon EC2

I’m involved with a project where our ruby/rails developer dropped out, so I decided to take on the job using node.js (rather than learn rails). We initially were using services from dotCloud, but it was too flakey from day to day and our demo was coming up. For hosting, Amazon’s EC2 was the obvious candidate, but I’d have to setup and provision the entire server from scratch. This is that story :)

Here’s what we’ll do

  • choose a Linux image
  • create a HelloWorld node.js server
  • use git to push code changes to the server
  • automatically restart node after pushing with git
  • set up node to run long term using supervisor

Setup a New EC2 Instance

launch a new Ubuntu instance

First things first, login to your AWS console and launch a new Ubuntu Linux image for your new EC2 server. Select the Community AMIs tab and search for this one:

099720109477/ebs/ubuntu-images/ubuntu-maverick-10.10-i386-server-20101225

I choose Ubuntu over other Linux distributions because more of what I needed was already available via the standard package manager (redis, couchdb, etc…). At this point, I usually assign an elastic IP address to my new instances before proceeding with ssh.

update your new system

Once your new instance is up and running, login and update the system. The upgrade might take some time.

$ sudo apt-get update
$ sudo apt-get -y upgrade

install the rcconf service utility

This will make it easy to manage services

$ sudo apt-get install rcconf

install some build tools including git

$ sudo apt-get install build-essential
$ sudo apt-get install libssl-dev
$ sudo apt-get install git-core

libssl-dev is needed to use the crypt node.js package

build node

You can view the [node.js installation instructions](https://github.com/joyent/node/wiki/Installation “Node.js installation”) or just follow what I did.

$ wget http://nodejs.org/dist/node-latest.tar.gz
$ tar xzf node-latest.tar.gz
$ cd node-v0.4.7
$ ./configure --prefix=/usr
$ make
$ sudo make install

I used –prefix=/usr to install node on the existing PATH. make install can take quite a while… go brew some espresso.

install the node package manager npm:

Get the latest npm from github and install.

$ cd ~
$ git clone http://github.com/isaacs/npm.git
$ cd npm
$ sudo make install

get some really good node packages :)

$ cd ~
$ npm install connect redis connect-redis jade express express-resource futures emailjs

install a web server

$ sudo apt-get install nginx

Edit the nginx default configuration file replacing the root (/) location section

$ sudo vi /etc/nginx/sites-enabled/default

Use a proxy passengry entry. This will forward your requests to your node server

location / {
    proxy_pass          http://127.0.0.1:8124/;
}

Restart ngnix

$ sudo /etc/init.d/nginx restart

Hello New Server!

create a directory for your node server

$ mkdir ~/www
$ cd ~/www

create a HelloWorld server

$ cat > server.js

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(8124, "127.0.0.1");
console.log('Server running at http://127.0.0.1:8124/');

Start the server

$ node server.js

test your new server

Test in a browser by navigating to http://your.static.ip/

Shutdown your new server before continuing…

Bring in Source Control!

Now we’ll get to the good part where we can leverage git to deploy new server code

create a remote repository for our node project

Create a bare repository outside the www folder

$ mkdir ~/repo
$ cd ~/repo
$ git init --bare

create a git hook

Create a post-recieve hook that will copy over new code after it’s been pushed to the repository

$ cat > hooks/post-receive

#!/bin/sh
GIT_WORK_TREE=/home/ubuntu/www
export GIT_WORK_TREE
git checkout -f

$ chmod +x hooks/post-receive

add this remote repository to your LOCAL repository:

On your local development machine, setup a repository for your new server

$ mkdir helloworld
$ cd helloworld
$ git init
$ git remote add ec2 ssh://ubuntu@your.static.ip/home/ubuntu/repo

create a local HelloWorld node server

$ cat > server.js

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(8124, "127.0.0.1");
console.log('Server running at http://127.0.0.1:8124/');

add and commit it to your local repository

$ git add server.js
$ git commit -m 'first'

push our local code changes to the remote repository:

$ git push ec2 +master:refs/heads/master

you only need to specify +master:refs/heads/mast the first time

test your new server

Test in a browser by navigating to http://your.static.ip/

Have Your Node Server Run Forever

dotCloud uses supervisor to keep node servers up after reboot and crashing. We’ll do the same. So, back on your EC2 instance:

$ sudo apt-get install python-setuptools
$ sudo easy_install supervisor

install it as a service

$ curl https://raw.github.com/gist/176149/88d0d68c4af22a7474ad1d011659ea2d27e35b8d/supervisord.sh > supervisord
$ chmod +x supervisord
$ sudo mv supervisord /etc/init.d/supervisord

set the service to start on boot

check the supervisord service in the services list using rcconf

$ sudo rcconf

create a configuration file

$ sudo echo_supervisord_conf > supervisord.conf
$ sudo mv supervisord.conf /etc/supervisord.conf

edit the new configuration file

$ sudo vi /etc/supervisord.conf

change the permissions of the [unix_http_server]

chmod=0777                 ; sockef file mode (default 0700)

set the user that supervisord runs as to be ‘ubuntu’

This is under the [supervisord] section

user=ubuntu

add a section that describes your node server

Note that we can set the NODE_ENV variable here.

[program:node]
command=node server.js
directory=/home/ubuntu/www
environment=NODE_ENV=production

reload supervisord

$ supervisorctl reload

You can also restart the service

$ /etc/init.d/supervisord restart

restart supervisord when changes are pushed

In the post-recieve git hook we previously created, append a final command that restarts supervisord

$ vi ~/repo/hooks/post-receive

#!/bin/sh
GIT_WORK_TREE=/home/ubuntu/www
export GIT_WORK_TREE
git checkout -f
sudo supervisorctl restart node

That’s It :)

Now you can work on a local development machine, push your changes to the server using git, sit back and relax while the post-recieve hook tells supervisor to restart your node server. Refresh the browser and throw the confetti!

  • Anonymous

    Did you configure iptables?

    • http://cuppster.com Jason

      I didn’t… just relying on EC2′s security groups, only letting web and ssh in. Should I apply both on a production server?

      • http://ceejayoz.com/ ceejayoz

        EC2′s security groups are great but they don’t regulate outgoing traffic, just incoming. Even for incoming traffic it’s probably wise to have iptables as a backup, in case there’s ever a hole discovered in EC2′s security groups.

        • http://cuppster.com Jason

          Ahh, yes… that makes sense… thanks for the tip!

  • http://twitter.com/fstrnet FSTR

    That’s the bit that stood out for me. It looks like the NGNIX is acting as a proxy forwarder. Requests to port 80 seem to be forwarded to localhost:8120 (someone correct me if I’m wrong)

    I’ve recently built an amazon micro node.js server in a similar fashion, but have used this article: http://www.debian-administration.org/articles/386 to configure IPTABLES. My express app runs on port 3000 and all requests to 80 are autoforwarded to 3000.

    Still there are some steps in this article that have taught me something. I like the install as service bit and the setting of an environment variable to production.

    With respect to running as a service though… what happens if the app crashes? Does it autorestart or should I still be using a nodemon / spark2 style option?

    • http://cuppster.com Jason

      NGNIX is definitely overkill, I’m only using it as a reverse proxy. I’m working on a haproxy version now and will post it when i’m done. Also I heard that ngnix doesn’t handle websockets well…

      Using supervisor is the best part. This is what dotCloud is using. When your server boots or your node server crashes, supervisor will start it back up… I only use nodemon for local development.

      • http://twitter.com/fstrnet FSTR

        I was watching Pedro Teixeira’s nondetuts where he tells us about different sorts of node tools. The up-to-the-minute recommendation* seems to be that we use Cluster** for allowing node to leverage the power of multiple cores.

        I wonder if there’s a way to set Cluster to work with Supervisor… or maybe it already works like supervisor. I’ve not fully researched either side of the equation yet, but there might be something there.

        * http://nodetuts.com/tutorials/14-some-nodejs-tools.html#video (Pedro answered the question about 4 days ago, but I only saw it just now).
        ** http://learnboost.github.com/cluster/

        • http://cuppster.com Jason

          Cluster and Supervisor, yes… that’s a good research project. If cluster’s ‘resuscitation’ works on crashed node workers, then supervisor won’t see it. But supervisor can still be configured to kick-off your cluster on reboot…

          • http://twitter.com/webguy Joel Haasnoot

            Cluster is nice, but left over the weekend or overnigth occasionally produced hundreds or thousands of reboots. Time for another research project to fix that one…

      • Ryan

        Calling nginx “overkill” is an engineering fallacy. Exactly *how* do you think it is overkill? You are wrong anyway but I would just like to know.

        • http://cuppster.com Jason

          just a guess, but nginx (or apache) probably has a lot more features than haproxy — a reverse proxy is all I need for the purposes of the most simple (but useful) example for this recipe… I’m still trying to get my head around the best ways to use varnish, haproxy, nginx/apache and node servers together for a larger deployment. If you have any tips, I’d love to know!

      • http://andrewseddon.com Andrew Seddon

        Yeah I had some trouble with NowJS (websockets) and Nginx so I’m just listening on port 80 now. Will let you know if I find a solution. Otherwise great recipe, very tasty!

        • http://cuppster.com Jason

          I’m working on a recipe that uses haproxy. Nearly there. Issues with keeping session cookies…

          • http://andrewseddon.com Andrew Seddon

            FYI just setup a proxy with https://github.com/nodejitsu/node-http-proxy it’s super simple and websockets work out of the box.

          • http://cuppster.com Jason

            Checking that out, thanks!

  • http://www.doorty.com Brent Daugherty

    I’ve been waiting for something as detailed as this for Node on EC2. I only had 2 snafus:

    1) git push ec2 +master:refs/heads/master ….. Permission denied (publickey). fatal: The remote end hung up unexpectedly
    2) supervisorctl reload …… error: , [Errno 2] No such file or directory: file: line: 1

    • http://cuppster.com Jason

      I remember getting that myself at some point, did you have a file at /etc/supervisord.conf?

      • http://www.doorty.com Brent Daugherty

        Yes. The file is there now, and I still get that error.

        • http://cuppster.com Jason

          Who’s the owner of and is /etc/init.d/supervisord executable?

          • http://www.doorty.com Brent Daugherty

            -rwxr-xr-x 1 ubuntu ubuntu 185 2011-05-18 03:11 supervisord

          • http://cuppster.com Jason

            Looks like you found a bug! The supervisord script is just an HTML page. The link should use ‘https’ instead of ‘http’. Take a look at yours and re-download it if it’s just HTML. I’ve updated the steps above.

            Also, /etc/init.d/supervisord restart may not work (I think the script is buggy). If you have to kill it, use ‘sudo pkill supervisord’, then issue ‘/etc/init.d/supervisord start’

            If all else fails, delete the .conf file, ‘apt-get’ remove supervisor and start again… :)

            Also, supervisord is not owned by root, I think this is ok (works for me), but maybe something more to this…

          • http://www.doorty.com Brent Daugherty

            Ahh. okay. I have the script now (not an html page), but now I get: -bash: /etc/init.d/supervisord: /bin/sh^M: bad interpreter: No such file or directory. Guess I’ll need to start over in the morning when I’m thinking clearly. Thanks for the help.

          • http://cuppster.com Jason

            should probably look into using ‘upstart’ instead of init… looks a lot simpler… found this: http://lincolnloop.com/blog/2010/jun/24/automatically-running-supervisord-startup/ will have to try it out…

          • http://cuppster.com Jason
          • http://www.doorty.com Brent Daugherty

            Worked like a charm. Thanks. Each git push now causes a restart. Now, will this restart Node if it crashes? How can I test that?

    • Anonymous

      Hi there! Just wondering how you got around the permission denied issue when pushing the master to ec2… I have the same problem and can’t seem to overcome the hurdle.

      Thanks,
      Banx

      • http://twitter.com/pablocantero Pablo Cantero

        I had the same problem.

        It answer worked for me.
        http://stackoverflow.com/a/5654728/464685

        cat ~/.ssh/id_dsa.pub | ssh -i amazon-generated-key.pem ec2-user@amazon-instance-public-dns “cat>> ~/.ssh/authorized_keys”

        :)

  • Stefan Richter

    Hi, node noob here. Why are you using nginx again? Could you quickly explain why it is needed/not needed in thsi setup? All I’m interested in is running node (but the git part is of course also very neat!). Cheers.

    • http://cuppster.com Jason

      You’ll likely want a front-end in production for caching, load-balancing, serving static files (so node.js doesn’t have to be configured to do it)… or all three. Instead of nginx, you could have substituted varnish or haproxy… I just used nginx as one example of a front-end reverse proxy.

      • Danislav Kostov

        Or gwan ;)

  • http://cuppster.com Jason

    I’ve got a new recipe up for using ‘upstart’ instead of classic init scripts to manage ‘supervisord’ http://cuppster.com/2011/05/18/using-supervisor-with-upstart

  • Ryan

    Upstart would be a much nicer solution to babysitting the node process. Bear in mind however that you will probably have to change over to systemd at some point.

    • http://cuppster.com Jason

      Thanks for that… didn’t know about systemd…

  • Michael Heuberger

    And what about databases? Possible to install the CouchDB for free there?

    • http://cuppster.com Jason

      try: $ sudo apt-get install couchdb

      I use it with my micro-instance. couchdb won’t cost you anything. You’ll just pay for amazon’s hosting…

      • Michael Heuberger

        Thanks for that. I assume “sudo” alone isn’t enough and some more changes on the server are required as well to make the couchdb work?

        • http://cuppster.com Jason

          I think it starts itself. You can check with rcconf, or just run: sudo /etc/init.d/couchdb restart It will be running on the default port without user/password
          Try the util interface in a browse: http://127.0.0.1:5984/_utils/

  • http://twitter.com/kvz Kevin van Zonneveld

    Ubuntu can also keep your node process running itself, through Upstart. Did a post about it here http://kevin.vanzonneveld.net/techblog/article/run_nodejs_as_a_service_on_ubuntu_karmic/

    • http://cuppster.com Jason

      Thanks for that! supervisor can be flaky at times… this looks like a better way to do it on distros that have upstart…

  • Stuart

    Great post! Thanks for the trailblazing. I ran in to some confusing problems… I built from the Ubuntu Maverick image you recommended. I installed what I needed plus some, but *not* nginix. When I ran the HelloWorld server it started, but I could not connect to it — I was able to ssh in to the instance no problem (using both the assigned external domain name, and an attached elastic IP). I battled this for hours thinking it was the port firewalling on ec2 of port 80… finally started poking around with netstat and noticed that ssh was listening at IP 0.0.0.0, not 127.0.0.1. I had two entries showing with ‘route -n’, one destination was the instances internal ec2 IP, the other was 0.0.0.0. So I edited the server.js file and changed the listen() to use “0.0.0.0″ IP — et voilla, I was then able to connect via http from the browser. And finally it strikes me as I go back and re-read your recipe that the 127.0.0.1 IP is where you have nginix forwarding to!! Let this be a lesson to anyone who skips part of an installation instruction without reading the part that’s being skipped :-)

    • http://cuppster.com Jason

      wow… sorry to hear all that! It *is* tricky… I myself had to do it a few times and again to verify for this blog post. I’ve done it to support other domains by adding ‘servers’ to my nginx conf forwarding them to other node servers on different ports… =D

  • http://twitter.com/clintandrewhall Clint Andrew Hall

    Just a heads up, as I used the latest micro ubuntu instance…

    1. my ubuntu instance only had yum installed, and I was unable to install apt-get, so I had to hunt down the alternate package names. No big deal, but it may confuse some folks. I may blog the differences at some point.
    2. the user for that instance was ec2-user; just a substitution wherever I saw “ubuntu”.
    3. In order to allow the git push to the remote repo, on my mac, I executed “ssh-add -K /path/to/pem.pem”… this added my creds and allowed a seamless push.
    4. I edited nginx.conf directly to add the proxy location.

    • Jason Cupp

      Thanks for that! Yes… I took the easy route with Ubuntu. The packages I needed (beyond the ones mentioned here) were already available.

    • http://twitter.com/chrisdeely Chris Deely

      Thanks for adding your notes, Clint. #3 was key for me, I was getting “Permission denied (publickey).” errors until I added the SSH key.

    • Afeez Aziz

      Clint, can you please explain in detail how to do this. I am having the same problem. My cant instance hang up when im trying to push my code from a local machine.

  • http://www.facebook.com/RobLourens Rob Lourens

    Thanks for the post! If you’re forwarding all requests to the node server, how do you set up nginx to serve the static files? I’m trying to do something like, serve a static file and have the client connect to the socket.io node server, but couldn’t get it to work, so I guess I’ll just skip nginx for the moment…

    • Jason Cupp

      I didn’t have to serve static files from nginx yet, but it seems this would be a very common scenario — should be docs about this. Probably a better idea than having node/express do it…

    • http://twitter.com/pablocantero Pablo Cantero

      To serve static files I use

      /etc/nginx/sites-enabled/default

      location ^(/images|/javascripts|/stylesheets)$ {
      root /home/ubuntu/www/public;
      autoindex on;
      }

  • Mike Newell

    Thank you soooooo much! this tutorial is amazingly detailed. I am a beginning linux user on centOS and I was able to follow along…

    Also, if anyone runs into a “file not in directory” error its because git redirects you on:

    curl https://gist.github.com/raw/176149/88d0d68c4af22a7474ad1d011659ea2d27e35b8d/supervisord.sh > supervisord

    so just go to that url, copy all the text and paste it into the file if the curl turns up empty or with html in it…

    also, you can check boxes in rcconf by hitting the space bar.

    Thanks again!
    Mike

    • http://cuppster.com Jason

      Thanks for that! I’ve updated the link in the instructions… Cheers!

  • http://www.xicom.biz/ web development services

    yeah..nice one

  • Pingback: NodeJS, Websockets and EC2 | Mike Newell

  • Anonymous

    Great article!!! I didn’t know about git hooks like that, until now, thanks!

    • http://cuppster.com Jason

      No problem! =D I admit I’m using rsync more often than git to deploy, but… there’s a time/place for everything!

  • http://www.kensingtondental.com/cosmetic_smile_smiledesign.shtml cosmetic dentistry

    loved it..

  • http://www.gofleet.ca/gps-tracking/vehicle-tracking-system.html gps vehicle tracking

    woaaah

  • http://twitter.com/vyumix vyumix

    Cool stuff.. thanks a lot for this write up

  • Anonymous

    for me supervisor didn’t work with error:

    $ sudo supervisorctl start node
    node: ERROR (no such file)

    until I changed “command node server.js” to “command /usr/local/bin/node server.js”

    i.e. full path to node.js

  • Anonymous

    Thanks! Definitely useful. I was clueless about a couple things until I used your post for reference. Haven’t done the Git part yet, but I did complete my proxy with nginx and your method of starting up node with supervisord with multiple node instances.

  • Guest

    Thanks! Your post made it easy for me to complete my project. I too kit one step further with using nginx as reverse domain proxy, also multiple nodejs instances with supervisor. Again, great article. Can not wait to view more!

    • http://cuppster.com Jason

      Cool! Glad to know it still works… there’s a more update-to-date AMI available, but I’m still using supervisor and nginx with my projects. To push code, I’m using git and rsync now instead pushing to a server repo and using commit hooks. That recipe is here: http://cuppster.com/2011/11/15/using-git-and-rsync-to-remote-install-code/

      There’s also a way to use git and rsync while maintaining timestamps on files (so rsync doesn’t always copy every tracked file) — a blog post on that is coming shortly =)

  • Christopher McCulloh

    Be sure that you open up inbound ports 22 and 80 in your security group in EC2. I couldn’t figure out why it was working and then I realized my EC2 firewall was blocking the requests. Yes, I’m a newb.

    For other newbs, how you do this is by logging into EC2, clicking “Security Groups” under “NETWORK & SECURITY” (on the left), selecting the group you used/created for your ubuntu instance, clicking the “Inbound” tab (at the bottom) and making a new “Custom TCP rule” with the port “20″ (click Add Rule) and then make another “Custom TCP rule” with the port “80″ (click Add Rule) and then click “Apply Rule Changes”. There may be another way, this is just how I did it…

  • Afeez Aziz

    Hi Jason,

    Thank you very much for writing this tutorial. I really appreciate the ability to use AWS as my node.js server. But i cannot push my git straight to AWS. Instead i have to push my code to Github first then using the CLI on the AWS, i have to clone the code into the development machine.

    I think this has something to do with the public key something. If you could help me on this, it would be appreciated.

    • http://cuppster.com Jason

      What kind of error are you getting during the git push to your server? If you can ssh to the server, you should be able to use git against it… you don’t need github.

      • Afeez Aziz

        Oh, it was a Public Key thinggy but i managed to solve the problem. Thanks by the way! =)

  • http://icewhite.us/ Ice White

    Thanks!

  • Amit

    Just what I needed and even more. thank you so much for this!

  • Pingback: Devops Weekly Digest 02/23 - pydelion.com