Gabe's Code

Stuff I've learned along the way


profile for Gabriel Luci at Stack Overflow, Q&A for professional and enthusiast programmers

Staticman: Setting up my own instance

I decided to use Staticman for comments on this site for reasons I described in my other article. Staticman has a public API that anyone is free to use. However, it’s in trouble: so many people are using it that it has started to hit the maximum usage that the GitHub API allows. So I decided to create my own instance of Staticman.

Of course, nothing is easy, so I had a ton of trouble setting it up. A couple issues I faced were:

So let me describe my adventure. Feel free to skip the sections that aren’t relevant to you.

Install NodeJS

Staticman is written in NodeJS, so I had to install it on my server. The VPS I already have ($15/year from ChicagoVPS) has only 128MB of RAM and runs Ubuntu Trusty. Not a whole lot of power, and an old version of Ubuntu, but it has served me well for the tiny websites I have running on there.

I found out the hard way that the version of NodeJS in the Ubuntu Trusty repository is a really old version that didn’t work at all with Staticman. Instead, I needed to add a repository to apt before installing NodeJS.

The process is described here. The instructions they give downloads and runs a script that attempts to add the correct repository for the version of NodeJS you want and the version of Ubuntu you’re running. But the error messages are sort of lacking, so it took me a while to figure out that there is no package for NodeJS 11.x for Trusty. So I had to install version 10.

The effect of all that was to create a file at /etc/apt/sources.list.d/nodesource.list with this content:

deb https://deb.nodesource.com/node_10.x trusty main
deb-src https://deb.nodesource.com/node_10.x trusty main

You could just do that yourself if needed, or just browse to that URL (and change the version number) if you need to experiment with which version is available for your version of Ubuntu (if you’re dealing with an older version of Ubuntu like me).

Once the repository is added, you can update and install it:

sudo apt-get update
sudo apt-get install nodejs

Cool. That finally worked.

Many NodeJS apps (including Staticman) rely on an environment variable called NODE_ENV, which indicates the environment you are running in (development, production, etc.). To add a system-wide environment variable in Ubuntu, edit the /etc/environment file and add this line (I only have one server for this, so I’m calling it “production”):

NODE_ENV=production

Download the Staticman code

The Staticman project on GitHub does have some instructions on Setting up the server, and it makes it sound easy. It wasn’t so easy for me.

I knew I wanted the code in /var/www/staticman, so I change directories to /var/www (the staticman directory will be created when we clone the repository):

cd /var/www

The Staticman instructions say to run this:

git clone git@github.com:eduardoboucas/staticman.git

However, that failed. Not being terribly experienced with Git, it took me a while to figure out that when you use git@github.com:, it is actually using the SSH protocol to connect, which requires that you already have an RSA key created on your computer and added to your GitHub account so it can authenticate you. We will actually end up doing that later, but for now, it makes a whole lot more sense to just use HTTPS:

git clone https://github.com/eduardoboucas/staticman

npm install Killed!

The next step is to run npm install to download all the dependencies. Well that just didn’t work. It ran for a couple seconds, then gave me the message: Killed

That, too, took me a while to figure out. It turns out that’s what happens when it runs out of memory. Y’know, because my VPS only has 128MB of RAM.

A popular recommendation is to just create a swap partition (or increase the size if you already have one), but my VPS didn’t allow me to have a swap partition.

I ended up finding this Gist that someone made for just this purpose. It’s a bash script that does everything that npm install would do, but without using as much memory. Someone who commented in that Gist provided the commands to download and run that script:

curl -o npm-f3-install.sh https://gist.githubusercontent.com/SuperPaintman/851b330c08b2363aea1c870f0cc1ea5a/raw/4d3e792c6a54def095f451eeedc50d33ae361339/npm-f3-install.sh
sudo chmod +x npm-f3-install.sh
./npm-f3-install.sh all -s

I added the -s to run it in silent mode, since even outputting to the console takes memory, which we want to avoid.

Finally, it worked.

Now let’s switch gears for a moment and

Create a new GitHub “bot” account

You could let Staticman use your own personal GitHub account, and I thought about doing that, but decided against it for one main reason: If this gets hacked or otherwise goes haywire, I don’t want my personal account affected.

GitHub does allow you to create a separate account for automation. To quote their Differences between user and organization accounts article:

User accounts are intended for humans, but you can give one to a robot, such as a continuous integration bot, if necessary.

A “bot” account on GitHub really is no different than any other user account. So to create it, just log out of your own GitHub account, then go through the sign-up procedure again. I just appended “-bot” to my normal username to create @gabeluci-bot.

The config file

Back at our server, we need to create our config file for Staticman and put the right values in it. Since we set the NODE_ENV variable to “production”, I created a production config file, based on the sample config:

cp config.sample.json config.production.json

The Staticman documentation simply says,

Edit the newly-created config file with your GitHub access token, SSH private key and the port to run the server.

So let’s look at each of those.

GitHub Access Token

GitHub’s documentation for creating an access token is pretty good, so do that and copy the new access token into the githubToken property in the config file.

RSA Key

Wait, I thought you said SSH key? I’ll explain.

The Requirements section of the Staticman docs says you need:

An SSH key (click here to learn how to create one)

That link goes to GitHub’s documentation on creating an RSA key and associating it to your GitHub account so that it can be used for authentication when connecting to GitHub via SSH. But Staticman doesn’t use it for that at all. Staticman only uses this key for encryption, nothing else.

At first, I followed GitHub’s documentation and got burned because Staticman (or more-specifically, the node-rsa module) expects the key to be in PEM format. But the instructions that GitHub gives you does not create it in PEM format. It took me a while to figure that out and convert my key to PEM format.

So save yourself some time and ignore GitHub’s documentation on the matter and just create an RSA key in PEM format:

openssl genrsa -out key.pem

Take the contents of key.pem, remove all line breaks (or replace them with \n - it doesn’t really matter which) and paste that into the rsaPrivateKey property of your config file.

Port

The port you choose is up to you. I already had Apache running on my server, so I knew I would have to use Apache as a proxy between the outside world and Staticman, so I chose port 8080.

At this point you should be able to run npm start and it should work! If so, hit Ctrl+C. There’s more work to do.

Since I will be using Apache as a proxy, I want to prevent the outside world from directly accessing Staticman on port 8080, I edited line 154 in server.js so that it would only listen on the local loopback IP (127.0.0.1). This ensures that only Apache can access it:

this.server.listen(config.get('port'),'127.0.0.1', callbackFn)

I did end up changing some other code in Staticman, which I described in my other article.

Make Staticman a Service

We want Staticman to start when the server starts and restart if it fails for whatever reason - just stay running! If you search, you will find various ways of doing this, like installing an NPM module called PM2, or using systemctl, but both of those required me to install something. I already knew I could run services (like Apache) with service apache2 start, so I wanted to set this up the same way.

Ubuntu’s service manager is called Upstart. Once I figured that out, I could search for the right thing and found this. It was as easy as creating a file called /etc/init/staticman.conf with this content:

start on filesystem and started networking
respawn
chdir /var/www/staticman
env NODE_ENV=production
exec /usr/local/bin/node /var/www/staticman/index.js

Then I could run the service:

service staticman start

Setup Apache as a Proxy

To let the world access my Staticman instance, I need to tell Apache to proxy any traffic looking for the domain name I setup (comments.gabescode.com) to Staticman. To do that, I created a file called /etc/apache2/sites-available/comments.gabescode.com.conf with this content:

<VirtualHost *:80>
    ServerName comments.gabescode.com

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/comments.gabescode.access.log combined

    DocumentRoot /var/www
    ProxyPass /.well-known/ !
    ProxyPass / http://localhost:8080/
    ProxyPassReverse / http://localhost:8080/
</VirtualHost>

The key is the last couple lines, which tells Apache to proxy the traffic to port 8080 (the trailing slashes are important, by the way).

These two lines:

DocumentRoot /var/www
ProxyPass /.well-known/ !

are for Let’s Encrypt, which I’ll talk more about later. That ProxyPass directive must come before the others. It tells Apache to not proxy any requests to the .well-known directory, which Let’s Encrypt uses to verify you own the domain. The DocumentRoot directive ensures those requests will serve any files in /var/www/.well-known.

Once that was setup, I could enable the site:

a2ensite comments.gabescode.com
service apache2 reload

At this point, I could go to http://comments.gabescode.com and see the message “Hello from Staticman version 2.0.0!”

Awesome!

Create the SSL Site

It’s a good idea to use HTTPS for this. We aren’t accepting credit card numbers, but we are submitting people’s email addresses, and it would be unfortunate if someone started intercepting those.

I was already using Let’s Encrypt for my other sites, so I already had acmetool installed. If you don’t, look at that documentation for instructions on installing.

To request a certificate, it was as easy as running this:

acmetool want comments.gabescode.com

That will create a special file in /var/www/.well-known/acme-challenge, then make a request to get the file through the domain you are requesting: https://comments.gabescode.com/.well-known/acme-challenge/{file}. That verifies that you own the domain. Then the certificate will be created in /var/lib/acme/live/comments.gabescode.com.

Note: If you just installed acemetool, make sure you setup a cron job that will run acmetool reconcile once a month so that it will automatically renew your certificates.

Now we can create our SSL site. To do this, I created a new site configuration file: /etc/apache2/sites-available/comments.gabescode.com.ssl.conf. It looks very similar to the last one, except for the port and the SSL-specific stuff at the end:

<VirtualHost *:443>
    ServerName comments.gabescode.com

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/comments.gabescode.access.log combined

    DocumentRoot /var/www
    ProxyPass /.well-known/ !
    ProxyPass / http://localhost:8080/
    ProxyPassReverse / http://localhost:8080/

    SSLEngine on

    SSLCertificateFile      /var/lib/acme/live/comments.gabescode.com/cert
    SSLCertificateKeyFile   /var/lib/acme/live/comments.gabescode.com/privkey
    SSLCertificateChainFile /var/lib/acme/live/comments.gabescode.com/chain
</VirtualHost>

Now I can enable the site:

a2ensite comments.gabescode.com.ssl
service apache2 reload

And test that I can visit the site: https://comments.gabescode.com.

Looks good!

Debugging

If you run into problems at any point, stop the service (service staticman stop) and just run npm start from the console. Then if any errors happen, you’ll see them on the console. This will be handy for debugging any errors later when you start using it.

Keep Other People Out

Staticman is designed to be a public API. As such, once you setup a Staticman instance, anyone can use it for their own website, if they know about it. If you’re ok with that, cool! But as discussed, my VPS is strapped for resources, and I just gave you all the information you need to know to use my instance :) so I decided to lock it down. I did that by adding this line to the Apache config files for both sites (HTTP and HTTPS):

ProxyPass /v2/connect !

That should go before the other ProxyPass directives.

A crucial step in setting up Staticman for your site is to make a request to /v2/connect/GITHUB-USERNAME/GITHUB-REPOSITORY, which tells Staticman to accept the invitation to become a collaborator in your GitHub repository. Adding this ProxyPass directive ensures these requests never actually make it to Staticman.

Just make sure you do this after you have completely setup your comments, since you will need to use this in your own setup.

In my other article, I show how I set up this site to use my new instance of Staticman.

3 comments

Leave a comment

Your email address is used to display your Gravatar, if applicable, and subscribe you to replies using the Mailgun web service, which you are free to unsubscribe from when you get any emails. Your email address will not be displayed publicly or shared with anyone else.