There are a couple of technologies I have recently become interested in, albeit for different reasons - Terraform and Hugo. Rarely does an opportunity present itself to me in which I can find a use case to try a couple of seemingly disparate technologies together. I decided to move my blog from WordPress on EC2 to a static website.
Here’s how I did it, and why.
Why Static Website?
Up until recently, I was hosting this blog on an EC2 instance running WordPress. I did not want to use WordPress.com or a similar managed hosting service, because I wanted to have finer-grained control over it. Even with a tiny EC2 instance, it seemed like overkill to have a full VM running WordPress to serve up a handful of static pages, and more expensive than needed.
I was turned on to the idea of static websites from reading through the Terraform documentation, which is built with Middleman. I found this to be a powerful way to manage product documentation, and started to research more about static website generators.
Terraform
Terraform is a configuration management technology for managing your infrastructure with code, with a stateful representation paradigm similar to how server configuration is commonly managed with technologies such as Chef, Puppet, Ansible, and others.
Hugo
Hugo is a super-fast static website generator written in Go, similar to Jekyll, Middleman, and others.
Goals
I wanted this exercise to be more than just a proof of concept for Terraform and Hugo. I had several things I wanted to improve over my EC2 hosted solution.
Cheaper Hosting
As far as cloud services go, file storage is among the cheapest. Most providers, including AWS, provide a simple website service on top of the file storage service, providing a cheap solution for hosting static websites. I chose Amazon S3.
High(er) Availabilty
Instances in EC2 (and in the public cloud in general, to be fair) are pretty volatile. They can reboot (or disappear) at random times. And if hosting a simple blog on WordPress on a single EC2 instance wasn’t overkill enough, adding multiple instances with load balancing across availability zones for high availability would be - more overkill?
Amazon S3 provides 99.99% availability, which is better than what you can expect from a single EC2 instance.
Simple Content Management
Now that I had somewhere to stash and serve up my soon-to-be static content, I needed a framework for producing, managing, and publishing content. One of the gripes I have with content management systems such as WordPress, is that the content is separated from the source. I didn’t want to be creating MySQL backups or XML exports and storing them somewhere else. I am more comfortable having my content be my source, so that I can version it using common revision control systems, like Git.
Simple Configuration Management
Even though an S3 bucket is simpler to manage than an EC2 instance, I wanted to have a way to manage it as well as read/write permissions to it. Terraform is a powerful technology that can manage complex infrastructures, but is also suited for simple configurations like mine.
Implementation
Directory Layout
The basic structure of the blog is shown below. I removed some of the detail to remove clutter:
├── Makefile
├── archetypes
│ ├── page.md
│ └── post.md
├── bin
│ └── admin
├── config.toml
├── content
│ ├── page
│ └── post
├── data
├── layouts
│ ├── partials
│ └── shortcodes
├── static
│ ├── css
│ └── images
├── terraform
│ ├── main.tf
│ ├── terraform.tfstate
│ ├── terraform.tfstate.backup
│ └── terraform.tfvars
└── themes
The Hugo source files are in the root, while the Terraform configuration resides in terraform/ directory.
Terraform Configuration
My Terraform configuration is a simple configuration that manages the following resources:
- S3 bucket - The website-enabled bucket that will host the static pages
- Publish User - The IAM user with read/write permissions to the bucket
First, I define the Terraform variables I want to use in my configuration, so that they can be separated from the configuration.
variable "region" { default = "us-east-1" }
variable "access_key_id" { }
variable "secret_key" { }
variable "bucket" { }
Then, I configure the Terraform AWS provider with AWS credentials that have sufficient privileges to perform the operations:
provider "aws" {
region = "${var.region}"
access_key = "${var.access_key_id}"
secret_key = "${var.secret_key}"
}
Next, I create the S3 bucket, configured with static web hosting:
resource "aws_s3_bucket" "site-bucket" {
bucket = "${var.bucket}"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::${var.bucket}/*"
}
]
}
EOF
website {
index_document = "index.html"
error_document = "404.html"
}
}
Next, I create an IAM user that will have read/write permissions to the bucket:
resource "aws_iam_user" "publisher" {
name = "${var.bucket}-site-publisher"
}
resource "aws_iam_access_key" "access-key" {
user = "${aws_iam_user.publisher.name}"
}
resource "aws_iam_user_policy" "publisher-policy" {
name = "FullAccess-${var.bucket}"
user = "${aws_iam_user.publisher.name}"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1471182003000",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::${var.bucket}",
"arn:aws:s3:::${var.bucket}/*"
]
}
]
}
EOF
}
Finally, I declare a few outputs to be used when publishing content via Hugo:
output "publish-user" {
value = "${aws_iam_user.publisher.name}"
}
output "publish-user-access-key" {
value = "${aws_iam_access_key.access-key.id}"
}
output "publish-user-secret-key" {
value = "${aws_iam_access_key.access-key.secret}"
}
And thats it! Below is the complete Terraform configuration:
terraform/main.tf
variable "region" { default = "us-east-1" }
variable "access_key_id" { }
variable "secret_key" { }
variable "bucket" { }
provider "aws" {
region = "${var.region}"
access_key = "${var.access_key_id}"
secret_key = "${var.secret_key}"
}
resource "aws_s3_bucket" "site-bucket" {
bucket = "${var.bucket}"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::${var.bucket}/*"
}
]
}
EOF
website {
index_document = "index.html"
error_document = "404.html"
}
}
resource "aws_iam_user" "publisher" {
name = "${var.bucket}-site-publisher"
}
resource "aws_iam_access_key" "access-key" {
user = "${aws_iam_user.publisher.name}"
}
resource "aws_iam_user_policy" "publisher-policy" {
name = "FullAccess-${var.bucket}"
user = "${aws_iam_user.publisher.name}"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1471182003000",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::${var.bucket}",
"arn:aws:s3:::${var.bucket}/*"
]
}
]
}
EOF
}
output "publish-user" {
value = "${aws_iam_user.publisher.name}"
}
output "publish-user-access-key" {
value = "${aws_iam_access_key.access-key.id}"
}
output "publish-user-secret-key" {
value = "${aws_iam_access_key.access-key.secret}"
}
Now that the configuration is setup, I need to supply values for the variables defined in the beginning. This is provided via the terraform/terraform.tfvars file:
region = "us-east-1"
bucket = "my-bucket"
access_key_id = "ACCESS_KEY_ID"
secret_key = "SECRET_ACCESS_KEY"
To view the changes that Terraform will apply, from within the terraform/ directory:
terraform plan
NOTE: The first time running this will show everything in the new state. Subsequent runs of terraform plan will show different output based on the state of the infrastructure.
To apply the changes, from within the terraform/ directory:
terraform apply
NOTE: If the above succeeded, the state will be saved in terraform/terraform.tfstate. Terraform will use this file when planning/applying subsequent changes.
IMPORTANT: The terraform/terraform.tfstate file contains the ACCESS_KEY_ID and SECRET_ACCESS_KEY values for the user created by Terraform. If you are hosting your source code in a public repository, you will want to store this file outside of your repository. See Additional Resources section below for how to store Terraform state files remotely.
Hugo Setup
I won’t get in to how to actually build sites with Hugo, as there are plenty of great detailed articles on the internet on how to do that, including the Hugo Docs.
Now that I can manage the hosting infrastructure, I need to publish the Hugo generated content to the S3 bucket. Building the static content via Hugo is extremely simple:
hugo -b http://my-domain-or-ip/
This will build the static content in the public/ directory. Now I need to push this content to the S3 bucket that will host our site. I will use the AWS CLI tools to synchronize the public/ directory with the bucket, as follows:
aws s3 sync public/ s3://my-bucket/
The AWS CLI requires AWS access keys to be configured (See Additional Resources). Because I am using access keys that belong to a user created by my Terraform configuration, I will use Terraform’s output command to retrieve the values from the state file at publish time. I will also use a Makefile to connect a few things for me:
Makefile
export AWS_ACCESS_KEY_ID := $(shell terraform output -state terraform/terraform.tfstate publish-user-access-key)
export AWS_SECRET_ACCESS_KEY := $(shell terraform output -state terraform/terraform.tfstate publish-user-secret-key)
.PHONY: build push
build:
hugo -b http://dotariel.com/
dev:
hugo server -t ghostwriter -b http://localhost:1313/ --buildDrafts
push: build
aws s3 sync public/ s3://dotariel.com/
Now, I can update my site and publish when ready:
make push
My next improvement will be to automate the deployment on commit ;)
Fun, right?