Essential Guide to the Twelve Factor App

Let’s say you have an idea that is game changing. Let’s assume the idea is Whats app, Facebook or Instagram or any other app you can assume.

Quick Question

What would you expect from any of those popular social media apps or any SAAS (Software-As-A-Service) apps apart from their core functionality..?

Obviously we expect the application to be

Portable: It means to run the app on any cloud platform or server seamlessly without making any changes in it.
Continuous Deployment: It means, No problem of updating an application from its

Development → Staging → Production

It simply says that we should easily update and deploy the changes in the production.
Scalability: It should be able to handle the traffic i.e number of users. The app should not crash when some 10 crore people use it.
Modern Cloud Platforms: The app should run or support all the modern cloud platforms like Amazon Web Services, Azure, Google Cloud Provider.

Definition

A Twelve-Factor App is a methodology to build an application or SAAS apps. These best practices are designed to enable applications to be built with portability and resilience when deployed in the web.

As the name suggest, we have 12 best practices that we should follow to build an app that is portable, scalable and resilient. Let’s see the 12 best practices

The Twelve-Factor App practices:

Codebase
Dependencies
Config
Backing Services
Build, Release and Run
Processes
Port Binding
Concurrency
Disposability
Dev / Prod Parity
Logs
Admin Processes.

Let’s look at them one by one with real-life example:

I hope you are familiar with python flask app or any node js app, with that said let’s discuss them one by one:

1. Codebase

Codebase is defined as, there should be single codebase created in a seperate folder or repository. The same codebase should be used for multiple deployments.

Let’s say you have an Node Js app, The app is in you system on localhost and needed to be developed.
Obviously one person won’t develop the app right, but the app is our system, How can others come and collaborate with us in the project. Here comes the tool called Git.
Git is a version control system and GitHub is the place where our projects are hosted. Authorized people can come and contribute to the project.(refer the image below)

As the first principle says, We should have one single codebase for a single project or app and the same app should be used for multiple deployments.

2. Dependencies

A Twelve-Factor app never relies on implicit existence of packages.

In Node Js app, Let’s say we are creating an express app. We need the dependency of express, We will install the dependency running
```
  npm install express
```
This will install dependency in the node_modules, where all the dependencies are present. In development we can use node_modules to develop app, but when deploying the application we should delete the node_modules as it increases our project size.
We should specify all our dependencies along with the version in which the app has developed, because all the dependencies keep updating, any change in the version of the dependency may affect the application to stop or crash.
As we said all our dependencies should not be installed implicitly, we should specify their name along with the version in seperate file. Thankfully in Node JS we have package.json file where all the dependencies are listed and then we say
```
  npm install
```
which results in downloading all the dependencies specified in package.json.

Package.json looks something like this:

  {
    "name": "Instagram",
    "description": "make your package easier to find on the npm website",
    "version": "1.0.0",
    "scripts": {
      "test": "echo \"Error: no test specified\" && exit 1"
    },
    "dependencies": { // These are the dependencies
      "body-parser": "^1.17.1",
      "express": "^4.15.2",
      "express-validator": "^3.1.2",
      "mongoose": "^4.8.7",
      "nodemon": "^1.14.12",
    },
    "keywords": [],
    "author": "",
    "license": "ISC",
  }

For python we should specify them in a file, generally it will be requirements.txt and then we just run
```
  pip install requirements.txt
```

Requirements.txt looks something like this:

  flask==1.0.0
  django==3.5.2
  numpy==1.2.3

Let’s say we have two node js apps
Both apps required different version of node Js, Hence it requires isolated environments to use.

In python we have virtual env (VENV) which can isolate an apps environment and ensures that no other dependencies interfere with our application, The same dependencies are consistently applied among all the stages of development and production.
What if we want the dependency outside the pip or python such as cURL. Here comes Docker.
Docker containers are kind of virtual machines but they are not..! isolated environments for running any app. It’s an self contained environment that is isolated with the host system, more efficient and reliable way to manage dependencies now.

We need to specify a docker file in an app to convert into an image and run it. Let’s write a sample docker file for Node Js app:

  FROM node:10-alpine
  # instruction to set the application’s base image
  RUN mkdir -p /home/node/app/node_modules && chown -R node:node /home/node/app
  # create a directory and give permissions
  WORKDIR /home/node/app
  # Set's the working directory
  COPY package*.json ./
  # Copies the package.json
  RUN npm install
  # Runs the npm install command to install all the packages
  COPY --chown=node:node . .
  # Copies the application director with permission
  EXPOSE 8080
  # Exposes the app to port 8080
  CMD [ "node", "app.js" ] 
  # starts the application

We run this docker file by docker build filename , This will create an image
docker run image_name we run the image to run an app.
Here the dependencies are installed explicitly through docker, which satisfies the second principle.

3. Config

Configuration that varies between deployments should be stored in the environment.

When we are building an app we will have some configuration data such as port values, api_keys, secret_keys, database_urls, which we do not want to expose them.
When deploying the application to different environments such as development, production and staging, which may use different instances require changes the host and port values etc…This is not a good practices and can lead to inconsistency and errors when deploying.
These configuration data should be stored in a seperate file called .env. So that we can hide the sensitive information.
```
  PORT=3000
  API_KEY="twelvefactorapp"
```
This allows us to use different configurations for different deployments such as testing, staging and production
Additionally, It makes it possible to make it open-source the need for any code level changes without exposing any sensitive information to the public.

4. Backing Service

All backing services are treated as attached resources and attached and detached by the execution environment.

Any services that our application uses over the network is known as backing services such as redis.
We use SMTP for sending an email which is an external service, we use the service outside the application.
We should be able to point to Local → Cloud or Cloud → Local for any instance of any backing service, It should work.

5. Build, Release & Run

The Twelve-Factor App uses strict seperation between the Build, Release & Run stages.

Let’s we wanted to add a new feature in our app, As this is simple project we can do it very quickly, But in real we will have huge codebase, We pushed a Commit → Testing → Deployment, This is possible when we have different stages of build & run. Let’s just walk through the phases of the release cycle:
Every minor or major change in code will result in new build process, release version & run phase.

6. Processes

Twelve-Factor processes are stateless and share nothing

Sticky sessions are the violation of the twelve-factor and should never be used or relied upon.

12-factor process are stateless and shared nothing, Let’s say we have decided to add a new feature to our app to show visitor count on our website. Every time a new visitor visits our page count increases
This works well when we have one process running because the visit count is stored in the memory of the process.
When we run the app in different servers multiple process run, all have their own version of this variable stored in them.
Same goes wit the session info, If this info is stored in the process memory or local process memory, when a future request of that user is requested to another process. The user may be considered logged out as the session info is not available in other processes.
So the session info or visit count should not stored locally instead stored in a place that can be easily accessed by all processes and it doesn’t matter which process a user routed to, It’s as if all requests are handled by the same process as all the processes have access to the same set of data.
we use some backing service such as Redis.

7. Port Binding

Twelve-Factor app is completely self-contained and does not rely on specific web server to function.

We know that locally we can access our app from http://localhost:3000 it’s as simple as that.
By default Node Js runs on PORT 3000
If we are running multiple instances on the same server we should be able to bind the port to other port servers such as 3001, 3002, etc…
Now our app exposes http as a service by binding to a specific port and listening for incoming requests on that port, unlike traditional web application

8. Concurrency

Twelve Factor app should be scalable horizontally. The application itself should built in this way by keeping in this mind.

When we containerize our application and run it. This executes one instance or one process of our application and this instance of our application will be served to several users visiting our site.
We can scale up the application vertically by increasing the resources in the server, This is not good practice because the number of resources are limited.
By vertically scaling we will eventually hit the downtime of server or maximum usage of resources at some point.
But by horizontally scaling we can add up new servers and spin up more containers or instances of application today with great ease. we can then have a load balancer in between the servers to balance the load among the different instances of the application.
To make our application this way we need to make our application stateless.

9. Disposability

Twelve-Factor app processes are disposable, meaning they can be started or stopped at a moments notice.

Apps use horizontal scaling and instances of the app in the servers are disposable, meaning can be terminated and initiated in seconds.

Why two signals, SIGTERM and SIGKILL to stop a container, we want enough time to shutdown the application itself gracefully or application may be be processing request from 100 of users at a time might get error suddenly.
A graceful shutdown allows the application enough time to stop accepting new request at the same time complete processing for all existing requests. This way users who are waiting for a response from the app are not impacted, for this app should be able to handle the SIGTERM signal to avoid any unexpected dataloss or resource loss, That can occur if the process is terminated forcefully via SIGKILL signal.

10. Dev / Prod Parity

The Twelve-Factor app is designed for continuous deployment by keeping the gap between development and production small.

The developer should resist the urge to use different backing services between development and production.

It may take weeks or months for an application to go from development → Production, because different people develop and different people deploy it, In development we use light database such as sq.lite and production will be such as postgressQL.
So there is a time gap from an app to go from dev to prod env, The problem with that is there may be other things changing in the app from the time it was developed to time it goes to production, that might affect the functionality in the change.
There will be personal gap ( Developer takes time to develop ) and tools gap ( Migrating from devtools to prod tools ). This is where the 12-Factor app comes in.
The developer who writes code is also involved in deploying and watching production environment is called DevOps Engineer. We mush aim to keep tools same as much as possible.
With modern tools like docker, making it easy to setup development env’s, This shouldn’t be hard to achieve.

11. Logs

The Twelve-Factor app never concerns itself with routing or storage of output streams

Store logs in a centralized location in a structural format

Logs requests, port starting, application address, If any errors in code this way we can trouble shoot issues or some failures
The problem with this approach is since we are living in the world of containers they are isolated and container may be killed any time and logs are lost.
More over the application is coded to write to specific log files on the file system, which is also another problem. Now in other cases applications try to push logs to certain central logging system like fluentd. while centralized management of logs encouraged tightly coupled specific logging solution to the app itself is discouraged. So we do not want our apps to be stuck to single solution.
It should write all its log to a stdout or to a local file in json forma, that can then be used by a agent to transfer to a centralized location for consolidation and analysis purposes.

12. Admin Processes

Administrative tasks should be kept seperate from the application process, specifically it recommends that anyone of periodic administrative task, such as DB migration or Server Restarts should be run as a seperate process or application.

They should be automated, scalable and reproducable.

Conclusion

I think thats all about the 12-Factor app. It is the standard rules that are followed while developing an application. You can read more about it 12factor.net

Subscribe to my email for more such blogs related DevOps, Cloud and Productivity..!

Master the Twelve Factor App: A Complete and Easy Guide

Table of contents