Getting Started with Docker Part 2: Building Images and Docker Compose

Published 11/02/2018 02:43 PM   |    Updated 11/16/2018 02:46 PM
This article originally appeared on April 28, 2017.
 
This is part two in a series on getting up and running with Docker. It would be best to walk through part one before continuing, but if you’re already familiar with Docker and running containers, feel free to pick up here. In this article, we’ll cover:
 
  • Building an image
  • Basic container orchestration with Docker Compose
  • Architecting an Application with Compose
 
Feel free to open your command prompt and favorite text editor before we get going. All three sections walk you through some hands-on activities. These exercises assume some familiarity in working with Command-Line Interface (CLI) tools and some basic knowledge of Linux system administration. You can get by without much of either.
 

Building an image

 
Now that we know a little bit about running containers and the different options available to us, let's try building our own image that we can run, distribute and deploy.
 
Building custom images is a great way to share exact environments and build tools with fellow developers, as well as distribute applications in a consistent way. If we publish that image to a public Docker registry such as Docker Hub, other individuals can pull our image down and run it on their machines. 
 
Those containers will have the exact replica of operating system, programs, configuration and application files that we built into the image. If someone has a compatible Docker host, he or she can run our image, wherever that environment is.
 
If you’re planning to use Docker in a public environment (say, a shared testing environment or production), custom images are probably the best way to deploy your application. The image can easily be dropped into any environment from an individual's machine running the Docker CLI, or via a continuous integration/continuous delivery pipeline tool, without having to worry about application build steps, dependencies or system configuration changes.
 
We create Docker images by writing a kind of script called a "Dockerfile." Dockerfiles have a relatively short list of commands at their disposal and allow us to specify how to build one image from another. There are several base images available on Docker Hub, and we can also derive our image from an application-specific image such as nginx if we want. We’ll start from the Ubuntu image and create our own nginx image to use for the purpose of this exercise.
 
Let’s create a new directory on our desktop or wherever you keep code projects. Now, we add a new file to this directory: Dockerfile. There is no file extension for Dockerfiles. Open this file in your favorite editor and add the following lines:
 
FROM ubuntu:xenial
RUN apt update && apt install -y nginx
EXPOSE 80
COPY . /var/www/html
CMD ["nginx", "-g", "daemon off;"]
 
Let's break down the Dockerfile line by line:
 
  • FROM tells Docker what image to start with. We start from the ubuntu image, specifying the xenial tag after the colon. The tag is a version identifier and, in this case, specifies the Ubuntu 16.04 release.
  • RUN allows us to execute commands and programs inside the container. This is frequently used to install packages like our file does, and it can be used to do a myriad of other tasks (e.g., create and change permissions of directories, download external files, run other setup scripts). Notice that we use the && operator to run the apt commands sequentially. This helps ensure the apt install command succeeds in getting the latest version of the package, and helps us manage our image size because of the way Docker executes RUN commands. We don’t need to get into the why here, but you can read up on the subject in the Docker docs on Dockerfile best practices.
  • EXPOSE lets Docker know which ports on the container should be exposed to Docker networks and available for mapping to the host. In our case, we just want to expose port 80 so that nginx's default configuration will work.
  • COPY copies files from your local directory to the specified path in the container. Make sure to use an absolute path for the container target. For our image, /var/www/html is the default path that nginx uses to serve files.
  • CMD instructs Docker to treat the specified command and subsequent arguments as the primary process in the container. When this process terminates, Docker stops the container. Our instructions tell Docker to start nginx in the foreground and let it govern the lifecycle of the container.
Before we build our image, let's add a default HTML page for the COPY command to incorporate. Create an index.html file in the same directory as Dockerfile and add the following content:
 
<html>
<head>
<title>Docker Hello World</title>
</head>
<body>
<h1>Hello World from the nginx Docker Container!</h1>
</body>
</html>
 
We’ll know our container launched successfully when we run it and can see this page in our browser. Now, we build the image by running docker build -t my-nginx. inside the directory with Dockerfile. The -t flag provides a “tag,” or name to use for your image within your Docker host, and the final argument “.” specifies the directory with the Dockerfile we want Docker to build. 
 
In our command prompt, we should see a lot of output from the build process, including output from the apt command (which is quite verbose). Look for the following lines indicating the steps in our Dockerfile that Docker is executing:
 
Step 1/5 : FROM ubuntu:xenial
...
Step 2/5 : RUN apt update && apt install -y nginx
...
Step 3/5 : EXPOSE 80
...
Step 4/5 : COPY dist /var/www/html
...
Step 5/5 : CMD nginx -g daemon off;
Successfully built 96047713afe8
 
This process may take a few seconds to a few minutes. When Docker is done building your image, we should see a line starting with "Successfully built," and Docker will return us to the command line. Now, we have our own Docker image named my-nginx stored in our Docker Engine's cache of images. 
 
Let’s run our new image with the command docker run --rm -d -p 8080:80 --name my-nginx my-nginx. Just like before, this tells Docker to throw away the container when it stops, run it in the background, and map port 8080 on the host to port 80 on the container. The name of the container is my-nginx just like the image so that we can easily find it. Run docker ps to check on our new container:
 
CONTAINER  ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                  NAMES

3f35e760ae10        my-nginx            "nginx -g 'daemon ..."   3 seconds ago       Up 3 seconds        0.0.0.0:8080->80/tcp   my-nginx
 
If we open our browser, navigate to http://localhost:8080 and refresh the page, we should see the message we wrote earlier in index.html: "Hello world from the nginx Docker container!" When we’re done testing the container, we can stop it with docker stop my-nginx.
 

Basic container orchestration with Docker Compose

 
A common discussion point when talking about Docker is "containerizing" applications. Containerizing essentially refers to the practice of porting a non-container-based application (i.e., traditional virtual machine or native platform environment) and reworking different components to run inside containers.
 
Almost all web applications consist of at least a database, an application runtime and a web server. Take the LAMP stack, for example:
 
  • Linux host
  • Apache web server
  • MySQL database
  • PHP process that executes the application
 
Porting this across to Docker containers might look like:
 
  • A MySQL container to run the database
  • A PHP-FPM container that includes the application source and connects to the MySQL container
  • An Apache container that runs the web server and connects to the PHP-FPM container
 
With this structure, each major component of the application runs in its own isolated environment with explicit connections to dependent containers. This allows components to be scaled independently as needed and upgraded on their own. For example, to deploy a new version of the PHP application, you just need to rebuild and redeploy the PHP-FPM image, not the whole stack. Similarly, Apache could be upgraded to address a security vulnerability without impacting the database or application configuration.
 
There are many open-source container orchestration tools out there, but recent versions of Docker come bundled with a Docker-native tool called Docker Compose. Compose allows you to describe an application's set of containers declaratively via a YAML file. Invoking Compose commands on the command line instructs Docker on how to manage the application's containers, including respecting dependencies between containers, mapping ports and negotiating rolling upgrades.
 

Architecting an application with Compose


Let's try writing our own Docker Compose file and fitting two system components together with it. The different containers Compose runs as part of your application are called "services." We’ll set up two simple services to demonstrate some basics of Compose.
 
Note: If you want to skip ahead or have a reference to compare your work against, check out the Docker 101 JSON API repository on GitHub. All of the files we write in the blog post are included in the repository, so you can always refer to it if you get stuck.
 
We start by creating a new directory to work in, calling it docker-101-json-api. Inside that directory, create an api directory and a webserver directory. Now, add a docker-compose.yml file in the root directory. Our file structure should look like this:
docker -101-json-api
| docker-compose.yml
+-- api
+-- webserver
 
The docker-compose.yml file will describe how our two services should be configured and relate. We’ll come back to this, but first, we should set up the two images we’ll use. Inside the webserver directory, let’s create a new Dockerfile and a nginx.conf file. Our Dockerfile will be very simple but allow us to provide custom configuration to nginx. Add the following lines to Dockerfile:
FROM nginx
COPY nginx.conf /etc/nginx/nginx.conf
 
All we’re doing is building our image from the nginx base image and overriding the default configuration file with our own. Add the following to nginx.conf:
 
user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    server {
        listen 80;

        location /api/ {
          proxy_pass http://api:8888/;
        }
    }
}
 
Almost all of this is taken from the default nginx.conf file inside the nginx image. We just modify the server block at the end to listen on port 80 (the default) and provide a proxy rule to direct all HTTP requests beginning with /api/ to http://api:8888/. This is the other service we’ll build and deploy with Compose.
 
Our api service is a simple Node app that serves a JSON file. We could evolve this to be a full Express-based application, but we just want to show two different images working together. Let’s start by adding a package.json file to the api directory with the following contents:
 
{
  "name": "docker-101-json-api",
  "version": "1.0.0",
  "description": "Sample Node JSON API server app.",
  "scripts": {
    "start": "static . --host-address \"0.0.0.0\" --port 8888"
  },
  "author": "your.email@address.com",
  "license": "MIT",
  "dependencies": {
    "node-static": "^0.7.9"
  }
}
 
This pulls in the node-static package to allow us to serve our JSON file and defines a start script for our image to run with npm. Download the movies.json file and add it to the api directory as well. These two files are enough for our Node API to run. Now we need to build the Docker image, but before we do, let’s add the following to a .dockerignore file:
 
node _modules
npm-debug.log
 
This prevents the node_modules directory from being copied into the image if we install them locally during development. Modules will be installed natively in the container when the Docker image is built. Add the following to a Dockerfile in the api directory:
 
FROM node:6
# Create app directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
# Install app dependencies
COPY package.json /usr/src/app/
RUN npm install
# Bundle app source
COPY . /usr/src/app
EXPOSE 8888
CMD [ "npm", "start" ]
 
We don’t need to get into the details of these instructions, especially if you’re not a Node developer, but the file follows a convention put forth by the Node group in its blog post Dockerizing a Node.js web app. Essentially, the build file installs the npm packages natively in the container, then copies over application source. We expose the port we need for our app and run npm start to kick things off.
 
Our full directory structure should now look like this:
docker -101-json-api
| docker-compose.yml
+-- api
    | .dockerignore
    | Dockerfile
    | movies.json
    | package.json
+-- webserver
    | Dockerfile
| nginx.conf
 
Now that we have Dockerfiles for both images set up, let's get back to the docker-compose.yml file. This YAML file serves as a manifest or blueprint for all of the containers in our application. These containers don’t all have to have custom images. In our case, docker-compose.yml looks like this:
 
version: "2"
services:
  api:
    build: ./api
    environment:
      NODE_ENV: production
  webserver:
    build: ./webserver
    ports:
      - "80:80"
    links:
      - api
 
We’ll review just a couple of features, but you can also review the full Compose file reference if you need to. The version property identifies which version of the Compose specification the file was written for. Version 3 is the newest version of the specification, but we only need features included in version 2, so we’ll stick with that for now.
 
The main property of interest is services. The services property describes the containers you want Docker Compose to run for you. The next property down gives each container a name. Incidentally, this is the same name we use to address one container from another within the network Docker creates. Within the definition of a service, we define the following properties:
 
  • Build tells Docker where the Dockerfile is for our image. If we were using an out-of-the-box image, we would instead specify image: mongodb. We can provide the same format for the image name as we do for the FROM directive in a Dockerfile.
  • Environment specifies environment variables that will be set for the container when it runs.
  • Ports identify port mappings between the container and the Docker host. This is just like the -p flag we pass to docker run.
  • Links specifies other services that should be exposed to this service via the network. For us, we want the Node app to be exposed to nginx, but only nginx is exposed outside the Docker host. Back in our nginx.conf file, we reference the api service with http://api:8888/. Docker adds services we name in the links property to the target service as named hosts so they’re easy to address.
 
Now let’s navigate to the root directory and run docker-compose up -d. This command performs several steps for us:
 
  • Check if any of the services you declare need to be built or rebuilt.
  • Build custom images for services as needed.
  • Set up internal networks.
  • Namespace all of the resources for your application in your Docker host.
  • Start all of your services in the background.
 
Both of our images will probably be built when we run this command. We may see a lot of output from the build process and, when it finishes, Compose will return us to the command line. We can check the status of our services with docker-compose ps.
 
      Name                     Command          State              Ports
-----------------------------------------------------------------------------------------
docker101jsonapi_api_1         npm start              Up      8888/tcp
docker101jsonapi_webserver_1   nginx -g daemon off;   Up      443/tcp, 0.0.0.0:80->80/tcp
 
We can see in the console output that each container has a specific name and, like docker ps, the exposed and mapped ports are shown. In our browser, we should now be able to navigate to http://localhost/api/movies.json and view our movies.json file being served by Node through nginx.
 
We can run docker-compose stop to halt all of the services in our docker-compose.yml file. The docker-compose rm command will remove them from our Docker host, just like the docker rm command. There’s also a shorthand command, docker-compose down, that stops and removes everything created by docker-compose up.
 
Container orchestration can be a lot more elaborate, and you can quickly outgrow the functionality of Docker Compose by itself. At this point, you’re equipped with the basics and can tackle more complex tools as you need them.
 

Next steps


Now that you know more about containers and have some hands-on experience with Docker, you may want to try incorporating it into your workflow on a new or existing project. Here are some ideas to get you going:
 
  • Try containerizing different components of your application. Can you build a Docker image that hosts your application? Do you need multiple containers for different services? Maybe you can use Docker Compose to orchestrate a few containers to support your application.
  • Level up your knowledge of container orchestration by learning about Docker Swarm mode, Apache Mesos or Kubernetes. These tools take the principles behind Docker Compose and apply them to managing production-grade systems that automatically scale, restart and distribute multiple instances of your containers across many host nodes.
 
We’ve only begun to scratch the surface of container technology. Hopefully you now have enough base knowledge to begin working with Docker and exploring ways it can benefit your development workflow.

Is this answer helpful?