Devops
Day 18: 90DaysOfChallenge

Day 18: 90DaysOfChallenge

Docker for DevOps Engineers

Shilpi Ekta's photo
Shilpi Ekta
·

8 min read

Table of contents

Docker Compose

Docker Compose is a tool that is used for making multiple containers and by using YAML file connections can be established amongst these multiple containers. All the services like build, connect, start, stop, etc can be done by running a single command. Compose works in all environments; production, staging, development, testing, as well as CI workflows.

The key features of Compose that make it effective are:

  • Have multiple isolated environments on a single host

  • Preserve volume data when containers are created

  • Only recreate containers that have changed

  • Support variables and moving a composition between environments

YAML

YAML stands for Yet Another Markup Language. It is a human-readable data serialization language, just like XML and JSON. It is used for writing configuration files for different DevOps tools, programs and applications. It uses Key-Value pairs separated by colon(:) followed by space. Ex-

 name: John Smith
 age: 23
 country: USA

There's no need to give double quotes for String as YAML understands on its own.

We can also create an Object. Ex- The value server1 gets mapped (or assigned) to the nameOfServer key, the value 20GB gets mapped to the capacitykey. Altogether, these create an object of server.

name: John Smith
age: 23
server:
   nameOfServer: server1
   capacity: 20GB

We can also create lists using - followed by space.

languages:
- HTML
- CSS
- JavaScript

Similarly, we can create a list of Objects. Ex-

name: John Smith
age: 24
social_media: 
  - LinkedIn
  - Facebook
  - Gmail
  - Discord

Indentation is very important in YAML. YAML doesn't allow you to use any tabs when creating indentation - use spaces instead. Whitespace doesn't matter as long as child elements are indented inside the parent element.

Task:

Docker compose is a tool that is used in making multiple containers and can establish connections amongst these containers using YAML.

Let's create a docker-compose.yml file using vi docker-compose.yml as below. Here, we have taken two containers - one for backend and one for database. The backend and database containers are written inside services containing two objects - backend and mysql.

version: '3'
services:

  backend:
    build:
      context: .
    ports:
      - "5000:5000"
    environment:
      MYSQL_HOST: mysql
      MYSQL_USER: root
      MYSQL_PASSWORD: test@123
      MYSQL_DB: twotier_database
    depends_on:
      - mysql

  mysql:
    image: mysql:5.7
    environment:
      MYSQL_ROOT_PASSWORD: test@123
      MYSQL_DATABASE: twotier_database
      MYSQL_USER: devops
      MYSQL_PASSWORD: devops

Let's view the docker-compose.yml using cat docker-compose.yml-

ubuntu@ip-172-31-40-139:~/dockerProjects/two-tier-app/two-tier-flask-app$ cat docker-compose.yml 
version: '3'
services:

  backend:
    build:
      context: .
    ports:
      - "5000:5000"
    environment:
      MYSQL_HOST: mysql
      MYSQL_USER: root
      MYSQL_PASSWORD: test@123
      MYSQL_DB: twotier_database
    depends_on:
      - mysql

  mysql:
    image: mysql:5.7
    environment:
      MYSQL_ROOT_PASSWORD: test@123
      MYSQL_DATABASE: twotier_database
      MYSQL_USER: devops
      MYSQL_PASSWORD: devops

Here, in the above docker-compose.yml each line has its meaning. Ex-

  • version: This denotes the version of docker-compose in use.

  • services: This denotes all the different containers that we are going to create.

  • build: This denotes that the image will be built using Dockerfile present inside that directory and . specifies the directory where docker-compose.yml is located.

  • ports: This defines the exposed port where the container will be running

  • Environment: This defines the different environment variables which are required to run mysql.

  • image: This defines the image of mysql through which container will be created.

Now, once the docker-compose file is ready, we can start the containers using - docker-compose up.

ubuntu@ip-172-31-40-139:~/dockerProjects/two-tier-app/two-tier-flask-app$ docker-compose up
Creating network "two-tier-flask-app_default" with the default driver
Creating two-tier-flask-app_mysql_1 ... done
Creating two-tier-flask-app_backend_1 ... done

We can view the list of running containers using docker ps.

ubuntu@ip-172-31-40-139:~/dockerProjects/two-tier-app/two-tier-flask-app$ docker ps
CONTAINER ID   IMAGE                        COMMAND                  CREATED          STATUS          PORTS                                       NAMES
70497e40b185   two-tier-flask-app_backend   "python app.py"          46 seconds ago   Up 45 seconds   0.0.0.0:5000->5000/tcp, :::5000->5000/tcp   two-tier-flask-app_backend_1
daec2effadcb   mysql:5.7                    "docker-entrypoint.s…"   47 seconds ago   Up 45 seconds   3306/tcp, 33060/tcp                         two-tier-flask-app_mysql_1

We can stop the running containers using - docker-compose down.

ubuntu@ip-172-31-40-139:~/dockerProjects/two-tier-app/two-tier-flask-app$ docker-compose down
Stopping two-tier-flask-app_backend_1 ... done
Stopping two-tier-flask-app_mysql_1   ... done
Removing two-tier-flask-app_backend_1 ... done
Removing two-tier-flask-app_mysql_1   ... done
Removing network two-tier-flask-app_default

Task 2:

Pull a pre-existing Docker image from a public repository (e.g. Docker Hub) and run it on your local machine. Run the container as a non-root user (Hint- Useusermodcommand to give user permission to docker). Make sure you reboot instance after giving permission to user.

To run the container as a non-root user, add the user to docker group so that user has all the permissions related to docker by giving the command - sudo usermod -aG docker $USER.

To check whether the user is added to the docker group, use cat /etc/group We can see below 'ubuntu' user is added to docker group.

ubuntu@ip-172-31-37-49:~$ sudo usermod -aG docker $USER
ubuntu@ip-172-31-37-49:~$ cat /etc/group
root:x:0:
daemon:x:1:
bin:x:2:
sys:x:3:
adm:x:4:syslog,ubuntuadmin:x:118:
netdev:x:119:ubuntu
lxd:x:120:ubuntu
_chrony:x:121:
ubuntu:x:1000:
docker:x:122:ubuntu

Now, reboot the system using sudo reboot and wait for some time.

ubuntu@ip-172-31-37-49:~$ sudo reboot

Login to Docker Hub and check for repositories if there are any. We can see we have a repository named flask-app in Docker Hub.

So, now we will login to Docker from our local by giving the username and password as given below.

ubuntu@ip-172-31-47-73:~/dockerProjects/flask-app$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: 
Password: 
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded

Now, we will run the command docker image pull userid/<image-name:tagname> to pull the image from the Docker Hub.

ubuntu@ip-172-31-47-73:~/dockerProjects/flask-app$ docker image pull shilpidns/flask-app:latest
latest: Pulling from shilpidns/flask-app
785ef8b9b236: Pull complete 
5a6dad8f55ae: Pull complete 
bd36c7bfe5f4: Pull complete 
4d207285f6d2: Pull complete 
9402da1694b8: Pull complete 
6fa59a7ce94b: Pull complete 
cc429e3ed9d5: Pull complete 
3bec4ce16e88: Pull complete 
bcf708e77743: Pull complete 
3a0c31a9f507: Pull complete 
51dd29c59ebf: Pull complete 
Digest: sha256:9f66627641f0f4001ee4e63be6e35eb1caeb6a4d8fab807d0c0440764df8b658
Status: Downloaded newer image for shilpidns/flask-app:latest
docker.io/shilpidns/flask-app:latest

Inspect the container's running processes and exposed ports using the docker inspect command.

We can inspect the container's running processes and exposed ports using the docker inspect <container-id>. We can also pick out any field from the JSON in a fairly straightforward manner.

ubuntu@ip-172-31-47-73:~/dockerProjects/flask-app$ docker inspect 5e0ace49240d
[
    {
        "Id": "5e0ace49240de8a09e60366390d16c7d40c93a2ff32e16a8943264f5a0395e80",
        "Created": "2023-09-01T13:53:54.421130702Z",
        "Path": "node",
        "Args": [
            "app.js"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 3669,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2023-09-01T13:53:54.773879686Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:384e0d23239216821c7a78219efbdf7ec05dd50869a42a85717466427ecd30ea",
        "ResolvConfPath": "/var/lib/docker/containers/5e0ace49240de8a09e60366390d16c7d40c93a2ff32e16a8943264f5a0395e80/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/5e0ace49240de8a09e60366390d16c7d40c93a2ff32e16a8943264f5a0395e80/hostname",
        "HostsPath": "/var/lib/docker/containers/5e0ace49240de8a09e60366390d16c7d40c93a2ff32e16a8943264f5a0395e80/hosts",
        "LogPath": "/var/lib/docker/containers/5e0ace49240de8a09e60366390d16c7d40c93a2ff32e16a8943264f5a0395e80/5e0ace49240de8a09e60366390d16c7d40c93a2ff32e16a8943264f5a0395e80-json.log",
        "Name": "/funny_nightingale",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "docker-default",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {
                "8000/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "8000"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "ConsoleSize": [
                25,
                139
            ],
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "private",
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": [],
            "BlkioDeviceWriteBps": [],
            "BlkioDeviceReadIOps": [],
            "BlkioDeviceWriteIOps": [],
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": null,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/4e300260b52bc504fe0af014afcd2f74aad8abd36eabc267fef1d3e878f8bf43-init/diff:/var/lib/docker/overlay2/2b4a8a7ceffbe1023cd90d95dadbf6e184795cc02eeaecd878ad72b5ad6e5623/diff:/var/lib/docker/overlay2/2bf687d574c553de733b007094d04daf7c124c7e369b54916690133ebd4f55f9/diff:/var/lib/docker/overlay2/45c8d7c54349a1e626edf03519cc51412338688cc66710e06c3b53f35bfe0a6f/diff:/var/lib/docker/overlay2/fc2d2ae38652cb18d19c170bfa8d91b5217fd21ac9a07b5e1accf863da19b641/diff:/var/lib/docker/overlay2/5b2c70efe250b3faab8f51d308bf4951c0394870e0a148ba5cb19badb070b2af/diff:/var/lib/docker/overlay2/42f9d7597834f1ce74c598714684fbdb6611233060a657a5ee79d94492b152bc/diff:/var/lib/docker/overlay2/2bd8844469ad7423c2a786fe1a098dce4f4f9aa4661f52dd6c3ad9359ddb4855/diff",
                "MergedDir": "/var/lib/docker/overlay2/4e300260b52bc504fe0af014afcd2f74aad8abd36eabc267fef1d3e878f8bf43/merged",
                "UpperDir": "/var/lib/docker/overlay2/4e300260b52bc504fe0af014afcd2f74aad8abd36eabc267fef1d3e878f8bf43/diff",
                "WorkDir": "/var/lib/docker/overlay2/4e300260b52bc504fe0af014afcd2f74aad8abd36eabc267fef1d3e878f8bf43/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [],
        "Config": {
            "Hostname": "5e0ace49240d",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "8000/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "NODE_VERSION=12.2.0",
                "YARN_VERSION=1.15.2"
            ],
            "Cmd": [
                "node",
                "app.js"
            ],
            "Image": "node-todo:latest",
            "Volumes": null,
            "WorkingDir": "/app",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "9183893de77684c2a0d52f10509a615445dfb28bdb23000344a7453f22e92467",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "8000/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "8000"
                    },
                    {
                        "HostIp": "::",
                        "HostPort": "8000"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/9183893de776",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "498e9a600e15c928a68ba84781d96cd4a884c523b4335abbe34a54b2ccfdabde",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "540907847aebf70d7cce63e9bf91a7b3efd0a025074d2b8a38c6bf0b319fb38d",
                    "EndpointID": "498e9a600e15c928a68ba84781d96cd4a884c523b4335abbe34a54b2ccfdabde",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null
                }
            }
        }
    }
]

To specifically check the Container's status, we can give the below command:

ubuntu@ip-172-31-47-73:~/dockerProjects/flask-app$ docker inspect --format='{{.State.Status}}' 5e0ace49240d
running

To specifically check the exposed ports, we can give the below command:

ubuntu@ip-172-31-47-73:~/dockerProjects/flask-app$ docker inspect --format='{{.Config.ExposedPorts}}' 5e0ace49240d
map[8000/tcp:{}]

Use the docker logs command to view the container's log output.

We can check the container's log output using the docker logs <conatiner-id> command as below:

ubuntu@ip-172-31-47-73:~/dockerProjects/flask-app$ docker logs 5e0ace49240d
Todolist running on http://0.0.0.0:8000

Use the docker stop and docker start commands to stop and start the container.

We can stop the docker using docker stop <container-id> and restart the docker using docker start <container-id>.

ubuntu@ip-172-31-47-73:~/dockerProjects/flask-app$ docker stop 5e0ace49240d
5e0ace49240d
ubuntu@ip-172-31-47-73:~/dockerProjects/flask-app$ docker start 5e0ace49240d
5e0ace49240d

Use the docker rm command to remove the container when you're done.

Using docker rm <container-id> we can remove the container permanently.

ubuntu@ip-172-31-47-73:~/dockerProjects/flask-app$ docker rm 5e0ace49240d
5e0ace49240d

How to run Docker commands without sudo?

Docker commands can be run without sudo by adding the user group to Docker Group. This is can be done by giving the below command-

sudo usermod -aG docker $USER

ubuntu@ip-172-31-37-49:~$ sudo usermod -aG docker $USER
ubuntu@ip-172-31-37-49:~$ cat /etc/group
root:x:0:
daemon:x:1:
bin:x:2:
sys:x:3:
adm:x:4:syslog,ubuntuadmin:x:118:
netdev:x:119:ubuntu
lxd:x:120:ubuntu
_chrony:x:121:
ubuntu:x:1000:
docker:x:122:ubuntu

We need to reboot the system after adding user to docker group so that the user is in sync with the group. This is done using sudo reboot command.

Thanks!

Happy Learning!

~Shilpi

Docker#90daysofdevopsDevopsDevops articlesLinuxsoftware developmentTrainWithShubham