![Docker CLI
docker pull image[:tag|@digest], aka docker
image pull
● tag can is a version, digest is a sha256
digest (like git commit hash)
docker push image:tag, aka docker image push
docker rmi image:tag, aka docker image rm
docker build, aka docker image build
● use --no-cache to rebuild from scratch
● use -t image:tag to add name and version
docker images, aka docker image ls
docker image inspect
docker image inspect <imageid> | jq -r '.[].RootFS'
shows all layers of an image](https://image.slidesharecdn.com/bgougsummer18-180609123200/85/Accelerate-your-development-with-Docker-22-320.jpg)
![Dockerfile Instructions
ARG <name>[=<default value>]
● Declares build time argument to the
Dockerfile. Pass valu to docker build.
FROM <image>[:<tag> | @<digest>] [AS <name>]
● Declares the base image to inherit from
● FROM can use ARG
● AS is for multistage builds
RUN ( <command> | [“exe”, “param1” …] )
● Execs a command in own layer
● ENV var setting is allowed by prefixing the
command with key=value
CMD
● The command to execute when starting
the container
● One per file
● This is not for executing statements
● See also ENTRYPOINT, it might use it
when no executable is declared
LABEL <key>=<value> <key>=<value> …
● For setting metadata which can be queried
later
● LABEL version=”1.0” vendor=”com.dnhsoft”
● Use LABEL instead of MAINTAINER](https://image.slidesharecdn.com/bgougsummer18-180609123200/85/Accelerate-your-development-with-Docker-26-320.jpg)
![But there is more...
EXPOSE
● Tell docker daemon the port will be
exposed
● Doesn’t expose the port automagically, to
do so use docker run -p XXXX:YYYY
ENV (key value | key=value …)
● Sets a ENV variable which is valid until the
end of the Dockerfile
● The ENV will also exist during container
runtime
COPY [--chown=<user>:<group>] <src>... <dest>
● Copies files, dirs into the container at <dst>
● Allows chowing to user:group
● Wildcards are possible
● If <dst> is relative than WORKDIR is used for
resolving the path
● You can’t send as <src> files/dirs up the tree
● Use .dockerignore if you want to skip files
when using wildcards.
ADD [--chown=<user>:<group>] <src>... <dest>
● Same as COPY but also
● Supports <src> from URL
● Local tar.gz|bz2|xz can be decompressed](https://image.slidesharecdn.com/bgougsummer18-180609123200/85/Accelerate-your-development-with-Docker-27-320.jpg)
![Hungry and ready for lunch?
ENTRYPOINT ["executable", "par1", "par2"]
● Makes from the container a command
● When you run a container the command
you pass is appended to the ENTRYPOINT
● http://www.johnzaccone.io/entrypoint-vs-
cmd-back-to-basics/
VOLUME /path/to/dir
● Shows the intent to mount at the location
● The real mount happens with docker run -
v hostdir:/path/to/dir
SHELL ["executable", "parameters"]
USER <UID>[:<GID>]
● Sets the uid:gid of subsequent commands
● Sets the uid:gid at container runtime
● Please use it, otherwise root = too much
rights
WORKDIR /path/to/workdir
● Sets $(PWD)
● Parameter can be absolute or relative
● When relative appended to current value
● Very much like cd /path/to/workdir](https://image.slidesharecdn.com/bgougsummer18-180609123200/85/Accelerate-your-development-with-Docker-28-320.jpg)
![Here come the last ones before the demo
ONBUILD [INSTRUCTION]
● Schedule INSTRUCTION to be executed
when building a child image. A trigger.
● Multiple ONBUILD triggers are executed in
the same order
● Allows one-liners child Dockerfiles : FROM
base-onbuild:1.2
STOPSIGNAL
● Sets the signal number to send when
stopping.
● Could be a number, like 9, or name
SIGKILL
HEALTHCHECK [OPTIONS] CMD
● Allows Docker to check the healthiness of
the container by executing CMD
● CMD should return 0 for healthy and 1 for
unhealthy
● docker ps shows the status
● --interval=TIME , runs every TIME
● --timeout=TIME, probe fails after TIME
● --retries=N , run the probe up to N times
consecutively
● --start-period=TIME , wait TIME after
container start before running the probe.
Useful for containers with long boot time](https://image.slidesharecdn.com/bgougsummer18-180609123200/85/Accelerate-your-development-with-Docker-29-320.jpg)
Accelerate your development with Docker
- 1. Accelerate your software development with Docker Andrey Hristov CTO, DNH Soft
- 2. $ whoami Started programming at age of 13 MSc in Computer Technologies (TU-Sofia) and Software Engineering (HFT Stuttgart) Professional developer since year 2000 PHP core developer since 2002 Spent 11 years working at MySQL, SUN Micro and Oracle improving the MySQL client and server side Last 2 years spent as freelancing technical team lead / consultant Lately became CTO of DNH Soft Software Development practices and architectures appassionato.
- 3. How many of you have used Docker / Linux containers?
- 4. What to expect from this talk? Quick intro into 12 Factor Applications What is Docker? Containers from Linux POV Description of technologies related to containers Overview of Docker Live Demo
- 5. 12 Factors App ● Methodology for building SAAS ● Drafted around 2011 at Heroku 1. Codebase 2. Dependencies 3. Config 4. Backing services 5. Build, release, run 6. Processes 7. Port binding 8. Concurrency 9. Disposability 10. Dev / Prod parity 11. Logs 12. Admin processes
- 6. Codebase / Dependencies / Config Codebase There should be exactly one codebase for a deployed service with the codebase being used for many deployments. Dependencies All dependencies should be declared, with no implicit reliance on system tools or libraries. Config Configuration that varies between deployments should be stored in the environment.
- 7. Backing Services / Build, Release, Run / Processes Backing Services All backing services are treated as attached resources and attached and detached by the execution environment. Build, release, run The delivery pipeline should strictly consist of build, release, run. Build stage artefacts should not be available to release and run stage. Build once, run everywhere. Processes Applications should be deployed as one or more stateless processes with persisted data stored on a backing service.
- 8. Port Binding / Concurrency / Disposability Port Binding Self-contained services should make themselves available to other services by specified ports. Concurrency Scale out via the process model Processes Maximize robustness with fast startup and graceful shutdown
- 9. Dev - Prod Parity / Logs / Admin Processes Dev / Prod Parity All environments should be as similar as possible. Logs Applications should produce logs as event streams and leave the execution environment to aggregate. Admin Processes Any needed admin tasks should be kept in source control and packaged with the application. They should run in the same environment as the application itself.
- 10. To begin with, what is Docker? Docker Inc. is a company, previously dotCloud However in the past 6 years the name meant containers Some people say dockerize when they mean containerize (similarly to the verb “to google”) Containers were not invented by Docker Inc. The company made them available to the masses.
- 11. Then, what is a container? Containerization is OS environment virtualization It feels like a VM but ain't one. Some people call them lightweight VMs. “One kernel to rule them all” compared to “one hypervisor to rule them all”. Can't boot a different OS or kernel. Can't load other kernel modules. Can boot different distro, however. Typically only one process / service (forking apps) runs inside the container. Examples of previous/other works : Solaris Zones, FreeBSD Jails
- 12. Containers on Linux Containers on Linux rely on a couple of kernel features Linux Namespaces, that provide isolation Currently existing namespaces are : cgroup, IPC, network, mount, PID, user (UIDs & GIDs), UTS Control Groups (cgroups), that provide means for hierarchical organization for metering and limiting of resources (memory, CPU, I/O, network) for group (collections of processes)
- 13. Who is running them? Containers are executed on lower level by runtimes LXC/LXD - LXD, written in Go, uses LXC rkt - App Container compliant, deprecated, by CoreOS, now Red Hat. Natively ACI, but supports also Docker and OCI images. Forked very recently runC - OCI compliant implementation in Golang by Docker Inc., a spin off from Docker Engine since Docker 1.11 containerD - works with runC for the high level details, while runC is low level railcar - OCI compliant implementation in Rust by Oracle OCI has two specs, released in July'17 : Image and Runtime CRI-O, implementation of the Kubernetes (1.5+) Container Runtime Interface (CRI) using OCI compatible runtimes.
- 14. But there is more! Containers are managed at a higher level by orchestrators. Docker Compose (single host only) and Docker Swarm both are part of Docker Engine Marathon on Apache Mesos Cattle, obsoleted, by Rancher. Rancher 2.0 runs k8s Kubernetes (k8s). Recently won the Orchestrator wars. If you plan to use containers k8s should be your orchestrator of choice KaaS is available from all major cloud providers - AWS (beta), Azure and GKE
- 15. In short, what’s in for me? Containers are lightweight, or at least lighter than VMs, both in run-time resources usage and size Containers are immutable Containers can be even read-only Containers are meant to be ephemeral Every container contains all needed dependencies and doesn't need anything else Implications: Dep hell is gone. DLL hell memories resurface? XAMPP is dead Linux distro software choice is dead Less software installed means less exploit surface
- 16. Hosting of container images (registries) Docker Inc. runs Docker Hub Library of public images Docker Store - commercially available containers and Docker plugins. Docker Hub supports automated builds triggered on a commit in Github / BitBucket. Storage for your images ● free of charge for you public ones ● has a cost for you private images Alternatives are: ● Host a registry in a container on own VPS or on premise ○ Docker Trusted Registry (Docker EE) ○ RedHat OpenShift CR ○ JFrog Artifactory ○ Sonatype Nexus ● Amazon Elastic Container Registry, you need AWS SDK ● Google Container Registry, you need Google Cloud SDK ● Azure Container Registry
- 18. Docker (the software) Flavors Supported OS for Docker CE: ● Linux (x86-64, ARM, ARM64, ppc64le, s390x( ● MacOS, comes bundled with k8s ● Windows, comes bundled with k8s ● AWS ● Azure Supported Platforms for Docker EE ● CentOS (x86-64) ● OL (x86-64) ● RHEL, SUSE Linux ES, Ubuntu (x86-64 / ppc64le / s390x) ● MS Windows Server 2016 (x86-64) ● AWS ● Azure ● IBM Cloud
- 19. Docker Compose Originally known as Fig Orchestrator that uses IaC “Cluster” configuration is stored in an YAML file ( ./docker-compose.yml ) Features are constantly added, thus there are many compose file versions. Latest is 3.6 as of 18.02 First line in the file states minimum version The file is split in 3 main sections - higher level abstractions, since 2.0 : services, networks, volumes If you plan to use Docker Swarm, then you have to use version 3. Docker EE also now supports K8s deployments from docker-compose.yml
- 20. Docker Compose Entities services - The containers = instances of images. With Swarm you can have multiple instance per service - scaling up and down. volumes / mounts - Persistently stored data. Otherwise data is gone when the container get removed. Mounts import data from the host and are shareable Volumes are BLOBs and are shareable too Volumes are abstracted thru plugins networks - The actual glue between the services DC creates a default network, if are lazy to not create one. This network is called <projectName>_default <projectName> is derived from CWD, pass -p to docker-compose for smth else. Networks can be seen by other projects and they are namespaced by project name. Network frontend in P1 can be attached in project P2 as external network under the foreign name P1_frontend.
- 21. Docker, where is my data? Container images are made of layers aufs (/var/lib/docker/aufs), superseed by overlayfs, shipped with Linux Kernel 4.0 cat /proc/filesystems to see what FSs your kernel support
- 22. Docker CLI docker pull image[:tag|@digest], aka docker image pull ● tag can is a version, digest is a sha256 digest (like git commit hash) docker push image:tag, aka docker image push docker rmi image:tag, aka docker image rm docker build, aka docker image build ● use --no-cache to rebuild from scratch ● use -t image:tag to add name and version docker images, aka docker image ls docker image inspect docker image inspect <imageid> | jq -r '.[].RootFS' shows all layers of an image
- 23. More Docker CLI docker run, aka docker container run docker exec, aka docker container exec docker rm, aka docker container rm docker ps, aka docker container ls docker stop, aka docker container stop (SIGTERM) docker kill, aka docker container kill (SIGKILL) docker kill `docker ps -q` to kill'em all (you might also need to remove them) docker inspect ● inspects networks, containers, images ● gives you tons of info in JSON format. Use jq to process it. docker container diff docker network ls docker network rm docker network prune docker system prune
- 24. Building a container image docker build ● Simple - just run the command ● Transparent - the recipe how to build is in the Dockerfile ● Self-contained everything is one place; the Dockerfile, the assets ONBUILD Strategy ● The Dockerfile is a simple “FROM baseimage” ● Intransparent, as the sysadmin defines what will happen Asset Generation Pipeline Strategy ● Run different asset generators as separate containers ● SASS, composer, etc. ● External driver is needed, like make, gulp, or just whatever your CI provides ● Pro - smaller images ● Con - complicated because of multiple moving parts Multistage Builds Strategy
- 25. Multistage builds Build different artifacts during different stages Opt-in what to pull from a previous stage In short, install the compile time deps in first stage, compile the app, pull only the compiled code in the next stage which will eventually be the delivered image Pro: No need for an external driver like make, gulp, etc Pro: The recipe is in one place - the Dockerfile Con: The Dockerfile become longish
- 26. Dockerfile Instructions ARG <name>[=<default value>] ● Declares build time argument to the Dockerfile. Pass valu to docker build. FROM <image>[:<tag> | @<digest>] [AS <name>] ● Declares the base image to inherit from ● FROM can use ARG ● AS is for multistage builds RUN ( <command> | [“exe”, “param1” …] ) ● Execs a command in own layer ● ENV var setting is allowed by prefixing the command with key=value CMD ● The command to execute when starting the container ● One per file ● This is not for executing statements ● See also ENTRYPOINT, it might use it when no executable is declared LABEL <key>=<value> <key>=<value> … ● For setting metadata which can be queried later ● LABEL version=”1.0” vendor=”com.dnhsoft” ● Use LABEL instead of MAINTAINER
- 27. But there is more... EXPOSE ● Tell docker daemon the port will be exposed ● Doesn’t expose the port automagically, to do so use docker run -p XXXX:YYYY ENV (key value | key=value …) ● Sets a ENV variable which is valid until the end of the Dockerfile ● The ENV will also exist during container runtime COPY [--chown=<user>:<group>] <src>... <dest> ● Copies files, dirs into the container at <dst> ● Allows chowing to user:group ● Wildcards are possible ● If <dst> is relative than WORKDIR is used for resolving the path ● You can’t send as <src> files/dirs up the tree ● Use .dockerignore if you want to skip files when using wildcards. ADD [--chown=<user>:<group>] <src>... <dest> ● Same as COPY but also ● Supports <src> from URL ● Local tar.gz|bz2|xz can be decompressed
- 28. Hungry and ready for lunch? ENTRYPOINT ["executable", "par1", "par2"] ● Makes from the container a command ● When you run a container the command you pass is appended to the ENTRYPOINT ● http://www.johnzaccone.io/entrypoint-vs- cmd-back-to-basics/ VOLUME /path/to/dir ● Shows the intent to mount at the location ● The real mount happens with docker run - v hostdir:/path/to/dir SHELL ["executable", "parameters"] USER <UID>[:<GID>] ● Sets the uid:gid of subsequent commands ● Sets the uid:gid at container runtime ● Please use it, otherwise root = too much rights WORKDIR /path/to/workdir ● Sets $(PWD) ● Parameter can be absolute or relative ● When relative appended to current value ● Very much like cd /path/to/workdir
- 29. Here come the last ones before the demo ONBUILD [INSTRUCTION] ● Schedule INSTRUCTION to be executed when building a child image. A trigger. ● Multiple ONBUILD triggers are executed in the same order ● Allows one-liners child Dockerfiles : FROM base-onbuild:1.2 STOPSIGNAL ● Sets the signal number to send when stopping. ● Could be a number, like 9, or name SIGKILL HEALTHCHECK [OPTIONS] CMD ● Allows Docker to check the healthiness of the container by executing CMD ● CMD should return 0 for healthy and 1 for unhealthy ● docker ps shows the status ● --interval=TIME , runs every TIME ● --timeout=TIME, probe fails after TIME ● --retries=N , run the probe up to N times consecutively ● --start-period=TIME , wait TIME after container start before running the probe. Useful for containers with long boot time
- 30. Live Demo
- 31. Q&A / Resources Anatomy of a container: https://bit.ly/2v0EEGj https://github.com/andreyhristov/bws2018-docker https://coreos.com/rkt/docs/latest/rkt-vs-other-projects.html https://docs.docker.com/install/linux/docker-ce/ubuntu/ https://docs.docker.com/compose/install/ https://docs.docker.com/compose/compose-file/ https://docker-software-inc.scoop.it/t/docker-by-docker https://nickjanetakis.com/blog/tag/docker-tips-tricks-and-tutorials https://opensource.com/article/18/5/navigating-container-security- ecosystem https://dashtainer.com/ https://landscape.cncf.io/ https://traefik.io/ https://leanpub.com/the-devops-2-toolkit https://leanpub.com/the-devops-2-2-toolkit https://leanpub.com/the-devops-2-3-toolkit https://thenewstack.io/ https://www.katacoda.com/ https://github.com/google/gvisor
Editor's Notes
- #5: After some introduction let’s move to the topic of the presentation. Docker in the title of the presentation is not a clickbait but before I go deeply into Docker I will cover some basic things about containers. I will start with what is a container from Linux point of view. After that I will give brief overview of the technologies related to containers. Later we will switch the gear and see what Docker offers. Finally, this presentation will finish with a live demo of development setup which uses Docker under Linux.
- #11: In first place Docker is a company. It created a product, which is easy to use and brings the software development and software operations to the next level. Because Docker revolutionized the container technology for the masses Docker equals containers. Similarly, to the verb “to google” is “to search”, “to dockerize” means to “containerize”. To be clear, containers were not invented by Docker, as GNU/Linux is not the first operating system of its kind, but both became widespread because of easy of use and thus cheap entry.
- #12: One gear back. What is a container? A container is an isolated environment. Compared to Virtual Machines it is lightweight and is OS environment virtualization compared to physical resources virtualization. A VM contains whole operating system. It is a virtualized computer. A container might contain full blown OS but this is rarely the case. What a container doesn’t contain is a kernel. The kernel of the host machine is used to execute the binaries of the container. So, with VMs there is a hypervisor to manage the usage of resources. With containers the OS kernel does it. There is a real implication because of that. You can’t boot a container which contains different OS. With VMs this is perfectly possible. However, it is possible to mix and match different Linux distributions. The host could be Ubuntu Server and in the container we might have Alpine or CentOS. Typically only one process or in case of an applications that uses fork only one application runs inside the container. Here I should mention that previous works in this world exist - Solaris Zones and FreeBSD Jails.
- #13: The Linux kernel has a couple of subsystems, which were not created directly with the idea of containers but when combined allow the implementation of containers. The first feature is namespaces. Namespaces provide isolation. This is very similar to the namespaces in the programming languages that provide them. For example, different network namespaces provide isolation and allow to run in parallel several network stacks without conflicts. Control Groups is another feature, which is essential for the containers. It provides means for metering resource utilization and limiting of it, if it needs be. This means that a container can have an upper limit of memory usage and CPU slices it will get. This is feature is on par with VMs.
- #14: Now, after we saw what features lay the ground for containers let’s talk a bit about how containers are run. Containers are not run by the kernel. We need a special process for this. This process is called a runtime. There are many runtimes out there. LXC/LXD is the very low level. LXC comes from Linux Containers. For example, already years ago it was possible to run Vagrant VMs with LXC as a backend (instead of Virtual Box or VMWare Player). Back in the days when Vagrant was the cutting edge, this was very useful as it was possible to run multiple virtual machines on non-server machines without killing the performace. Rkt is another runtime, which is deprecated It has been developer by CoreOS. CoreOS was recently acquired by RedHat. CoreOS was/is a product with containers only in mind. Based on systemd and fleet as orchestrator. I will talk more about orchestrators a bit later. Rkt supports both Docker and OCI images. More about images later in the talk. runC is a OCI (Open Container Initiative) compliant implementation of a runtime. Developed by Docker Inc, and implemented in Golang. Golang is the base technology for a lot new projects which we put in the category system programming. runC is a spin-off from Docker Engine. Docker Engine was the monolith product developed by Docker Inc. Because of fears from the community about vendor lock-in there were talks about creating a whole new stack. Because of the that Docker started splitting the monolith into separate products which build then the whole stack. runC and containerD aber both such products. containerD is a layer above runC but has higher level abstractions. Railcar is OCI compliant runtime, implemented in Rust (another language for system programming) by Oracle. I suppose Oracle Cloud uses railcar. So far the open container initiative has released two specifications - image and runtime. CRI-O is a runtime, that is product specific. It is a implementation of the Kubernetes Container Runtime Interface to use OCI compatible runtimes. More about Kubernetes later in the talk.
- #15: As we talked about runtimes we should also mention orchestrators. Orchestrators are higher level product in the stack. This is where the real differentiation happens. Let’s start with Docker Compose. Docker Compose is massively used during development and can be used for production deployments of relatively simple setups, as it supports only single hosts. D-C started as product called Fig and was acquired by Docker Inc. Docker Swarm is the orchestrator implemented by Docker Inc. for distributed and highly available setups. For ease of use, the deployments are described in declarative way and are extension to the configuration used by Docker Compose. Declarative, also known, as infrastructure as code, deployments are currently very popular, especially for cloud development. They allow to keep the procedures as part of the version control system. Terraform and Puppet are well known products for IaC for VMs. Marathon is the orchestrator of Apache Mesos platform. Cattle was the orchestrator of Rancher OS v1. Rancher v2 uses Kubernetes. RancherOS is a OS based on Linux for running containers. The last and the very important one is Kubernetes. It has recently won the Orchestrator war. Kubernetes as a service is now offered by all big three clould providers - AWS (as beta), Azure and GKE. CoreOS and Rancher switched to Kubernetes. Apache Mesos allows supports Kubernetes. Even Docker Inc. integrated K8s and the users of Docker Engine can choose whether to use Swarm or K8s.
- #16: So, let’s see what containers offer to the IT crowd Containers are lightweight compared to VMs still allowing resource isolation, metering and limits. The isolation is not on hardware level, which could be problematic at times. Containers are immutable and also read-only. In the first case the container in memory could be changed but the changes are gone if the container is restarted or killed. The container is the in memory representation of an image. A container is a process, an image is an executable as a comparison. Containers can be even read-only. Well, the file systems can be mounted read-only. This is very good from security point of view. A non-read-only container can be inspected for the changes compared to the image used to start the container. Perfect for security audits. Containers are meant to be ephemeral although nowadays even database servers are ran as containers with the storage mounted from outside. Ephemeral containers could be killed at any time. This might happen because there is a new version of the container, or the container will be migrated to a different host (pod), or because the container misbehaves. This feature allow also blue/green deployments (rolling updates) to be easily implemented. Every container comes with all dependencies it needs. There is no DLL hell anymore. However, unlike real or virtual machines, it is not possible to centrally update libraries in case of security problems and fix for every program. Another implication is that for example XAMPP is dead. There is no need for pre-bundled distros. Just pull on yourself the language runtime, web server and database server of choice. Mix and match controlled directly by the developer. In this regard the choice of software of a Linux distro is no longer a problem. Recall that Debian switched from MySQL to MariaDB. Who cares what comes with Debian when one can pull a container, and even doesn’t need apt or yum repos when the latest version is pushed as a container. A container contains only the libraries needed for the particular service. Which in turn means that there is less attack service. A container might be few megs in size or tens of megs, rarely over a gigabyte.
- #17: Containers are started from images. These images in most cases are stored somewhere in a centralized manner. Images are hosted in registries. There are two versions of the registries. Currently v2 is in use. Docker Inc. runs a registry called Docker Hub. Docker Hub can host public and private images. Private images come at a cost. Public images are either free of charge or can cost something to the end-user. Similarly to how AWS AMIs could be used for a charge. Docker Hub can be used to build images, thus being part of a CI pipeline waiting on a changes coming from GitHub and BitBucked. Alternatives to Docker Hub is a self hosted one - either in the cloud or on premises. In the cloud all three big players offer Container Registries to be used with their container as a service products but not limited to this usage.
- #18: Here is a graphic from a recent survey regarding the usage of Docker Registries
- #19: Docker comes in two flavours. Community Edition and Enterprise Edition. CE is free of charge and supports these platforms. EE costs money, supports K8S and is available on these platforms.
- #20: The easiest way to get usage of containers when doing software development, especially when doing web development, is by using Docker compose. D-C was called Fig before acquired by Docker Inc. D-C is a infrastructure as code tool. The infrastructure of the project is statically defined in a declarative file in YAML format. The name of the file is docker-compose.yml D-C contantly adds features, mostly regarding Swarm, while deprecating prevous ones. For this reasons there are multiple versions of the D-C file. There are 3 major versions. V3 is the latest major version. In v2 the file format was reorganized to create higher level abstractions called services, networks and volumes. If you plan to use Docker Swarm as an orchestrator then only major version 3 is available for you
- #21: The container instances are called services. Every service configures the container. For example mounts, volumes, ports, networks attached to etc. Volumes are for persistently stored data. Volumes can be either blobs or mounted directories. There are volume plugins so the volume might not be local at all. Networks are the glue between the services. There is always a default network, if you are lazy to create your own. The default network name is projectName_default. ProjectName comes from the CWD. Networks are used as VLANs to separate the traffic. For example a typical Web project will have a frontend and backend network or even more. The database will sit on one network connected to the application. The web clients will connect to the frontend network, where the web server and the application will reside. For logging with ELK there might be another network and so on. Other projects can attach to existing networks. Thus, the interoperability between projects is possible.
- #22: Docker uses union file systems for data storage. Union file systems work like a series of diffs. If I file is wanted it will be searched for through a linked lists of layers.
- #23: When doing development on the local machine the interaction with the containers happen through the docker and docker-compose CLIs Here are some of the most commonly used CLI commands Docker pull is used to pull a container from a remote registry. If it is a public image on Docker Hub there is no need to log in. However if it is a private one or you are using a private one (on premises or in the cloud) the first thing is to log into the registry. An image always has a tag and digest. In most cases it will be pulled by a tag. If tag is omitted latest that is used. For stable environments latest should be avoided and you should stick to particular tag. When an image is pushed a digest is created. This digest can be used to pull the image. The digest can never be changed thus you will always pull the same image. Tags can be changed and so different environments might get different images. Tags should not be abused in this way, but sometimes it happens. Docker push pushes an image. As already stated, the digest of the newly pushed image will be shown. Docker rmi removes an image from the local machine. It won’t delete an image. Deleting image in the registry is often hard if not impossible. Docker build creates an image by using Dockerfile as a bluprint. The Dockerfile might refer to other files too. Here you see that there are two commands that do the same thing. At some point of time the CLI was refactored so more commands can be easily added. A level of indirection was added - namespacing. Docker images shows all images which exist locally. Some of them might not exist anywhere else. This might happen if the image was never pushed to a registry or was created on demand by docker-compose with a build directive. This usage of docker-compose is better not used as two different machines could end with different images. Images are better pre-built by one person in a team and then pushed to a registry. So every developer uses the same image. Docker image inspect shows a lot of useful information about an image. The information is in JSON format. By using jq as JSON filter we can easily extract particular information from the JSON object without using more / less.
- #24: Docker run starts a new container Docker exec executes are command in an existing container Docker rm removes a container (not the image) Docker ps shows the running containers. -a shows all containers, even the stopped ones. -q is quiet, which means it shows only IDs. This is useful for kill. Docker stop stops a container. It can be later resumed with docker start Docker kill stops a container with SIGKILL (9). As an example to kill and remove all containers use this command Docker inspect shows information about different entities. Can be used for networks, containers and images. Docker container diff shows the changes in the container (the mounted FS) compared to the image used to start the container. The changes are in own layer. This layer might be committed but it is strongly advised not to do so. Docker network ls shows all existing networks. When stopping a project the project networks continue to exist. These networks might be manually removed with docker network rm orl docker network prune. Docker system prune cleans up the system from unused resources. This can often release gigabytes of data from old unused images.
- #25: Building an image might be compex operation. The docker build command is used for this, as already shown. The Dockerfile is the recipe that is read and executed line by line. Every line is a different layer in the image. Thus it is advised to join multiple commands. Commands that should create only temporary command should also be joined so after the command finishes the temporal data is not committed to the layer. This is typically used with apt-get update, which downloads tons of info about packages. This info is not needed in the final container. The Dockerfile might refer to external files, for example assets to be copied into the container. This assets might be source files or scripts, or images, etc. At the end the container image is self-contained There are a few strategies for building containers. When using ONBUILD a base image is being inherited. This base image is created by some skillful person. The derived Dockerfile is pretty simple. The problem is that the skillful person might become a SPoF. Also the knowledge is not spread. Another strategy is to run different asset generators as a part of a external pipeline (for example with Make). These generators create end files which are then copied into the container. Images are small. The third strategy is the multistage builds.
- #26: Multistage builds allow to skip the usage of an external pipeline and rely only on the Dockerfile. The multistage builds allow to cherry pick assets from particular image from previously built image in the same Dockerfile. So let’s say in a stage we load the vendors/ directory with composer (PHP). The we copy the vendors/ but not composer. When using C/C++, the whole build toolchain is installed into one image but in the next image only the executable is copied from the former. At the build toolchain gets removed as it is not copied. The Dockerfiles when using this strategy tend to become long.