Why Docker Exists
Introduction
Docker has become one of those technologies that almost every developer eventually encounters. Whether you’re building a small side project, deploying a production API, or preparing for a DevOps interview, Docker is everywhere.
Yet many developers use it every day without fully understanding why it exists.
You have probably seen commands like:
docker build
docker run
docker compose upand perhaps even copied a Dockerfile from another repository without thinking much about what happens behind the scenes.
That works—for a while.
Sooner or later, however, you’ll run into questions like:
- Why does my application work on my machine but fail in production?
- Why is this Docker image 3 GB?
- Why does every build take several minutes?
- Why did updating a base image suddenly break my application?
- What exactly is the difference between an image and a container?
This guide answers those questions from first principles.
Rather than memorizing commands, you’ll learn how Docker actually works, why its design solves real engineering problems, and how experienced developers structure their containerized applications.
By the end of this guide you’ll understand:
- why containers became the industry standard
- what Docker actually does
- how images are built
- how containers run
- how isolation works
- how to write cleaner Dockerfiles
- how to avoid common beginner mistakes
Why Docker Was Created
Before Docker became mainstream, deploying software was surprisingly fragile.
Imagine a simple Python web application.
It depends on:
- Python 3.11
- PostgreSQL
- Redis
- several third-party libraries
- operating system packages
- environment variables
- configuration files
Everything works perfectly on your laptop.
Then you send the project to a teammate.
Five minutes later they message you:
“It doesn’t work.”
Nothing changed in your code.
Yet somehow everything is broken.
This situation became so common that developers jokingly called it:
“Works on my machine.”
The problem wasn’t the application.
The problem was the environment.
Different computers had:
- different operating systems
- different package versions
- different runtime versions
- different system libraries
- different configurations
Even reinstalling your own project six months later could produce completely different results.
Docker was designed to eliminate this entire category of problems.
One Application, One Environment
Instead of installing everything directly onto the operating system, Docker packages an application together with everything it needs to run.
That package is called an image.
An image contains:
- your application
- the runtime
- installed dependencies
- configuration
- operating system libraries
- startup commands
Everything is bundled into a predictable environment.
If the image contains Python 3.11, Docker will always use Python 3.11.
If it contains Node.js 22, every container created from that image will use Node.js 22.
It doesn’t matter whether the image runs on:
- your laptop
- a teammate’s computer
- GitHub Actions
- AWS
- DigitalOcean
- a Kubernetes cluster
The environment remains identical.
That’s Docker’s biggest promise:
Build once. Run anywhere.
Stability Through Reproducibility
Modern software changes constantly.
Package managers receive updates.
Operating systems evolve.
Libraries introduce breaking changes.
Configuration defaults change.
Without isolation, even a small system update can unexpectedly break a working application.
Docker avoids this by freezing the execution environment inside an image.
Instead of relying on whatever happens to be installed on the host machine, every container starts from the exact same filesystem and dependency versions every time.
This makes deployments reproducible.
If version 1.2.0 of your application worked yesterday, it will continue to work next month—as long as you’re running the same image.
For teams, this dramatically reduces debugging time because everyone is running the same environment.
Isolation: Keeping Applications Out of Each Other’s Way
Stability is only half of Docker’s value.
The other half is isolation.
Imagine a server hosting several applications.
One service requires:
- Python 3.9
Another requires:
- Python 3.12
A third depends on:
- PostgreSQL 14
Another expects:
- PostgreSQL 16
Installing everything directly on one operating system quickly becomes difficult.
Different versions overwrite one another.
Configuration files collide.
Library updates break unrelated services.
Docker solves this by giving every application its own isolated environment.
Each container has its own:
- filesystem
- processes
- network interfaces
- installed libraries
- environment variables
Containers don’t interfere with one another.
You can run multiple versions of the same runtime side by side without conflicts.
This isolation makes Docker particularly useful for:
- microservices
- CI/CD pipelines
- development environments
- testing multiple software versions
- self-hosted infrastructure
Resource Limits
Isolation isn’t just about software.
Docker can also isolate hardware resources.
A container can be limited to:
- a fixed amount of RAM
- a specific number of CPU cores
- storage quotas
- process limits
For example, a logging service can be restricted to 512 MB of memory, while a database receives several gigabytes.
If one container starts consuming excessive resources, Docker prevents it from affecting the rest of the system.
Instead of allowing a single runaway application to crash an entire server, each workload stays within its assigned limits.
This is one of the reasons Docker has become such a common foundation for modern cloud infrastructure.
Why Developers Love Docker
After understanding these ideas, Docker becomes much easier to appreciate.
Its popularity isn’t about learning new commands.
It’s about solving real engineering problems.
Docker gives developers:
- predictable environments
- repeatable deployments
- isolated applications
- easier collaboration
- safer experimentation
- simpler onboarding
- cleaner production servers
Once you’ve experienced working in fully reproducible environments, it’s surprisingly difficult to go back to manually configuring applications on every machine.
Docker Terminology Every Developer Should Know
Before writing your first Dockerfile or running your first container, it’s important to understand Docker’s vocabulary. Many beginners struggle not because Docker is difficult, but because its documentation assumes you already know what terms like image, container, layer, and volume mean.
Once these concepts click, Docker becomes much easier to reason about.
Image vs Container
The two terms that cause the most confusion are image and container.
A simple analogy is to think of an image as a recipe and a container as the finished meal.
The recipe describes how something should be prepared.
The meal is the actual running result.
Docker works the same way.
An image is a read-only blueprint that contains everything needed to run an application.
A container is a running instance of that image.
Docker Image
│
▼
docker run
│
▼
Docker ContainerOne image can create hundreds of containers.
For example:
docker run nginx
docker run nginx
docker run nginxAll three containers come from the exact same image, but each runs independently.
What Is a Docker Image?
A Docker image is a packaged filesystem.
It contains everything your application requires:
- operating system files
- runtime (Node.js, Python, Java, Go…)
- installed packages
- application source code
- configuration
- startup command
Unlike a virtual machine, an image doesn’t contain an entire operating system kernel.
Instead, it shares the host kernel while keeping its own isolated filesystem.
Think of an image as a snapshot.
Once built, it doesn’t change.
If you want to modify it, you build a new image.
This immutability is one of Docker’s biggest strengths because deployments become completely reproducible.
What Is a Container?
A container is a running process created from an image.
When Docker starts a container, it adds one additional writable layer on top of the image.
Everything that happens while the application is running occurs inside this writable layer.
For example:
- log files
- temporary files
- uploaded images
- cache
- generated reports
If the container is deleted, that writable layer disappears.
The original image remains untouched.
This explains why developers often lose data after deleting containers.
The container is temporary.
Persistent data belongs somewhere else—which we’ll discuss when talking about volumes.
The Lifecycle of a Container
Containers have a predictable lifecycle.
Image
│
docker run
│
Running Container
│
docker stop
│
Stopped Container
│
docker start
│
Running Again
│
docker rm
│
RemovedNotice something important.
Stopping a container does not remove it.
Many beginners assume these commands are equivalent:
docker stopand
docker rmThey are not.
docker stop
- stops the running process
docker rm
- permanently removes the container
After removal, every change made inside the writable layer disappears.
Images Are Built from Layers
Docker images are not stored as one giant file.
Instead, they’re built from layers.
Imagine this Dockerfile:
FROM node:22
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .
CMD ["npm", "start"]Docker builds something similar to this:
Layer 1
Ubuntu / Debian Base
↓
Layer 2
Node.js Runtime
↓
Layer 3
Working Directory
↓
Layer 4
package.json
↓
Layer 5
npm install
↓
Layer 6
Application Source CodeEach layer is immutable.
Docker stores them separately and reuses them whenever possible.
This dramatically speeds up builds.
Why Layers Matter
Layers are one of Docker’s biggest performance optimizations.
Imagine two projects.
Project A
FROM node:22Project B
FROM node:22The Node.js base image only exists once on disk.
Docker doesn’t download or store duplicate copies.
The same applies to every unchanged layer.
If Docker already has a layer, it simply reuses it.
That’s why pulling a second image is often much faster than the first one.
Docker Build Cache
Docker caches layers aggressively.
Consider this Dockerfile:
COPY package.json .
RUN npm install
COPY . .If only your application code changes, Docker skips reinstalling dependencies because the package.json layer hasn’t changed.
Only the final layer is rebuilt.
Now compare it to this:
COPY . .
RUN npm installEvery code change invalidates the cache.
Dependencies must be installed again.
Builds become much slower.
This is why experienced Docker users always copy dependency files before copying the application source.
A small change in Dockerfile order can reduce build times from minutes to seconds.
Docker Registry
Docker images need somewhere to live.
That place is called a registry.
The most popular registry is Docker Hub.
Developer
│
docker push
│
Docker Hub
│
docker pull
│
Production ServerPublic registries include:
- Docker Hub
- GitHub Container Registry
- GitLab Registry
- AWS ECR
- Google Artifact Registry
- Azure Container Registry
Many companies also run private registries for internal software.
Image Tags
Every image has a name and a tag.
For example:
node:22
python:3.12
nginx:1.27
postgres:17The tag identifies a specific version.
Many beginners write:
docker pull nginxDocker automatically interprets this as:
docker pull nginx:latestWhile convenient, latest is rarely a good choice in production.
The maintainer can move the latest tag to a completely different version at any time.
That means today’s deployment might not be identical to tomorrow’s.
Instead, pin explicit versions whenever possible:
node:22.17.0
python:3.12.5
postgres:17.5
nginx:1.27.1Predictable versions make deployments predictable.
Dockerfile
A Dockerfile is simply a build recipe.
It tells Docker how to create an image.
A minimal example looks like this:
FROM node:22
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD ["npm", "start"]Every instruction creates another step in the build process.
Some instructions also create filesystem layers.
Understanding which instructions affect caching is essential for writing efficient Dockerfiles—a topic we’ll explore in detail later in this guide.
Docker Engine
People often say “Docker” when they actually mean several different components working together.
Docker Engine consists of:
- Docker CLI
- Docker Daemon
- Docker API
When you type:
docker run nginxthe CLI doesn’t launch the container itself.
Instead, it sends a request to the Docker Daemon.
The daemon performs all the heavy lifting:
- downloading images
- building images
- creating containers
- configuring networks
- mounting volumes
- managing resources
The CLI is simply the interface you interact with.
Docker Compose
Most real-world applications require more than one container.
A typical web application might include:
- backend
- frontend
- PostgreSQL
- Redis
- Nginx
Starting everything manually would be tedious.
Docker Compose lets you describe the entire application in a single YAML file.
docker-compose.yml
↓
Backend
↓
Frontend
↓
Database
↓
Redis
↓
NginxThen everything starts with a single command:
docker compose upCompose has become the standard way to run local development environments and small production deployments.
Build Context
One often-overlooked concept is the build context.
When you execute:
docker build .the final dot (.) matters.
It tells Docker to send the entire current directory to the Docker daemon.
That includes every file unless excluded by .dockerignore.
Large folders like:
node_modules.git.cachedist
can unnecessarily increase build time.
This is why a properly configured .dockerignore file is considered a best practice for every project.
Key Takeaways
By now, you should understand the core building blocks of Docker:
- Image — the immutable blueprint.
- Container — a running instance of an image.
- Layer — reusable filesystem changes that make builds efficient.
- Registry — where images are stored and distributed.
- Tag — the version identifier for an image.
- Dockerfile — the recipe for building images.
- Docker Engine — the system responsible for creating and managing containers.
- Docker Compose — the tool for orchestrating multiple containers.
- Build Context — the files Docker receives during image creation.
These concepts form the foundation for everything else in Docker. Once they’re clear, writing Dockerfiles and debugging containerized applications becomes far more intuitive.
Installing Docker the Right Way
Before you can build images or run containers, you need a working Docker environment.
Fortunately, installing Docker has become much easier over the years. The recommended setup depends largely on your operating system and how you plan to use Docker.
For learning and day-to-day development, almost any modern machine can run Docker comfortably. However, understanding the differences between Docker Desktop and Docker Engine will save you time and help you avoid unnecessary resource usage.
Docker on Linux
If you’re using Linux, you’re getting the best Docker experience.
Docker runs directly on the Linux kernel without requiring any virtualization layer.
Your Application
│
Docker Engine
│
Linux Kernel
│
HardwareThis architecture provides several advantages:
- lower memory usage
- faster startup times
- better filesystem performance
- native networking
- fewer compatibility issues
On a typical Linux installation, the Docker daemon itself usually consumes only a small amount of memory when idle.
This is one of the reasons many developers eventually move their development environment to Linux.
Docker on macOS
macOS doesn’t run Linux containers natively.
Instead, Docker Desktop creates a lightweight Linux virtual machine behind the scenes.
The architecture looks more like this:
Your Application
│
Docker
│
Linux VM
│
macOS
│
HardwareFor most developers this is completely transparent.
You still use exactly the same commands:
docker build
docker run
docker compose upThe only noticeable differences are:
- higher memory usage
- slightly slower filesystem access
- Docker Desktop must be running
Despite this, Docker Desktop provides an excellent developer experience and is the recommended option for macOS.
Docker on Windows
Windows offers two different approaches.
Option 1 — Docker Desktop (Recommended)
Most developers install Docker Desktop.
Internally it uses WSL 2 (Windows Subsystem for Linux) together with a lightweight Linux virtual machine.
This provides excellent compatibility while keeping the developer experience simple.
For the vast majority of users, this is the correct choice.
Option 2 — Docker Engine inside WSL
Advanced users often skip Docker Desktop entirely.
Instead they install Docker Engine directly inside a Linux distribution running under WSL 2.
The benefits include:
- lower memory usage
- fewer background services
- more control
- Linux-native workflow
The downside is additional setup complexity.
If you’re just learning Docker, Docker Desktop is perfectly fine.
Minimum Hardware Requirements
Docker itself is lightweight.
The applications you run inside Docker usually consume far more resources than Docker does.
A reasonable development machine today should have:
| Resource | Recommended |
|---|---|
| RAM | 8 GB minimum |
| CPU | 4 cores |
| Storage | 20–50 GB free |
| Operating System | Linux, macOS, or Windows with WSL 2 |
If you’re planning to run databases, Elasticsearch, Kubernetes, or multiple services simultaneously, more memory is always helpful.
Installing Docker
Docker provides official installation instructions for every supported platform.
In general, the installation process follows the same pattern:
- Install Docker.
- Verify the installation.
- Start the Docker daemon.
- Run the first container.
After installation, verify everything is working.
docker versionExample output:
Client:
Version: 28.x.x
Server:
Engine: 28.x.xYou can also inspect available commands:
docker --helpIf both commands work without errors, Docker is ready.
Your First Container
The traditional first example is the official hello-world image.
docker run hello-worldIf everything is configured correctly, Docker will:
- search for the image locally
- download it if necessary
- create a container
- execute it
- display a success message
- exit
During this process you’ll see Docker downloading several layers.
Something similar to:
Unable to find image locally...
Pulling from library/hello-world...
Downloading...
Extracting...
Hello from Docker!Congratulations—you’ve just run your first container.
Understanding What Just Happened
A surprising amount of work happened behind a single command.
docker run hello-worldDocker performed all of the following automatically:
Search Local Images
│
Not Found
│
Download Image
│
Verify Layers
│
Create Container
│
Start Process
│
Print Output
│
ExitThis simple workflow is exactly what Docker does for every application, whether it’s a tiny demo container or a production service handling millions of requests.
Docker Desktop vs Docker Engine
Many beginners think these are the same thing.
They’re related—but not identical.
Docker Engine
Docker Engine is the actual container runtime.
It includes:
- Docker Daemon
- Docker CLI
- Docker API
This is all Linux needs.
Docker Desktop
Docker Desktop is a complete desktop application built around Docker Engine.
It adds features like:
- graphical interface
- integrated settings
- container browser
- image manager
- extension marketplace
- automatic updates
- WSL integration
- Kubernetes integration
Docker Desktop includes Docker Engine—but Docker Engine does not require Docker Desktop.
Should You Use Linux?
Many experienced DevOps engineers eventually switch to Linux as their primary Docker environment.
Not because Docker Desktop is bad—but because Linux provides:
- native performance
- lower memory consumption
- fewer abstraction layers
- better filesystem performance
- production-like behavior
That said, Docker Desktop on macOS and Windows is excellent for learning and everyday development.
Your skills transfer directly because Docker commands are identical across operating systems.
Common Installation Problems
Even modern installations occasionally run into issues.
“docker: command not found”
The Docker CLI is either not installed or not available in your system’s PATH.
“Cannot connect to the Docker daemon”
The daemon isn’t running.
On Linux:
sudo systemctl start dockerPermission Denied
On Linux, users often need to add themselves to the Docker group:
sudo usermod -aG docker $USERThen log out and back in before trying again.
Virtualization Disabled
On Windows and macOS, Docker requires hardware virtualization.
If Docker Desktop refuses to start, check that virtualization is enabled in your BIOS or UEFI settings.
Writing Your First Dockerfile
A Docker image doesn’t appear out of thin air.
Every image begins with a Dockerfile—a plain text file containing a sequence of instructions that Docker follows to build a reproducible environment.
Think of it as a recipe.
Instead of telling a chef how to prepare dinner, it tells Docker how to assemble an application.
A simple Dockerfile can look like this:
FROM node:22
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD ["npm", "start"]Although it’s only a few lines long, this file completely describes how to build and start a Node.js application.
Understanding what each instruction does is far more valuable than memorizing Docker commands.
How Docker Builds an Image
Docker executes instructions one by one.
Every instruction produces a new state of the image.
You can imagine the build process like this:
FROM node:22
│
WORKDIR /app
│
COPY package.json
│
RUN npm install
│
COPY source code
│
CMD npm startDocker saves each step.
If nothing changes, it can reuse previously built steps instead of rebuilding everything from scratch.
This is known as the build cache, and it’s one of Docker’s biggest performance optimizations.
FROM
Every Dockerfile starts with a base image.
FROM node:22or
FROM python:3.12or
FROM nginx:1.27The base image provides the initial filesystem and runtime.
Without a base image, Docker has nowhere to begin.
Choosing a base image is similar to choosing the operating system for a virtual machine.
Different images serve different purposes.
Some are optimized for development.
Others are minimal and designed specifically for production deployments.
WORKDIR
Instead of repeatedly writing absolute paths, Docker allows you to define a working directory.
WORKDIR /appEvery following instruction executes inside that directory.
Without it, your Dockerfile quickly becomes messy:
COPY package.json /app
RUN cd /app && npm install
COPY . /appWith WORKDIR, everything becomes cleaner:
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .It also makes Dockerfiles easier to maintain.
COPY
The COPY instruction transfers files from your machine into the image.
COPY package.json .Later you might copy the rest of the project:
COPY . .One of the biggest beginner mistakes is copying the entire project immediately.
For example:
COPY . .
RUN npm installIt works.
But it’s inefficient.
Every code change forces Docker to reinstall dependencies.
There’s a better approach.
RUN
The RUN instruction executes commands during image creation.
Examples:
RUN npm installRUN pip install -r requirements.txtRUN apt-get updateOnce the command finishes, Docker stores the result as another image layer.
That means the command doesn’t execute every time the container starts.
It executes once during the build.
This distinction is important.
Many newcomers confuse RUN with CMD.
CMD
Unlike RUN, the CMD instruction specifies what should happen when the container starts.
For example:
CMD ["node", "server.js"]or
CMD ["python", "app.py"]or
CMD ["npm", "start"]A good rule to remember is:
- RUN builds the image
- CMD starts the application
One happens during image creation.
The other happens every time the container launches.
EXPOSE
Many Dockerfiles include:
EXPOSE 3000This often confuses beginners.
EXPOSE does not publish a port.
It simply documents that the application listens on a specific port.
To actually make the service accessible from your machine, you still need:
docker run -p 3000:3000 my-appThink of EXPOSE as documentation for humans and orchestration tools.
ENV
Applications often require configuration.
Examples include:
- database URLs
- API endpoints
- secret keys
- environment names
Docker allows you to define environment variables inside the image.
ENV NODE_ENV=productionYour application can then read them normally.
For example, in Node.js:
process.env.NODE_ENVOr in Python:
os.environ["NODE_ENV"]Environment variables make images reusable across different environments without changing application code.
Why Dockerfile Order Matters
Instruction order has a huge impact on build speed.
Consider this Dockerfile:
COPY . .
RUN npm installEvery source code modification invalidates the cache.
Docker must reinstall every dependency.
Now compare it with this version:
COPY package*.json ./
RUN npm install
COPY . .Here the dependency installation depends only on package.json.
If you edit a JavaScript file, Docker reuses the cached dependencies.
Only the final source code layer changes.
This simple optimization often reduces build times dramatically.
Visualizing the Cache
Suppose your Dockerfile looks like this:
FROM node:22
COPY package.json .
RUN npm install
COPY . .
CMD ["npm", "start"]Initial build:
FROM
✓ Build
COPY package.json
✓ Build
RUN npm install
✓ Build
COPY source
✓ Build
CMD
✓ BuildNow imagine you edit only one JavaScript file.
Second build:
FROM
✓ Cached
COPY package.json
✓ Cached
RUN npm install
✓ Cached
COPY source
✓ Rebuild
CMD
✓ ReuseDocker skipped the expensive dependency installation entirely.
That’s exactly what you want.
The Problem with Large Images
Another beginner mistake is creating unnecessarily large images.
Imagine this Dockerfile:
FROM ubuntu
RUN apt-get update
RUN apt-get install -y curl
RUN apt-get install -y git
RUN apt-get install -y vimEvery RUN creates another layer.
More layers aren’t inherently bad.
The problem is that package manager caches remain inside previous layers.
A better version combines commands:
RUN apt-get update && \
apt-get install -y curl git vim && \
rm -rf /var/lib/apt/lists/*Everything happens in a single layer.
The cache is removed before Docker saves the filesystem.
The resulting image is smaller.
Every Layer Is Immutable
One of Docker’s most important concepts is immutability.
Once a layer is created, Docker never changes it.
Instead, every modification creates a new layer.
That’s why rebuilding an image after changing one instruction invalidates every layer that follows.
This behavior explains why Dockerfile organization matters so much.
A well-structured Dockerfile builds in seconds.
A poorly structured one rebuilds everything.
A Production-Friendly Dockerfile
A clean Node.js Dockerfile often looks something like this:
FROM node:22
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
EXPOSE 3000
CMD ["npm", "start"]It’s simple, readable, cache-friendly, and works well for many applications.
As your projects grow, you’ll eventually adopt more advanced techniques such as multi-stage builds, non-root users, health checks, and smaller production images—but this foundation remains the same.
Key Takeaways
By now, you should understand the purpose of the most common Dockerfile instructions:
| Instruction | Purpose |
|---|---|
FROM | Selects the base image |
WORKDIR | Sets the working directory |
COPY | Copies files into the image |
RUN | Executes commands during the build |
ENV | Defines environment variables |
EXPOSE | Documents the application’s listening port |
CMD | Defines the default startup command |
More importantly, you should understand that a Dockerfile is not just a list of commands.
It’s a carefully ordered build process where every instruction affects caching, image size, and deployment speed.
Layers, Build Cache, and Multi-Stage Builds
Once you’ve written your first Dockerfile, the next step is understanding how Docker builds images internally.
This is where many developers make the jump from simply using Docker to using it efficiently.
A poorly optimized Docker image might:
- take several minutes to build
- consume gigabytes of storage
- include unnecessary files
- expose sensitive information
- deploy much slower than necessary
A well-designed image can be dramatically smaller, faster, and easier to maintain.
The difference comes down to understanding layers.
What Is a Layer?
Every significant change Docker makes while building an image becomes a layer.
Imagine this Dockerfile:
FROM node:22
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .Docker doesn’t create one large filesystem.
Instead, it builds something like this:
Layer 1
Base OS
↓
Layer 2
Node.js Runtime
↓
Layer 3
Working Directory
↓
Layer 4
package.json
↓
Layer 5
Installed Dependencies
↓
Layer 6
Application Source CodeEach layer is stored independently.
Once created, a layer never changes.
Instead, Docker simply stacks new layers on top.
Immutable Layers
Docker layers are immutable.
That means they cannot be edited after they have been created.
Suppose Docker has already built this image:
Layer 1
Ubuntu
↓
Layer 2
Node.js
↓
Layer 3
npm install
↓
Layer 4
ApplicationNow you modify one JavaScript file.
Docker doesn’t edit Layer 4.
Instead it creates a completely new Layer 4.
The previous one still exists until Docker decides it is no longer needed.
This immutability makes Docker images:
- reproducible
- cacheable
- easy to distribute
- easy to verify
Why Layers Are Shared
One of Docker’s biggest optimizations is that layers are shared across images.
Suppose you have two projects.
Project A
FROM node:22Project B
FROM node:22Docker downloads Node.js only once.
Both images reference exactly the same layer.
Likewise, if ten different applications use Ubuntu as their base image, Docker stores Ubuntu once.
This dramatically reduces storage usage.
The Build Cache
Docker remembers every previously built layer.
This is called the build cache.
Consider this Dockerfile.
COPY package.json .
RUN npm install
COPY . .First build:
COPY package.json
✓ Build
RUN npm install
✓ Build
COPY source
✓ BuildNow imagine you only edit one file:
src/routes/users.jsDocker checks every step.
Did package.json change?
No.
Reuse cache.
Did npm install need to run again?
No.
Reuse cache.
Only the source code layer changes.
Second build:
COPY package.json
✓ Cached
RUN npm install
✓ Cached
COPY source
✓ RebuiltInstead of reinstalling hundreds of packages, Docker copies only the updated application files.
Cache Invalidation
Now let’s reverse the order.
COPY . .
RUN npm installLooks harmless.
It isn’t.
Every code modification changes the first instruction.
Docker sees:
COPY . .has changed.
Everything after it becomes invalid.
Now the build looks like this:
COPY source
✓ Rebuild
RUN npm install
✓ RebuildEven though dependencies haven’t changed, Docker has no choice but to reinstall them.
That’s why Dockerfile order matters.
Organizing Dockerfiles for Maximum Cache Efficiency
Experienced developers usually structure Dockerfiles like this:
FROM node:22
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
CMD ["npm", "start"]Notice what changes frequently.
Source code changes constantly.
Dependencies change occasionally.
Therefore:
- copy dependency files first
- install dependencies
- copy application code last
This simple optimization often reduces build times by more than 80%.
COPY vs ADD
Docker provides two similar instructions.
COPYand
ADDMost of the time, use COPY.
It does one thing.
Copies files.
COPY . .ADD has additional behavior.
It can:
- extract local archives
- download remote URLs
- perform automatic unpacking
Example:
ADD archive.tar.gz /appDocker automatically extracts the archive.
While this sounds convenient, it also makes builds less predictable.
The official recommendation is simple:
Use COPY unless you specifically need one of ADD’s extra features.
Image Size Matters
Imagine these two images.
Image A
3.2 GBImage B
180 MBBoth run exactly the same application.
Which one would you rather deploy?
Smaller images:
- download faster
- deploy faster
- consume less storage
- reduce registry costs
- improve CI/CD speed
Optimizing image size becomes increasingly important as projects grow.
Package Manager Cache
A classic mistake involves package managers.
Consider this Dockerfile:
RUN apt-get update
RUN apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/*Looks fine.
But Docker stores every RUN instruction separately.
The package cache already exists inside the previous layer.
Deleting it later doesn’t shrink the image.
Instead combine everything:
RUN apt-get update && \
apt-get install -y curl && \
rm -rf /var/lib/apt/lists/*Everything happens inside one layer.
The cache never becomes part of the final image.
Multi-Stage Builds
One of Docker’s most powerful features is the multi-stage build.
Many applications require build tools that should never be shipped to production.
Consider a React application.
During development you need:
- Node.js
- npm
- source code
- TypeScript
- build tools
In production you only need:
- static HTML
- CSS
- JavaScript
Why ship Node.js if the application is already compiled?
That’s exactly what multi-stage builds solve.
Traditional Build
Without multi-stage builds:
Node.js
↓
npm
↓
Source Code
↓
Dependencies
↓
Build Tools
↓
Production ImageEverything ends up inside the final image.
Even tools that are no longer needed.
Multi-Stage Build
Instead:
Stage 1
Node.js
↓
Install Dependencies
↓
Compile Application
↓
Build Output
====================
Stage 2
Nginx
↓
Copy Build Output
↓
Production ImageOnly compiled files move into the final image.
Everything else stays behind.
Example
FROM node:22 AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/htmlDocker creates two completely different environments.
The first image builds the project.
The second image serves only the compiled files.
The Node.js runtime never reaches production.
Benefits of Multi-Stage Builds
Compared with traditional Dockerfiles, multi-stage builds provide:
- much smaller images
- fewer security vulnerabilities
- faster deployments
- cleaner production environments
- fewer unnecessary dependencies
Today, they’re considered a best practice for almost every compiled application.
Cleaning Up Your Images
Over time Docker accumulates:
- unused images
- stopped containers
- dangling layers
- build cache
- unused volumes
Eventually you’ll reclaim disk space with:
docker system pruneor
docker image pruneBe careful.
These commands permanently remove unused Docker resources.
Always review what Docker plans to delete before confirming.
Key Takeaways
Docker layers are far more than an implementation detail.
They’re the reason Docker can:
- build images quickly
- reuse previous work
- distribute images efficiently
- minimize downloads
- support immutable deployments
Understanding how layers interact with caching will help you write Dockerfiles that are faster, smaller, and easier to maintain.
Volumes, Bind Mounts, and Persistent Data
One of the biggest misconceptions new Docker users have is believing that containers behave like virtual machines.
They don’t.
A container is designed to be temporary.
You should always assume that a container can disappear at any moment.
If deleting a container means losing important data, your application architecture probably needs improvement.
This is why Docker provides several mechanisms for storing data outside the container itself.
Understanding them is essential before running databases, file uploads, or any stateful application.
Containers Are Disposable
Imagine you start a PostgreSQL container.
docker run postgresEverything seems fine.
You create databases.
Insert records.
Run your application.
Then one day you accidentally remove the container.
docker rm postgresEverything is gone.
Why?
Because all changes were stored inside the container’s writable layer.
Remember this diagram from earlier?
Docker Image
│
Read-only Layers
│
───────────────
Writable Layer
│
Your ChangesDeleting the container deletes that writable layer.
Docker behaves exactly as designed.
Containers are ephemeral.
Data should not live inside them.
Where Should Data Live?
Docker offers three primary storage mechanisms.
Application
│
Container
│
┌───────────────┬────────────────┬───────────────┐
│ │ │
Volume Bind Mount tmpfsEach serves a different purpose.
Choosing the correct one makes development and production environments much easier to manage.
Docker Volumes
Volumes are Docker’s preferred solution for persistent data.
Instead of writing files inside the container, Docker stores them in a dedicated location managed by the Docker Engine.
You don’t need to know where the files live.
Docker handles everything.
Example:
docker run \
-v postgres-data:/var/lib/postgresql/data \
postgresHere:
postgres-datais the volume.
The PostgreSQL container stores its database inside the volume rather than inside the container itself.
Now imagine deleting the container.
docker rm postgresThe database remains.
Create a new container using the same volume and everything is still there.
How Volumes Work
The relationship looks like this.
Container
│
/var/lib/postgresql/data
│
Docker Volume
│
Host StorageThe application believes it’s writing to its local filesystem.
Docker transparently redirects those writes into the volume.
This separation allows containers to come and go without affecting stored data.
When Should You Use Volumes?
Volumes are ideal whenever the application owns its own data.
Examples include:
- PostgreSQL
- MySQL
- MariaDB
- MongoDB
- Redis persistence
- Elasticsearch
- RabbitMQ
- uploaded user files
- CMS content
A simple rule:
If the application creates the data, it probably belongs in a volume.
Bind Mounts
Volumes are managed by Docker.
Sometimes that’s not what you want.
Suppose you’re developing a Node.js application.
Your source code already exists on your laptop.
You don’t want Docker to make another copy.
Instead, you want the container to read your existing files directly.
That’s where bind mounts come in.
Example:
docker run \
-v $(pwd):/app \
node:22Now the container uses your local project directory.
Any file you edit immediately becomes visible inside the container.
No rebuilding required.
How Bind Mounts Work
Unlike volumes, bind mounts point to a specific path on the host.
Your Project
C:\Projects\App
│
Bind Mount
│
/app
│
ContainerDocker doesn’t manage the data.
It simply connects the two locations.
Common Development Workflow
This is the most common Docker development workflow.
VS Code
│
Local Files
│
Bind Mount
│
Container
│
Node.jsYou save a file.
The container immediately sees the update.
Hot reload works exactly as if the application were running locally.
Volume vs Bind Mount
Developers often ask which one is better.
Neither.
They solve different problems.
| Volume | Bind Mount |
|---|---|
| Managed by Docker | Managed by you |
| Portable | Depends on host paths |
| Great for databases | Great for source code |
| Better for production | Better for development |
| Easier backups | Easier editing |
A practical rule:
Use volumes for application data.
Use bind mounts for project files.
tmpfs Mounts
Docker also supports memory-backed storage.
Instead of writing to disk, data exists only in RAM.
Container
│
tmpfs
│
MemoryNothing survives container shutdown.
This makes tmpfs useful for:
- temporary caches
- cryptographic operations
- sensitive temporary files
- high-performance workloads
Because nothing touches the disk, access is extremely fast.
However, data disappears as soon as the container stops.
Why Databases Should Never Store Data Inside Containers
Consider two PostgreSQL deployments.
Bad
PostgreSQL
↓
Container Filesystem
↓
Delete Container
↓
Database LostGood
PostgreSQL
↓
Docker Volume
↓
Delete Container
↓
Database Still ExistsThis is why every official PostgreSQL example uses volumes.
The container is disposable.
The data is not.
Sharing Data Between Containers
Volumes also make it easy for multiple containers to access the same files.
For example:
Application
│
Shared Volume
/ \
Nginx WorkerBoth containers can read and write the same persistent storage.
This pattern is common for:
- uploaded images
- generated PDFs
- media processing
- backups
- shared configuration
Inspecting Volumes
Docker provides commands for managing volumes.
List existing volumes:
docker volume lsInspect one:
docker volume inspect postgres-dataCreate manually:
docker volume create postgres-dataDelete unused volumes:
docker volume pruneRemember that pruning removes unused data permanently.
Always verify that a volume is no longer needed before deleting it.
Common Beginner Mistakes
Storing databases inside containers
This works—until the container is removed.
Using bind mounts everywhere
Bind mounts are excellent during development but often make production deployments harder to reproduce because they depend on specific host paths.
Forgetting file permissions
Containers and host operating systems may use different user IDs.
This can lead to confusing “Permission denied” errors.
We’ll discuss Docker users and permissions in a later chapter.
Mounting over existing files
Suppose your image contains:
/app/config.jsonThen you bind mount:
/appEverything originally inside /app becomes hidden by the mounted directory.
This surprises many beginners.
Docker isn’t deleting the files—they’re simply masked by the mount.
Named Volumes vs Anonymous Volumes
Docker actually supports two kinds of volumes.
Named volume:
docker run \
-v postgres-data:/var/lib/postgresql/data \
postgresAnonymous volume:
docker run \
-v /var/lib/postgresql/data \
postgresAnonymous volumes receive random names.
Over time they accumulate and consume disk space.
Named volumes are easier to:
- identify
- reuse
- back up
- migrate
- remove safely
For long-lived applications, named volumes are almost always the better choice.
Best Practices
A few simple rules will save you many headaches later:
- Never rely on a container’s writable layer for important data.
- Use named volumes for databases and persistent application data.
- Use bind mounts during local development.
- Avoid hardcoding host-specific paths in production.
- Regularly clean up unused anonymous volumes.
- Treat containers as disposable and storage as permanent.
If you follow these principles, replacing a container should become a routine operation rather than a stressful event.
Key Takeaways
Persistent storage is one of the first concepts that separates production-ready Docker deployments from simple experiments.
The most important thing to remember is this:
Containers are temporary. Data is not.
Once you embrace that mindset, Docker’s storage model becomes straightforward.
- Volumes preserve application data.
- Bind mounts connect existing host directories.
- tmpfs provides temporary in-memory storage.
- The container itself should never be considered permanent storage.
Docker Networking Explained
Most real-world applications consist of more than a single container.
A typical web application might include:
- a frontend
- a backend API
- PostgreSQL
- Redis
- Nginx
- a background worker
These services need to communicate with one another.
Docker Networking provides the infrastructure that makes this possible.
Fortunately, you don’t need to become a network engineer to understand it.
Once you learn a few core concepts, Docker networking becomes surprisingly intuitive.
Every Container Gets Its Own Network
When Docker starts a container, it creates an isolated network environment.
Each container receives:
- its own IP address
- its own network interfaces
- its own routing table
- its own ports
Think of every container as a tiny computer connected to a virtual network.
Host Machine
│
──────────────────────────
Container A
IP: 172.x.x.x
Container B
IP: 172.x.x.x
Container C
IP: 172.x.x.xAlthough everything runs on one physical machine, Docker makes each container feel like an independent server.
The Default Bridge Network
If you don’t specify anything, Docker places containers into the bridge network.
You can inspect available networks:
docker network lsTypical output:
NETWORK ID NAME DRIVER
xxxx bridge bridge
xxxx host host
xxxx none nullThe default bridge network allows outbound internet access.
For example, an Ubuntu container can install packages:
apt updateA Node.js application can call an external API.
A Python script can download dependencies.
However, containers launched separately on the default bridge network cannot automatically discover each other by name.
That’s why user-defined networks are usually a better choice.
User-Defined Networks
Creating your own network is simple.
docker network create app-networkNow launch containers inside it.
docker run -d \
--network app-network \
postgresdocker run -d \
--network app-network \
redisdocker run -d \
--network app-network \
my-apiNow all three containers can communicate directly.
No manual IP configuration required.
Containers Can Talk by Name
One of Docker’s most useful features is automatic DNS.
Instead of remembering IP addresses, containers simply use container names.
For example:
Frontend
↓
api
↓
postgresThe backend connects to PostgreSQL like this:
Host:
postgres
Port:
5432Not
172.18.0.14Docker automatically resolves:
postgresto the correct container.
This makes applications much easier to configure.
Why Container IPs Should Never Be Hardcoded
Container IP addresses are temporary.
Suppose your PostgreSQL container restarts.
Yesterday:
172.18.0.4Today:
172.18.0.7Nothing guarantees the address remains the same.
Container names, however, stay stable.
Always configure applications using service names.
This is exactly what Docker Compose does.
Publishing Ports
A common misunderstanding is that containers are automatically accessible from your computer.
They’re not.
Imagine a Node.js application listening on port 3000.
Inside the container:
localhost:3000Only the container itself can reach that address.
To expose it externally:
docker run -p 3000:3000 my-appDocker creates this mapping:
Host
localhost:3000
↓
Container
3000Now your browser can reach the application.
Understanding Port Mapping
The syntax is:
HOST_PORT:CONTAINER_PORTExample:
docker run -p 8080:80 nginxResult:
Browser
↓
localhost:8080
↓
Container Port 80Notice that the ports don’t need to match.
Docker simply forwards traffic.
EXPOSE vs -p
Earlier we saw:
EXPOSE 3000Many beginners think this publishes the application.
It doesn’t.
EXPOSE
- documents which port the application uses
-p
- actually publishes the port
Without -p, your application remains private inside Docker’s virtual network.
Container-to-Container Communication
Suppose your API needs PostgreSQL.
Inside Docker:
API
↓
postgres:5432The API never needs to know the database IP.
Docker’s internal DNS handles everything.
Likewise, Redis becomes:
redis:6379Nginx becomes:
nginx:80This naming system makes Compose files remarkably readable.
Docker Compose Networking
Imagine this Compose file.
services:
api:
postgres:
redis:Docker automatically creates a private network.
Every service joins it.
Communication becomes:
api
↓
postgres
↓
redisNo extra configuration required.
This automatic networking is one of the reasons Docker Compose is so popular.
Is the Database Public?
Not necessarily.
Consider this Compose file:
services:
postgres:No published ports.
Result:
Browser
❌
Cannot Access DatabaseOnly containers inside the network can reach PostgreSQL.
Now compare:
ports:
- "5432:5432"Result:
Internet
↓
Your Computer
↓
PostgreSQLThe database becomes externally reachable.
In development this is useful.
In production it often isn’t.
Many production deployments never expose database ports publicly.
Multiple Networks
A container can join more than one network.
Example:
Internet
↓
Nginx
↓
Frontend Network
↓
API
↓
Backend Network
↓
PostgreSQLHere:
- Nginx talks to users
- API talks to Nginx
- PostgreSQL talks only to the API
This creates natural security boundaries.
Host Network Mode
Docker also supports:
--network hostIn this mode the container shares the host’s network stack.
Advantages:
- slightly lower latency
- no port forwarding
- simpler networking
Disadvantages:
- less isolation
- possible port conflicts
- Linux only
Most applications should continue using bridge networks.
The None Network
Docker even supports complete network isolation.
--network noneResult:
Container
×
Internet
×
Other Containers
×
Host NetworkThe container has no network access whatsoever.
This can be useful for highly restricted workloads.
Inspecting Networks
Useful commands include:
List networks:
docker network lsInspect one:
docker network inspect app-networkConnect an existing container:
docker network connect app-network my-apiDisconnect:
docker network disconnect app-network my-apiRemove unused network:
docker network rm app-networkCommon Networking Mistakes
Using localhost inside containers
Inside a container:
localhostmeans:
the container itself
It does not refer to another container.
To connect to PostgreSQL:
Wrong:
localhostCorrect:
postgresHardcoding IP addresses
Never use:
172.18.0.5Always use service names.
Docker manages the addresses automatically.
Publishing Everything
Many beginners expose every service.
Frontend
Backend
Redis
PostgreSQL
RabbitMQOnly publish ports that users actually need.
Internal services should remain inside Docker’s private network.
Forgetting Firewalls
Publishing a port makes it reachable from the host.
It does not automatically make it safe.
Production servers should still use firewalls, reverse proxies, and proper authentication.
Docker networking is not a replacement for network security.
Best Practices
A few habits make Docker networking much cleaner:
- Use user-defined bridge networks instead of the default bridge.
- Connect services using container names, never IP addresses.
- Publish only the ports that must be accessible externally.
- Keep databases and internal services on private networks.
- Let Docker handle DNS resolution.
- Use Docker Compose whenever multiple services need to communicate.
Key Takeaways
Networking is what transforms Docker from a tool that runs isolated containers into a platform capable of hosting complete distributed applications.
The most important ideas are:
- Every container has its own network namespace.
- Docker provides automatic DNS for user-defined networks.
- Containers should communicate using service names.
EXPOSEdocuments ports;-ppublishes them.- Internal services don’t need to be publicly accessible.
- Docker Compose automatically creates secure, private networks for your applications.
Once you understand these concepts, building multi-container applications becomes significantly easier.
Docker Best Practices
By this point, you’ve learned the core concepts behind Docker: images, containers, Dockerfiles, storage, and networking. Before you start containerizing your own projects, it’s worth covering a few best practices that will save you time, reduce security risks, and make your images easier to maintain.
These aren’t advanced tricks—they’re habits that experienced Docker users follow every day.
Never Use latest in Production
Docker automatically assumes the latest tag when no version is specified.
docker pull nginxis equivalent to:
docker pull nginx:latestWhile convenient during experimentation, latest introduces uncertainty. The image behind that tag can change at any time, meaning two deployments executed on different days may run different software.
Instead, pin explicit versions whenever possible.
nginx:1.27.1
postgres:17.5
node:22.17.0
python:3.12.5Immutable versions make deployments predictable and simplify rollbacks.
Always Create a .dockerignore
Many developers pay close attention to .gitignore but forget about .dockerignore.
During every build, Docker sends the entire build context to the Docker daemon. That includes every file in the current directory unless it has been excluded.
A typical .dockerignore might look like this:
.git
.gitignore
.env
.env.*
node_modules
dist
build
coverage
*.log
.DS_StoreIgnoring unnecessary files provides several benefits:
- faster builds
- smaller build contexts
- reduced image size
- lower risk of accidentally shipping secrets
As a rule of thumb, every Docker project should include a .dockerignore file.
Don’t Run Applications as Root
By default, containers run as the root user.
While this is convenient during development, it’s rarely a good idea in production.
If an attacker compromises your application, running as root gives them far more privileges than necessary.
Instead, create a dedicated application user.
RUN addgroup --system app \
&& adduser --system --ingroup app app
USER appRunning with the minimum required privileges follows the principle of least privilege and significantly improves container security.
Keep Images Small
Smaller images offer several advantages:
- faster downloads
- quicker deployments
- lower storage costs
- reduced attack surface
Some simple ways to reduce image size include:
- choosing lightweight base images
- removing package manager caches
- using multi-stage builds
- excluding unnecessary files with
.dockerignore
Small improvements quickly add up, especially in CI/CD pipelines.
Take Advantage of Docker’s Build Cache
Docker’s build cache exists to make rebuilds fast.
Structure your Dockerfiles so that rarely changing instructions appear first.
For example:
COPY package*.json ./
RUN npm ci
COPY . .This allows Docker to reuse the dependency layer whenever only your application code changes.
Poor instruction ordering can easily turn a ten-second rebuild into several minutes.
Treat Containers as Disposable
One of Docker’s most important design principles is that containers are temporary.
You should be able to stop, remove, and recreate any container without losing important data.
If deleting a container breaks your application, reconsider where that data is being stored.
Persistent data belongs in volumes, object storage, or external services—not inside containers.
Learn the CLI
Graphical tools are helpful, but sooner or later you’ll find yourself connected to a remote Linux server over SSH.
Knowing a handful of Docker commands will take you much further than relying on a desktop interface.
The commands you’ll use most often are:
docker ps
docker logs
docker exec
docker images
docker build
docker run
docker compose up
docker compose down
docker system pruneMastering these commands provides a solid foundation for everyday Docker development.
Final Thoughts
Docker has fundamentally changed how modern software is developed, tested, and deployed.
It solves one of the oldest problems in software engineering: ensuring that applications behave consistently across different environments.
More importantly, Docker encourages a different way of thinking.
Applications become self-contained.
Infrastructure becomes reproducible.
Deployments become predictable.
Once you understand images, containers, networking, storage, and Dockerfiles, you’ll discover that Docker isn’t a complicated technology—it’s a collection of simple concepts that work together remarkably well.
Whether you’re building personal projects, deploying production APIs, preparing for DevOps interviews, or exploring cloud-native technologies, Docker is a skill that will continue to pay dividends throughout your career.
The best way to learn from here is simple: install Docker, containerize one of your own applications, experiment with different Dockerfiles, and don’t be afraid to break things. Every container can be rebuilt, and that’s one of Docker’s greatest strengths.
What you’ve learned in this guide
✅ Why Docker exists
✅ Images vs Containers
✅ Docker Layers
✅ Dockerfiles
✅ Build Cache
✅ Volumes and Bind Mounts
✅ Docker Networking
✅ Production Best Practices
With these fundamentals in place, you’re well prepared to explore more advanced topics such as Docker Compose, image optimization, CI/CD pipelines, container orchestration, and Kubernetes. Those tools build on the same concepts you’ve learned here—they simply apply them at a larger scale.
