Docker Volumes: An Introduction to Data Persistence in Docker Containers

Docker Volumes provides an easy way to persist data in Docker containers. Learn about volumes and how they can be used with Docker Volumes and more in this article.

If you're new to Docker, you may wonder how data persistence works in containers. After all, containers are designed to be lightweight and portable, so it might seem like storing data in them would be difficult.

Enter Docker volumes. Volumes allow you to store data outside of your container and mount it into your container at runtime. This makes it easy to persist data between container restarts or even between deployments.

In this blog post, we'll provide an introduction to Docker volumes and explain how they work. We'll also show you how to create and manage volumes using the Docker CLI. Whether you're a student just getting started with Docker or a seasoned pro looking for a refresher on data persistence, this blog post is for you. So let's get started!

Table of Contents

Persistence Storage Docker Docker Data Storage Docker Volumes How do docker volumes function? Bind Mounts Conclusion

Introduction: Persistence Storage Docker

Containers are a lightweight and efficient way to package applications along with their dependencies, allowing developers to easily deploy and manage them in different environments. This approach offers some significant benefits, such as increased flexibility, scalability, and portability. With containers, developers can build once and run anywhere, making it easier to move applications from development to production environments without any additional configuration or setup.

Containers have revolutionized the software development industry with their ability to package applications and their dependencies into portable, lightweight units. 

Data persistence ensures that important information can be stored and retrieved even after a container has been deleted. Docker volumes provide a solution for managing persistent data in containers.

Why is data persistence important in containerized environments?

While containers are great for running and scaling applications, they are usually ephemeral, meaning that any data or changes made within the container are lost once it's deleted. This makes data persistence critical in containerized environments, as important information needs to be stored and retrieved even after a container has been destroyed. Docker volumes offer an effective solution for managing persistent data in containers and can help ensure that your data is always available when you need it.

In order to understand how Docker volumes work, it's helpful to first understand the concept of Docker data storage. In a containerized environment, there are two main types of data storage: ephemeral and persistent. Ephemeral storage is stored within the container itself and is lost when the container is stopped or deleted. Persistent storage, on the other hand, is stored outside of the container and can persist even when the container is restarted or replaced.

Docker volumes are a way to implement persistent storage in containers. They allow you to store data outside of your container and mount it into your container at runtime. This provides a more flexible and scalable approach to data storage, as your data can now exist independently from your container. Docker volumes can be managed and manipulated using the Docker CLI, making it easy for developers and operations teams to use and deploy them with ease.

One key benefit of Docker volumes is that they can support multiple containers. By mounting the same volume into different containers, you can share data between them without having to copy it or synchronize it manually. This is particularly useful in microservice architectures, where services often need to communicate and share data with each other.

Docker Data Storage

Docker containers keep all the content being used on the container readable level, which is only stored till the lifecycle of the container, i.e. it is no longer available after the container is deleted. Docker images are built-in forms of layers. Also, if other processes need the data, it becomes challenging to remove it from the container due to this.

Docker offers two techniques to persist data independent of the container's lifetime so that the files are still accessible on the host directory even if the container is no longer active:

1. Docker Volumes

2. Bind Mounts

When the storage level, such as mount or volume, is not defined before the Docker container is launched. The in-memory storage file system will be used to operate the docker container. The information will be temporarily stored. The data will disappear whenever the corresponding Docker container is deleted or stopped. It does not contain the current functioning data.

Docker Volumes

Four major Docker Objects are:

• Containers 

• Volumes 

• Images 

• Networks

In Docker, volumes are the chosen method of data persistence. Compared to bind mounts, they are simpler to back up or move. Either the Docker API or the CLI may be used to manage them. They can be distributed among several containers more securely. You may store volumes on distant servers or cloud providers, secure the information inside of volumes, and add additional functionality with volume drivers. A container can pre-fill the content of new volumes. improved Docker Desktop responsiveness on Windows and Mac

Volumes are kept in a section of the host filesystem that is under Docker's management. The operating system determines the precise folder. On Linux, for instance, it's:

These folders should never be altered by non-Docker processes.

How do docker volumes function?

The primary method for information exchange across containers is Docker volumes. Volumes have one benefit over bind mounts: they automatically generate themselves the initial time they are attached to a container, eliminating the need for explicit creation.

The volume and the files continue to exist when that specific container is stopped. An additional advantage of volumes is that they may be mapped to numerous containers, both in read-write or read-only mode.

The permanent data storage level of Docker is another name for volume storage. The standard directory for the docker volume is /var/lib/docker/volumes. whenever a volume is being created or linked to a Docker container. By default, the volume will be stored in the /var/lib/docker/volumes directory. 

Volume mounting and migration are really simple. whether the volume is still there on the storage layer even after removing the docker container. Afterward, you may utilize the same storage for the next or a different Docker container. The identical information that was previously utilized by the earlier container will be utilized.

When you build a volume, it is kept on the Docker host in a folder. This folder is placed in the container when the volume is mounted. Docker manages these volumes that are isolated from the major features of the host machine.

Example 1:

On the command line of your Docker host, type the following to display volumes:

My Computer/mnt/C/Users/FirstLearning docker volume Is DRIVER VOLUME NAME My Computer/mnt/C/Users/FirstLearning

As you can see, this host doesn't have any volumes.

To create a volume, type the following command:

My Computer /mnt/C/Users/FirstLearning docker volume ls My Computer /mnt/C/Users/FirstLearning docker volume create volume1 Volume1 My Computer /mnt/C/Users/FirstLearning docker volume ls DRIVER VOLUME NAME Local     volume1

For the reasons mentioned above, volumes are the recommended method for storing data produced by and consumed by Docker containers.

You may find out crucial details about the volume.

My Computer /mnt/C/Users/FirstLearning docker volume inspect volume1 [ { "CreatedAt": "2023-03-08T08:58:24Z", "Driver": "local",

Here, certain information is shown. However, the newly generated volume is not linked to any containers.

What is the best way to use a volume with a container then?

The container will now utilize that volume to persist data if you launch a container with the option -v specified:

The folder inside the container is called /app, and the volume is named volume1.

My Computer /mnt/C/Users/FirstLearning docker run -it -v volume1:/app ubuntu bash root@845faeee3d63:/#

Since the Ubuntu image is not already saved on my machine, Docker will first retrieve it from Docker Hub, the company's primary open container repository where many images are kept.

Examining the volume now:

"Mounts": [ { "Type": "volume", "Name": "volume1", "Source": "/var/lib/docker/volumes/volume1/_data" "Destination": "/app", "Driver": "local", "Mode": "z", "RW": true, "Propagation": "" } ]

As you can observe from my test, several Docker containers can share a volume that has been produced and associated to one of them. One container may write content to the volume while the other retrieves it, based on the mounting style you choose. You truly have a choice.

Tip: Always use a database to create a use case for docker volumes.

Example 2:

To illustrate how Docker volumes work, let's consider a simple web server application that needs to persist user data. By creating a Docker volume and mounting it in the container, you can ensure that the data remains accessible even if the container is deleted or recreated.

First, create a volume using the "docker volume create" command:

$ docker volume create mydata

This creates a new volume named "mydata". You can then start a container and mount this volume using the "-v" option:

$ docker run -d --name myapp -v mydata:/ app/data myimage

This command starts a new container named "myapp" from the image "myimage" and mounts the "mydata" volume at the "/app/data" directory within the container. Any data stored within this directory will be persistent even if the container is deleted or recreated.

You can then verify that our data is persisting by creating a file within the mounted volume:

$ docker exec -it myapp bash $ cd /app/data $ touch myfile.txt

If you now delete and recreate the container, you can see that our file "myfile.txt" still exists in the volume when you start a new container and mount the same volume:

$ docker rm myapp $ docker run -d --name myapp -v mydata:/app/data myimage $ docker exec -it myapp bash $ cd /app/data $ ls myfile.txt

As you can see, by using Docker volumes, you have ensured that our data is persistent and always accessible, even if the container itself is destroyed or recreated. This makes Docker volumes an essential tool for managing persistent data in containers.

How to create and use a Docker volume in a simple application like a web server?

To create and use a Docker volume in a simple web server application, follow these steps:

1. Create a new directory on your host machine where you will place all the necessary files for your web server.

2. Write the code for your web server and save it in that directory.

3. Create a Dockerfile in the same directory with the following instructions:

FROM nginx COPY index.html /usr/share/nginx/html

4. Build the Docker image with the following command: 

docker build -t my-web-server .

5. Create a Docker volume with the following command:

docker volume create my-volume

6. Run the Docker container and attach the volume to it with the following command:

docker run -d -p 80:80 --name my-container -v my-volume:/usr/share/nginx/html my-web-server

7. Verify that the container is running and that the volume is attached correctly by visiting http://localhost in your web browser.

8. Additionally, you can modify files on the volume by running another container that mounts the same volume with the following command:

docker run -it --name my -second-container -v my-volume:/usr/share/nginx/html busybox sh

9. Now you can modify files in the volume from the second container and see the changes reflected in the web server running in the first container.

Using Docker volumes allows for easier management of data persistence and makes it simpler to share data between containers. By following these steps, you can easily create and use a Docker volume in a simple web server application.

Bind Mounts

Bind mounts are an extremely ancient technique for mounting a file or directory to a Docker container. As compared to the volume, the bind mount's capability is restricted. When you are utilizing the bind mount mechanism, you are using the native file or the local path to the docker container. 

The entire location of the local folder or local subdirectory on the local system must be specified in the docker container. You must depend on the local machine configuration files while working with the bind mounts. You must abide by the norms and guidelines that are placed on the local system.

When a docker volume is generated, it is kept on the host computer since docker volumes are typically handled by Docker processes and are a component of the host's filesystem. These are a few essential qualities of volumes:

• By using the docker volume create function, volumes are expressly established; they have a one-to-many relationship, allowing one volume to be associated with one or many processes.

• Volumes are nameless by default. By utilizing the name option in the following syntax: 

docker volume create --name flaskapi-vol;

• Containers believe that volumes are the best means of data persistence.

Although Docker bind mounts provide fewer features than volumes, they let you persist data from containers. A document or folder from the host operating system must be installed into the containers for the bind mount to work. If such a directory or file doesn't already exist, it will be created upon request. A bind mount is always identified by the host system's complete path. The host operating system (containing files, applications, and others) is exposed to the containers when you use bind mounts, which is a significant consideration. The capacity to create, alter, or remove files on the host operating system means that programs running inside the container have this capability. Since non-Docker OS processes may be impacted, this is a security problem. The -v or --volume options can be used to mount bind volumes.

Another option called --mount does the same thing. The --volume/-v option will always generate the directory you want to mount if it doesn't already exist on the Docker host. Nevertheless, using the --mount flag will result in an error and prevent you from creating that directory. Because of this, using --volume/-v to mount a file or directory is recommended.

Utilization Bind Mounts

Despite their restricted functionality, bind mounts can be used in the following situations. -

• It may be used to give the host and container access to shared system settings. For DNS resolution, Docker mounts "/etc/resolv.conf" to containers.

• You can utilize bind mounts to keep records if the parent file system is assured to find consistency and the mounting volume is not accessible to non-Docker processes.

Difference between volumes and bind mounts

Volumes and bind mounts in Docker are two ways to persist data between container instances.

Parameter	Docker Volumes	Bind Mounts
Definition	A volume is a managed storage location, which allows data to be stored independently of the container, making it easier to manage and share data between containers.	Bind mounts are simply a direct link between a directory on the host machine and a directory inside the container. This means that changes made on the host machine to the files in the linked directory will be reflected inside the container, and vice versa.
Lifecycle	Volumes have a lifecycle independent of any container, meaning that they can exist even when no containers are using them.	Since bind mounts are nothing more than a link between two directories, they are not managed by Docker, and their lifecycle is tied to that of the container.
Back up	Volumes can be backed up, restored, or migrated between hosts more easily than bind mounts.	If a container using a bind mount is deleted or recreated, the data stored on the host machine can potentially be lost.
Application	Volumes are recommended for managing persistent data in Docker containers as they provide better isolation and management capabilities.	Bind mounts are useful for certain use cases where direct access to host files is required, such as when developing code on the host machine and testing it inside a container.

Ultimately, both volumes and bind mounts have their strengths and weaknesses, and the choice between them depends on the specific needs of your application. It's important to weigh the pros and cons of each option and make an informed decision based on your requirements.

Conclusion

You have seen the complete idea of "Docker Volumes" together with the proper illustration, justification, and command with various results. Data is accessible on the Docker host computer or the remote Docker machine when permanent storage is defined in Docker. They will still be there if you remove the Docker container. From the developer's perspective, docker persistent storage is more crucial.

With any luck, this post can help you get started using Docker volumes and conduct your experiments. As always, I sincerely hope that you have learned something from this, and I want to thank you for reading.