Saturday, December 15, 2018

How To Use A Tintri VMstore For Docker Persistent Volumes

Note:  EVERYTHING IN THIS POST IS UNSUPPORTED BY TINTRI

After I did some lab work with the Minio Gateway using NFS it occurred to me that something similar could be done but using a Tintri VMstore as the target for Docker Volume persistent Storage.  While this seems easy on the surface it has a lot of considerations which is why it’s not something currently supported by Tintri, but it does in fact work.  The premise is built on the creation of a Docker Volume that is mounted to an NFS share first then used to attach to a container.  Most people know that Tintri uses NFSv3 to connect so I found that this does work but has some very strict considerations.

Docker Persistent Volumes on Tintri Considerations

  1. You should always create a quota based Tintri Service group to ensure you do not consume all the available storage on the array.  This is a safety net for the most part.
  2. You can only mount with the “nolock” switch since Tintri is NFSv3 based so you should create other subfolders in the service group folder and mount each of those individually to prevent potential lock issues.  Without locking you cannot have multiple containers access the same data for any large scale use.  This problem is actually solved with NFSv4.
  3. These Docker Volumes not able to take advantage of advanced functions like snapshots, but I’m not able to tell if de-duplication or compression could be applied but could very well come into play behind the scenes.
  4. The Docker host must have access to the Tintri Data IP address
  5. Since this is not akin to a vDisk and it is a directory tree of files, it’s worth warning against using large numbers of small files.  The Tintri filesystem is optimized toward fewer, large files rather than billions of small files. Large numbers of small files could cause some internal resource pressure on supported workloads.

Assuming you have network access the easiest way to test this if you don’t have a docker host is by deploying PhotonOS as in my previous post.  The commands are very similar with the exception of creating the Tintri service group and subfolders first.

Create Tintri Service Group With Quota

Using Tintri Global Center (TGC) click Service Groups and click New Service Group and choose option 3

Select the VMstore you want to use and then assign a sub mount folder name for the group.  It will always be a sub folder to the root “/tintri” folder

Once you complete these steps you can assign the quote to the group to protect the storage from being consumed too much.  Simply edit the settings and add a Quota of your choice.

At this point you can mount the folder and create additional sub-folders to use for your individual Docker volumes.  Some people may be able to mount these with a standard NFS client, but for some of you these may only be visible through the vSphere Datastore browser so that works too.  You can see below I’ve created two sub folders for a Unifi and NGINX container.  These folders will be passed into the Docker host to be used as persistent storage for these containers.

Now we can execute the Docker Volume commands to set up the mount points to this Tintri VMstore and the associated folders.

docker volume create --driver local --opt type=nfs \
   --opt o=addr=<Tintri_DataIP>,rw,nolock \
   --opt device=:/tintri/Colotti-Docker/unifi unifi

docker volume create --driver local --opt type=nfs \
   --opt o=addr=<Tintri_DataIP>rw,nolock \
   --opt device=:/tintri/Colotti-Docker/nginx nginx

Then we can deploy the containers themselves inserting the new volume as the persistent storage

docker create \
  --name=unifi \
  --restart unless-stopped \
  -v unifi:/config \
  -p 3478:3478/udp \
  -p 10001:10001/udp \
  -p 8080:8080 \
  -p 8081:8081 \
  -p 8443:8443 \
  -p 8843:8843 \
  -p 8880:8880 \
  -p 6789:6789 \
  linuxserver/unifi

docker create \
  --name=nginx \
  --restart unless-stopped \
  -v nginx:/config \
  -p 80:80 -p 443:443 \
  linuxserver/nginx

Once the container is deployed you can refresh the vSphere datastore browser and see that the configuration data is now in that folder and can be mounted later to ensure that changes are kept when a new version of the container is deployed.

What is cool about this from my PhotonOS post is that all the NFS mounting is being controller by Docker with no need for NFS-Utils or other package requirements on the PhotonOS virtual machine.  This approach is all within the constructs of Docker and both of these containers are bound by the 500GB quota of the Tintri service group as you can see from the console of the NGINX container.

Conclusion of Docker Persistent Volumes on a Tintri VMStore

I need to say this again that this is UNSUPPORTED by Tintri, but I’ve proven this does in fact work and could have some cool potential use in the future.  Containers and their data could be treated like any other virtual workload and getting them on the filesystem is one thing.  Applying all the advanced Tintri functions to them is a completely other animal.  Bottom line is this does work and it is cool to play with in a lab, especially if that is where you are playing with containers anyhow.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Scroll To Top
%d bloggers like this: