Filesystems
Understanding filesystems that are being used in containerization is essential to design and through debugging. Specifically, we will focus on how Snap and Docker will utilize these concepts.
FUSE
FUSE (Filesystem in USErspace) is a software interface for Unix and Unix-like computer operating systems that allows non-privileged users to construct their own file systems without altering kernel code. This is accomplished by executing file system code in user space, with the FUSE module simply acting as a bridge to the real kernel interfaces. FUSE is available for the following operating systems: Linux, FreeBSD, OpenBSD, NetBSD (as puffs), OpenSolaris, Minix 3, macOS, and Windows. FUSE is free software licensed under the GNU General Public License and the GNU Lesser General Public License.
The FUSE system was initially a component of AVFS (A Virtual Filesystem), a filesystem implementation greatly influenced by the GNU Hurd translator idea. It replaced the Linux Userland Filesystem and included a translational interface in libfuse1. In kernel version 2.6.14, FUSE was integrated into the mainline Linux kernel tree.
A handler application connected to the provided libfuse library must be created in order to implement a new file system. This program's primary function is to describe how the file system should respond to read/write/status requests. In addition, the software is needed to mount the new file system. The handler is registered with the kernel when the file system is mounted. When a user submits read/write/status requests for this freshly mounted file system, the kernel transmits these IO-requests to the handler and then returns the handler's answer to the user. FUSE is especially handy when creating virtual file systems. Unlike typical file systems, which interact with data on mass storage, virtual filesystems do not actually store data. They function as a representation or translation of an existing file system or storage device. Any resource available to a FUSE implementation can, in theory, be exported as a file system.
Union mounts
Union mounting is a method of merging several directories into one that seems to contain the contents of all of them. Union mounting was first introduced in Linux 0.99 in 1993, under the name the Inheriting File System, but its creator abandoned it due to its complexity. UnionFS, which developed out of the FiST project at Stony Brook University, was the next significant implementation. aufs, an effort to replace UnionFS, was published in 2006, followed by OverlayFS in 2009. OverlayFS union mount support was added to the standard Linux kernel source code in 2014.
SquashFS
SquashFS is a Linux compressed read-only file system. Squashfs compresses files, inodes, and directories and supports block sizes ranging from 4 KiB to 1 MiB for maximum compression. There are several compression algorithms offered. Squashfs is also the name of free software for accessing Squashfs filesystems that is released under the GPL. Squashfs is designed for usage as a general-purpose read-only file system as well as in limited block-device memory systems (e.g., embedded devices) where little overhead is required.
Squashfs is used by Arch Linux, Debian, Fedora, Gentoo Linux, HoleOS, Linux Mint, Salix, Ubuntu, Clonezilla, and embedded distributions such as OpenWrt and DD-WRT router firmware. To offer a read-write environment for live Linux distributions, it is frequently coupled with a union mount filesystem, such as UnionFS, OverlayFS, or aufs. This makes use of Squashfs's high-speed compression capabilities as well as the ability to change the distribution while running it off a live CD. The AppImage project, which seeks to develop portable linux programs, creates appimages with squashfs. Squashfs is also used by the Snappy package management for its .snap file format.
Squashfs was originally kept as an out-of-tree Linux patch. The first version, 1.0, was published on October 23, 2002. Squashfs was integrated into Linux mainline as part of Linux 2.6.29 in 2009. The backward-compatibility code for previous formats was deleted during this procedure. The Squashfs kernel space code has since been maintained in the Linux mainline tree, while the user space tools are still available on the project's GitHub website.
Squashfs's original version used gzip compression, but Linux kernel 2.6.34 added support for LZMA and LZO compression, Linux kernel 2.6.38 added support for LZMA2 compression (used by xz), Linux kernel 3.19 added support for LZ4 compression, and Linux kernel 4.14 added support for Zstandard compression. The Linux kernel 2.6.35 now supports extended file attributes.
AUFS
aufs (short for advanced multi-layered unification filesystem) is a Linux file system union mount. Until version 2, the name stood for AnotherUnionFS. aufs, created in 2006 by Junjiro Okajima, is a full rewrite of the previous UnionFS. Its goal was to increase reliability and speed, but it also included several novel concepts, such as writable branch balancing, and other enhancements, some of which are currently incorporated in the UnionFS 2.x branch. aufs was denied for inclusion in the core Linux kernel. Its code has been chastised for being "dense, illegible, and uncommented." OverlayFS was instead included into the Linux kernel. After several attempts to merge aufs into mainline kernel, the author has given up.
Aufs is pre-installed in Debian "Jessie" and Ubuntu 16.04. Debian "stretch" no longer includes aufs, but it does contain the package aufs-dkms, which auto-compiles the aufs kernel module using Dell's dkms. Aufs was first used by Docker for container filesystem levels. It is still accessible as a storage backend, but it has been deprecated in favor of the overlay2 backend, which utilizes OverlayFS.
OverlayFS
OverlayFS is a Linux union mount filesystem implementation. In late 2009, the necessity for the definition of a kernel mode Linux union mount filesystem was recognized. Miklos Szeredi provided the original RFC patchset for OverlayFS in 2010. By 2011, it has already been adopted for usage by OpenWrt. It was integrated into the mainstream Linux kernel in 2014, with kernel version 3.18. It was enhanced in version 4.0, delivering enhancements required for the overlay2 storage driver in Docker, for example.
OverlayFS's core mechanics revolve around the merging of directory access when both filesystems offer a directory with the same name. Otherwise, OverlayFS displays the item produced by one or the other, with the "higher" filesystem having priority. Unlike some other overlay filesystems, OverlayFS does not need the directory subtrees being merged to originate from different filesystems. To facilitate file and directory deletion, OverlayFS allows whiteouts and opaque directories in the top filesystem. OverlayFS does not enable renaming files without doing a full copy-up; however, it does support changing directories in an upper filesystem. OverlayFS does not support merging changes from an upper filesystem to a lower filesystem.
References
Some materials on this document is a repharasing and cherrypicking from the following Wikipedia articles:
[1] Wikipedia contributors, "Filesystem in Userspace," Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Filesystem_in_Userspace&oldid=1039229134 (accessed August 22, 2021).
[2] Wikipedia contributors, "Union mount," Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Union_mount&oldid=1024874785 (accessed August 22, 2021).
[3] Wikipedia contributors, "SquashFS," Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=SquashFS&oldid=1035018570 (accessed August 22, 2021).
[4] Wikipedia contributors, "Aufs," Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Aufs&oldid=1025516828 (accessed August 22, 2021).
[5] Wikipedia contributors, "OverlayFS," Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=OverlayFS&oldid=1026312016 (accessed August 22, 2021).