If I have some number of source directories, e.g. dir1, dir2... dir5, how can I create an automatically syncing merged destination directory that has all the files and directories of the source directories but doesn't involve duplicating files?For example, I have the following source directory structure...dir1/  - a  - bdir2/  - b   <-- Note the duplicate name, this one is more recent than dir1\b  - c  - dir2.1/   <-- Subdirectories present too.      - zdir3/  - d... which when merged would look like this:merge/ - dir2.1/     - z - a - b   <-- Which one to show is based on modified time; most recent first. - c - d(Assume there could be dozens of directories with thousands of files and subdirectories.)All the files and directories in the source list should remain in place and unchanged and the merged destination directory should take up no addition storage; i.e. It's possibly just all symlinks managed by some daemon using inotify. The source directories are also having files and subdirectories added and removed frequently and this needs to be reflected, as soon as possible, in the merged directory too.Some usage examples:I create a new file, dir3/e, and it automatically and immediately (or within a few seconds) appears in the merge directory.I remove the file dir1/a and it automatically and immediately (or within a few seconds) disappears from the merge directory.I edit dir3/d by opening merge/dI call touch on dir1/b so it it has a newer modified date than dir2/b and so merge/b automatically updates to point to dir1/b as it is the most recent.I remove dir1/b and now merge/b will point to the older file dir2/b.I attempt to create a file in merge but get an error because doing so makes no sense!
How do I create and automatically sync a merged directory from multiple other directories without duplicating files?
linux;debian;directory
Dynamically combining the content of several directories is exactly what a union mount (or union filesystem) is about. There are several implementations on Linux, but the usual ones resolve duplication between branches by always preferring one particular branch (e.g. dir2/file always has precedence over dir1/file if both exist). However I've just discovered mergerfs which has more flexible policies, and in particular allows picking the newest file on a file-by-file basis.The setup:mkdir dir1 dir2 mergeecho 1 >dir1/b; echo 2 >dir2/asleep 1echo 1 >dir1/a; echo 2 >dir2/b; echo 2 >dir2/cNow we mount. We set the newest policy for file access (search) and metadata modification (action), and forbid creation (create) (you could make creation pass down to one of the branches, either always the same, or based on available disk space, or even randomly!).$ mergerfs -o category.action=newest,category.search=newest,category.create=erofs dir1:dir2:dir3 merge$ head merge/?==> merge/a <==1==> merge/b <==2==> merge/c <==2$ rm dir2/c$ echo merge/?a bTo unmount: fusermount -u merge