Minor questions in Linux file semantics

I’ve known for a long time that if you delete a file on Unix / Linux but that file is open somewhere, the blocks used by the file aren’t freed until that user closes the file (or is terminated), but I was left wondering about some other edge cases.

Shaken Fist has a distributed blob store. It also has a cache of images that virtual machines are using. If the blob store and the image cache are on the same filesystem, sometimes the image cache entry can be a hard link to an entry in the blob store (for example, if the entry in the blob store doesn’t need to be transcoded before use by the virtual machine). However, if they are on different file systems, I instead use a symbolic link.

This raises questions — what happens if you rename a file which is open for writing in a program? What happens if you change a symbolic link to point somewhere else while it is open? I suspect in both cases the right thing happens, but I decided I should test these theories out.

First off, let’s cover the moving a file which is being written to case. Specifically, moving the file on the same filesystem. I wrote this little test program:

#!/usr/bin/python3

import datetime
import time

with open('a', 'w') as f:
    try:
        while True:
            f.write('%s\n' % datetime.datetime.now())
            time.sleep(1)

    except KeyboardInterrupt:
        f.close()

In one terminal I set it running. In another I then renamed ‘a’ to ‘b’ and waited a bit. The short answer? The newer writes from my script ended up in ‘b’ correctly. This makes sense when you remember that files don’t have names in most Unix filesystems — a directory has dirents with names, and they point to an inode. The open program is changing the content of an inode and associated blocks, and that’s quite separate from changing the dirent that points to that inode.

Secondly, what happens if I have a symlink to a different filesystem, move the file on that other filesystem and then update the symlink? All of course while the file is in use?

Unsurprisingly it works just like the previous example — the open file continues to be updated regardless of the move and the change of symlink.

This is good, because it makes re-sharding the blob store in Shaken Fist much easier. So there you go.