Transaction In File Operations

As a developer, we all have used transactions in our Db interactions over the years. When it comes to writing to File System, most often than not, this has not been a cause of concern. But there are situations wherein, one might require support for Transactions in File System. In other words, there might be scenarios when one might require atomic file operations in the File System. For example,

using(TransactionScope scope = new TransactionScope()) {
    File.WriteAllText(filePath, fileContent);
    RaiseSomeException();
    scope.Complete();
}

In this above code, the call RaiseSomeException() raises an exception. In this scenario, the File Write Operation should be reverted. This type of transactional support for File System operations doesn't work out of the box. But .Net provides us enough ammunition to make this possible easily.

Transactional Resource manager

Microsoft has introduced Transactional NTFS with Windows Vista. These were a set of powerful APIs to support transactional atomicity in file operations. But as per the Alternatives to using Transactional NTFS in Microsoft's documentation, Microsoft strongly recommends not to use the functionality as this might not be available in the future versions of Windows.

The alternative involves developing one's own Transactional Resource Manager that supports transactional file access. The Resource Manager should be able to manage data (both old and new) and should be able to commit/rollback the transaction as per the notifications received from the transaction manager. To ensure that the Resource manager receives notification from the current transaction, the resource manager needs to implement a contract that would allow the transaction manager to provide 2 phase commit notification callbacks. Additionally, it should also ensure any resources involved in the transaction are enlisted by the Transaction.

Two-Phase Commit Protocol (2PC)

To ensure atomicity of the operation, the Two-Phase Commit Protocol (2PC) breaks the commit operation into two parts or phases

Prepare Phase

In the preparation phase, the resource manager or the coordinator asks all the participating clients if it is ready to commit the changes. This signals the clients to prepare all the necessary steps required for the eventual commit or abort of the operation. Each client finally responds to the voting request raised by the coordinator by

yes: ready for commit
no: problem has been detected in the client and cannot commit

Commit Phase

Based on the voting response, the resource manager/coordinator decides whether to commit or abort the operation. This is then conveyed to all the clients.

The Transaction class acts as the coordinator in .Net and is responsible for coordinating the clients. This is done through a set of callbacks, which is provided by the IEnlistmentNotification interface.

IEnlistmentNotification

The IEnlistmentNotification interface is a special interface within the System.Transactions.Local which a resource manager should implement to receive 2 Phase Commit Notification callbacks from the Transaction Manager.

public interface IEnlistmentNotification {
    void Prepare(PreparingEnlistment preparingEnlistment);
    void Commit(Enlistment enlistment);
    void Rollback(Enlistment enlistment);
    void InDoubt(Enlistment enlistment);
}

For any resource to receive notification, it needs to be enlisted for participation. This is done using the Transaction Class, which supports methods like EnlistVolatile and EnlistDurable, depending on whether your resource is volatile or durable. The durability of the resource refers to the ability of the Resource Manager to recover from failure. The Resource Manager can opt to persist the data involved such that if the resource manager goes down for any reason, it can still re-enlist the operation and complete them after it recovers.

Resource Manager

Let us now write our Resource Manager class. We will create our implementation as simple as possible as the core intent of this article is to give the readers an understanding of how the resource manager should work, rather than build a foolproof system.

public class ResourceManager: IEnlistmentNotification, IDisposable {
    private List < CreateFileOperation > _operations;
    private bool _disposedValue;
    public void EnlistOperation(CreateFileOperation operation) {
        Console.WriteLine("Enlisting Operation");
        if (_operations is null) {
            var currentTransaction = Transaction.Current;
            currentTransaction?.EnlistVolatile(this, EnlistmentOptions.None);
            _operations = new List < CreateFileOperation > ();
        }
        _operations.Add(operation);
    }
    public void Commit(Enlistment enlistment) {
        Console.WriteLine("Initiating Commit Operation");
        foreach(var createFileOperation in _operations) {
            createFileOperation.DoOperation();
        }
        enlistment.Done();
        Console.WriteLine("Commit Operation Completed. Ensuring cleanup");
        foreach(var createFileOperation in _operations) {
            createFileOperation.Dispose();
        }
    }
    public void InDoubt(Enlistment enlistment) {
        foreach(var createFileOperation in _operations) {
            createFileOperation.RollBack();
        }
        enlistment.Done();
    }
    public void Prepare(PreparingEnlistment preparingEnlistment) {
        try {
            Console.WriteLine("Preparing for commit.");
            PrepareForCommit();
            Console.WriteLine("Client is ready for commit Notify Cordinator");
            preparingEnlistment.Prepared();
        } catch {
            Console.WriteLine("Client is NOT ready for commit Notify Cordinator");
            preparingEnlistment.ForceRollback();
        }
    }
    public void Rollback(Enlistment enlistment) {
        Console.WriteLine("Initiating Rollback");
        foreach(var createFileOperation in _operations) {
            createFileOperation.RollBack();
        }
        enlistment.Done();
        Console.WriteLine("Rollback completed");
    }
    private void PrepareForCommit() {
        // Prepare for commit
    }
    public void Dispose() => Dispose(true);
    // Protected implementation of Dispose pattern.
    protected virtual void Dispose(bool disposing) {
        if (_disposedValue) return;
        if (disposing) {
            foreach(var operation in _operations)
            operation.Dispose();
        }
        _disposedValue = true;
    }~ResourceManager() => Dispose(false);
}

As you can observe, we have implemented the IEnlistmentNotification interface in the method. We will not go in deep into the implementation of each of the methods, but as a sample, let us look into Prepare() and Commit() methods.

The Prepare() method receives PreparingEnlistment instance as the parameter of the callback. It then performs any pre-commit operations it needs to do before the commit phase. It then notifies the coordinator/transaction manager that it is ready using the PreparingEnlistment.Prepared() method. In case, a problem has been noticed and the Resource Manager is ill-prepared for commit, it can inform the Transaction Manager using the PreparingEnlistment.ForceRollback() method.

public void Prepare(PreparingEnlistment preparingEnlistment) {
    try {
        Console.WriteLine("Preparing for commit.");
        PrepareForCommit();
        Console.WriteLine("Client is ready for commit Notify Cordinator");
        preparingEnlistment.Prepared();
    } catch {
        Console.WriteLine("Client is NOT ready for commit Notify Cordinator");
        preparingEnlistment.ForceRollback();
    }
}

The Commit() method ensures the actual commit operation has been carried out. Once done, it informs the Transaction Manager that the operation has been completed using the Enlistment.Done() method. It should also ensure any resources consumed are disposed of.

public void Commit(Enlistment enlistment) {
    Console.WriteLine("Initiaitng Commit Operation");
    foreach(var createFileOperation in _operations) {
        createFileOperation.DoOperation();
    }
    enlistment.Done();
    Console.WriteLine("Commit Operation Completed. Ensuring cleanup");
    foreach(var createFileOperation in _operations) {
        createFileOperation.Dispose();
    }
}

In addition to the methods supported by IEnlistmentNotification, the ResourceManager exposes EnlistOperation() method. As mentioned earlier in the article, resources involved in the transaction should enlist themselves to the current Transaction. This is done using the Transaction.Current.EnlistVolatileOperation() method. For sake of this example, we are using Volatile enlistment.

private List < CreateFileOperation > _operations;
public void EnlistOperation(CreateFileOperation operation) {
    Console.WriteLine("Enlisting Operation");
    if (_operations is null) {
        var currentTransaction = Transaction.Current;
        currentTransaction?.EnlistVolatile(this, EnlistmentOptions.None);
        _operations = new List < CreateFileOperation > ();
    }
    _operations.Add(operation);
}

CreateFileOperation

The CreateFileOperation class is where the actual operation is carried out.

public class CreateFileOperation: IDisposable {
    private string _backupPath;
    private string _actualPath;
    private byte[] _content;
    private bool _disposedValue;
    public CreateFileOperation(string filePath, byte[] content) {
        (_actualPath, _content) = (filePath, content);
        _backupPath = Path.GetTempFileName();
    }
    public void DoOperation() {
        if (File.Exists(_actualPath)) File.Copy(_actualPath, _backupPath, true);
        File.WriteAllBytes(_actualPath, _content);
    }
    public void RollBack() {
        File.Copy(_backupPath, _actualPath, true);
        File.Delete(_backupPath);
    }
    public void Dispose() => Dispose(true);
    // Protected implementation of Dispose pattern.
    protected virtual void Dispose(bool disposing) {
        if (_disposedValue) return;
        if (disposing) {
            File.Delete(_backupPath);
        }
        _disposedValue = true;
    }~CreateFileOperation() => Dispose(false);
}

The key methods to focus here are DoOperation() and Rollback().

public void DoOperation() {
    if (File.Exists(_actualPath)) File.Copy(_actualPath, _backupPath, true);
    File.WriteAllBytes(_actualPath, _content);
}
public void RollBack() {
    File.Copy(_backupPath, _actualPath, true);
    File.Delete(_backupPath);
}

The DoOperation() method creates a copy of the existing file (if it exists) in the user temporary folder. It then does the required Write Operation. The Rollback() method ensures the backup file is restored. Once the backup file is restored, the original backup is removed from the internal cache.

Client Code

Let us write some client code now which consumes our example ResourceManager.

var bytesToWrite = Encoding.UTF8.GetBytes("hello world!!");
var createFileOperation = new CreateFileOperation("sample.txt", bytesToWrite);
try {
    using
    var resourceManager = new ResourceManager();
    using
    var transactionScope = new TransactionScope();
    resourceManager.EnlistOperation(createFileOperation);
    // throw new Exception();
    transactionScope.Complete();
} catch {
    Console.WriteLine("Some error has occurred. Transaction has been aborted and rolled back");
}

In case of a successful commit operation, you can receive the log information as the following.

Hello, World!
Enlisting Operation
Preparing for commit.
Client is ready for commit Notify Coordinator
Initiating Commit Operation
Commit Operation Completed. Ensuring cleanup

If you uncomment the throw new Exception() line, the Rollback() is ensured.

Hello, World!
Enlisting Operation
Initiating Rollback
Rollback completed
Some error has occurred. The transaction has been aborted and rolled back

Summary

As mentioned earlier in the article, the code has been simplified to a large extent here. In the real-life scenario, one needs to ensure issues like cross-thread calls and nested transactions are addressed.

Also, the code needs to ensure it works when the scope doesn't involve a transaction. This could be easily done by exposing a ResourceManager.CreateFile() method which enlists the operation if the scope of code is within a transaction (you can use Transaction.Current!=null to ensure this). If the code is not executed within the scope of a transaction, instead of enlisting the operation, the actual Operation could be invoked directly.

All code described in this article could be accessed from my Github.