Microsoft Sync Framework - A primer to the file sync provider


Introduction

Currently I am working on a project that requires some files/folders to be synchronized at regular intervals across networks ranging in size from an inexpensive LAN to and expensive WAN. The problem was to search for a technology that would be effective and at the same time gel well with the .Net environment, as the rest of the application(s) were to be written in C#. In this process I came across the Microsoft Sync Framework which is still in the Community Preview phase but thought it would be a good idea to share my research with the community.

What is the Sync Framework?

The Sync Framework is a Microsoft framework for developing systems that can synchronize any type of data between any type of device and across any type of network. This is the objective behind the framework. In reality it is an extensible framework which can be used by developers to write sync providers that can sync data between devices/data sources or just use existing sync providers to create applications with data synchronization capabilities. When we talk about synchronization, the things that come to our mind is efficiency, reliability, conflict resolution, etc and the Sync Framework handles all of these parameters gracefully with features like: updating incremental data changes instead of the entire dataset, defining conflict resolution policies, graceful cancellation of synchronizations when required without data inconsistencies, support for recycle bin, etc.

Microsoft currently offers 3 types of providers that can be used by developers to write applications that can leverage the functionality in these providers. The sync providers that are currently available from Microsoft are:

Provider for ADO.Net: For synchronizing data between ADO.Net supported data sources.

Provider for File Systems: For synchronizing Files and Folders on an NTFS or FAT file system.

Provider for Feeds like RSS: For synchronizing data from Feeds.

This article does not focus on writing a sync provider and just focuses on creating a small application to demonstrate the sync provider for File Systems. Probably sometime soon I will write another article explaining how to utilize additional sync providers.

The File Sync Provider

As mentioned above, the File Sync provider is for synchronizing files and directories on the FAT or NTFS file systems. The way the File Sync works is by comparing the metadata for the replica and detecting changes in files from the last time the replica was synchronized. The metadata is persisted in a file named "filesync.metadata" both in the source and destination replicas. If you are wondering what is a Replica, it is the locations between which the synchronization would happen. For example if your application uses File Sync to synchronize data between folder C:\Sync\Folder1 and C:\Sync\Folder2, then these two folders can be termed as replicas.

Now I will write a simple application to demonstrate the File Sync Provider. There are numerous ways of implementing this technology. The following program is just to give you a brief understanding of the technology.

Let's say I want to synchronize the contents of the folder C:\Sync\Destination with the contents of the folder C:\Sync\Source.

  1. Create a new windows application in Visual studio.
  2. Add reference to the assemblies "Microsoft.Synchronization" and Microsoft.Synchronization.Files".
  3. Place a button named "btnSynchronize" on a form.

Write the following code in the Click handler of the button:

private void btnSynchronize_Click(object sender, EventArgs e)

{

    //Generate a unique Id for the source and store it in a file for

    //future reference.

    SyncId sourceId = GetSyncID(@"C:\Sync\Source\File.ID");

 

    //Generate a unique Id for the destination and store it in a file

    //for future reference.

    SyncId destId = GetSyncID(@"C:\Sync\Destination\File.ID");

 

    //create a FileSyncProvider object by passing the SyncID and the

    //source location.

    FileSyncProvider sourceReplica = new FileSyncProvider(sourceId,@"C:\Sync\Source\");

 

    //create a FileSyncProvider object by passing the SyncID and the

    //destination location.

    FileSyncProvider destReplica = new FileSyncProvider(destId,"C:\Sync\Destination\");

 

    //Initialize the agent which actually performs the

    //synchronization.

    SyncAgent agent = new SyncAgent();

 

    //assign the source replica as the Local Provider and the

    //destination replica as the Remote provider so that the agent

    //knows which is the source and which one is the destination.

    agent.LocalProvider = sourceReplica;

    agent.RemoteProvider = destReplica;

 

    //Set the direction of synchronization from Source to destination

    //as this is a one way synchronization. You may use

    //SyncDirection.Download if you want the Local replica to be

    //treated as Destination and the Remote replica to be the source;

    //use SyncDirection.DownloadAndUpload or

    //SyncDirection.UploadAndDownload for two way synchronization.

 

    agent.Direction = SyncDirection.Upload;

 

    //make a call to the Synchronize method for starting the

    //synchronization process.

    agent.Synchronize();

}

 

//This is a private function I have created to generate the SyncID if

//it is the first time a Sync is happening on the source and

// destination replicas or else retrieve the SyncID stored in a

// pre-defined file (File.ID) on the source and destination replicas.

// I have used the GUID for generating the SyncID, whereas you may also

// use a unique string, byte array etc. for generating the SyncID.

// The objective here is to have a unique SyncID.


private
static SyncId GetSyncID(string syncFilePath)

{

    Guid guid;

    SyncId replicaID = null;

    if (!File.Exists(syncFilePath)) //The ID file doesn't exist.
    //Create the file and store the guid which is used to 
    //
instantiate the instance of the SyncId.

    {

        guid = Guid.NewGuid();

        replicaID = new SyncId(guid);

        FileStream fs = File.Open(syncFilePath, FileMode.Create);

        StreamWriter sw = new StreamWriter(fs);

        sw.WriteLine(guid.ToString());

        sw.Close();

        fs.Close();

    }

    else

    {

        FileStream fs = File.Open(syncFilePath, FileMode.Open);

        StreamReader sr = new StreamReader(fs);

        string guidString = sr.ReadLine();

        guid = new Guid(guidString);

        replicaID = new SyncId(guid);

        sr.Close();

        fs.Close();

    }

    return (replicaID);

}

The source code above is just a simple example of synchronizing contents in one folder with that of another. There can be more advanced requirements like excluding files from synchronization, recovering files which are deleted in synchronization, etc. For these you have to use the FileSyncScopeFilter and the FileSyncOptions while creating the FileSyncProvider object for source and destination replicas.

For excluding files from synchronization, you have to create an object of the FileSyncScopeFilter, add the filenames to be excluded from the synchronization and then pass the FileSyncScopeFilter object to the FileSyncProvider constructor while creating an object for the source replica.

For example if I was to exclude the file named "File.ID" from the synchronization in the above source code, I would have to do the following:

Add the following code just before creating the object named sourceReplica.

FileSyncScopeFilter scopeFilter = new FileSyncScopeFilter();
scopeFilter.FileNameExcludes.Add("File.ID");

And modify the code for creating the sourceReplica object by passing the scopeFilter object to the constructor as follows:

FileSyncProvider sourceReplica = new FileSyncProvider(sourceId, @"C:\Sync\Source\",scopeFilter, FileSyncOptions.None);

That's all you do to exclude a file from synchronization. You would be wondering what the FileSyncOptions.None does in the code above. Well FileSyncOptions is an enumeration that you use to specify additional information on how the synchronization handles deletes, overwrites etc. For example the synchronization process may lead to deletion of file(s) from the destination replica if the same no longer exists in the source replica. In such cases you may want to send the deleted file to the recycle bin for future reference. The FileSyncProvider does not send the deleted file to the recycle bin by default, but you can set the FileSyncOptions.RecycleDeletes option while creating the destReplica object to send all deleted files to the recycle bin as follows:

FileSyncProvider destReplica = new FileSyncProvider(destId, @"C:\Sync\Destination\", scopeFilter, FileSyncOptions.RecycleDeletes);

Other FileSyncOptions that you may use are:

FileSyncOptions.RecycleOverwrites

FileSyncOptions.CompareFileStreams
FileSyncOptions.ExplicitDetectChanges

Resources

You will need the Microsoft Sync Framework to create sync programs in .Net. You can download the same and other related resources from
http://msdn2.microsoft.com/en-us/sync/default.aspx

Conclusion

As the intention of this article is just to be a primer on the Microsoft Sync Framework, there are numerous options apart from the ones in this article that you can explore to make powerful synchronization applications.