How to Improve Responsiveness of Objects that do not Guarantee Responsiveness


Introduction

In this article we will analyze a problem which often strikes software solutions. That is how to improve responsiveness of objects that are generally not guaranteed to reply to every request immediately.

For example, in network communications proxy class may be in a blocking call and hence completely unaware of the outer world until current operation completes. If it happened that some network operation took longer to complete, e.g. due to temporary network delays, then network client might block the calling thread for so long that user observes interface freezing. Even worse, if connection is dropped without notice, client might have to wait for half a minute or so until network timeout is reported back.

It would be utterly inappropriate to keep UI thread blocked or in any other way disabled due to waiting for a network operation to complete. On the contrary, it is of great importance to ensure that application will not freeze and that user is able to initiate other network operations in parallel, even while previous operation is currently blocked and waiting for the error to finally pop up or a lengthy transfer to complete successfully.

Responsive application should be able to show progress of all active operations and to allow user to cancel any operation gracefully without freezing. In the text that follows we will demonstrate how responsiveness of the application can be guaranteed when working with an object which is not fully responsive by its own design. All operations initiated from main application thread (which might include requests made by the user through GUI) will be executed with no delay. User would never notice any problems regarding application responsiveness. Under the hood, objects that are performing lengthy operations might become unresponsive from time to time and specific wrappers will be used to control the situation so that user does not notice these delays.

Example Model

In this section we will present a class which simulates a process executing in time slices. It means that object of the class might execute a method in a blocking mode, so that method does not exit immediately but only after a certain amount of time. This is quite typical situation when contacting Web service or sending data over the socket. However, code which is using this class would like to be able to retrieve current status even when the class is executing the blocking method. This is not a simple request because another call to the class would have to be made from a different thread, which implicitly means that the class must be thread safe. However, it is not the case in general. Here is the listing of the class which simulates the process with a blocking method.

class TimeSlicedWorker
    { 
        public void BeginWork(int totalSize)
        {
            _totalSize = totalSize;
            _progress = 0;
            _speedFactor = _rnd.Next(10) + 1;
        }
 
        public void Advance(int amount)
        {
            _progress = (_progress + amount > _totalSize ? _totalSize : _progress + amount);
            System.Threading.Thread.Sleep(amount * _speedFactor);
        } 
        public int GetWorkStatus() { return _progress; }
        public int GetProgressPercentage() { return (_totalSize > 0 ? 100 * _progress / _totalSize : 0); }
        public void CancelWork() { _progress = _totalSize = 0; }
        public bool IsComplete() { return _progress >= _totalSize; } 
        private int _progress;
        private int _totalSize;
        private int _speedFactor;
        private static Random _rnd = new Random(); 
    }


This class simulates a lengthy work which can be modeled as increasing an integer value (_progress) from zero to a predefined upper limit (_totalAmount). Work begins with a non-blocking call to BeginWork method which specifies the upper limit of the task. Further on, Advance method would be invoked potentially many times, each time increasing current progress towards the _totalAmount value, but also every time blocking for an uncertain period of time. Hence Advance method simulates the blocking method which may freeze the calling thread for some period of time.

In addition to these methods, caller may inspect the return value of IsComplete method, which indicates whether _progress has reached _totalAmount. If so, the work is done and caller may dispose the object. Otherwise, caller is free to call CancelWork, which would reset the object and after this call no more calls should be made to the object. Remaining two methods, GetWorkStatus and GetProgressPercentage, are used to read current progress and show it to the user.

Now what happens in practical case is that calling thread becomes blocked in the Advance method for, say, couple of seconds, and during that time other thread should not call CancelWork or GetProgressPercentage methods, because this class is not thread safe. As the result of this unfortunate situation, user interface is not able to provide current progress to the user, nor is it able to allow user to cancel operation at any time. Even if user requests cancellation of current operation, that request will not be executed until current blocking call exits, and that might not be any time soon. And that is the behavior of the application which users regard as unresponsive interface.

In the next section we will provide another class which wraps TimeSlicedWorker and provides everything it lacks: thread safety and responsiveness.

Wrapper Class


To resolve the responsiveness issue described above, we will provide another class named Smoother, which will wrap an instance of TimeSlicedWorker class. In addition to simple wrapping of the object, Smoother will also provide a thread-safe interface, exposing non-blocking methods to the user interface. Internally, calls to the TimeSlicedWorker would be made from a different thread. That thread would be blocked most of the time, waiting for the TimeSlicedWorker to complete blocking operations. But every time when it has control over contained object, it will collect latest status information from it. In that way, Smoother will be capable to provide current, or at least last known status, of the worker at all times, even when its own thread which communicates to the worker is blocked.

In addition to these features, Smoother is also able to receive the cancel request. This request will be routed to the worker as soon as current blocking operation ends. This means that worker will be informed to cancel further work as soon as it becomes responsive. In the meantime, Smoother's CancelWork method would immediately exit, allowing the user interface to continue operations without freezing.

Here is the full listing of the Smoother class.

    class Smoother
    { 
        public Smoother()
        { 
            _worker = new TimeSlicedWorker();
            _worker.BeginWork(_rnd.Next(10000)); 
            _sync = new object();
            _isComplete = new System.Threading.ManualResetEvent(false); 
            _thread = new System.Threading.Thread(new System.Threading.ParameterizedThreadStart(ThreadProc)); 
        }
 
        public System.Threading.ManualResetEvent Start()
        {
            _thread.Start(this);
            return _isComplete;
        }
        private void ThreadProc(object state)
        { 
            Smoother smooth = (Smoother)state;
 
            bool complete = false
            lock (_sync)
            {
                _tsLastActivity = DateTime.UtcNow;
                _percentProgress = _worker.GetProgressPercentage();
            }
 
            while (!complete)
            {
                if (IsWorkCancelled())
                {
                    _worker.CancelWork();
                    complete = true;
                }
                else if (_worker.IsComplete())
                    complete = true;
               
else
                   _worker.Advance(_rnd.Next(400));   // At this line _worker becomes unresponsive for indefinite period of time
                 lock (_sync)
                {
                    _percentProgress = _worker.GetProgressPercentage();
                    _tsLastActivity = DateTime.UtcNow;
                } 
            }
 
            _isComplete.Set(); 
// Signal the outer world that working is over
        } 

        public int GetPercentProgress() { lock (_sync) return _percentProgress; }
        public void CancelWork() { lock (_sync) _workCancelled = true; }
        private bool IsWorkCancelled() { lock (_sync) return _workCancelled; }
        public DateTime GetLastActivityTs() { lock (_sync) return _tsLastActivity; }
        public System.Threading.ManualResetEvent GetWaitHandle() { return _isComplete; } 
        private object _sync;
        private System.Threading.Thread _thread;
        private TimeSlicedWorker _worker;
        private bool _workCancelled;
        private System.Threading.ManualResetEvent _isComplete;
        private int _percentProgress;
        private DateTime _tsLastActivity;
        private static Random _rnd = new Random(); 
    }


In the constructor, worker instance is allocated and initialized and thread is created (but not started). Also, objects used in threads synchronization are initialized. If TimeSlicedWorker needs to receive any initialization parameters, this might be the place to pass them from the caller - either as Smoother constructor arguments, or via additional Smoother's methods and properties.

When everything is in place, caller can invoke the Start method, which starts the working thread and returns manually reset event on which caller can wait for the worker to complete all work. This event, when set, signals that work was completed, either by performing it to the end, or by successfully cancelling the remaining work.

ThreadProc is the main method in the Smoother class. It calls worker's methods until either all work is done or cancel is signaled. When either of the conditions is met, thread procedure would simply set the event indicating work completion and exit.

The Smoother class also provides public method GetPercentProgress, which is a non-blocking method returning last known percentage progress. Note that result of this method might not be up to date, but at any time it is the last available value. Next method is CancelWork and it can be used to cancel further operations on the contained worker. Note that this method only sets the internal flag indicating that worker should cancel. It does not wait for the worker to effectively exit. This is what makes the Smoother class responsive at all times. However, caller must be aware that calling the CancelWork method does not guarantee that work will be effectively cancelled at that specific moment. Working might continue for quite a while after this method returns. So caller must always wait at the event returned by the Start method (or by GetWaitHandle method) in order to ensure that worker has been disposed. Last public method is GetLastActivityTs, and it provides UTC timestamp of last communication of the Smoother with its contained worker. If this timestamp differs too much from current time, then worker can be declared unresponsive.

User Interface Example

We will not provide complete user application listing in this text because of its length. However, this code is included in the attached source code, so please feel free to download it and try it.

The example client allows user to start multiple workers in parallel. Each worker will be wrapped in an instance of the Smoother class, ensuring that interface will be responsive at all time. User will be able to contact each of the objects, to read progress and to cancel work at will.

Here is the sample output of the program, which shows all available information.

Press 0-9 for unused slot to initiate worker
Press 0-9 for used slot to cancel work
Press Q to quit application

[0]
[1]
[2] Running (5%)
[3] Running (16%) [Not responding]
[4]
[5]
[6]
[7]
[8] Running (6%) [Not responding]
[9]
Pending close workers: 4

This output shows three workers started and progressing, two of them declared as non responsive (i.e. they did not respond in more than one second). The last line indicates that there are total of four workers waiting to cancel their work. Their wrappers have already received the CancelWork request, but these requests did not take effect yet because blocking operations have not been completed on the workers.

Once again, please feel free to download and try the attached source code.

Conclusion

In this article we have demonstrated how a single-threaded blocking class can be wrapped into a class with thread-safe non-blocking interface as a step towards building a fully responsive interface. The source code provided is for demonstration purposes only. It cannot be used as is in any particular application, but ideas employed to design it are quite general and thus applicable in wide range of practical applications.