Comparison Of Microsoft Windows Tools For Waiting Time Management

Introduction

Even if Microsoft Windows is not a real-time operating system and cannot, by design, guarantee real-time constraints, it is sometimes necessary for software to provide a cyclic behavior with the best possible frequency accuracy. This article aims to compare some solutions provided by Microsoft Windows to manage time, time precision, and the impact of CPU overload on the frequency accuracy.

Terminology

In this article, each cycle has two parts: the active and the passive part. The active part represents the time spent doing some real computing in the application. The time spent waiting for the next cycle to begin is in the passive part. CPU resources should be freed by the current thread in the passive part to allow other processes to use it.

The period will be the duration between two beginning of cycles, including both the active part and passive part, and the frequency is defined as the reverse of the period.

For success rate calculation, we will consider as a success a cycle that lasts 100ms ± 1ms.

Environment

The experiment code is in C#, targeting .NET Core 3.1.

The laptop used is a Dell Precision M4700. The processor is an IntelCore i7-3820QM with a fixed frequency of 2.68Hz without turbo and hyperthreading.

Experiments description

Experiments are done on only one processor core by modifying the current process processor affinity at the beginning of the program (see ProcessorAffinity). The testing program has two threads, one running the expected cyclic behavior and the other simulating a punctual processor overload.

In each experiment, the period is 100ms, which means a frequency of 10 Hz. We recorded 600 cycles in which the processor overload is simulated from the 200^th to the 400^th cycle. By doing this, we can evaluate the frequency accuracy of our cyclic behavior. On the charts provided thereafter, this processor overload will be represented between vertical blue lines.

In the case of an external timer, we used a semaphore. Each time the timer raised an event, it gives a token to the working thread to allow it to do one cycle. The working thread executes its active part, and then wait for one semaphore resource, which represents its passive part. This behavior is shown in the next figure.

Remark

Measures are taken by the millisecond because it is not useful to be more accurate on a .NET program running under Windows OS, which is not a real-time OS.

Test cases

In this article, we will compare a few solutions:

Measure the active part duration of the cycle. Use a Thread.Sleep to suspend the current thread for the rest of the period.
The same manipulation as in 1. but setting thread priority to “AboveNormal” in the initialization.
The same manipulation as in 1. but setting thread priority to “Highest” in the initialization.
Use an external cyclic System.Timers.Timer provided by a C# library to trigger the start of each cycle. The timer thread is running in Normal priority.
Use an external cyclic System.Timers.Timer provided by a C# library to trigger the start of each cycle. The timer thread is running as the Highest priority.
Use the old Windows Multimedia Timer API to make an external cyclic timer with a resolution of 1ms.
Use the old Windows Multimedia Timer API to make an external cyclic timer with a resolution of 20ms.
Use ThreadPoolLegacyApiSet to make an external cyclic timer with a resolution of 1ms.
Use ThreadPoolLegacyApiSet to make an external cyclic timer with a resolution of 20ms with a Normal priority thread.
Use ThreadPoolLegacyApiSet to make an external cyclic timer with a resolution of 20ms with a Highest priority thread.

Results and Interpretation

Thread.Sleep with normal priority thread

We can distinguish two cases,

When the processor is overloaded, most cycles last 140ms.
When the processor is not overloaded, most of the cycles last exactly 100ms, which is the expected duration. However, there are many spikes for durations between 110 and 120ms, and a spike at 200ms.

We obtain a success rate of 52% with 1ms tolerance. The average period is 111.44ms and the standard deviation is 16.77ms, which are bad results that are even worse when the processor is overloaded.

Success rate (%)	Average period (ms)	Standard deviation (ms)
51.50	111.44	16.77

Thread.Sleep with AboveNormal priority

By using an above priority thread, 76.5% of cycles last 100ms ± 1ms. This is a great result. However, we can see many punctual spikes between 101 and 120ms, which represent 22.8% of cycles. The average is 102.6ms and the standard deviation is 5.32ms.

A surprising fact is that we obtain more accurate results when the processor is overloaded.

Success rate (%)	Average period (ms)	Standard deviation (ms)
76.50	102.60	5.32

Thread.Sleep with Highest priority

We obtain similar results as in part 3., but with a better accuracy: 81.5% of the cycles last 100ms ± 1ms. The average is 102.02ms and the standard deviation is 4.83ms, which are a bit better than with a thread priority set to AboveNormal.

We also see a better accuracy when the processor is overloaded:

Success rate (%)	Average period (ms)	Standard deviation (ms)
81.50	102.02	4.83

Use the classic C# external timer with a Normal priority for the timer thread

By using the classic C# external timer, we see an important difference when the processor is overloaded. The cycle duration repartition shows that only 41.5% of the cycles effectively last 100ms (± 1ms).

However, by looking at errors, we can observe that a delay is often followed or preceded by a “compensating” advance. Therefore, the total average is 101.12ms which is great when thinking about 58.5% of errors on cycle duration. Inversely, the standard deviation is 10.34ms, which is an important number compared to the previous cycle management (using a Thread.Sleep).

Success rate (%)	Average period (ms)	Standard deviation (ms)
41.50	101.12	10.34

Use the classic C# external timer with a Highest priority for the timer thread

When reading these results, the first observation is that this timer is a little less efficient with the highest priority thread than with a thread with a normal priority. That is surprising, and even more when looking at the chart where we could see that the timer is less performant when the processor is overloaded.

Success rate (%)	Average period (ms)	Standard deviation (ms)
37.83	100.79	12.04

Multimedia Timer API with a 1ms resolution

By using Windows Multimedia Timer API, the accuracy is exemplary. In most cases, each cycle lasts 99ms with regular spikes at 100ms. A thread overload has no impact on cycle duration, except for the first cycle where it implies an advance of 44ms. By considering 99ms as a valid value for the period, we have a success rate of 99%.

However, these results are obtained with a 1ms resolution. We can nuance them by looking at the article “Why are the Multimedia Timer APIs (timeSetEvent) not as accurate as I would expect?”. The main default of this timer is related to its working principle. Setting the resolution to 1ms reduces the period between two calls to the OS scheduler. Consequently, the proportion of time where CPU is allocated to the scheduler increases, and other processes have less time to be executed. In brief, increasing timer accuracy implies a global system slowing down. In case of OS saturation (that means when processes ask for more CPU time than what is available), the timer will become useless due to high response time.

Following this article, we can also see the advantage of a timer that works on fixed dates. Indeed, the next cycle will come faster if the timer is late for one event. In the worst case, many events might be raised at the same time. This is a good point for monitoring applications where cumulative lateness is worse than punctual delays compensated by an advance on the next cycle.

This article also shows that there will regularly be delays until 20ms. It is a Windows constraint. Whatever timer we use, running a program under this OS makes it impossible to guarantee an accuracy better than 20ms.

Nonetheless, Microsoft does not support this timer anymore, recommending to use the ThreadPoolLegacyApiSet instead. We will come back on this API further in this document.

Note

The timer thread priority is Highest by default for Multimedia Timers.

Success rate (%)	Average period (ms)	Standard deviation (ms)
99.00	99.04	1.49

Multimedia Timer API with a 20ms resolution

By using this timer with a 20ms resolution, which is 20 times less demanding for the OS, we obtain a satisfying global accuracy. 80.33% of the cycles last 100ms ± 1ms. 99% of bad cycle durations have an error which value is lower than 20ms, which is a nice value when looking at the article referenced in part 5. All delays of more than 20ms are compensated in the following cycle. The standard deviation is 4.63ms, which is an admissible value.

Success rate (%)	Average period (ms)	Standard deviation (ms)
80.33	99.62	4.63

Use ThreadPoolLegacyApiSet with a 1ms resolution

We can easily distinguish the part where the processor is overloaded of the one where it is not. In this last case, cycle duration is quasi constant with a duration of 99ms, like its predecessor the Multimedia Timer API. However, 75% of cycles last around 95ms when the processor is overloaded, and the remaining 25% cycles have punctual delays that compensate exactly this cumulated advance. Therefore, the mean is around 99ms, and the standard deviation is 6.37ms.

Note

The timer thread priority is Normal.

Success rate (%)	Average period (ms)	Standard deviation (ms)
66.50	99.04	6.37

Using ThreadPollingLegacyApiSet with a 20ms resolution and Normal thread priority

When looking at the chart, we can see that the cycle duration is not the same for each cycle and the variation is important. The mean duration is 99.7ms. It is nearly the expected duration of 100ms but the standard deviation is 6.64ms, which is quite elevated.

Nonetheless, these results are unsatisfying compared to other results. Even the Multimedia Timer which is no longer supported has better results in terms of success rate and standard deviation, but this is normal due to timer thread priority. Indeed, the Multimedia timer uses the Highest priority thread to manage its timer, while the ThreadPoolLegacyApiSet only uses a Normal priority one. So, we must increase the ThreadPoolLegacyApiSet thread priority to have comparable results.

Success rate (%)	Average period (ms)	Standard deviation (ms)
67.83	99.69	6.64

Using ThreadPollingLegacyApiSet with a 20ms resolution and Highest thread priority

On these results, we can see that the timer thread priority has a huge impact on performances. Unlike on Normal priority, executing in Highest priority significantly increases the success rate and the standard deviation from (respectively) 67.8% to 99.72% and from 6.6ms to 3.36ms.

However, there is no way to increase properly the thread priority from Normal to Highest because it is hidden beside the imported API. To make this test, we changed the currently executing thread priority to Highest when the timer event is received, like on the line of code above.

Thread.CurrentThread.Priority =ThreadPriority.Highest;

Success rate (%)	Average period (ms)	Standard deviation (ms)
87.83	99.72	3.36

Experience results in one table:

	Success rate (%)	Average period (ms)	Standard deviation (ms)	Advantages	Disadvantages
1: Thread.Sleep, Normal thread priority	51.50	111.44	16.77	Easy to develop	Fluctuating period Incorrect average Not resilient to concurrency
2: Thread.Sleep, AboveNormal thread priority	76.50	102.60	5.32	Easy to develop Resilient to concurrency	Fluctuating period Incorrect average
3: Thread.Sleep, Highest thread priority	81.50	102.02	4.83	Easy to develop Almost stable period Resilient to concurrency	Incorrect average
4: Using C# external timer with Normal priority for the timer thread	41.50	101.12	10.34	Easy to develop Almost correct average	Fluctuating period Not resilient to concurrency
5: Using C# external timer with Highest priority for the timer thread	37.83	100.79	12.04	Moderate development Correct average	Fluctuating period Not resilient to concurrency
6: Using external Multimedia Timer with 1ms resolution	99.00	99.04	1.49	Stable period Correct average Resilient to thread concurrency	Difficult to develop May cause a global system slowing down or freezing
7: Using external Multimedia Timer with 20ms resolution	80.33	99.62	4.63	Almost stable period Correct average Resilient to concurrency	Difficult to develop
8: Using a timer based on ThreadPoolLegacyApiSet with 1ms resolution	66.50	99.04	6.37	Without concurrency: Stable period Correct average	Difficult to develop Not resilient to concurrency May cause a global system slowing down or freezing
9: Using a timer based on ThreadPoolLegacyApiSet with 20ms resolution and Normal thread priority	67.83	99.69	6.64	Without concurrency: Almost stable period Correct average	Difficult to develop Not resilient to concurrency
10: Using a timer based on ThreadPoolLegacyApiSet with 20ms resolution and Highest thread priority	87.83	99.72	3.36	Almost stable period Correct average Resilient to thread concurrency	Difficult to develop

Conclusion

Microsoft provides several APIs to manage cyclic aspects of an application developed within Windows OS. However, when looking at the accuracy, we can see that depending on our application the choice may be important. In fact, for managing time to wait to provide a cyclic aspect, the more instinctive choice to do is to use an external timer.

Timers provide effectively the right average period, but the standard deviation could be pretty much heavy than the one provided by simply using a call to Thread.Sleep at the end of each cycle for waiting during the remaining time. The Thread.Sleep implies a greater average that will be a little over the expected one, but its standard deviation is remarkably lower when increasing the thread priority.

Without manually changing thread priority, the Multimedia Timer has better results than its successor, the ThreadPoolLegacyApiSet, but this is due to the default timer thread priority. The Multimedia Timer has a timer thread with Highest priority while the ThreadPoolLegacyApiSet has a normal priority timer thread. By manually changing the thread priority on this last timer to Highest, we obtain the best results of the experiment. However, the only way to do that is to set the “Thread.CurrentThread.Priority” when the timer is initializing, which could be seen as a bad practice.

One other bad aspect of Multimedia Timer and the timer based on ThreadPoolLegacyApiSet is that overriding the resolution to have better results implies a global system slowing down.

To conclude, there are our recommendations by use case,

For those who want a cyclic behavior with high accuracy on the frequency and which task has a high priority and want a solution easy to develop and to maintain: use a Thread.Sleep and increase the thread priority to AboveNormal or Highest.
For those who want a cyclic behavior with high accuracy on the frequency and which task has a high priority and want the best possible results: use the ThreadPoolLegacyApiSet with a resolution over 20ms by setting the thread priority to Highest at timer initialization.
For those who want a cyclic behavior for a normal or low priority task for which frequency accuracy is not important: use the C# provided Timer (System.Timers.Timer) which is the simplest to use and maintain.
For those who want a cyclic behavior for a normal or low priority task for which frequency accuracy is important, with limited CPU resource and high process concurrency: use the ThreadPoolLegacyApiSet with a resolution over 20ms.
For those who want a cyclic behavior for a normal or low priority task for which frequency accuracy is important and for which CPU resource is not a limitation: use the ThreadPoolLegacyApiSet with a resolution bellow 20ms.

Acknowledgment

This work has been done within Agileo Automation for an Industrial Internet of Things (IIoT) application. The aim was to analyze the best possible choice to implement cyclic data collection plans for industrial equipment data acquisition within Agileo Automation Equipment Controller Framework (A²ECF).