WCF Concurrency (Single, Multiple and Reentrant) and Throttling

Introduction and Goal

In this article, we will concentrate on WCF concurrency and throttling. We will first try to understand what are WCF concurrency and the 3 important types of WCF concurrency. We will then see a small sample of WCF concurrency with single and multiple. We will then go through 9 combinations of WCF concurrency and instancing. Finally we will try to understand how to configure throttling using WCF 'web.config' file.

This is a small Ebook for all my .NET friends which covers topics like WCF, WPF, WWF, Ajax, Core .NET, SQL, Design patterns, UML, Azure etc you can download the same from here or else you can catch me on my daily free training @ from here

Courtesy: - http://www.nassaulibrary.org/ncla/nclacler_files/LILC7.JPG

Pre-requisite

In order to understand WCF concurrency well, it's important to understand concepts around WCF instancing. So before moving ahead with this article please do once read WCF instancing from here

Why do we need concurrency in WCF?

If you search on the web for the dictionary meaning of concurrency you will find the following definition:-

"Happening at the same time"

WCF concurrency helps us to configure how WCF service instances can serve multiple requests at the same time. You will need WCF concurrency for the below prime reasons, there can be other reasons as well but these stand out as important reasons:-

Increase through put: - Many times you want to increase the amount of work your WCF service instance does at any moment of time, in other words you would like to increase the through put. Throughput means how much work a given thing can do.

By default WCF service handles only one request at given moment of time.

Integration with legacy system: - Many times your WCF services interact with legacy systems like VB6 COM etc. It's very much possible that these systems are not multithreaded, in other words they handle only one request at any given moment of time. So even though your WCF service has concurrent capabilities you would like still to handle one request at a one time. This is achieved by using throttling in combination with WCF concurrency capabilities.

WCF concurrency and instancing - two different things

In our previous article we had discussed about WCF instances. WCF instancing and WCF concurrency are two different things. WCF instance dictates how the objects are created while WCF concurrency dictates how many requests can be handled by the WCF objects.

I do understand that many of our developer's friends know this difference, but as you go deeper in WCF concurrency there is lot of connection with WCF instancing also, so wanted to just emphasize the differences.

WCF instancing gives three levels of WCF object instance controls per call, per session and single. In case you are not aware of the same do read my article 3 way to do WCF instance management. In similar lines WCF concurrency also have 3 ways of implementing WCF concurrency.

Three types of WCF concurrency

There are three ways by which you can handle concurrency in WCF Single, multiple and reentrant. To specify WCF concurrency we need to use the 'ServiceBehavior' tag as shown below with appropriate 'ConCurrencyMode' property value.

Single: - A single request has access to the WCF service object at a given moment of time. So only one request will be processed at any given moment of time. The other requests have to wait until the request processed by the WCF service is not completed.

Multiple: - In this scenario multiple requests can be handled by the WCF service object at any given moment of time. In other words request are processed at the same time by spawning multiple threads on the WCF server object.

So you have great a throughput here but you need to ensure concurrency issues related to WCF server objects.

Reentrant: - A single request thread has access to the WCF service object, but the thread can exit the WCF service to call another WCF service or can also call WCF client through callback and reenter without deadlock.

By default WCF services are Single concurrency - Sample code demonstration

By default WCF services are set to concurrency type 'single' and instancing mode 'per call'. In order to demonstrate the same, let's create a simple sample code as shown below. We will create a simple WCF service with a method name called as 'Call'. When any client calls this WCF service it will display the following details:-

Client name that made the call. This value will be provided as an input when the client wants to make call to the WCF service.
Instance number, this will represent number of WCF instance count which is serving the request.
Thread number which is executing the method.
Time when the 'Call' method was actually called.

Below is the service contract of the 'Call' method. Please note that the 'OperationContract' is defined as 'IsOneWay' true.

[ServiceContract]
public interface IHelloWorldService
{
[OperationContract(IsOneWay=true)]
void Call(string ClientName);
}

Below is the simple implementation of the service contract 'IHelloWorldService' interface defined below. It has a instance variable 'i' which helps us to maintain the instance counter and simple console.writeline which displays client name, instance number , thread number and time when the method was called.

This method waits for 5 seconds using the 'Thread.Sleep' function.

public class HelloWorldService: IHelloWorldService {
// maintain instance count
public int i;
public void Call(string ClientName) {
// increment instance counts
i++;
// display client name, instance number , thread number and time when
// the method was called
Console.WriteLine("Client name :" + ClientName + " Instance:" + i.ToString() + " Thread:" + Thread.CurrentThread.ManagedThreadId.ToString() + " Time:" + DateTime.Now.ToString() + "\n\n");
// Wait for 5 seconds
Thread.Sleep(5000);
}
}

This WCF service will be self hosted using 'WsHttpBinding' as shown below. We have chosen self hosting so that we can monitor the time, threads and number of instances of WCF service.

static void Main(string[] args) {
//Create a URI to serve as the base address
Uri httpUrl = new Uri("http://localhost:8010/MyService/HelloWorld");
//Create ServiceHost
ServiceHost host = new ServiceHost(typeof(ClassLibrary1.HelloWorldService), httpUrl);
//Add a service endpoint
host.AddServiceEndpoint(typeof(ClassLibrary1.IHelloWorldService), new WSHttpBinding(), "");
//Enable metadata exchange
ServiceMetadataBehavior smb = new ServiceMetadataBehavior();
smb.HttpGetEnabled = true;
host.Description.Behaviors.Add(smb);
//Start the Service
host.Open();
Console.WriteLine("Service is host at " + DateTime.Now.ToString());
Console.WriteLine("Host is running... Press <Enter> key to stop");
Console.ReadLine();
}

From the client side we will make 5 continuous calls simultaneously on the WCF service. Below is the code snippet for the same.

Console.WriteLine("Enter Client name");
string str = Console.ReadLine();
ServiceReference1.HelloWorldServiceClient obj = new ServiceReference1.HelloWorldServiceClient();
for (int i = 0; i < 5; i++) {
obj.Call(str);
}

If you run the above project and monitor your WCF server service you will see the screen shot. There are 2 important points to note:-

By default the WCF service is configure to run with instance mode of per call. So new instances are created every time the service runs. Instance 1 indicate new instances created for every method call.
By default the concurrency is single so same thread is use to service all the 5 request's which are sent from the WCF client side. You can see the value of the thread is always 4.
The most important factor is time. As the concurrency is configured as single it will be called one after another sequentially. You can notice the same from the time difference of method call of 5 seconds.

Let's go and change the concurrency mode to multiple. In order to change the concurrency mode to multiple we need to specify 'Multiple' in the concurrency mode as shown in the below code snippet.

If you run the client now you can see different threads (i.e. thread 4, 5, 6, 7 and 8) are spawned to serve the request and the time of the method calls are almost same. In other words the methods are executed concurrently on different threads.

In short you are now having higher throughput in less amount of time.

9 combinations of Instancing and Concurrency

There are 9 combination of concurrency and instancing methodology as shown in the below table. In the further coming section we will discuss in more detail about the same.

InstanceContext Mode	ConcurrencyMode
InstanceContext Mode	Single (Default)	Multiple	Reentrant
Single (Single instance for all client)	Single thread for all clients.	Multiple threads for all clients.	Single threads for all clients, locks are released when calls diverted to other WCF services.
PerSession (Multiple instance per client)	Single thread for every client.	Multiple threads for every request.	Single threads for all clients, locks are released when calls diverted to other WCF services.
PerCall (Default) (Multiple instance for every method call)	Single thread for every client	Single thread for every client	.Instance mode = Per Call and Concurrency = Single Single threads for all clients, locks are released when calls diverted to other WCF services.

Instance mode = Per Call and Concurrency = Single

Instance mode 'PerCall' with 'Single' concurrency is the default setting in WCF. We have already seen and example and demonstration of the same.

With per call new WCF instances are created for every method calls made to the WCF server service. The default concurrency is single so only one thread will be used to serve all instances.

Below is a simple pictorial representation of what will happen in per call and single concurrency scenario:-

For every client instance a single thread will be allocated.
For every method call a new service instance will be created.
A single thread will be used to serve all WCF instances generated from a single client instance.

If you refer the previous sample you can see how threads are same and the halt of 5 seconds on the WCF service.

Instance mode = per Call and Concurrency = Multiple

[ServiceBehavior(InstanceContextMode=InstanceContextMode.PerCall, ConcurrencyMode = ConcurrencyMode.Multiple)]
public class HelloWorldService : IHelloWorldService
{
}

In this combination multiple instances are created for every call but multiple threads serve every method call to WCF service instance. You can see in the below figure we have two WCF service instance and every instance has multiple WCF service objects created for every method call. Every method call is handled by multiple threads.

In the above sample if you remember we have multiple threads serving every method call and the time difference between every method call is reduced considerably. The method calls are fired approximately same time.

Instance mode = per session and Concurrency = single

[ServiceBehavior(InstanceContextMode=InstanceContextMode.PerSession, ConcurrencyMode = ConcurrencyMode.Single)]
public class HelloWorldService : IHelloWorldService
{
}

In this combination one WCF service instance is created for every WCF client session because the WCF instance mode is set to per session. All the method calls are executed in a sequential manner one by one. In other words only one thread is available for all method calls for a particular service instance.

If you run the sample code with the per session and single combination mode you will find the same thread executes all request with same WCF instance per session.

Instance mode = per session and Concurrency = Multiple

[ServiceBehavior(InstanceContextMode=InstanceContextMode.PerSession, ConcurrencyMode = ConcurrencyMode.Multiple)]
public class HelloWorldService : IHelloWorldService
{
}

In this combination one WCF instance is created for every WCF client session and every method call is run over multiple threads. Below is the pictorial representation of the same.

If you run the sample code attached with this article you will find same instance with every method call running on different methods.

To get a better idea you can run with different client exe instance with different names as shown below. You can notice how every client get his own WCF service instance with every method allocated to run on different threads.

Instance mode = Single and Concurrency = Single

[ServiceBehavior(InstanceContextMode=InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Single)]
public class HelloWorldService : IHelloWorldService
{
}

In this combination only one instance of WCF service instance is created which serves all requests which are sent from all WCF clients. These entire requests are served using only one thread.

You can see in the below figure approximately one thread is assigned for every WCF client call and only one instance of the WCF service is created.

Instance mode = Single and Concurrency = Multiple

[ServiceBehavior(InstanceContextMode=InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)]
public class HelloWorldService : IHelloWorldService
{}

In this combination one WCF service instance is created for serve all WCF clients. All request are served using multiple/different threads.

You can see from the output we have 6 threads serving 10 requests as compared previously we had approximately only 2 threads.

Reentrant

In mode only thread runs to serve all requests. If your WCF service makes an outbound call to some other WCF service or makes a client call it releases the thread lock. In other words until the outbound call is completed other WCF clients can make call.

Throttling behavior

WCF throttling settings helps you to put an upper limit on number of concurrent calls, WCF instances and concurrent session. WCF provides 3 ways by which you can define upper limits MaxConcurrentCalls, MaxConcurrentInstances and MaxConcurrentSessions.

MaxConcurrentCalls: - Limits the number of concurrent requests that can be processed by WCF service instances.

MaxConcurrentInstances: - Limits the number of service instances that can be allocated at a given time. When it's a PerCall services, this value matches the number of concurrent calls. For PerSession services, this value equals the number of active session instances. This setting doesn't matter for Single instancing mode, because only one instance is ever created.

MaxConcurrentSessions: - Limits the number of active sessions allowed for the service.

All the above 3 setting can be defined in the 'servicebehaviors' tag as shown in the below XML snippet.

<serviceBehaviors>
<behavior name="serviceBehavior">
<serviceThrottling maxConcurrentCalls="16"
maxConcurrentInstances="2147483647" maxConcurrentSessions="10" />
</behavior>
</serviceBehaviors>

Default values for WCF throttling

Below is a simple table which shows default settings for throttling as per different WCF versions.

	MaxConcurrentSessions	MaxConcurrentSessions	MaxConcurrentSessions
WCF 3.0 / 3.5	6	26	10
WCF 4.0	16 * processorcount MaxConcurrentCalls	MaxConcurrentCalls + MaxConcurrentSessions 100 * processorcount	100 * processorcount

Reference

Nice information on throttling details by Mr. Kenny wolf http://kennyw.com/work/indigo/150

Nice article by Mr. Rick Rain in 3 parts which explains Concurrency and throttling in depth.

http://blogs.msdn.com/b/rickrain/archive/2009/06/15/wcf-instancing-concurrency-and-throttling-part-1.aspx

Mr. Michele Leroux Bustamante talks about settings which influence how throughput of WCF services can be increased using WCF concurrency.

http://www.windowsitpro.com/article/net-framework2/concurrency-and-throttling-configurations-for-wcf-services.aspx

Justin smith explains demonstrates a sample code on how 'InstanceContextMode' and 'ConcurrencyMode' property affects concurrency.

http://www.wintellect.com/CS/blogs/jsmith/archive/2006/05/16/instancecontextmode-and-concurrencymode.aspx

Mr. Glav talks about throttling and legacy system http://weblogs.asp.net/pglavich/archive/2007/06/27/wcf-and-concurrent-usage-throttling.aspx

Default values for WCF throttling http://blogs.msdn.com/b/wenlong/archive/2009/07/26/wcf-4-higher-default-throttling-settings-for-wcf-services.aspx

Download code

You can download the WCF concurrency code from top of this article.

4 Comments

Join the conversation! Your thoughts help the community grow.

sky none13y ago
It's a good article. Thank you for sharing.
Michal Szychowiak15y ago
You have presented here very interesting experiments. Good job! I think, it may be valuable for the overall benefit from this presentation, if you can make some part of it a little bit clearer. First suggestion concerns example “Instance mode = Per Call and Concurrency = Single”. In my understanding, the picture you include for this example (http://www.c-sharpcorner.com/UploadFile/shivprasadk/1802/Images/8.jpg) seems to represent more likely the per-session case, is it not? The picture doesn’t undoubtedly expose any sequentiality between (apparently) simultaneous calls made by the two distinct clients, i.e. method calls 1 and 2 are clearly sequential, but method calls 1 and 3 are depicted rather in parallel (in spite of the annotation between curly brackets). Probably I can’t resist interpreting the picture from a time axis perspective, and that’s what makes my confusion (well, compare that picture with another one: http://www.c-sharpcorner.com/UploadFile/shivprasadk/1802/Images/16.jpg to see my point). The confusion gets deeper, as you run only one client in the experiment (http://www.c-sharpcorner.com/UploadFile/shivprasadk/1802/Images/111.jpg). Unfortunately, you also run a single client in another example, “Instance mode = per session and Concurrency = single” (http://www.c-sharpcorner.com/UploadFile/shivprasadk/1802/Images/12.jpg) so it really doesn’t prove what is actually “single” here – any call for any instance (i.e. for any session) or only each call for the particular instance (session), which means, in the latter case, parallel calls (threads) possible for distinct instances. Similar experiment as the following one (http://www.c-sharpcorner.com/UploadFile/shivprasadk/1802/Images/15.jpg) would be much more useful here. Second thing. In the first example you state that “by default the WCF service is configured to run with instance mode of per call”. However Microsoft declares it to be „per session” (http://msdn.microsoft.com/en-us/library/cc681240.aspx). Obviously, since you run only a single client in that experiment, no difference can be observed between „per call” and „per session”. Regardless of the above criticism, I have found your work very valuable and important. Please consider my post only as a suggestion to possible improvement of the sole presentation.
Manish Chadha16y ago
Thank you. It is a good article with very good images to drive home the point. But i think the image for Instance mode = Per Session and Concurrency = Multiple, the image shows a the same instance being used even if the calls are from a different client. This i believe is incorrect. Every new client, in this context, should have their own new instance.
shiva kumar16y agoedited
thanku for posting