Please form a queue, for poison.

In the previous post about using MSMQ to facilitate One-Way-Calls I covered some of the basics of setting up a MSMQ binding. In that scenario if a consumer of that end point sent a malformed message or the message became corrupted in transmission, the service would either discard it and there would be no feedback on the problem or far worse the service may attempt to process the poison message again and again getting stuck in a loop. Such a processing loop would prevent a service from continuing normal operation of being able to handling subsequent messages. MSMQ 3.0 under Windows XP and Windows Server 2003 has only the ability to retry a failed message once, before it either simply drops the message or faults the channel. A message that is un-processable in this fashion is referred to as a poison message. The wise choice is therefore to use MSMQ 4.0 under Windows Server 2008/Vista/Windows 7 which offers more suitable alternatives.

One alternative I would like to discuss here is using a poison queue. The basic concept is; the primary (msmq based) service which has an “important function” (unlike our unimportant notification service in the previous msmq post) has an associated poison queue. Messages are moved into this queue when they are determined to in-fact be poison. These messages can then be dealt with by a separate service or just human monitoring.

A setup of a poison queue is achieved by exposing a second service and endpoint binding with an identical address but with a ;poison suffix (as well as making use of a different bindingConfiguration):

<services>
   <service name = "MyService">
      <endpoint
         address  = "net.msmq://localhost/private/ImportantQueue" 
         binding  = "netMsmqBinding"
         bindingConfiguration = "importantMsgHandling"
         contract = "IImportantService" 
      />
      <endpoint
         address  = "net.msmq://localhost/private/ImportantQueue;poison"
         binding  = "netMsmqBinding"
         contract = "IImportantService"
      />
   </service>

   <bindings>
      <netMsmqBinding>
         <binding name = "importantMsgHandling"
            maxRetryCycles= "2" 
            receiveRetryCount = "3"
            receiveErrorHandling="Move"
            retryCycleDelay = "00:00:10">
         </binding>
      </netMsmqBinding>
   </bindings>
</services>

I hit a few hiccups in this run through so I’ve added a trouble shooting section at the bottom of the post.

The other key configuration features to note on this configuration setup are (and illustrated below):

  • receiveErrorHandling – with the options of: Fault, Drop, Reject and Move. With our chosen option of move meaning once the error handling process has completed it will be moved to our poison queue
  • receiveRetryCount – number of immediate attempts to process the message.
  • maxRetryCycles – number of subsequent attempts to process message.
  • retryCycleDelay – time between retry cycles.
Poison Queue Message Processing

Poison Queue Message Processing

Once our poison message has failed to be processed, it is shifted to the “poison” sub-queue as shown in this Computer Management screen shot:

Computer Management - Poison Queue

Computer Management - Poison Queue

At this point we can do a few things, from the simplest option of having an admin user review the messages in this poison queue, to a more sophisticated approach of having a separate service attempt to process these poison messages with some more sophisticated logic. Juval Lowy (a while ago now) has published an MSDN Magazine article on an even more sophisticated error handling approach for dealing with a poison messages dubbed a “Response Service”. In essence a (potentially) disconnect client is given the ability to receive feedback via a separate queue. Message responses are generated based on the original flawed message. I plan to implement this approach if at some point the medical demo application I started a few weeks ago warrants it.

Troubleshooting:
On a side note under Windows 7 and Vista the user you are logged in as will not have access to listen for messages on a given port. You’ll quickly see the AddressAccessDeniedException:
HTTP could not register URL http://+:8000/. Your process does not have access rights to this namespace (see http://go.microsoft.com/fwlink/?LinkId=70353 for details)

AddressAccessDeniedxception

AddressAccessDeniedxception


To resolve this simply grant yourself permission to access the port via an administrator command prompt call to netsh.exe

netsh http add urlacl url=http://+:8000/ user="DOMAIN\User Name"

Note: “Domain” can be the machine name if you’re not on a domain. Add a few other ports you might be using too, i.e 8005 for the ServiceModelEx logbook service. For more info refer to this blog post, which has a detailed explanation and in the comments there is discussion of issues on other systems such as Windows Server 2003.

ServiceModelEx

I have brought up ServiceModelEx several times already, and thought I’d dedicate a post that I can link back to other times when I mention ServiceModelEx.

ServiceModelEx is a set of WCF extensions primarily written by Juval Lowy from IDesign.net. This library of extensions is called ServiceModelEx it is available as a free download. The library is accompanied by an extensive collection of demo applications covering the use of WCF in conjunction with ServiceModelEx.

To quote the front page of the demo and utility section:
IDesign serviceware are a set of original techniques, tools, utilities and even breakthroughs developed by the IDesign architects. The utilities are largely productivity-enhancing tools, or they compensate for some oversight in the design of WCF.

IDesign WCF Tools Interface

IDesign WCF Tools Interface

At the time of writing the current version of the resources-cd demo count was 133, with some overlaps and but all demonstrating some specific concept or approach.

One Way Street

One-Way-Calls in WCF are an effective way to perform an action that you don’t care when (or even if) it completes. If you do care about tracking success or logging failure I’ll have a follow up post to cover that. The simplest example is something along the lines of a non-critical system status update; achieved by sending an email, a text message or an even more modern tweet post, informing someone of an system state/event.

One Way

To begin with lets get a one way method call sent off to a service that may (or may not) be currently active. To achieve this I will be making use of Microsoft’s Message Queuing system (MSMQ). At the end of the post I’ll have a brief discussion on MSMQ.

Now, the key attribute to achieve one-way is:

[OperationContract(IsOneWay = true)]

It is applied to methods on the Service Contract interface. A quick thing to note here is for the NetMsmq Binding in WCF the IsOneWay is required as the queue does not support a duplex channel scenario, and if you miss-configure your methods this exception will quickly pop up on load:

Contract Requires Two-Way Exception

Contract Requires Two-Way Exception

To configure your App.Config file we make use of the net.Msmq binding. This config follows the same pattern in previous posts. This endpoint can exist alongside other types of endpoints, but should be separated into it’s own manager that will handle all the OneWay endpoints/methods.

<services>
   <service name="Notification.NotificationImplementation.OneWayNotificationManager">

      <endpoint name="netMsmq_IOneWayNotificationService"
         address="net.Msmq://MACHINE_NAME/oneWayNotificationService"
         binding="netMsmqBinding"
         bindingConfiguration="netMsmq"
         contract="Notification.NotificationServiceContract.IOneWayNotificationService" />

   </service>
</services>

Now for the full interface definition, it’s as just like the previous examples except with the IsOneWay attribute. Another addition I’ve included here is DeliveryRequirements.Required it assist in defining a solid rule that this interface is designed only for MSMQ calls. The other 2 options on the attribute are ‘NotRequired’ and ‘Allowed’.

[ServiceContract(Name = "IOneWayNotificationService", Namespace = "http://your.domain/2009/")]
[DeliveryRequirements(QueuedDeliveryRequirements = QueuedDeliveryRequirementsMode.Required)]
public interface IOneWayNotificationService
{
   [OperationContract(IsOneWay = true)]
   [TransactionFlow(TransactionFlowOption.Allowed)]
   void SendNotification(NotificationRequest request);
}

Creating the host is identical to previous examples:

ServiceHost oneWayNotificationManager = new ServiceHost();

One additional approach to make life just that little bit easier, making use of ServiceModelEx‘s method VerifyQueue() to create the actual queue and verify access to it. As a reminder my ServiceModelEx overview is here.

//With a quick and dirty zero index array element if you only have 1 endpoint.
ServiceModelEx.QueuedServiceHelper.VerifyQueue(oneWayNotificationManager.Description.Endpoints[0]);

//Or iterate over each endpoint in a loop
foreach(ServiceEndpoint endpoint in oneWayNotificationManager.Description.Endpoints)
   ServiceModelEx.QueuedServiceHelper.VerifyQueue(endpoint);

oneWayNotificationManager.AddErrorHandler();
oneWayNotificationManager.Open();

Now calling the method from a client application (or other service) is just like any other call except there will be no response.

using ( OneWayNotificationServiceProxy proxy = new OneWayNotificationServiceProxy() )
{
   NotificationRequest request = new NotificationRequest();
   request.someProperty = true;

   proxy.SendNotification(request);
}

The client sending the call will need to have an net.Msmq endpoint configured in it’s App.Config:

<system.serviceModel>
   <client>
      <endpoint name="netMsmq_IOneWayNotificationService"
         address="net.Msmq://MACHINE_NAME/OneWayNotificationService"
         binding="netMsmqBinding"
         bindingConfiguration="netMsmq"
         contract="Notification.NotificationServiceContract.IOneWayNotificationService" />
      </client>
   </system.serviceModel>

It’s as simple as that, open up the Computer Management Console to view the messages show up in the queue. In the next post I’ll go into some more detail about the queues.

Computer Management Console

Computer Management Console

MSMQ

There’s a lot to say about MSMQ, and a variety of ways to use with WCF. In this example basically it will hold the message until our service is ready to use it, and in this post we’re not concerned if something goes wrong. Two quick things I want to bring up about MSMQ are:

  1. It is not installed by default; you need perform a few steps – MS Technet has the instructions.
  2. To make use of “Public Queues” the system needs to be part of a domain with Active Directory, otherwise all you have access is to “Private Queues”. To directly quote the MSMQ Faq:
    • “MSMQ utilizes Active Directory for security information related to encryption and decryption, or message authentication. MSMQ also uses Active Directory for public queue lookups – in this way the sending or receiving applications do not need to know the machine-name of the computer hosting the destination queue.”

Being a Good Host 2 – Grouping Multiple Services

Previously I gave a simple overview of hosting a single services in a form. If you’re developing a larger system and you have broken down business across several services, for reasons ranging from code management, to partitioning development tasks across teams, to reducing the compile time for a visual studio solution. You’re likely to want to keep the services separate but kick them off in groups; “Core System Services”, “Supporting Services”, “Reporting/Administration Services”, etc.

The most manual part is to merge the endpoints from all your individual config files into 1 single App.Config file that will accompany your production level deployment approach. This can also be used to package up the initial services mid-way through development when they are unlikely to change and it saves hassle running them as a group.

Watch-out-for: the contract definition in the config file; make sure it’s a fully qualified namespace for each service end point. You add individual references to your host application for each service you’re including.

<endpoint  
  address="net.tcp://localhost:8001/Patients"  
  binding="netTcpBinding"  
  bindingConfiguration="myNetTcpBinding"
  contract="Server.Patient.IPatientServiceContract" 
 />

For this example I’ve simply chosen to run it as a standard windows service, where all I need to do is implement OnStart() and OnStop() and if you think you need to OnPause() and OnContinue() or any of the other ServiceBase methods.

public partial class MultiServiceHost : ServiceBase
{
   private IList<ServiceHost> hosts = new List<ServiceHost>();
}

The OnStart() is simply a list of each of your services you’re grouping to run together.

protected override void OnStart(string[] args)
{
   try
   {
      hosts.Add(new ServiceHost(typeof(PatientService)));
      hosts.Add(new ServiceHost(typeof(MedicalProcedureService)));
      hosts.Add(new ServiceHost(typeof(OtherService)));
      //more hosts...
      StartServices();
   }
   //Catch ()...
}

A loop to kick off all the services, that’s called after they’re all added.

private void StartServices()
{
   foreach (ServiceHost h in hosts)
   {   h.Open();   }
}

Closing them all off upon shutdown of the host.

protected override void OnStop()
{
   foreach (ServiceHost host in _hosts)
   {
      if (host != null)
         host.Close();
   }
}

It’s that simple.

Talking Without Speaking, Hearing Without Listening.

In this blog I publish information and ideally have people that subscribe to it. For the blog I use the WordPress hosting site (service) you may use an RSS aggregator to subscribe (client). In a business application you can achieve this same concept easily with WCF. In this post I will run through an example of the Medical System Client application subscribing to patients of interest. When other clients update those same patients with business critical data the active clients can receive an ‘alert’ immediately at easily.

We begin with the client side setup. First we define the contract interface that contains our operational methods (excluded below, but examples include Save(), Retrieve(), etc). The key things to note on the interface is the ‘CallbackContract’ attribute, and the subscribe/unsubscribe methods. The subscription methods use a Patient Id key but this can be whatever is most suitable in your situation, for example an enum will work fine for more generalised event types.

[ServiceContract(CallbackContract = typeof(IPatientEvents))]
public interface IMyContract
{
   [OperationContract]
   void Subscribe(int patientId);
   [OperationContract]
   void UnSubscribe(int patientId);
}

The next step is to define an events interface. This interface outlines expected methods the client must implement to be able to handle the events raised by the service. Note that the event handler has an attribute defining it as a OneWay call. This event is lab results for a patient have been entered into the system, and are of a critical nature.

public interface IPatientEvents
{
   [OperationContract(IsOneWay = true)]
   void LabResultsUpdated(int id, string text);
}

Then you simply create your client side class against the events interface. In this example the client is a WPF page class.

class PatientDetails : Page, IPatientEvents
{
     MyContractClient proxy;
	 
	 //other methods
}

The client will need to supply a context to the service so it can maintain an association on the channel. This is simply achieved through a constructor call when initialising the proxy. Also at this point we’re subscribing to the events for the given ID. This subscription can be client triggered this way. Alternatively certain actions can have the server subscribe the client server side. An example would be during a data retrieval call for a patient; the server forcing the setup up the subscription.

  InstanceContext context = new InstanceContext(this);
  
  proxy = new MyContractClient(context);

  proxy.Subscribe(id);

Now on the service we implement the same contract IMyContract we also define the signature of the delegate Action that will be raised as an event. Along with implementing the method that will attach the event handler to the appropriate client. The key thing to note here is the Current.GetCallbackChannel this is linking via the context the client supplied in the proxy initialisation.

[ServiceBehavior(InstanceContextMode
      = InstanceContextMode.PerCall)]
class PatientService : IMyContract
{
    static Action<int, string> m_EventLabs = delegate { };

    public void Subscribe(int patientId)
    {
        IPatientEvents subscriber = 
          OperationContext.Current.GetCallbackChannel<IPatientEvents>();

        m_EventLabs += subscriber.LabResultsUpdated;
    }
}

All that’s left now is to write the actual event firing code, that can be triggered after an appropriate event. In this example the firing takes place after the Patients Lab Results have been updated. Thereby allowing any other users currently viewing this patient’s details to be alerted.

public static void FireEvent(LabResult labResult)
{
    m_EventLabs(labResult.PatientId, labResult.ToString());
}

public bool SaveLabResults(LabResult lr)
{
    //save code...

    PatientService.FireEvent(labResult);
}

Back on the client we have defined a method that will be executed upon the event. Remember the service has attached the handler on the Subscribe() method.

public void LabResultsUpdated(int id, string text)
{
    this.displayAlerts.Text = text;
}

That is all that is required to have your client quickly and easily subscribe to and handle events. It’s thread-safe and efficient with the service doing all the hard work.

I will expand on [ServiceContract(CallbackContract … ] and [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)] in future posts, and I’ll try to remember to update this post with direct links.

Just for some reference sake here are the objects used above that were not defined in the code blocks:

[DataContract]
public class Patient
{    [DataMember]
    public int PatientId { get; set; }
    //more members
}
[DataContract]
public class LabResult
{    [DataMember]
    public int LabResultId { get; set; }
    //more members
}
//displayAlerts is just a page <TextBlock> object

This MessageHeader will self destruct in 5 seconds

Let’s get our hands dirty with an example you can follow along with. With the basic concepts covered we can whip up something to send some data back and forth. We will start with a simple winform to run our service. Another winform (acting as the client application) to connect to it so our service can do some intended job.

In this post I’m also going to introduce a refined and well developed library of enhancements for WCF. It is created by the very talented Juval Lowy you can find his work at IDesign.net. This library is called ServiceModelEx it is available as a free download, and it is very, very useful. I hope to demonstrate its usefulness in this and future posts. ServiceModelEx is designed to help simplify some of the more complex aspects of WCF along with adding extension to existing features. It makes some aspects of WCF more robust, such as adding TypeSafe alternatives and often adding an overlooked sub-feature (e.g. NetNamedPipeContextBinding I’ll discuss this in a future post).

Once you have ServiceModelEx downloaded and included in your Visual Studio project all you need to do is add a reference to it:

//WCF Service Model:
using System.ServiceModel;
//iDesign Service Model Extensions:
using ServiceModelEx;

In this post I’m introducing a theme for the demos; they will follow the ideas of a medical system. With concepts such as Patient, TreatmentSchedule, MedicalProcedure, Doctor, and other types of system Users (Nurses, LabTechnicians, etc).

On with it.
We are setting up a simple call to a service to create/update/delete a new patient.

To begin we will need to setup some data constructs (they are simplified to keep the code sections short, i.e. no extensive code to setup private members and public properties). A class MsgHeader will act as a common object sent ‘out-of-band’ on all calls to the service. The ‘out-of-band’ concept is used for the purpose of keeping data contracts free of plumbing-related operations. This example uses the header to supply something specific about the user, and the type of operation we are performing (enum OperationTypes). Another example of advanced control data you can pass via headers is a ‘message priority’ value.

[DataContract]
public class MsgHeader
{
    [DataMember]
    public int userRoleId;
    [DataMember]
    public int operationType;
}

A snippet of the data contract, and other setup data:

[DataContract]
public class Patient
{
   [DataMember]
   public int PatientId;
   //more members...
}

public enum OperationTypes { 
   Update = 0, Insert = 1, Delete = 2 }
public enum UserRoles { 
   Reception = 1, Nurse = 2, Doctor = 3 }

The Service is set up with a simple save/retrieve interface:

[ServiceContract]
interface IPatientContract
{
   [OperationContract]
   void SavePatient(Patient p);
   [OperationContract]
   Patient RetrievePatient(int patientId);
}

Via the Client proxy constructor we pass in our populated MsgHeader object. Then call any required methods on the proxy, in this case the save.

//class instance of patient details
public Patient CurrentPatient;

public void Save()
{
   MsgHeader header = new MsgHeader();
   //Set the properties accordingly ...

   //Setup and Call Proxy
   MedicalSystem proxy = new MedicalSystem(header);

   proxy.SavePatient(this.CurrentPatient);

   proxy.Close();
}

Finally on the service implementation to access the MsgHeader you simply use HeaderContext.Current.

class MedicalSystem : IPatientContract
{
   public void SavePatient(Patient p)
   {
      switch (HeaderContext.Current.dataOperation)
      {
         case (int)DataOperations.Delete:
            DeletePatient(p.id);
            break;
         case (int)DataOperations.Insert:
            CreateNewPatient(p);
            break;
         // ...
      }
   }
   // ...
}

There we have it – ‘out-of-band’ passing of data to your service to keep system entities free of the clutter of control data.

Putting all the basics together

Before we get stuck into a full scale example, I’ve prepared a simple checklist of tasks required to get a basic service up and running.

Refer to this checklist to get your first WCF solutions up and running quickly.

Service & (Host)

  1. Service Contract IMyServiceContract
  2. Implementation of that Contract
  3. Service Proxy Implementing ServiceContract & ClientBase<IMyServiceContract>
  4. Endpoints via App.Config
  5. Form/Console to kick off Host

Client

  1. Proxy implementing IMyServiceContract
  2. Endpoints to match those specified in service via App.config

If you haven’t done things right one of the very early exceptions you might receive is an InvalidOperationException and it will name the Contract that you haven’t implemented correctly. This is likely to be caused by a flaw in your app.config (server side).

A Common Exception from a Simple Misconfiguration

A Common Exception from a Simple Misconfiguration

Other exceptions to look out for are those when you attempt to call the service and either endpoints are misconfigured or implementations aren’t complete.