How much memory are you currently using?

Following on from my previous post about a demo PLINQ application, I had some small discoveries about memory usage and wanted to blog them.

As I was loading large chunks of data into memory and started monitoring my little demo applications RAM footprint, I discovered that even tho the garbage collection system had already cleaned up my now out-of-scope variables, the RAM utilisation as reported to windows did not necessarily drop.

This actually turns out is quite a logical thing to have happen and works this way for good reason, I just hadn’t thought about it. I was simply expecting an action of explicit free-up of memory to instantly translate into free’d memory in Windows.

The process was just holding onto a reserved amount of memory the .NET runtime assuming I would soon need that space again, and it was correct in its assumptions; I did need that memory very soon.

Use More Memory!

Use More Memory!

To prove this: subsequent button clicks to load more data (roughly the same size volume), didn’t make the application’s memory footprint grow even larger. Another interesting thing I noted with the task manager window open all the time, was when I loaded additional applications and my demo app didn’t need the memory, it would free up the few hundred mb it had a hold of but wasn’t currently making use of.

Demo Application Memory Usage

Demo Application Memory Usage

Advertisements

LINQ Basics (Part 2) – LINQ to SQL in .NET 4.0

Continuing my 2 part series on LINQ Basics, here I will attempt to discuss a few improvements in LINQ to SQL as part of the .NET 4.0 release scheduled soon. A key post on the 4.0 changes is by Damien Guard who works on LINQ at Microsoft along with other products. This will allow me to discuss some of the changes but also go into a bit more detail extending the “basics discussion” started in the previous post.

Note: In October 2008 there was a storm of discussion about the future of LINQ to SQL, in response to a focus shift from Microsoft for “Entity Framework”, the key reference post for this is again one by Damien G.

The upcoming 4.0 release doesn’t to cause damage to the capability of LINQ to SQL as people may have been worried about (i.e. disabling/deactivation in light of the discussions of Oct 2008). There are actually a reasonable amount of improvements, but as always there’s a great deal of potential improvements that there was not capacity/desire to implement for a technology set that is not a major focus. But LINQ to SQL is still capable enough to function well to be used for business systems.

First up there are some performance improvements in particular surrounding caching of lookups and query plans in it’s interaction with SQL Server. There are a few but not all desired improvements in the class designer subsystem, including support for some flaws with data-types, precisions, foreign key associations and general UI behaviour flaws.

There is a discussion on Damien’s post about 2 potential breaking changes due to bug fixes.

  1. A multiple foreign key association issue – which doesn’t seem to be a common occurrence (in any business systems I’ve been involved in).
  2. A call to .Skip() with a zero input: .Skip(0) is no longer treated as a special case.

A note on .Skip(), such a call is translated into subquery with the SQL NOT EXISTS clause. This comes in handy with the .Take() function to achieve a paging effect on your data set. There seems to be some discussion on potential performance issues using large data sets, but nothing conclusive came up in my searches, but this post by Magnus Paulsson was an interesting investigation.

As for the bug It does seem logical to have treated it as a special case and possibly help simplify code that might be passing a zero (or equivalent null set), but if it was an invalid case for .Skip() to be applied to with a value greater than zero it will fail forcing you to improve the query or its surrounding logic.

Just for the sake of an example here is .Skip() & .Take(). Both methods have an alternate use pattern too: SkipWhile() and TakeWhile(). The while variants take as input a predicate to perform the subquery instead of a count.

var sched = ( from p in db.Procedures  
             where p.Scheduled == DateTime.Today  
             select new {  
                      p.ProcedureType.ProcedureTypeName,  
                      p.Treatment.Patient.FullName  
                     };
                ).Skip(10).Take(10);

There are also improvements for the use of SQL Metal which is the command line code generation tool. As a side note to make use of SQL Metal to generate a DBML file for a SQL Server Database execute the command as such in a Visual Studio command prompt:

    sqlmetal /server:SomeServer /database:SomeDB /dbml:someDBmeta.dbml

LINQ Basics

As part of preparation work I’m doing for a presentation to the Melbourne’s Patterns & Practices group in October on LINQ and PLINQ. I thought I would cover off the basics of LINQ in .NET 3.5 and 3.5 SP1 in this post. The changes in .NET 4.0 in the next post, and then a discussion about PLINQ in a third post.

O.k. so let’s sum up the basics. The core concept of LINQ is to easily query any kind of data you have access to in a type-safe fashion. Be it SQL Server stored, a collection (i.e. something implementing IEnumerable) or an xml data structure. A further addition to this power is the ability through C# 3.0 to create projections of new anonymous structural types on the fly. See the first code sample below for the projection; in that simple examples it’s the creation of an anonymous type that has 2 attributes.

Continuing on with my “medical theme” used for the WCF posts, here is a simple schema layout of the system, consisting of a patient and a medical treatment/procedure hierarchy. This is given the title of ‘MedicalDataContext’ to be used in our LINQ queries.

Medical System Basic Schema

Medical System Basic Schema

These items have been dragged onto a new ‘LINQ to SQL Classes’ diagram from the Server Explorer > Data Connections view of a database.

Server Explorer Window

Server Explorer Window

To create the ‘LINQ to SQL Classes’ diagram simply add …

Add New Item

Add New Item


a new …
Linq to SQL Classes

Linq to SQL Classes

Back to the logic. We have a Patient who’s undergoing a certain type of treatment, and that treatment has associated procedures. To obtain a collection of procedures for today, and the name of the patient who will be attending we simply build up a new query as such:

var db = new MedicalDataContext();

var sched = from p in db.Procedures
            where p.Scheduled == DateTime.Today
            select new {
                     p.ProcedureType.ProcedureTypeName,
                     p.Treatment.Patient.FullName
                    };

Note: The patient table structure doesn’t have a field called ‘FullName’ I make use of a partial class extending its properties to add a read-only representation, check out this post by Chris Sainty for more info on making use of partial classes with LINQ.

At this point we can now iterate over each item in our ‘sched’ (scheduled procedures) collection.

foreach (var procedure in sched)
{
   //process/display/etc
}

This brings me to another key point ‘Delayed Execution’ or (‘Deferred Execution’) check out Derik Whittaker’s: Linq and Delayed execution blog post for a more detailed walk through.

Basically the query we defined earlier is only a representation of the possible result set. Therefore when you first make a call to operate on the variable representing the query results, that’s when execution will occur.

So it becomes a program flow decision whether to always execute the query live when it’s being processed (i.e. most up-to-date data) or to force a single execution then make use of that data in the current process flow. A forced execution can be easily achieved several ways, the simplest choice is to just create a new list object via ToList() to execute the fetching of the data.

var allProcedures = todaysProcedures.ToList();

So far this has all revolved around accessing data in a SQL Server Database (the setup of a LINQ to SQL class). LINQ’s purpose is to be able to query any form of collection of data.

Now let’s say we wanted to obtain some information through recursion of a class using reflection.

Note: a business case for the use of reflection is often tied very deeply into some limitation, special case, etc. So a more specific example would take us well away from the topic of LINQ. So will keep this example trivial, this could just as easily manipulate a more useful class object to interrogate it.

var staticMethods = 
    from m in typeof(string).GetMethods()
    where !m.IsStatic
    order by m.Name
    group m by m.Name into g
    select new 
      { 
         Method = g.Key, 
         Overloads = g.Group.Count()
      };

Which output element by element will generate this output:

  { Method = Clone, Overloads = 1 }
  { Method = CompareTo, Overloads = 2 }
  { Method = Contains, Overloads = 1 }
  { Method = CopyTo, Overloads = 1 }
  { Method = EndsWith, Overloads = 3 }
  . . .

For an off-topic discussion, check out this article by Vance Morrison post titled Drilling into .NET Runtime microbenchmarks: ‘typeof’ optimizations, as a discussion of the use of ‘typeof’ in my above query.

For a more typical business case example using a LINQ statement to fetch a collection of controls and operate on them (in this case disable them). Here we’re operating on a Control collection.

Panel rootControl;

private void DisableVisibleButtons(Control root)
{
   var controls = from r in root.Controls
                       .OfType()
                   where r.Visible
                   select r;

    foreach(var c in controls)
    {
        if(c is Button) c.Enabled = false;
        DisableVisibleButtons(c);  //recursive call
    }
}

//kick off the recursion:
DisableVisibleButtons(rootControl);

Summary:

  • LINQ to SQL generates classes that map directly to your database.
  • LINQ helps you manipulate strongly typed results (including intellisense support).
  • “LINQ to SQL” is basically “LINQ to SQL Server” as that is the only connection type it supports.
  • There is a large set of extension methods out of the box check out for samples: 101 LINQ Samples.

WPF Community Workshop

I attended a WPF Community Workshop yesterday (Sat 27/Mar/09). It was organised by The Australian Computer Society with help from some Readify guys; Mahesh Krishnan and David Burela. Our attendance fee was generously donated to the Red Cross Victorian Bushfire Appeal.

The day was targeted at people new to WPF. Which I knew going in but I still enjoy going over basic concepts, always learn a few extra little things (or at least get reminded of them). I would like to see more in depth workshops run, I’ll provide this feedback to the organisers.

As it turns out we were a test group for a set of lab activities titled “WPF Reactor”, I say this because the lab source code was out of sync with the lab instructions. Meaning it wasn’t subjected to any extensive Quality Assurance process before being burnt to little 2 disc foldout packages. With problems ranging from; lab task ordering issues (preceding labs had complete code for subsequent labs) to misleading (non-detailed or even erroneous) instructions. None the less it’s still a learning experience.

I managed to locate a download link for the lab material online (~410mb) on GainSecure.net full download here, lite download here. Those who have physical access to me, can get one of a few copies of the discs off me, not sure if it would be wise for me to upload 400mb to wordpress.

Some of the things to check out are:
– As an absolute must see of cool WPF tricks – Kevin’s WPF Bag-O-Tricks (a full VS solution of demos).
– The WPF toolkit on CodePlex.
– David Burela’s “WPF links for beginners” post – contains lots of links.
– A WPF/Silverlight tutorial site – NibblesTutorials.net.
– Expression Blend 3 Beta preview download – with a notable TFS integration amongst all the other enhancements.

This MessageHeader will self destruct in 5 seconds

Let’s get our hands dirty with an example you can follow along with. With the basic concepts covered we can whip up something to send some data back and forth. We will start with a simple winform to run our service. Another winform (acting as the client application) to connect to it so our service can do some intended job.

In this post I’m also going to introduce a refined and well developed library of enhancements for WCF. It is created by the very talented Juval Lowy you can find his work at IDesign.net. This library is called ServiceModelEx it is available as a free download, and it is very, very useful. I hope to demonstrate its usefulness in this and future posts. ServiceModelEx is designed to help simplify some of the more complex aspects of WCF along with adding extension to existing features. It makes some aspects of WCF more robust, such as adding TypeSafe alternatives and often adding an overlooked sub-feature (e.g. NetNamedPipeContextBinding I’ll discuss this in a future post).

Once you have ServiceModelEx downloaded and included in your Visual Studio project all you need to do is add a reference to it:

//WCF Service Model:
using System.ServiceModel;
//iDesign Service Model Extensions:
using ServiceModelEx;

In this post I’m introducing a theme for the demos; they will follow the ideas of a medical system. With concepts such as Patient, TreatmentSchedule, MedicalProcedure, Doctor, and other types of system Users (Nurses, LabTechnicians, etc).

On with it.
We are setting up a simple call to a service to create/update/delete a new patient.

To begin we will need to setup some data constructs (they are simplified to keep the code sections short, i.e. no extensive code to setup private members and public properties). A class MsgHeader will act as a common object sent ‘out-of-band’ on all calls to the service. The ‘out-of-band’ concept is used for the purpose of keeping data contracts free of plumbing-related operations. This example uses the header to supply something specific about the user, and the type of operation we are performing (enum OperationTypes). Another example of advanced control data you can pass via headers is a ‘message priority’ value.

[DataContract]
public class MsgHeader
{
    [DataMember]
    public int userRoleId;
    [DataMember]
    public int operationType;
}

A snippet of the data contract, and other setup data:

[DataContract]
public class Patient
{
   [DataMember]
   public int PatientId;
   //more members...
}

public enum OperationTypes { 
   Update = 0, Insert = 1, Delete = 2 }
public enum UserRoles { 
   Reception = 1, Nurse = 2, Doctor = 3 }

The Service is set up with a simple save/retrieve interface:

[ServiceContract]
interface IPatientContract
{
   [OperationContract]
   void SavePatient(Patient p);
   [OperationContract]
   Patient RetrievePatient(int patientId);
}

Via the Client proxy constructor we pass in our populated MsgHeader object. Then call any required methods on the proxy, in this case the save.

//class instance of patient details
public Patient CurrentPatient;

public void Save()
{
   MsgHeader header = new MsgHeader();
   //Set the properties accordingly ...

   //Setup and Call Proxy
   MedicalSystem proxy = new MedicalSystem(header);

   proxy.SavePatient(this.CurrentPatient);

   proxy.Close();
}

Finally on the service implementation to access the MsgHeader you simply use HeaderContext.Current.

class MedicalSystem : IPatientContract
{
   public void SavePatient(Patient p)
   {
      switch (HeaderContext.Current.dataOperation)
      {
         case (int)DataOperations.Delete:
            DeletePatient(p.id);
            break;
         case (int)DataOperations.Insert:
            CreateNewPatient(p);
            break;
         // ...
      }
   }
   // ...
}

There we have it – ‘out-of-band’ passing of data to your service to keep system entities free of the clutter of control data.

Putting all the basics together

Before we get stuck into a full scale example, I’ve prepared a simple checklist of tasks required to get a basic service up and running.

Refer to this checklist to get your first WCF solutions up and running quickly.

Service & (Host)

  1. Service Contract IMyServiceContract
  2. Implementation of that Contract
  3. Service Proxy Implementing ServiceContract & ClientBase<IMyServiceContract>
  4. Endpoints via App.Config
  5. Form/Console to kick off Host

Client

  1. Proxy implementing IMyServiceContract
  2. Endpoints to match those specified in service via App.config

If you haven’t done things right one of the very early exceptions you might receive is an InvalidOperationException and it will name the Contract that you haven’t implemented correctly. This is likely to be caused by a flaw in your app.config (server side).

A Common Exception from a Simple Misconfiguration

A Common Exception from a Simple Misconfiguration

Other exceptions to look out for are those when you attempt to call the service and either endpoints are misconfigured or implementations aren’t complete.

Being a Good Host

Hosting a WCF application is quite simple. There are 3 available hosting options IIS (Internet Information Services), WAS (Windows Activation Service) under Vista/Server 2008 and an ordinary (windows form/console) application.

The quickest to getting your project up and running is a windows form (or console) app. This way you can begin development quickly.

//program.cs
static class Program
{
   [STAThread]
   static void Main()
   {
      Application.Run(new FormWcfHost());
   }
}

Then for the windows form:

public partial class FormWcfHost : Form
{
   private ServiceHost<MyService> serviceManager 
      = new ServiceHost<MyService>();

   public FormWcfHost()
   {
      serviceManager.Open();
   }
		
   private void FormWcfHost_FormClosed(object sender, 
       FormClosedEventArgs e)
   {
      serviceManager.Close();
   }
}

It’s that simple.

The production level hosting decissions are just slightly more involved. Basically if you have an installation of Windows 2008 Server available then it’s ideal to make use of WAS to host it. Otherwise the choice can be argued between IIS6 hosting or just running it as a standard windows service on a Windows 2003 depolyment.

The advantages that WAS offers are; application pooling, isolation, identity management. So is the more scalable choice.