Queuing ajax calls to ASP.NET WebApi Controllers

Objective

To halt processing of subsequent ajax calls after one causes an error.

Why

Any actions related to a similar set of data (or concept) will all go through the same queue. This way if something falls over during processing of any given request, halting the processing of further requests should help not worsen things, or at least avoid subsequent errors that are a direct result of the first error.

When something unexpected has happened it’s likely the underlying system data is not in a valid state. Further actions may complicate things, possibly making matters worse. But more likely further actions will also error and we shouldn’t subject the system to having to handle them. So the solution is to get the user to reload the application into a known good state, i.e. complete fresh request of data.

There’s a competing concern that particular user actions may be desirable to have processed even after particular unknown/unexpected errors occur, but for now we’re going to assume these are rarer and can be dealt with specifically simply by having them bypass this queue (or make use of an alternate queue).

How

A toggled case
– After an error set a flag that will keep prompting the user to reload the page after an error.

A fixed case
– Do not process items already queued up in the processing queue.

Queue entry management

This part is simple. Most of the code is plumbing around building up the ajaxQueue call.

#CoffeeScript
class ActionCommandQueue

    validState: true
    
    #todo: surface the console log issues to the user

    sendData: (url, data, type, settings) =>
        settings = settings || {}
        settings.url = url
        settings.data = JSON.stringify(data)
        settings.type = type
        settings.contentType = 'application/json; charset=utf-8'
        settings.error = @onError
        if @validState
            $.ajaxQueue(settings)
        else
            console.log "You need to reload the page to continue. (click here)."

    onError: (xhr, status, error) =>
        @validState = false
        console.log "Sorry that action failed. Please reload the page and try again"

Usage

    #CoffeeScript

    sendData '/api/Tasks', { description: "new task" }

http://api.jquery.com/queue/

The queue iteself

This part is also simple, seeing how the hard work in creating a jQuery .ajaxQueue() method has already been done by gnarf, and here’s the blog post outlining the code. This came out of a great stackoverflow answer and this one.

Except one modification

The .then( next, next ) call was replaced with a failure case, in production code it’s likely to be a silent failure because there’s an error handling function higher up in the call chain that will report the problem to the user (no need for repeating the message).

    function doRequest( next ) {

        fail = function () { console.log('cannot continue to process queue after an error');

        jqXHR = $.ajax( ajaxOpts )
            .done( dfd.resolve )
            .fail( dfd.reject )
            .then( next, fail /*was next*/ );
    }

An added benefit here is not just around safety for the error case, but this queue approach ensures important user actions will get dispatched and arrive in a correct order.

Example

Scenario
1. Create a Task.
2. Update task details
3. Add Sub Task
4. Edit Sub Task [Fails]
5. Some other action (possibly related to the sub task)
6. 7. 8. 9. all same as 5.

At step 4, it fails, because something prevented the sub task being added. But before Task 4 failed, the user also issued action 5 very quickly. Step 5 is what the above code is preventing from running because we’re not sure what the impacts may be.

Before I release some Unit Tests to demonstrate this. I have a crude annotation on the Chrome Developer tools console output. The proof is lack of calls to ‘doneCb‘ after the error. Note the 500 error, then the 2 error messages that follow.

Ajax Queue Drop

Chrome debug output showing the error output.

Queues, Deferred and Promises

It’s beyond the scope of what I wanted to talk about here, but the jQuery.ajax wrapper jQuery.ajaxQueue makes use of Deferred, Deffered.then() and Promises, they’re worth looking into to gain a detailed understanding. Some reading about the queue itself.

Conclusion

So now a user can click to their hearts content on a variety of partially related actions on a page, having each fire off an ajax() request. If something goes wrong they can be alerted, and you can save a flood of subsequent errors happening.

Capturing client side JavaScript errors for later analysis

We’re getting close to pushing an application to a larger set of test users, and we’ll be interested in what happens when a larger variety of machine configurations (browsers and operating systems) and of course user actions encounter errors.

We started with simply catching any server side errors, and log them to a single database table (separate database to application).

To achieve this part of it, it is simple as having a very lightweight database connection mechanism to insert this log entry record, this way if there was an error with our applications standard database access mechanism we could also capture that.

In this fashion we can be reasonably sure that anything short of major production environment failure (network infrastructure / machine specific) that we’ll be able to capture errors while the system is being used. Those types of issues will be handled differently.

Then we realised it would be just as helpful to capture and store client side JavaScript errors in a similar fashion.

Capture

The JavaScript error handler looks like this, making use of window.error:

window.$debug.globalErrorHandler = function () {
  window.onerror = function (m, u, l) {
    
    $.ajax({
      url: '/Error/Occurred',
      type: 'POST',
      dataType: 'json',
      data: JSON.stringify({ errorMsg: m, url: u, line: l, uri: window.location.href }),
      contentType: 'application/json; charset=utf-8',
    });
    
    return true;
  };
};

An extra note here is I came across a JS library that offered a common ‘printStackTrace()’ method. Created by Eric Wendelingithub project. Which doesn’t work how I would have hoped it would have worked in our global error handler, but that’s the nature of the error handler event.

Never the less it does look quite helpful, to make use of it you need to have a specific try { } catch { } block around something that may fail. Checkout the readme on the github project page.

Back to the main focus of this post…

We can also decide to call $debug.globalErrorHandler() method in some central part of the web application, so that later it doesn’t have to be always turned on.

Receive

The MVC side of this is even simpler, since our users are logged in while using the application, we have ‘context’ information about them, so that’s one extra useful piece of information we can capture as part of the error.

public class ErrorController
{
  public void Occurred(ClientSideJavaScriptException error)
  {
    // this context in our application represents the user, so we know who experienced the error.
    error.User = _userContext.UserName;

    // we send this off elsewhere to be persisted, you could simply persist it here
    _service.Handle(error);
  }
}

The ClientSideJavaScriptException class simply has the properties required to send over the information from the ajax post.

public class ClientSideJavaScriptException
{
  public string ErrorMsg { get; set; }
  public string Url { get; set; }
  public string Line { get; set; }
}

Process

Finally the persistence logic here is via the Micro.ORM NPoco selected because Adam said it was good 😉 and the Nuget Package helped.

//setup
Db = new Database("configuration_key_name");

Db.Insert(new Details
{
  OccurredWhere = "client-side-js",
  ExceptionMessage = errorDetails.ErrorMsg,
  When = DateTime.Now,
  StackTrace = errorDetails.Stack,
  Method = "Line Number:" + errorDetails.Line
}

Unit Testing JavaScript methods that contain jQuery ajax calls

Objective

Unit test a JavaScript method which contains a $.ajax() call. Using QUnit.

Details

This was supposed to be a simple task, and if I didn’t have a few (now) obvious faults in my JavaScript code, it would have been completed more quickly. I have gone through several answering my own StackOverflow questions on the next day here’s the one in question for the curious ones.

I will hide behind the excuse of it being the early days of the new year and still finding my coding groove after almost two weeks on holiday…

So in this post I’ll summarise the two approaches I came out with in the end. It’s two because neither worked for me first time, and in my attempt to solve the first discovered the second. In the end resolving both. I won’t bother going into much detail about QUnit, as there are many posts out there about it; here, and here and here and official documentation here, and other interesting things to do with it.

As a quick aside we use the NuGet package NQUnit.NUnit to help us integrate QUnit into our Visual Studio projects.

Basic Approach

Solution 1 – basic way as is shown on this StackOverflow answer

// Arrange
    var options,
        jsonText = '{"id": "123", "fieldName": "fbs", "data": "fbs-text"}',

// Set up to 'capture' the ajax call (this forms the mock)
$.ajax = function (param) {
    options = param;
};

// Act - call the method which is 'under test'
/* ... */

// Call the success (or failure) method to complete the mock of handling of the 'response'
options.success(expectedData);

// Assert - verify state of system is as expected
/* ... */

Alternate Approach MockJax

Solution 2 – using the MockJax library with a great walk-through on how to use it here.

There are several advantages in using MockJax, that can be summarised as just having more control over mocking the ajax portion of the method under test, including but not limited to; timeouts, introducing latency, returning HTTP status codes.

After having included MockJax in your project, the solution 1 code gets replaced with 1 method call to $.mockjax() and looks like this.


// Arrange 
    var jsonText = '{"id": "123", "fieldName": "fbs", "data": "fbs-text"}';

// The call to mockjax replaces the need to overwrite the jQuery ajax() method
$.mockjax({
    url: '/Your/Regular/Action',
    dataType: 'json',
    contentType: 'application/json',
    responseText: jsonText
    }
});

// Act - call the method which is 'under test'
/* ... */

//perform test assert state of system is as expected
/* ... */

Demonstration

Full QUnit test code is up as a Gist.

Find a working copy of the code here in it’s simplest form: jsfiddle.net/NickJosevski/9ZZmc/4/

Tweaking a VS2010 plugin to run JSLint in the command line

We’ve gone to some lengths at work to automate, and have a continuous delivery style pipeline for all things build and deployment related. It’s well on its way, but not ‘perfect’ yet. Maybe perfection isn’t attainable, or maybe when you have a red button on the desk that when pushed does *everything*. None-the-less aiming for perfection will continue to drive us to improve.

So here I wanted to discuss what steps I took to get a nice JSLint Visual Studio plugin to form part of our build process. It was a bit of fussing about getting it to work, and it’s an example of still being imperfect, but for now it serves the build pipeline well enough.

If you don’t use JSLint for your JavaScript code, you probably should. It’s a static analysis tool to analyse and report on broken coding rules for JavaScript. Try it out on the creator (Douglas Crockford’s) site JSLint.com.

There is an easily accessible Visual Studio plugin called exactly what you would expect “JSLint for Visual Studio 2010” and here is the direct link for it on the VS Gallery for your installation pleasure.

If you do anything and stop reading here you’ve done well, install it and happy JavaScripting.

JSLint for Visual Studio 2010 Extension

But what about continuous integration?

For us it wasn’t enough, we wanted our build process which runs as psake scripts to fail if JSLint rules were broken.

Just for a bit more completeness on the psake digression and how the command line tool will execute under MSBuild, here’s a snippet of the ‘targets’ code that is referenced by the .csproj files that contain JavaScript. The input parameter on the executable being the directory containing the script files relative to .csproj file.

<Target Name="AfterBuild"> 
    <Message Text="Running jslint..." Importance="Normal" /> 
    <Exec Command="&quot;..\jslint\JSLint.CLI.exe&quot; .\Scripts\ " /> 
</Target>

So the search for command lines tools began. I found some existing command line tools, some which had more complex dependencies, others which seemed to be more ‘primitive’, i.e. did not report on errors the IDE based JSLint plugin was reporting.

There were 2 main objectives driving the choice of the tool

  1. Similarity and accuracy to JSLint.com and the Visual Studio extension
  2. Ease of setup

There was some discussion happening on StackOverflow, here and here but nothing I tried digging deeper into seemed suitable.

I then had an idea…

Take the core logic of the Visual Studio extension and wrap it in a very simple console application to execute as part of the build process.

With the approach I took, it seems even option 2 was difficult to achieve (consumed some time), but at least it had the least external dependencies to the other options out there.

The very first step was to obtain and install the VS2010 SDK, as this was an project which referenced many interop assemblies for interacting with the IDE. It needs to be the Service Pack 1 SDK in fact, and here’s a direct link. Once I was able to compile the extension it was then a matter of understanding how it operates and how to access a method to perform the ‘linting‘.

There were 2 major hacks to get access to some inner workings of the extension to operate:

  1. Making some ‘protected/internal’ methods and properties public
  2. Modifying where the JSLint logic obtained the settings file from (JSLintOptions.xml).

Locating JSLintOptions.xml proved somewhat difficult at first, as it was tucked away hiding in the Roaming section of my user folder on Windows (\Users\*\AppData\Roaming\). These hacks could greatly benefit from some re-factoring effort if I have ever have the time, or someone else is so inclined. But after an initial attempt to refactor out the most core of logic things fell apart in the land of SDK dependencies, so I rolled back and opted for a less elegant approach which was the 2 hacks listed above.

The logic for the console application is then trivially simple:

  1. Read .js files.
  2. Using the JSLinter class method Lint() supply the .js file content/
  3. Write errors to console
  4. Return error code, 0 (Success), 1 (JavaScript warnings), etc.

Show me the code!

If you want to see the hacks I had to make to get this to work head on over to this git repository on BitBucket. I offer no warranty it may be fragile so manipulate with caution.

If you just want the command line executable, then it’s built and stored in the repository in the /output folder, if I make any updates to the executable, I’ll also update that executable.

Areas for improvement in the console app:

  • Taking path locations as input parameters/external file of locations
  • Taking alternate settings files for rules to ignore/include
  • General tidyup

Outstanding issue

The final hiccup is not directly related to the use of the command line tool itself, but building the command line tool from source as part of a more comprehensive-all-inclusive build process. Adding commands in the .csproj file post-build settings was not sufficient. The build and copy of the executable needs run as a directive in the targets ‘afterbuild’ section of the .csproj file.

CSProject Settings Dialog - Post Build Events

Not suitable to place actions in 'post build' section

This led to a conflict between MSBuild and the VS2010 SDK, which was very frustrating and currently isn’t solved yet. The question is up on StackOverflow.

AutoSave Form Fields using jQuery – .change(), focusout() and the deferred .when() functions

I have been prototyping some general approaches, some new functionality, and other various concepts, all part of how a larger system will work. After seeing some interesting and nostalgic things being done with jQuery, it gave me an idea for an approach that may be suitable to achieve some delayed asynchronous actions. Thanks to Aaron Powell for those neat walk-throughs.

My Idea is probably even simpler but focussed more around having elements of a form “submit themselves” in a way making use of new features of jQuery 1.5. I have not yet completed the polished version of this functionality, where it will handle errors and other complexities to be a robust solution. I do believe this approach does align with the progressive enhancement objectives of good websites, if a clients browser does not support this functionality there is still a standard submission of the form data available. But none-the-less would like to share/document my approach and direction.

Basic Flow

  1. User types-in/modifies an input field.
  2. jQuery code detects this.
  3. Asynchronously sends this piece of data.
  4. Provide elegant feedback when this is complete.

Improved flow (a future exercise)

  • Step 2 above, should count down and reset counting if that same field is changed before the data is sent. Still a “to do” item.
  • Step 4 isn’t exactly elegant at the moment; a confirmation message or icon shows up, which is rather crude.

Demo

Again thanks to Aaron for reminding me of the awesome site: jsFiddle, I used it to speed up my jQuery experiments without relying on Visual Studio and running an MVC application to create and test the jQuery logic.

$(':input').change(function() {
    var c = $(this);
    $.when(
    c.focusout()).then(function() {
        $('#last-action').append(c.attr('name') + ' = ' + c.val() + '<br />');
    });
});

Here is the jsFiddle demo of this functionality, simply type in any of the input boxes, and tab away and see the results be “submitted”. Of course no submit code is firing yet, we’ll look at that now.

Let’s refactor a little to support some more important functionality to get ready to submit our form. The line in particular about just appending the data can be replaced with a function to handle submitting the form, and just for the demo it will also still output the data in the little debug div. This is where I make use of Aaron’s demonstration of the recursive setTimeout pattern.

$(':input').change(function() {
    var c = $(this);
    $.when(
    c.focusout()).then(function() {
        startCountDownAndSendData(c, 5) //delay by 5 seconds
    });
});

With this function, we’re going to supply the html element (c) and an integer (waitDuration) representing how many seconds to wait before “submitting”.

function startCountDownAndSendData(c, waitDuration) {
    function go(){
        if(waitDuration <= 0) //count down from a supplied duration value (in seconds)
        {
            $('#last-action').append(c.attr('name') + ' = ' + c.val() + '<br />');
            $('form#MyFormId').trigger('submit');
            return;
        }
        waitDuration--;
        setTimeout(go, 1000);
    }
    go();
}

You can see this refactoring in operation here, but it won’t submit anywhere, that’s what we will handle next time in a post about Asynchronous MVC controllers. So steps from 3 and 4 are still to demonstrate to you dear reader, stay tuned… part two is here.

Using jQuery Templates to .appendTo() an <option> on a <select> list via .render()

I couldn’t find a good example that shows how to achieve this, so I thought I would quickly outline the process along with some “gotchas” I encountered.

First, an outline of the technology/components/elements I’m talking about:

  • Templates is the Microsoft proposed templating solution for jQuery.
  • This library can be obtained from github.
  • I want to add <options> on the <select> list.
  • I want to achieve this using the .render() method demonstrated on ScottGu‘s blog post on jQuery Templates.
  • In conjunction with using the .appendTo() method in jQuery.

The desired end result, outputting items into the select list via jQuery:
Select List

There’s actually several similar ways this can be achieved as can be seen on this StackOverflow question. I’ve put a summary of this post as an answer on that question also.

To add scope to this post, here is a quote from Scott Gu’s blog post on the purpose of the templating system.

Client-side templates enable jQuery developers to easily generate and render HTML UI on the client. Templates support a simple syntax that enables either developers or designers to declaratively specify the HTML they want to generate. Developers can then programmatically invoke the templates on the client, and pass JavaScript objects to them to make the content rendered completely data driven. These JavaScript objects can optionally be based on data retrieved from a server.

At this point I’m going forward on the assumption you now have a rough understanding of the purpose of templating so I can focus primarily on getting this to work with a <select> list.

The first step is to setup the appropriate template structure. In this scenario, it’s the <option> block with the ‘value’ and ‘text’ properties needing substitution. The syntax for the substitution variables is wrapped by the two opening curly-braces and an equal sign, it’s name then followed by 2 closing curly-braces. A note here as this is JavaScript performing the substitutions, it is in fact case sensitive.

<script id="templateOptionItem" type="text/html">
    <option value=\'{{= Value}}\'>{{= Text}}</option>
</script>

Zoom in, enhance!

Ensure you escape single quotes

Ensure you escape single quotes otherwise the templating render() action will fail.

The next step is to “fetch” your data (input data), in my real world example this comes back as JSON. But for now let’s just hard code the data array.

    function renderTest() {

        var someData = [
            { Text: "one", Value: "1" },
            { Text: "two", Value: "2" },
            { Text: "three", Value: "3"}];

        //on the template, apply the render with the data input, and then append to the html element.
        $("#templateOptionItem").render(someData).appendTo("#theSelectList");
    }

That last line with the .render() and .appendTo() is where the elegance lies, the set-up of reusable templates is extracted out of the actual call to map the data to the html elements. There are more features in the templating library, so check out this post for more examples.

The aim is to keep this post on the very narrow topic of just applying to a select list. So the actual implementation of the ajax-y request and response of JSON will come in a future post. The impact on the actual templating logic is trivial and limited to the what you pass to the .render() call.

Summary of the gothcas:

  • Escape your quotes (single and doubles).
  • The template must be in it’s own script block, not as a JavaScript var (string). Not sure why that is, I’m still investigating.
  • Ensure your selectors are working.
  • Ensure your output is wrapped in valid html elements, at the least a <div>.
  • Template place holders are case-sensitive.