API call based Azure Functions with DocumentDB

Azure Functions caught my eye recently, mainly because of the F# support where I was hoping to write some F# based integrations. But the F# support is classified as experimental, so while I learn the capabilities of Azure Functions, I didn’t want to get held up on issues that may be only specific to their support of F#. Once I get my core objective operational via C#, I’ll attempt to rewrite the functions later, and I’ll share that too (in fact I’ll likely just update this post with re-written at some point).

I’m taking my simplest application and migrating that to make use of Azure Functions, it’s the trusty Used Guid service. If you haven’t heard of it… you’re missing out!

Playing with AppHarbor twitter and WebAPI

untitled_clipping_091816_103748_am

Background

So in the original WebAPI app, it was straight forward enough process:

The dependencies are:

  • DataBase (Read + Write)
  • Twitter (Write)

Function Based Architecture

The initial challenge here with the Azure Function approach is off the back of the user request, how to do the Guid lookup. The integrations offered by Azure Functions are designed in a way that it’s INPUT + OUTPUT(S).The first function has to be the input from the user and that’s an HTTP call.

I started thinking about the coordination between multiple azure functions early in this process. I thought the second function could just operate on the back of a new document showing up in the Azure DocumentDB. But upon digging into the documents I could not see any examples of how to “subscribe” to the feed of new documents. After not being able to solve it via experimentation it started to look unsupported. So I went to Stack Overflow to get confirmation (or what I was really hoping to hear: “yes, this is coming soon”.)

That was not the case. The answer now (Sept 2016) is NO: not supported. So the list of supported bindinds in the documentation was accurate and up to date:untitled_clipping_091716_100900_pm

So now the first function that takes the http INPUT, needs to have 2 OUTPUTs; DocumentDB and Queue.

Function 1

Input – HTTP

It’s going against the ease of use of the Azure Functions to do a database read and return the failure cause, though I may not have a choice but to do that.

I wrestled with the architectural approach to this problem, and it’s because this problem space is absolutely contrived and doesn’t lend itself to an elegant solution. When you actually step back at the problem domain / business’s requirement 2 users really won’t have colliding data (… the Guid). They would just ask a service to deliver them the next datam of vulue.

So with that I’ll continue on following the happy path, because the core objectives are to get to deployment concerns around functions, and just lay some groundwork here.

Outputs – DocumentDB + Queue

They simplest way to write a DocumentDB document from your Azure Function is to have it as out out object. Now in many cases you only want to write the document if you pass initial validation. It seems valid to just assign null to the out paramters you don’t want to pass data to on the invalid/error cases. To feed data to the subsequent functions, a second out paramter is needed which is the queue.

Code:

Function 2

Output – Tweet

I thought this was going to be the simpler of the 2; I wanted to look at how to get secrets (API keys, OAUTH etc) into the functions. But when I went to write the function, oh that’s right I don’t a 1 step approach to fetch NuGet packages. So making use of TweetSharp to do the authenticated twitter API call, will take a bit of extra time too.

Well I started digging around that code, and to extract out exactly what I need is taking a while. Below is a link to the original code in the WebAPI app, where making use of the library makes producing a tweet quite easy (once configured with authentication).

So the options I’ll investigate later will be:

  1. The minimum set of code that can do the authentication and post the tweet so it’s all embedded in the 1 function.
  2. Making use of Azure Logic Apps (which I need to investigate more), but look to offer some abstraction around common integrations.

Original C# code in WebAPI app:

https://github.com/NickJosevski/usedguids/blob/master/UsedGuidTwitter/Logic/Tweeting.cs

Azure Function:

For now just proving can read off the queue. The basic set up is the data on the queue being a string.

With the integration panel looking like this:

untitled_clipping_091816_103117_am

Summary

What’s working:

  1. API endpoint go get user requests in
  2. Writing to DocumentDB
  3. Writing to a Queue
  4. A second function reads from that Queue

Initial Frustrations

The Azure portal is quite nice, the effects, the theming, it does look nicer than the AWS console which I’m much more familiar with. But deep linking into Azure functions doesn’t work as expected, say you duplicate a tab, you either end up back at the dashboard level, or on the create new function screen. Sometimes it would just spin/hang for a while.

untitled_clipping_091816_102055_am

Pricing

I wanted to add a quick note on pricing, I’m no expert in this yet, but when I first start playing with the functions, I had a dedicated app instance, that was draining my balance, when I realised I switched to dynamic pricing model which I thought would have been the default.

It’s good to track the cost of running features, especially while trying out new ones, but one thing that kept showing up in the notifications (bell area) was my current balance, it would always try to get my attention, but more often than not the outstanding balance did not chang.

untitled_clipping_091816_101845_am

What’s Next?

In the coming posts I’ll be covering the deployment pipeline for these functions, stay tuned.

Advertisements

JIRA Cookie-based auth API calls in F# with RestSharp

Today I was trying to create a quick integration with a bug tracking tool, as a little spike, unfortunately as is often the case, out of date documentation, vague errors, etc held up the task. None-the-less I got something working with cookie based authentication (I’ll be switching to OAuth based soon I’m sure that’ll go just as smooth). I’ll also submit a report about the problems with the documentation I discovered.

The rest of the documentation seems ok, but only time will tell as I get further with it, but from my experience it’s usually the initial steps that result in the most frustrations, as you get reminded not to trust the documentation…

Following along with this guide – JIRA REST API Example – Cookie-based Authentication.

What’s not clear in the documentation:

  • Even though you sign in with your email address, you really need to use your username (which is different), at least is for the `admin` account.
  • Error 1: there is a leading `/jira/` element in the auth API route, this may be old or may be specific to self hosting, it’s not part of the URI for on-demand, it should be `http://jira.example.com:8090/rest/auth/1/session`
  • Error 2: (the big one) what is returned is a different `session` object and it not only contains the required `JSESSIONID` but also another key,value pair you need to have in your cookie when you make a subsequent request: `studio.crowd.tokenkey`.

The core objective was to get specific events from 1 system to be reflected in new or existing locations in a second system. In this case the second system is JIRA the bug tracker.

Complete Solution

The break down will follow below.

Sorry this code is a bit awkward because RestSharp is written for fluent C# style usage, I’ll be looking for an F# focused rest client, and I only chose ReshSharp because I had used it ages ago.

Walk Through

Some types to send along, notice the casing, that’s because I don’t have any JSON serialization code wired up yet to do the case change from .NET uppercase convention to lowercase JSON style.

type PostData = {
    body: string
}

type Login = {
    username : string
    password : string
}

Using these records, against your on-demand account make the auth request, and inspect what you get back (see Error 2) and you’ll discover more cookie details come back than documented. As an extra note as to why it was even more frustrating if you capture and review the calls from the web UI those calls supply even more cookie details such as: `ondemand.autologin`, `xsrf.token` and others.

let uname = "admin"
let pw = "your-password"

let restClient = 
    RestSharp.RestClient("https://your-account.atlassian.net")

let authReq = 
    RestSharp
        .RestRequest("/rest/auth/1/session")
        .AddJsonBody({ username = uname; password = pw })

Issue that auth request and now you’ll have the session cookie values you’ll need.

let authResponse = restClient.Post authReq
let cookiesToAdd = 
    authResponse.Cookies 
    |> Seq.map (fun x -> (x.Name, x.Value))

In this case I’m updating an existing comment to which I know the identifier.

let addCommentReq = 
    RestSharp
        .RestRequest("/rest/api/2/issue/Issue-Id/comment")
        .AddJsonBody({ body = "a new comment" })

for (name, value) in cookiesToAdd do
    addCommentReq.AddCookie(name, value) |> ignore

Finally issue that add comment request, and the status code should be ‘Created’, I saw ‘Unauthorized’ responses for far to long.


let commentResp = restClient.Post addCommentReq

Lastly another fun hiccup was discovering that curl on Windows doesn’t support https “Protocol https not supported or disabled in libcurl”, but that was the least of my problems.

IIS, Visual Studio, unable to start debugging on the web server.

Background

So this is another logging this issue after some frustrating problem sovling process. This is to help future me when I forget the same windows configuration step in another 8-15 months, maybe it’ll help you too.

On Windows 8 you have go in and turn on many individual items to just get an ASP.NET IIS hosted web app running, I’ve a variation of this problem in the past and blogged about it.

This time round it’s basically the same problem, but with Visual Studio upon trying to spin up and attach to an IIS hosted web project kicks up 1 of 3 errors.

Issue

The most common one was:

Unable to start debugging on the web server. The debugger cannot connect to the remote computer. The debugger was unable to resolve the specified computer name.

Which is not helpful, and the posts I found were people trying to remote debug machines or some thing else not helpful.

Unable to start debugging on the web server

Other variations included:

… error occured on a send

Unable to start debugging on the web server.

If you dig further into the windows event store you’ll be sent down an even further wrong path with errors such as:

The program can’t start because SecRuntime.dll is missing from your computer. Try reinstalling the program to fix this problem.

Solution

Ensure that you turn on ‘Internet Information Services Hostable Web Core’ along with all the other .NET / IIS feautes you need that I mentioned in the previous post.

Windows Features

Thinking in a document centric world with RavenDB @ ALT.NET

Last night (25th Feb 2014), I presented on RavenDB at ALT.NET Melbourne.

I got some great feedback from the audience and was happy to share my experience so far with RavenDB. If you were there / watch the recording and have some suggestions good or bad would love to hear them so I can improve.

Here’s the ALT.NET recording with slides, plus me up at the projector screen.

If you just want the slides and audio then here’s an alternate recording.

I’ve also put the slides up on slide share.

F# Newbie

On the 5th of June* at work we decided to give F# a go and by ‘go’ I/we mean really use it in the most important parts of our application. Some of the team has had greater exposure to functional languages like Haskell and Scala, they are the ones being our champions and guiding the rest.

*I know the exact date because it is marked by a small commit of a blank F# project added to the solution I went back and checked.

This was not a snap or rushed decission but one with a fair few chats over code examples and a proof of concept.

If I have to boil down the decission to go ahead with F# at this early stage it is:

  • Record Types
  • Patern Matching
  • Immutability

These language features combined with team motivation to be working with something new and interesting “sold” the rest of us who haven’t didn’t have the exposure yet.

At this point I want to confess that it’s a strange feeling to be working with F# code and not having the same level of confidence to make changes and jump in and write like I would with my near decade of experience in C#. Many would argue that that’s actually quite a good thing, and I’m starting to agree – for one thing it means I’m thinking longer about the code.

It’s very easy to just write some procedural logic, call some methods, change variables, and just keep writing until it works and think about tidying it up after the fact, or worse leaving a “//TODO: clean this up” comment for a future self or peer to rage over.

Take the preceding paragraphs and the title of the blog post as my disclaimer that this is all new to me but here goes… My take on the concepts I understand and apprecaite about F# so far are simple and based on and how it’s helping me and the team so far to write code more easily.

Record Types

Concise / less code

In our CQRS style application we have lots of little commands (that’s what the C stands for), and lots of events so being able to very succinctly create these is a huge win.

More info here.

Pattern Matching

Concise / less code

In particular there must always be some branch that matches and if there isn’t you get a compile time error.

incomplete pattern match

Go here to learn all about it.

Immutability

Concise

Of the 3 things I this is the most fundamental F# concept that I can appreciate so far, at a mimimum it’s about avoiding side effects in your code, and makes you think about transforming data as it flows through your application. In our CQRS world this is a great match, a command gets populated, it shouldn’t change. An event gets produced, that’s even more definate it happened – here’s what happend it doesn’t change.

Just like all the links about F# a great source is fsharpforfunandprofit.com

Did you notice the trend for each of the 3 items I chose to cover, that’s right it’s about being concise. The immutabilty may not always be less code but you absoultely do get conciseness. If you take 1 thing away from this is that how could less code possibly be bad, less to write, less to mainting, win, win.

That’s it for now, we haven’t had to give up and re-write all this code in C#.