AspComet - High Level Architecture Overview

I have received a few requests to give one of these, and I promised over a year ago that I would do! Being a typical developer I don't like writing documentation, but I think this will help with anyone doing any work with AspComet to understand what's going on under the hood.

I assume you have basic knowledge of the Bayeux protocol before reading this document. If not, then give it a read as it will help to understand what I’m talking about!

Bayeux “Clients”

When a browser first connects to AspComet it will enter a series of negotiation requests. The server is at this point creating a Client object to represent that browser and giving it a unique ID, which is communicated back to the browser and understood and stored by the Bayeux JavaScript implementation. This client ID is used in all subsequent requests between browser and server to marry up the requests.

On the server side, the client class stores, amongst other things, the list of channels to which the browser has subscribed, such that it can send the appropriate messages to that browser when required.

Async Http Handlers

As I mentioned in a previous blog post, we implement async http handler which is provided by the ASP.NET pipeline. These handlers are designed for sites which perform long running operations and wish to avoid starving the ASP.NET thread pool during these operations (by returning threads back to ASP.NET during the time that said operations are being made). AspComet piggybacks off of this and uses it in a slightly different way as probably intended.

When a request comes into the browser, it’s picked up by the BeginProcessRequest method where we create an AsyncResult to return back to ASP.NET. At this point, the thread will be returned to the request pool.

Two things can now happen here – if there is data already there to be sent to the browser (or the incoming messages generated responses which need to be sent straight back) then the AsyncResult will be completed immediately with those messages (raising the callback which ASP.NET gave us to call when we’re ready). Otherwise the AsyncResult gets stored along with the client and will be used at the point a message is ready to be sent.

To illustrate this with an example, client A wants to send a message to client B. Client B connects, and has no pending messages, so his AsyncResult is stored inside his Client instance. At this point client A’s browser sits there waiting for a response to come back. Client A sends a publish message which needs to be sent to B, at that point from within Client A’s handler thread Client B’s callback is called, sending the message straight to Client B’s browser.

Message Handlers

Within Bayeux there are a number of predefined channels which are used for passing specific requests to the server. These are called Meta channels. Within AspComet there is a Message Handler for each one.

The HttpHandler receives an array of messages from the browser and passes them all to the MessageBus. This splits up the array and passes each message to the appropriate handler in turn. Once all messages have been handled, it’s up to the message bus to decide whether to complete the AsyncResult immediately or store it in the client to be called at a later date. I won’t go into detail about each MessageHandler is doing, as it’s basically here that the Bayeux protocol implementation comes into play – the protocol is essentially providing a spec for what each of the handlers should be doing.

Application Interfaces

The primary way for applications to interface with AspComet is via Events and the EventHub. Events are raised by the message handlers for each type of message being handled, which go through the central EventHub static class. This enables pretty much any scenario you can think of!

Applications will typically want to be notified when messages are published to specific channels, when new clients connect and when clients subscribe and unsubscribe to channels. This is all supported via Events. Some events also support cancellation, so an application could subscrivbe to those and set Cancel=True, for example to cancel subscription or block certain messages reaching channels.

For complete control, it’s also possible to use dependency injection to inject alternate implementations of some of the types used by AspComet.

The best place to learn is to look at the Chat sample which comes bundled with AspComet. That shows how to use the Dojo and JQuery JavaScript Comet frameworks to interface with AspComet, and has some good (but contrived) examples of subscribing to events and injecting alternate dependencies.

AspComet now on Github

Just a quick announcement here. I have decided to move AspComet from Google Code to Github. You can get it here: http://github.com/nmosafi/aspcomet

I've been tinkering with Git over the last few weeks trying to get to grips with it and I can already see how extremely powerful it is.

Github is a very exciting to me in the way it forms a kind of social network around open source projects and allows people to collaborate and share in such a seamless and well presented way. It's never been particularly transparent when you put an open source some project on Google Code what people are doing with it - with aspComet on Github it's already been forked and I only put it up 24 hours ago - impressive!

There's also a little README file up there I added (with Github you just check a README file into your repo and it appears as the synopsis on the project homepage - simple!) and there's a good bit of information in there about getting started with the framework.

If you've any questions on AspComet let me know, otherwise happy forking! :-)

BDD Everywhere - don't underestimate your target audience

I have recently moved from doing TDD to BDD, using Scott's excellent SpecUnit.Net with, I have to say, some real success.

BDD was the subject matter being discussed at May's London Altnet beers. After the talk I was discussing with some people about their approach to doing BDD. What was being presented was the idea that BDD specs are useful only at the outer layers of the application and once you get more low level, you should revert to normal style TDD tests.

I don't agree. I have been writing BDD specs for every single class I write, and have recently upgraded our build system (rock on TeamCity) to make the spec reports. Saying BDD is just for the outer layers is ignoring the fact that the users of application know a lot more about their business than you do.

I'll give an example. We are building a system for the traders to view their trades - in what is known in the financial world as a "blotter". Being a WPF application, we build the ViewModels first and then build some sort of data access component. So the requirement is that all the trades made today should be displayed on the blotter when the user refreshes it. Here is a set of specs which say that

when the user refreshes the blotter and no trades have been made today
- should display a message saying there are no trades
- refresh button should be enabled

when the user refreshes the blotter and two trades have been made today
- should be two trades displayed in the blotter
- should display the earlier trade on the first row
- should display the later trade on the second row

So now we write some tests for that, and flesh out our implementation. During this time we create an interface ITradeProvider with a method GetTradesForToday(), and use a mocking framework to stub out the calls to that method using our favourite mocking framework.

Our user/customer/business analyst would definitely find those specs useful, and may even give some feedback or changes, which is great. But what about the TradeProvider implementation? Is that worth doing it for? Some would say no, but I would say. We could start off with something like this:

when loading todays trades
- should load todays trades
- should not load trades from a previous date

That's not really saying anything useful is it. Or is it? Imagine you show this to your users and/or business analysts. They're gonna ask:

"What on earth does that mean? When loading today's trades, should load today's trades? It doesn't make sense! We're a bank, we have loads of trade systems, which one are you getting the trades from?"

"Oh right, well I was going to get them from the MYZ team's Oracle database which we got the trades from on the last project I worked on"

"Ok, that's fine, but that's only going to work for trades outside of Asia. For the trades on the Asian stockmarkets, we're going to have to get the data from the AYN team located in Singapore. They have a web interface for looking at the trades and I think they provide some sort of service for you to retrieve it from"

"Oh OK, thanks! I'll update the specs and get back to you to review them again!"

You may think this is a fairly contrived example, but it's the sort of thing you will come up with time and time again. As developers, we move from project to project all the time, and we often make assumptions based on previous projects. Users, on the other hand, are getting you to build a system to provide value to their business. They are likely to know it a lot better than you! Business Analysts especially have a very detailed knowledge of all the internal systems and what their responsibilities are (though they may not have the kind of deep technical knowledge you have of communications protocols, etc).

So think twice about abandoning BDD for the lower layers of the system. Write the specs, share them, invite everyone you possibly can to review them. The earlier problems are found in a system, the cheaper they are to fix. That's been proven through countless studies.

Evidence Based Scheduling

A co-worker just sent me this article from Joel Spolsky: http://www.joelonsoftware.com/items/2007/10/26.html.

As I'd expect from Joel, it's a very well written article, and definitely worth a read. Has anyone got any experience doing this? I've always thought that tracking individual velocity was a bit fascist, but perhaps it could prove useful?

Also, has anyone tried running a project using FogBugz. Is it any good?

Extension methods make your code more readable

There has been a long standing argument for and against using extension methods. Recently I am on the quest to write code which is as readable as possible. That really is my prime aim - I want someone who is simply scanning my code to be able to, at a high level, understand exactly what is going on in that function. All other aims come second to that (e.g. until a requirement comes along which says otherwise).

Well, six months ago, I would never have made this statement - "Extension methods are a massive help here!". Honestly, they just are. I know people hate them because you can't really tell where the method comes from, and it looks wierd, and it breaks OO because you should inherit to add functionality (well actually, I disagree with that statement being made blindly)...

So I have a function which takes in some dates and returns a status depending on how the dates compare. Here is a simplified version of the function:

public Status GetStatus(DateTime now, DateTime modifiedOn, DateTime createdOn)
{
  if (modifiedOn.Date == now.Date)
    return Status.Amended;
  else if (createdOn.Date == now.Date)
    return Status.New;
  else
    return Status.Unchanged;
}

So what's wrong with that? Well you have no idea what the if statement is actually doing! Tell me, if you scan that quickly, can you tell what the business rule is here? I am pretty sure most people will have to scan that a few times to get what is going on there. I considered adding a comment, but comments are a definite code smell (see DRY) so this is how I changed it (using an extension method)

public Status GetStatus(DateTime now, DateTime modifiedOn, DateTime createdOn)
{
  if (modifiedOn.SameDayAs(now))
    return Status.Amended;
  else if (createdOn.SameDayAs(now))
    return Status.New;
  else
    return Status.Unchanged;
}

Simple but effective methinks. I think it's definitely clearer (though could perhaps be made even better). Do you agree?

A shout out to Benjamin Mitchell

Benjamin has recently started blogging (and about time too!). I have worked as a developer on one of his teams before, and he is a good guy to work for and really knows his stuff. He's one of those PMs who is always looking to improve his team's knowledge, skills and processes.

I hope to see postings about Agile techniques and Lean Development, and hopefully some good anecdotes too!

I am on Twitter

Well, after much nagging by my good friend Mr Norton, followed by a number of people at the AltNet Beers the other night ("what, you aren't on Twitter mate???"), I have finally succumbed and signed up. You can find me at http://twitter.com/nmosafi. I hope to be posting about all my fun and frustrations at work, and hopefully will avoid doing so whilst completely hammered!