Modelling accounts, event driven

I’d like to share results of my research on modelling an account. It is based mainly on this discussion on CQRS mailing list and specifically by Greg’s example. I’ll start with a slightly naïve model.

Account as an aggregate

Let’s model Account as an aggregate. What interface would an Account need? How about Debit and Credit? Here’s a sketch of how a command handler would look like for a TransferFunds command


var uow = new UnitOfWork();
var from = accountRepository.GetAccountForNumber(fromNumber);
var to = accountRepository.GetAccountForNumber(toNumber);

from.Debit(amount);
to.Credit(amount);

uow.Complete();

If you don’t like this approach you are right. The problem with this model is it requires two aggregates to be modified in one transactions. This violates the good practice of treating aggregate boundaries as transaction and consistency boundaries. Also, many event stores would not allow even allow you to create such transaction in the fist place. So, what can we improve?

Eventual consistency

We can take advantage of the idea of eventual consistency. Let’s refactor the command handler code:

var from = accountRepository.getAccountForNumber(fromNumber);
from.TransferTo(amount, toAccount);
accountRepository.PersistChanges(from);

The effect of this code is a AccountDebited event published by source account aggregate. This event then needs to be dispatched to a handler that would look like this:

var to = accountRepository.GetAccountForNumber(evnt.DestinationAccount);
to.ReceiveFunds(evnt.Amount);
accountRepository.PersistChanges(to);

Can you spot a problem here? While eventual consistency is generally a great tool, as all great tools it can be easily misused. The code above ensures that eventually, given infinite time, the transaction will be balanced. Can we wait so long? Usually it will happen very quickly but sometimes, under heavy load, it can take a while. Can we tolerate this? The answer, as always, is, it depends. We need to go back to the business problem we are solving and ask ourselves a question if we are not violating aggregate’s invariants by introducing eventual consistency.

In our example the invariant states that at any point in time credit and debit operations must balance (sum to zero). Clearly, after the command is processed but before the event processing happens, this rule is violated. What can we do with it?

Event driven modelling

Instead of trying to identify aggregates, let’s start with commands and events. Here’s what happens in the system when we want to transfer money:

The domain expert tells us that the rule we must comply with states that AccountDebited and AccountCredited events always happen together and they always balance to zero. What aggregate can we create here so that the rule is honoured? Clearly not Account. The aggregate we are looking for is Transaction. The interaction can be pictured as

The new command handler code would look like this:

var tx = new Transaction();
tx.Post(amount, fromAccount, toAccount);
transactionRepository.Store(tx);

and here’s the brand new Transaction aggregate:

public void Post(decimal amount, string fromAccount, string toAccount)
{
   this.Apply(new AccountDebited(amount, fromAccount));
   this.Apply(new AccountCredited(amount, toAccount));
}

Now the following question arises: if account is not an aggregate, what is it? Clearly, end users want to see accounts, balances and history records. The answer is provided by CQRS approach. Accounts exist only on the read side of the system. There should probably be a couple of them, one for each screen.

Takeaways

The takeaway from this a bit lengthy blog post is, if you are using Event Sourcing approach you should not start by identifying the aggregates by rather by identifying sequences of commands and events. Only then you can ask business expert what are the consistency boundaries in particular sequence. The result will be the definition of an aggregate.

VN:F [1.9.22_1171]
Rating: 5.0/5 (4 votes cast)

Greg Young’s Advanced Class

Last week, from Wednesday to Friday I attended Greg Young’s new advanced class. The event took place in Pod Wawelem hotel, 50 meters away from the Castle. I haven’t been to Greg’s previous training but I heard it was about Domain-Driven Design and CQRS. I was expecting something similar, but more in-depth. It turned out I was wrong but for sure not disappointed. The new content is great! Here’s a brief summary.

Day 1 – Alarm clock

We spend almost whole day discussing one topic — time. We learnt how hard it is to unit test time-dependent logic. Those who tried putting Thread.Sleep or ManualResetEvent in a test know how painful it is. Then we discovered that every time-dependant piece of logic (e.g. if cook does not respond in 5 minutes then…) can be transformed into order-of-messages problem. Instead of waiting for some period of time, we send a message to our future selves that will inform us we should take some action. I don’t have to convince you that sending and receiving messages is super easy to test, right?

To prove it, we spent some time doing one of well-known katas. I won’t tell which one to not spoil a surprise.

The last assignment of the day was a modelling task. We were modelling a business process of a restaurant. The funny/new/inspiring part of this assignment was that we were doing it by trying to figure out how would a Word template for document describing an order look like and how it would be passed between various people in a restaurant. We analysed what parts of the document are filled by each person and in what order.

Day 2 – Restaurant

We spent whole day implementing the solution for the restaurant according to the model we created a day before. We went through quite a few patterns from Enterprise Integration Patterns:

  • document message – that’s how we represented our order
  • adapter – waiter
  • enricher – cook and assistant manager, adding, respecively, preparation time and prices
  • competing consumer – when we have more then one cook
  • message dispatcher – a better (does not require locks) solution for load balancing cooks
  • message broker – centralize setup of queues

Day 3- More restaurant & Event Store

Before lunch we managed to cover some more advanced messaging patterns, namely:

  • correlation id – how we trace orders
  • routing slip – move responsibility for order flow from broker to a dedicated entity to add flexibility (e.g. normal flow & dodgy customer flow)
  • process manager – when routing slip is not enough (e.g. smart failure handling)
  • publish-subscribe – how to distribute information about changes in order state
  • event message – as above
  • wire tap – centralize diagnostics

Quite impressive, isn’t it?  After lunch we started playing with the Event Store. We managed to implement a simple chat application using Event Store’s ATOM APIs. Cool, but the rest was even better. We learnt how all the concepts we learnt during last two days can be easily implemented inside the Event Store using Projections. This stuff is really amazing and I say it despite my antipathy for JavaScript.

Summary

The cost of the training for early birds was just 300 euros. In exchange we got

  • three days of super high quality workshops with over 60 percent of coding time
  • gorgeus view of Wawel Castle during coffee breaks
  • three course lunches every day
  • after-party & beer

If this was not enough, Greg managed to give away three free tickets for CS students. Kudos man, you rock!

VN:F [1.9.22_1171]
Rating: 5.0/5 (6 votes cast)

Mechanics of durable subscription

<Announcement>

There are still three spots left on Greg Young’s Advanced Class in Krakow (February 6th-8th). Go register here before they are gone.

</Announcement>

Today I’d like to talk a bit about the mechanics of a robust durable subscription system. First, what is a durable subscription? Here’s a description from Enterprise Integration Patterns (I really recommend this book):

Use a Durable Subscriber to make the messaging system save messages published while the subscriber is disconnected.

The goal is not to loose messages while subscriber is offline (for whatever reason). There are many possible implementations based on the specifics of a messaging system we are working with. The main distinction between them is who, publisher or subscriber, is responsible for maintaining the state of the subscription.

In theory, the mechanics are simple, no matter who’s responsible for the state. In case subscriber goes down, the state is simply not updated while new messages are generated (and stored). When subscriber is up again the publisher picks up where he left based on the state and sends all the missed messages. In case where publisher is responsible the only additional step is an acknowledgement that has to be sent to publisher every N messages (where N depends on how much you don’t like duplicates).

In practice however, many messaging platforms (including EventStore which is recently my favourite) offer two distinct APIs for implementing subscriptions:

  • push API where a client (subscriber) opens a persistent channel (TCP connection) via which a server (publisher) sends messages immediately as they are generated
  • pull API where a client (subscriber)  periodically polls a server (publisher) asking if there are any messages newer than X where X is the current state of the subscription

The advantage of push API is, it is usually a lot faster (less effort on publisher side and minimum latency). The problem is, it works only when both parties are up and running and there is no lag. If the subscriber processes messages slower than the publisher generates, them sooner or later buffers on the publisher side become too big to fit in the memory. EventStore is smart enough to drop the subscription automatically in such case.

There  is no such problem with the pull API because the client requests messages in its own pace and can even crash every now and then without the publisher ever noticing. This comes with a price however. Pull API is inherently slower (meaning higher latency) and it requires additional reads on the publisher side.

As usual, the best solution combines both approaches. The idea is simple: when the subscriber is up to date with the message stream, it switches to push API and whenever it is not — switches back to pull. The actual working algorithm is slightly more complex:

  1. The subscriber always starts with pull assuming there were some messages generated while it was offline
  2. The subscriber pulls messages until there’s nothing left to pull (it is up to date with the stream)
  3. Push subscription is started  but arriving messages are not processed immediately but temporarily redirected to a buffer
  4. One last pull is done to ensure nothing happened between step 2 and 3
  5. Messages from this last pull are processed
  6. Processing messages from push buffer is started. While messages are processed, they are checked against IDs of messages processed in step 5 to ensure there’s no duplicates.
  7. System works in push model until subscriber is killed or subscription is dropped by publisher drops push subscription.

This algorithm is the single most important thing in my new project.

VN:F [1.9.22_1171]
Rating: 5.0/5 (4 votes cast)

IDDD Tour @ Kraków

I’d like to announce that May 6-9 Vaughn Vernon is coming to Kraków to teach his Implementing Domain Driven Design course (based on the book with same title). The course is 4 day long (as you can easily guess from dates;) ). Expected lecture to coding ratio is around 50%. The total price for early bird registration is 400 EUR.

You can register via Eventbrite.

See you there!

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Services & Bounded Contexts

TL;DR

There is a common misconception that a Service in a Service Oriented Architecture is equal to a Bounded Context in Domain-Driven Design. It is not.

Explanation

Bounded Contexts govern the way we speak about things and how we model them. They are about the language and meaning of words (e.g. what account means within the bounded context of a banking versus what it means within the bounded context of e-mail).  In other words, bounded contexts are used to break down the problem domain.

Services on the other hand structure the way we implement the models. They are used to break down the solution domain. They should adhere to The Four Tenets of Service Orientation

  • Boundaries are explicit
  • Services are autonomous
  • Services share schema and contract, not class
  • Service compatibility is based on policy

Example

Let’s examine a distributed application server management tool as an example of a simple SOA. There are only two services: a Node service which is installed on each machine and acts an agent and a Supervisor service which controls and coordinates all nodes. There are also two bounded contexts: a context of Node and a context of the whole Environment.

An example of the term which is used in both contexts but has different meaning is the verb to deploy. Within the Node BC, to deploy means to install a Windows Service for particular component on current machine while within the Environment BC it means to deploy and start all service components on all nodes according to the plan.

The Environment BC is implemented only in Supervisor service but the Node BC is implemented in both services. In the Supervisor service there is a Node aggregate that represents and single machine. It has a method Deploy that implements exactly the semantics of deploy as meant by Node BC — it uses REST API to ask the Node service to deploy a component. The rest of the Supervisor service implements the Environment BC.

Disclaimer

Throughout the post I used a term to implement a bounded context. Wherever I used it, I meant that the code in the service implements the model that was created within the bounded context.

VN:F [1.9.22_1171]
Rating: 5.0/5 (1 vote cast)