Enterprise Service Buses (ESBs) Drive SOA Adoption Part 4
Blog Post 4 – Publish-Subscribe Messaging and Neuron Topic Networks
Introduction
In the last installment, I took a very high-level look at the Neuron architecture. In particular, I spoke of the “internal view” of the bus: Neuron’s internal publish-subscribe messaging model. Most ESBs provide some publish-subscribe abstraction over raw communications.
Publish-Subscribe Pattern
Publish / Subscribe is just a fancy name for events in distributed systems. Unlike the quick and dirty event handler, a good publish / subscribe infrastructure assumes that systems might be unavailable and can work around temporary failures by automatically resending messages when the external system comes back up.
Since events are an important part of building distributed systems, and publish / subscribe is how we handle events in a distributed system, it makes a lot of sense not have to code and recode this “plumbing” over and over. Neuron provides a very rich publish-subscribe model in terms of “Topic Networks.” But first, let’s dive a bit deeper into the pattern itself.
I believe I have referred to the seminal work in this space, Hohpe and Woolf’s Enterprise Application Patterns. Gregor, along with some folks at Microsoft Patterns & Practices built upon this work with the 2004 release of the book Integration Patterns. The EAP book defines a “Publish-Subscribe Channel” as:
“A Publish-Subscribe Channel works like this: It has one input channel that splits into multiple output channels, one for each subscriber. When an event is published into the channel, the Publish-Subscribe Channel delivers a copy of the message to each of the output channels. Each output channel has only one subscriber, which is only allowed to consume a message once. In this way, each subscriber only gets the message once and consumed copies disappear from their channels.”
I prefer the definition in the Integration Patterns book:
“Publish/Subscribe: A communication infrastructure that uses topics or dynamically inspects message content, enables listening applications to subscribe to specific messages, and sends messages to all interested subscribers. Three variations of Publish/Subscribe are List-Based Publish/Subscribe, Broadcast-Based Publish/Subscribe, and Content-Based Publish/Subscribe.”
List-Based Publish-Subscribe
The Integration Patterns book goes on to describe the “List-Based Publish-Subscribe” Pattern as “A List-Based Publish/Subscribe pattern advises you to identify a subject and to maintain a list of subscribers for that subject. When events occur, you have the subject notify each subscriber on the subscription list.”
The first point I want to make is that you can implement this pattern in WCF. In fact, it’s implemented as a sample! But it’s going to take me 480 lines of WCF code that I should never be writing. I should be writing business logic for my customer instead. I would argue that coding at the WCF level is in fact too-level and does not present the right level of abstraction for distributed communication. I would argue that a service bus and publish-subscribe messaging to be the *basic* level of communication and that the focus should be on events between business systems rather than crafting SOAP or REST packets on the wire. In particular, what’s wrong with the 480 line WCF version (some of which is shown below):
[ServiceBehavior(InstanceContextMode=InstanceContextMode.PerSession)]
public class SampleService : ISampleContract
{
public static event PriceChangeEventHandler PriceChangeEvent;
public delegate void PriceChangeEventHandler(object sender, PriceChangeEventArgs e);
ISampleClientContract callback = null;
PriceChangeEventHandler priceChangeHandler = null;
public void Subscribe()
{
callback = OperationContext.Current.GetCallbackChannel<ISampleClientContract>();
priceChangeHandler = new PriceChangeEventHandler(PriceChangeHandler);
PriceChangeEvent += priceChangeHandler;
}
public void Unsubscribe()
{
PriceChangeEvent -= priceChangeHandler;
}
public void PublishPriceChange(string item, double price, double change)
{
PriceChangeEventArgs e = new PriceChangeEventArgs();
e.Item = item;
e.Price = price;
e.Change = change;
PriceChangeEvent(this, e);
}
public void PriceChangeHandler(object sender, PriceChangeEventArgs e)
{
callback.PriceChange(e.Item, e.Price, e.Change);
}
Requires lots of infrastructure code to write
• Subscribe/Unsubscribe methods
• Three configuration files!
• Requires tightly-coupled callback channels and contracts
• Have to maintain client list
• How do I maintain client list reliably?
• I can’t add/subtract subscribers while running
• I can’t change QOS/transports while running
I can’t base a real distributed system on this and write this over and over. ESBs, with publish-subscribe messaging offer a powerful advantage and in particular, Neuron’s Topic Networks and offering all three variants of publish-subscribe (among many other communication patterns) allows you to focus on your business problem.
There are two ways to look at Neuron’s implementation: logically and architecturally.
A Logical View of Topics
The whole idea of a Topic is that it is a business conversation and should be named accordingly. Topics can have simple names like Orders or Suppliers, or can have a hierarchy of subtopics such as Orders.New, Orders.Approved and Orders.Shipping, and so forth. Parties express interest in a topic with a subscription. Subscriptions can also be wild carded. The whole idea to success here is to model your solution after actual business conversations.
Another important thing to point out is that Neuron does not limit you to the Publish-Subscribe messaging pattern. You can have point-to-point messaging, request-response messaging and Itinerary-based routing amongst other patterns. Moreover, unlike the fragile WCF code in the List-Based Subscribe sample, Neuron will allow you to add, change and delete parties, topics and subscriptions even while running! This is very powerful.
Architectural Views of Topics
Architecturally, each individually configured topic is called a Topic Network. The important differentiator for Neuron here is that each topic network is assigned a communication technology to drive it, such as WCF TCP, peer networking, MSMQ, BizTalk and others. By being based on WCF, it’s easy for Neuron to use a different channel. My personal favorite part of this feature is that it can be configured and changed while the system is running! Moreover, it can be simply changed from a list box in the ESB Explorer, where graphically the Explorer gives a visual representation of speed versus reliability.
The big difference here between Neuron and many “ESBs” is that you can run each topic with a mixture of qualities of services. Each topic network can also refer to a separate server infrastructure, making topics the primary division for scaling out. This distributed topic architecture very different from classic EAI Hub-and-Spoke. There is no central hub or store that messages must pass through.
Implementing List-Based Publish-Subscribe with Neuron
In contrast to all the WCF code shown above, Neuron requires no code to implement list-based publish-subscribe and can be handled through the GUI. However, using the Neuron API, we can do the Send side in three lines of Neuron code:
using (Publisher publisher = new Publisher())
{
publisher.Connect();
foreach(var change in changes)
{
publisher.Send("PriceChange", change);
}
}
Where Are We?
In this article, I extensively covered the Publish-Subscribe pattern and its variant List-Based Publish-Subscribe. I showed some of the code required to implement this pattern in WCF and how fragile it is. I then showed how Topic Networks implement Publish-Subscribe in Neuron and how easy they are to use.
In the next installment, I will look at something that is commonly built on top of WCF, Message Routers, and how message routing is a core function of Neuron.
Filed under: Neuron ESB
Pingback from NServiceBus 1.9
Introduction In the last article, we started to implement messaging applications, specifically Message Routers. We first turned, as usual to the Hohpe and Woolf book to look at the Message Router pattern. We then looked at variants of routers. I then