SOA: Making the Paradigm Shift Part 10 of N
This is the 10th article in the series. I should mention that much of the series so far has been geared at a high level and strategic focus for IT Decision Makers rather than for those writing code. This is deliberate as much of SOA is an "Enterprise IT" activity. There has been a fair amount geared towards Architects as well. That will change as I get into Indigo, but today's topic is again a strategic one. Given that we have looked at the current state of SOA, how to make the paradigm shift and SOA design approaches, we are now faced with the questions of what should I do "Short Term" and what investments should I make "Long Term."
As a reminder, the series so far is:
As we went along, I talked about the need to build a Capability Driven strategy, as IT needed to become more in sync with the needs of the business, and the capabilities that the business offers. I talked about ensuring that SOA is part of the implementation in current and upcoming projects. Becoming Service Oriented is an important first step, realizing that exposing business assets as reusable services is more important than creating another island. This "paradigm shift" requires developers and architects to think beyond one application and to the needs of the enterprise as a whole, in support of business drivers. I talked about using Microsoft IO to access the current state of your IT as part of a maturity model and using it as step-by-step guidance in creating an Agile IT that is Service Enabled.
Short-Term vs. Long-Term
It can become difficult, and sometimes impossible to measure SOA benefits using traditional Return On Investment (ROI) approaches because SOA projects remain a moving target. This is the nature of SOA; the need to enable, support, and manage more-frequent changes. Many people have experiences with SOA that run that gamut from spotty (we wrote a web service and it worked - we'll try again next year) to scared (analysts quoting cost of SOA to be larger than the companies' entire budget). Even though many CIOs see moving to SOA inevitable, most are still challenged to articulate what the business is going to get for its money if they allow their IT people to go off on yet another initiative.
In this regard. Long-Term "Investment" projects can be difficult to recover. Companies need to deliver value right now and yet contain or control longer-term management costs.
As enterprise adoption of services continues, the need for a service-oriented architecture will start to be felt. There's a big difference between the casual use of services and running your enterprise primarily over services. As enterprises travel down the road that will take them from light use of services to deep use of services, many issues will arise, such as how to manage large numbers of services well; how to overcome differences between services; how to enforce SLAs; and how to enforce enterprise policies across distributed collections of services
The Enterprise Service Bus
Pinning down the definition for ESBs can be just as elusive as pinning down the definition for SOA. There is certainly definitions and redefinition's by ESB vendors to suit their purposes. Some of these definitions are:
"A Web-services-capable infrastructure that supports intelligently directed communication and mediated relationships among loosely coupled and decoupled biz components." - Gartner Group
"The ESB label simply implies that a product is some type of integration middleware product that supports both MOM and Web services protocols." –Burton Group
"A standards-based integration backbone, combining messaging, Web services, transformation, and intelligent routing." –Sonic Software
"An enterprise platform that implements standardized interfaces for communication, connectivity, transformation, and security." - Fiorano Software
"To put it bluntly: If you have WebSphere MQ and other WebSphere brokers and integration servers, you have an ESB." –Bob Sutor, IBM
"The Enterprise Service Bus is a uniform service integration architecture of infrastructure services that provides consistent support to business services across a defined ecosystem. The ESB is implemented as a service oriented architecture using Web Service interfaces." –CBDI
However, I do think one can be practical about it, stay out of the wars and see what's common in the definitions. One practical view of the "common" ESB Functions can be found in the Wikipedia definition [1]
| Category |
Function |
| Invocation |
Support for synchronous and asynchronous transport protocols, service mapping (locating and binding) |
| Routing |
Addressability, static/deterministic routing, content-based routing, rules-based routing, policy-based routing |
| Mediation |
Adapters, protocol transformation, service mapping |
| Process Choreography1 |
Implementation of complex business processes |
| Service Orchestration |
Coordination of multiple implementation services exposed as a single, aggregate service |
| Complex Event Processing/EDA |
Event interpretation, correlation, pattern matching |
| Other Quality of Service |
Security (encryption and signing), reliable delivery, transaction management |
| Management |
Monitoring, audit, logging, metering, admin console, BAM |
| |
|
1 Some do not consider Process Choreography to be an ESB function.
² While Process Choreography supports implementation of complex business processes that require coordination of multiple business services (usually using BPEL), Service Orchestration enables coordination of multiple implementation services (most suitably exposed as an aggregate service) to serve individual requests.
In this article, we are interested in looking how an ESB can help us over the longer-term as an "investment" made now to help us accelerate our SOA efforts. In other words, buying an ESB will not implement a service-oriented architecture, but provide the features in which one may be built
But what is an Enterprise Service Bus and what is the value of it to my infrastructure and to my SOA? For our purposes, I am going to focus on three main areas:
An ESB is an operating environment for Services
An ESB provides the architecture needed for deep adoption of services. As I stated earlier, once your organization begins to move from casual use of services to running your enterprise primarily on services, the need for something like an ESB will be more acutely felt. As stated above, many issues will arise, such as how to manage large numbers of services well; how to overcome differences between services; how to enforce SLAs; and how to enforce enterprise policies across distributed collections of services. An ESB is a backbone for connecting and integrating an enterprise's applications and services. An ESB accelerates SOA by providing infrastructure services to complement business services.
What are these "infrastructure" services? They are a whole set of valuable services that you need to use over but should not have to write. These include routing, storing, and forwarding of messages, business activity monitoring, transformations and EAI functions. This is not rocket science as most enterprises contain common infrastructure services like domain controllers and directory services like LDAP or AD.
An ESB is a Common Messaging Fabric
In the ESB model, most or all applications and services in the enterprise connect to the ESB and communicate with
each other over the ESB. Applications and services usually connect using SOA standards, whereas legacy systems require integration via EAI technologies such as adapters. The communication between endpoints is handled by message-oriented middleware arranged in a bus topology. The primary advantage of such an approach is that it reduces the number of point-to-point connections required to allow applications to communicate. This, in turn, makes impact analysis for major software changes simpler and more straightforward. By reducing the number of points-of-contact to a particular application, the process of adapting a system to changes in one of its components becomes easier. [2]
The ESB serves as a common messaging fabric for the enterprise. Programs connect to the ESB and send or receive messages. The ESB handles routing details, mediation of differences between endpoints, and the physical details of "It's far more sensible to put such matters in the hands of business analysts and I.T. personnel who can make enterprise-level decisions than having them controlled by developers at the application level. " [3]
Much of the value an ESB provides is in the area of centralized management. An ESB provides a single place to handle management functions such as configuration, deployment, monitoring, and control. Having a central facility like this makes change management straightforward and rapid response possible. Not having centralized management creates a significant problem for I.T. departments and imposes unnecessary cost and delays when change is required. [4]
An ESB is a Long-Term Architectural Pattern
A subject of much debate is whether an ESB is a pattern or product. The most obvious answer would seem to be "both": an ESB can certainly be described as an architectural pattern and there is more than one path to creating one. You have to build your ESB out of something however, so its natural that one or more products would be used to accelerate the process.
It is my feeling that the ESB is first and foremost a pattern. Like most patterns in this space, it is cataloged in Hohpe and Woolf's Enterprise Integration Patterns as Message Bus. As such, an ESB for the long-term should be possible to survive incremental and generational changes in technology without having to throw away the pattern.
Indigo On the Way!
There is so much more I could and want to say about ESBs and I will return to look at example ESBs like Neuron, nServiceBus and Tibco later in the series. Next up, finally you may say, is looking at Indigo, Microsoft's programming API and Framework for (possibly) creating Service Oriented Architecture. My love affair with Indigo goes back four years now as a member of the SDR program, but we will return to the concepts in this article to suggest that powerful as Indigo is, it is low-level and one may take advantage of frameworks like nServiceBus and ESBs like Neuron (built on WCF) to avoid having to write a lot of the "plumbing" code yourself.
References
[1] http://en.wikipedia.org/wiki/Enterprise_service_bus
[2] http://en.wikipedia.org/wiki/Enterprise_service_bus
[3] Neuron ESB Whitepaper - "Understanding the Enterprise Service Bus" available at http://neuronesb.com/NeuronESB.pdf
[4] same