Understanding the Current Integration Technologies Puzzle (cloud and on-premise)

Published on : Jul 1, 2013

Category : BizTalk Server

Saravana

Author

In the recent months, there were overwhelming amount of improvements in the Microsoft integration space both on-premise and in the cloud and it’s very difficult to keep up with the various technologies and how they all relate to each other.  In this article, we are going to take a brief look at each one of them to give you an idea, show how they are related and make some suggestions on when to choose what. Integration is everywhere these days, this article is written keeping in mind to help people coming more from BizTalk Server background and things revolving around BizTalk Server. There are other integration technologies available in Microsoft stack like Workflow, Workflow Manager, SSIS, Business Data Connectivity Service (BDS) in SharePoint etc. which we will not cover.  The following picture shows you stuff that are present in the cloud and on-premise and some of the management tools available in relation to BizTalk Server. Understanding the current Integration Technologies Puzzle (cloud and on-premise)

Integration Solutions in Azure Cloud

Let’s start of from the cloud. Currently there are three core integration technologies available as part of the Windows Azure cloud offering
  • Microsoft BizTalk Server 2013 in Azure VM (IaaS)
  • Windows Azure Service Bus (SB)
  • Windows Azure BizTalk Services (WABS)

Microsoft BizTalk Server 2013 in Azure VM (IaaS)

Microsoft BizTalk Server 2013 in Azure VM (IaaS)Windows Azure is offering running Virtual machines for a while now. You can now run the full version of BizTalk Server (Standard, Enterprise and Evaluation) in a Virtual machine. This configuration is not much different to provisioning a virtual machine on-premise. But there are few advantages running BizTalk Server Virtual machine in cloud.
  • Fully functional BizTalk Server can be provisioned in few minutes. Helping to provision machines for development and test purposes
  • Customers can start using BizTalk Server for production without huge upfront cost
  • Opportunity for customers to quickly spin evaluation version for Proof of Concept.
The important thing to note here is,  customer are responsible for managing the virtual machines (ex: patching the servers) themselves. The offering is purely providing Infrastructure as Service (IaaS).

Windows Azure Service Bus (SB)

Windows Azure Service Bus (SB)Windows Azure Service Bus is the messaging infrastructure in the cloud that allows applications to exchange messages in a loosely coupled way at very high scale and resilience. There are four main offering in Service Bus:
  • Queues
  • Topics (Subscriptions)
  • Notification hubs
  • Relay
SB is not directly related to BizTalk Server and it’s more of a complimenting technology for BizTalk Server.  Let’s take a quick look at the four capabilities:

Queues:

This is more or less identical to MSMQ or any queuing technology.  It supports variety of protocols like REST/HTTP, AMQP, SB-Proprietary to push and pull messages into the Queue.

Topics:

Topics are identical to Queues, but it comes with the additional capability of publish/subscribe mechanism. You can set rules/filter in the topics and messages gets routed automatically based on predicates.

Notifications hubs:

Notification hubs solves one basic problem of sending notification messages via different push notification services (Windows Phone, Windows Store, Apple and Google). It provides very simple API’s to send messages at large scale (to millions of devices if required) which would be very difficult to achieve. Under the hood it takes advantage of the Queue/Topics in Service Bus.

Relay:

Relay is very unique concept, it allows to expose a service running on-premise seamlessly in the internet. The Service Bus relay will take care of dealing with firewall, NAT etc.

Windows Azure BizTalk Services (WABS)

Windows Azure BizTalk Services (WABS)Windows Azure BizTalk Services is one of the true PaaS (Platform as a Service) offering from Microsoft for EAI/EDI integration scenarios running on Azure Cloud. WABS covers three important integration scenarios
  • Cloud based Enterprise Application Integration (EAI)
  • Cloud based Electronic Data Interchange (EDI)
  • Building Hybrid-Integration Solutions

Cloud based Enterprise Application Integration (EAI) :

WABS introduces a new concept called Bridges, which basically is a combination of a Source System, a Pipeline (Validate, Enrich, Transform, Enrich, Route – VETER), and a Destination system connected together. Source system can be FTP, SFTP or HTTP. The destination system could be variety of sources like Service Bus Queues/Topics, LOB systems using BizTalk Adapter Service (BAS), Web Service, HTTP Endpoint, Azure blob storage etc.

Cloud based Electronic Data Interchange (EDI):

EDI standards like X12, AS4, EDIFACT plays a key role in exchanging messages (documents)between various trading partners. WABS comes with a brand new trading partner management portal (not the standard Windows Azure Management portal) that allows to create/manage new partners, their agreements etc.

Building Hybrid Integration Solutions:

WABS makes hybrid connectivity scenario seamless with the help of BizTalk Adapter Service (BAS) as shown in the above figure running on on-premise and connecting to LOB systems like SQL, Siebel, SAP, Oracle, etc.

Integration Solution On-Premise

Customers will typically have many disparate Line of Business (LOB) applications running on their premise and integration is always a key driving factor in bringing all these isolated application to work together and create business value. Now let’s take a look at the various integration technologies available on-premise. Understanding the current Integration Technologies Puzzle (cloud and on-premise)

Windows Server Service Bus (SB)

Microsoft is taking the approach of cloud first and then move the technology to on-premise.  Windows Server Service Bus is a classic example of this, where the technology was first introduced in the Azure cloud and later made it available on the Windows Server. Obviously the server version will be bit behind the cloud version and there will be some features missing and eventually catch up. Microsoft recently announced the availability of “Windows Azure Pack”, which for example allows you to install a similar Windows Azure Management portal in your own server for managing artifacts like Service Bus running on premise.

Microsoft BizTalk Server 2013

Microsoft BizTalk Server is the premium integration product from Microsoft and there is not much introduction required. The product is in the market for several years with multiple versions released since 2000. The latest version is BizTalk Server 2013. It comes with various LOB/Protocol  adapters to connect to various systems like SQL, Oracle, Siebel, SAP, IBM MQ Series, MSMQ, Web Services, etc. It comes with very powerful orchestration engine, pub/sub engine, Business Activity Monitoring, Business Rules Engine, etc making it one of the powerful integration product available in the market. If you integrations needs are limited to within your own enterprise, then BizTalk Server may be the only product you’ll require.

BizTalk Adapter Services

The BizTalk Adapter Service feature allows an application in the cloud to communicate with a Line-of-Business (LOB) system on-premise, in your network, behind your firewall. Using the LOB adapters in the BizTalk Adapter Pack (BAP), a BizTalk Adapter Service application can execute LOB operations to the following on-premise LOB systems:
  • Microsoft SQL Server
  • Oracle Database
  • Oracle E-Business Suite
  • SAP
  • Siebel eBusiness Applications

Developer Tools

There are various developer tools available for development. Visual Studio being the key tool, you still need to download and install the latest Windows Azure SDK. On the cloud side in addition to the standard Azure management portal, there is also an Trading Partner Management BizTalk portal for Windows Azure BizTalk Services. Like any other Azure services, you can also use PowerShell for some management. Recently Microsoft also introduce “Windows Azure Pack”, which brings some of the Azure management to on premise like “Azure Management Portal” to run within on-premise.

Guidelines on using right Integration Technology

Microsoft clearly “One solution is not going to fit all”, as you can see from the article there are various options available for the customer when it comes to integration. Sometime it may result in confusion on choosing the right technology. Here is a very high level guidance
  • Point-to-point & pub/sub messaging  – Service bus
  • Application integration on-premises – BizTalk Server
  • Agile development/testing or scenario requiring dynamic scale – BizTalk Server on Azure VMs
  • Cloud-based integration – Windows Azure BizTalk Services