Messaging or data exchange between business can be done in various ways. Most of you are familiar with FTP (file transfer) or HTTP(S) with XML, JSON or flat file (EDI, HL7) as the message format. With Azure, new options arrived where you can leverage the Service Bus Azure Platform Service, which offers brokered and pub/sub type of messaging capabilities. In general, Azure offers besides Azure Service Bus Queues and Topics, SQL Databases, and Blob storage to enable message routing.
In this post, we will be discussing the Azure Service Bus Queues (brokered) and Topics (Pub/Sub).
Using Azure Service Bus and BizTalk 2016 for messaging between enterprises.
The Azure Service Bus can be a broker between enterprises, each sending their messages to either a queue or topic. A third party can broker between the enterprises to facilitate the messaging and offer a type of trust similar to a certificate authority (CA) in Public Key Infrastructure (PKI) scenario’s. A third party can manage Azure service bus namespace and distribute the Shared Access Signature (SAS) keys to each enterprise required to send/receive messages. Each enterprise doesn’t know the key of the others.
Note: On premise, you can have custom code implementation and hosting of it, to send and/or receive messages from the Azure Service Bus queue or topic/subscription. In this post, we will be discussing using BizTalk Server 2016 for communication leveraging SB-Messaging adapter.
Setting up connectivity with Azure Service Bus using SB-Messaging Adapter
A message can be sent to the queue and a consumer can pick up the message for further processing. In our use case this will be a Logic App that mediates between the in- and outbound queue. Enterprise A can send a message to the inbound queue by configuring the Send Port with the SB-messaging adapter. The consumer of the message, Enterprise B, can configure a Receive Location within a Receive Port configured using the SB-Messaging adapter that enables communication with a Azure Service Bus Queue.
In Send Port and Receive Location the SB-Messaging configuration consists of two steps:
The addressing, where does queue and/or topic reside in Azure, is done in the General tab. You need to specify the namespace of the Azure Service Bus and the Queue Name.
In this General tab, where you specify the address, you can observe a few other settings. The message size for instance is an interesting one. The Maximum Message Size is 256KB (Standard tier) / 1MB (Premium tier), for more quotas and settings see Service Bus quotas on Microsoft Docs. In the second tab, you specify the security settings, i.e. this will be SAS key that the third party can provide. The SAS Key will be created in the Azure Portal based on a SAS policy that allows to write only (SEND).
The generated key (connection string) looks like:
Endpoint=sb://securemessagehub.servicebus.windows.net/; SharedAccessKeyName=write_message;SharedAccessKey=qDTuVypr2jgLp1HH5kbG76DikLLmLMuqpBHAhug/OBs=; EntityPath=inboundqueue
- Address: sb://securemessagehub.servicebus.windows.net/inboundqueue
- SharedAccessKeyName: write_message
- SharedAccessKey: VVuHLN9l42o4jViS0Krye4MnO/GDcV3OxvqzSX5OhRE=
The generated key (connection string) for Enterprise B to receive the message from the outbound queue looks like:
Endpoint=sb://securemessagehub.servicebus.windows.net/; SharedAccessKeyName=message_reader; SharedAccessKey=Ux6QIl35b62fyEzHJRoUhEXJmARf3AlolRy2ciIeJ6U=; EntityPath=outboundqueue
- Address: sb://securemessagehub.servicebus.windows.net/outboundqueue
- SharedAccessKeyName: message_reader
- SharedAccessKey: Ux6QIl35b62fyEzHJRoUhEXJmARf3AlolRy2ciIeJ6U=
The above settings can be used to configure the SB-Messaging adapter on the consumer side i.e. Enterprise B.
The scenario with sending a message to a topic is a bit different. Forwarding is not necessary in this case, as the message is distributed to multiple subscriptions based on a filter. The message will require a property for routing purposes within the topic. Topics and subscriptions provide a pub/sub mechanism comparable to BizTalk, however the message doesn’t require an active subscriber i.e. the subscription is a queue that subscribes to the message and the actual subscribers Enterprise B and C can pick the message any time they want (as long as they pick it up within the retention time (time to live).
To write to the topic the following details from connection string in bold are required:
Endpoint=sb://securemessagehub.servicebus.windows.net/; SharedAccessKeyName=write_topic; SharedAccessKey= lkJ8n02Q2SYas/XcfzN290XxI+9k8DpeoOb8E4jMSPM=; EntityPath=multidistribution
To receive the messages, the Receive Ports at Enterprise B and C will be configured using the details from the following connection string:
Endpoint=sb://securemessagehub.servicebus.windows.net/; SharedAccessKeyName=read_subcription; SharedAccessKey=dUdzOEGbYhUbLfArXQWA3vrrtHEkYIIoIq7Qv7wQibk=; EntityPath=multidistribution
The string above can provide details for address and authentication. However, the address for the topic is as follows:
The authentication settings are with the SharedAccessKeyName=read_subcription and SharedAccessKey=dUdzOEGbYhUbLfArXQWA3vrrtHEkYIIoIq7Qv7wQibk=.
Setting up the Logic App Forwarder
In the Azure Portal, you can provision a Logic App. To support our use case, we provisioned one with the name “Forwarder”. We will be using the Peek-lock receive template to pick up the message from the inbound queue to forward it to the outbound queue.
Once the template is chosen, the first step is to set up the connection to the Service Bus. For the connectivity with the Service Bus we will use the RootManageSharedAccessKey, which provides us access to all the artefacts within the Service Bus namespace.
Endpoint=sb://securemessagehub.servicebus.windows.net/; SharedAccessKeyName=RootManageSharedAccessKey; SharedAccessKey=CmtkK8nZsA/MdREdTeycWAfNKnPLQ/Eh5wG6gXUJI6A=
Our definition of the Logic App will look like below:
Message is received from the inboundqueue and completed in the inboundqueue and send to outboundqueue.
The use case does bring a few aspects to bear in mind. One is message size, which is a limiting factor depending on the Service Bus tier you choose.
With 10 USD a month you can have a nice set up between two parties exchanging messages with 256kb max in size. For a 60-fold in costs you can send messages with large size.
The second aspect, an important one not to overlook is security. If you (Enterprise A) or the other party (Enterprise B, C or the third party) control the Azure subscription, hence the Service Bus namespace the responsibility of creating and exchanging the keys can be challenge. Therefore, a certain process and SLA need to be in place. From time to time you need to regenerate the keys and distribute them again to prevent exposure of SAS key over time. An alternative to the distribution of the SAS keys could be Key Vault. A request can be sent to Key Vault to get the appropriate SAS key, which is tied to the appropriate SAS policy (see How to setup Key Vault with end to end key rotation and auditing). This approach will add complexity and customization to support our use case, yet can mitigate the risk of exposure of the SAS key.
Another aspect will be scalability, which falls in line with message size, and pricing. Ergo which tier you choose impacts capability, message size, pricing (variable or fixed), and which type of capacity (shared or dedicated). Most of the details per tier can be examined through Service Bus Pricing, the FAQ at the bottom of the page and additional FAQ.
Setting up queues, topics and subscription is another key aspect with regards to this scenario in particular. The queues can be set up to support sessions, partitioning, size, time to live (retention), locking, duplicate detection, and dead letter queue.
Similarly, for a topic you can specify size, time to live, duplicate detection and partitioning. And finally, for subscription you need to specify time to live, lock duration, dead letter, and sessions. Therefore, you need to design your queues, topics and subscriptions to your needs or towards implementations of this use case.
A final aspect is the management and monitoring of the Service Bus and the relying solution building blocks i.e. BizTalk Server. The latter can be managed and monitored using the BizTalk Management Console and BizTalk360, which are a killer combination. The Service Bus can be managed and monitored through the Azure Portal.
The Azure Service Bus provides you an easy and straight forward way to setup messaging between enterprises. Different than classic implementations using FTP or HTTP(s) and with support for multiple protocols (SB-messaging, AMQP, HTTPS) and message formats (flat file, XML, JSON). The TCO of the Azure Service is much lower and capabilities offer a great value. For a secure type of messaging, this scenario offers value in terms of decoupling, separation of concerns and costs. Regardless of a BizTalk Server, enterprises can set up a secure means of message exchange leveraging the Service Bus capability.
Using Azure Service Bus and BizTalk 2016 for messaging between enterprises.