Push documents to Window Azure Blob Storage using BizTalk Server

Push documents to Window Azure Blob Storage using BizTalk Server

Published on : Mar 16, 2017

Category : BizTalk Server

steef

Author

BizTalk Server 2013 and later versions support connectivity with Windows Azure through its adapters WCF-BasicHttpRelay, WCF-NetTcpRelay, WCF-WebHttp and SB-Messaging. The relay adapters enable connectivity with the relay service in the Windows Azure Service Bus. SB-Messaging adapters provides the means to connect to queues and subscriptions (topic). Finally, the WCF-WebHttp supports connectivity to REST endpoints. Windows Azure Storage, other services on Windows Azure and the cloud support a REST Interface to connect to. The WCF-WebHttp adapter can be used to connect to the Windows Azure Storage Service. A benefit of connecting to Windows Azure Storage and making use of this service is that you can store large amounts of unstructured text or binary data. In case you need to pull or push documents to Windows Azure you can connect to the Windows Azure SQL Database or Windows Azure Storage. The latter offers storage of documents at lower costs. In this blog, we will discuss the connection with the Windows Azure Storage through the WCF-WebHttp Adapter in BizTalk Server 2016.
You can download this article as a PDF document Download now.

Windows Azure Storage

The Storage service in Windows Azure provides a way to store and access data. This data can be unstructured binary data like videos, audio and images. Besides binary data you can store and access text data like XML files, PDF’s, Word documents and so on. This can be done by making use of what is called Binary Large Object (BLOB) storage. The service offers more than just Blob storage:
  • Queues to store messages
  • Tables to store non-relational structured data
In this post however, we will be using Blob storage. To connect to the Windows Azure Storage to store a Blob like an XML or text file you will need a Windows Azure Account and subsequently a storage account. So in case you do not have a storage account, you need to create one, which is described below.

Create a Windows Azure Storage Account

To create a Windows Azure Storage Account, go to the Azure Market Place, type Storage Account, select it in the list and click Create. Specify a name, deployment model, account kind (general purpose, which include tables, queues and blobs), replication, encryption, subscription, resource group and location. Subsequently click Create.

Create a container

 Within a Windows Azure Storage account, you can create a container. A container gives you the ability to group a set of Binary Large Objects (BLOBs). To create a container, select the recently created storage account, choose blobs, and click Add Container in the top left corner. Specify a name, leave access type to Private and click Create. Note: A storage account can contain an unlimited number of containers, as long as their total size is under 100TB.

Configuring the WCF-WebHttp Adapter

You can access and store data in a container in Windows Azure Storage. That can be done through either code like .NET code, Node.js or through using BizTalk Server 2013 and up by leveraging the WCF-WebHttp adapter. In case you want to support a business process or messaging solution with BizTalk that needs to send a document to the Cloud (Windows Azure Storage), than the WCF-WebHttp adapter can be of great value. The configuration of the WCF-WebHttp adapter in a messaging scenario is as follows:
  • A document needs to be sent to a container in Windows Azure Storage. Therefore, a send port will be created and configured with WCF-WebHttp
  • In the General Tab of the WCF-WebHttp Transport properties, the address of the service can be specified (URI). This address can be found in the container.
configuration of the WCF-WebHttp adapter in a messaging scenario
  • Since we are going to make use of the blob service, the endpoint (address) we will use ishttps://mydocument.blob.core.windows.net or in case the Transport mode is set to Transport (i.e. HTTPS) than the address will be https://mydocument.blob.core.windows.net
  • In theGeneral Tab of the WCF-WebHttp Transport properties we will also specify the HTTP Method and URL MappingURL Mapping will be applied to dynamically have the file name, that is the name of the file being offered to BizTalk Receive Port. The method is going to be PUT as we will create (place) the file in the container.
wcf webhttp endpoint address
  • In the HTTP Method and the URL Mapping sections you specify the method (operations) you are going to perform. In this scenario case, it is going to be only PUT. At the URL mapping you define what is going to be added after the specified URI. To make it more dynamically, instead of hard-coding in general, you can make use of the Variable Mapping configuration feature. Moreover, what’s between the brackets is a variable that can be mapped to promoted properties. The HTTP Method and URL Mapping will look like what is visible in the picture above.
  • Variable mapping is a powerful technique to define any custom variable (or place holder) in your URL. in this case{filename}, and map that variable to any context property with the property name and namespace. The Variable Mapping can be specified by clicking the Edit…
defining custom variable in url
  • In the Binding Tab, you could specify time-out properties or message size.
  • In the Security tab, you need to specify the security mode. You can set this to transport. Nowadays most cloud services have their own authentication scheme (like SalesForce that has OAuth protocol, or Service Bus that leverages the Access Control Service (ACS)). The Windows Azure Storage uses a key/value pair, which is the Storage Account and the key. Therefore, you need that authentication scheme by using an endpointbehavior. Fortunately, you do not have to create your own behavior to support authentication to the Azure Storage. Microsoft has created an azureStorageBehaviour you can use to authenticate with Windows Azure Storage. Here you can specify the name of the storage and the storage account key.
  • In the Behaviour Tab you right click EndPointBehaviour and click Add Extension.
endpoint behaviour
  • You select the azureStorageBehaviour and specify the Azure Account Key and Name.
azure storage behaviour
  • You can find the details, when you go to your Storage Account and in the blade, select under Settings, click Access Keys.
document storage
  • In the  Access Keys dialog you will find the Storage Account name and access keys. You can paste the key and name in AzureStorageBehaviour.
  • Proxy and Messages Tab, you do not need to specify anything and leave it with default settings.
Now that the WCF-WebHttp Send port is configured, we can look at the complete BizTalk messaging solution that consists of a Send port and Receive port.
You can download this article as a PDF document Download now.

BizTalk Messaging Solution

The messaging solution is a very basic (see diagram below). A file will be dropped into a folder and a BizTalk receive port that listen to that folder will pick the file up. It will then be routed to a send port that subscribes to the files coming in through the receive port. The send port is configured with WCF-WebHttp adapter and will send the file, according to its configuration, to a container in Windows Azure Storage. wcf webhttp send port However, the way the adapter is configured the name of the file will be containing the path of the folder and the file name. Something like C:\foldername\filename. To prevent that from happening in a message based solution you will need a simple custom pipeline component that will strip off the path name. Below you can see the code to strip the path from the ReceivedFileName context property.
public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)

{

IBaseMessageContext contextReceivedFileName = pInMsg.Context;

 

//extract the filename

string value = contextReceivedFileName.Read("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties").ToString();

 

//set filename only, strip path

FileInfo file = new FileInfo(value);

string fileString = file.Name;

 

//write updated value back to context

pInMsg.Context.Write("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties", fileString);

 

//return the message with modified context

return pInMsg;

}
When sending a document to Windows Azure Storage, a file will be dropped in the folder and will end up in Windows Azure Storage Container named documents. windows azure storage container

Wrap Up

This blog demonstrated how to send a document (text file) to Window Azure Storage Container by means of a BizTalk messaging solution. The WCF-WebHttp Adapter is used to connect to Windows Azure Storage. This service, like other Azure Services, provides a REST interface that support operations on the service. In case of the Azure Storage Service it will support operations like accessing or storing data. The Windows Azure Storage has its own authentication scheme and an out of the box behavior extension that needs to be added to configure the access to the storage. Once the behavior in the adapter is configured with all the other necessary properties, the data or document can be sent to the cloud as described in the messaging solution in this blog.

Call for action

In case you want to try it yourself, you will need a virtual machine with BizTalk Server 2013 or up (either local or a VM Image in Azure), a Windows Azure account and within your Azure account a Windows Azure Storage account. You can find the source code belonging to this blog for BizTalk 2013 on MSDN Code Gallery: BizTalk Server 2013: Connect to Window Azure Blob Storage Sample. You can port the code to a higher version of Visual Studio for BizTalk Server 2013R2 or 2016. [adrotate banner=”4″]