REST (WCF-WebHttp) adapter in BizTalk Server 2010 R2

REST is getting wider adoption recently mainly due to the popularity of mobile devices and light weight rich (Ajax) web applications. In such scenarios usage of SOAP based services looks bit of overhead with lot of additional elements and attributes. Majority of the open API’s these days are exposed as REST based end points. Example Twitter, eBay, Google, etc. We are not going to focus too much on what’s REST, we safely assume people are aware of it’s usage.

Until now it was a challenge to consume REST based services within Microsoft BizTalk Server. There were no out of the box native adapters to communicate with REST based services. Even though BizTalk server comes with first class support for WCF, it’s was a not smooth experience consuming a REST based service. In the past people used variety of custom techniques to consume REST services. If you are on older versions of BizTalk (prior to 2010 R2) you have couple of custom options. There is an excellent article by Nithin Mehrotra from Microsoft explaining the process on 2010 Invoke ReSTful Web Services with BizTalk Server 2010 and also it was later enhanced and packaged into a nice REST Start Kit for BizTalk Server by Mikael Håkansson

BizTalk Server 2010 R2 addresses this limitation by bringing a native REST adapter (WCF-WebHttp) as part of the product.  We will various capabilities in series of articles. In this article, we will cover the basics and get started with GET operations.

GET Request:

A simple GET request is the fundamental, widely used operation in REST world. You simply visit a URL in browser (or programmatically) and the browser returns a machine understandable structured data. The structured data returned could be of any format (as long as the consumer can understand it), but XML and JSON are the two widely used formats. XML for messaging scenarios and JSON for web/mobile applications.

GET request is not any different to browsing a web page in your browser. When you visit a web page from your browser, the server returns HTML content, which your browser can understand, parse and display it nicely (pictures, text, videos, etc.). In a messaging world you will get XML/JSON response instead of HTML. Let’s taken an example, if you just visit this twitter user time line URL

http://api.twitter.com/1/statuses/user_timeline.xml?screen_name=biztalk360 in your favourite browser, you are going to receive just a structure XML data as shown below

image

The only varying parameter here in the URL is the “screen_name” query string added to the end. This makes it very easy to consume this end point in mobile devices and messaging scenarios.

We are going to use the same example  and see how we can consume that URL using the new BizTalk 2010 R2 REST (WCF-WebHttp) adapter.  All three of the following example follow the same BizTalk pattern (messaging only scenario, no orchestrations)

image

A one-way file receive adapter will be used to kick of the process. The two-way REST (WCF-WebHTTP) adapter will pick up the message (filter: BTS.ReceivePortName=one-way file receive port name), make a synchronous call to external REST end point, receive the response message and publish it into message box. The final one-way file send port will pick up (fiter: BTS.SPName=wcf-webhttp two-way send port) the published response and write it to disk. The last send port is important, otherwise BizTalk will not make the external call and simply through an exception “Published message could not be routed…”

Remove Body Custom Pipeline Component

One of the basic requirements of a GET request is there should not be any content in the body. But in BizTalk case when the REST send port receives the message the message. the bodypart of the message will be added to the HTTP request body. This is against the HTTP GET request specification and the underlying code will through the protocol violation exception as shown below.

The adapter failed to transmit message going to send port “Twitter-Parameterized.Get” with URL “http://api.twitter.com/1/statuses/user_timeline.xml”. It will be retransmitted after the retry interval specified for this Send Port. Details:”System.Net.ProtocolViolationException: Cannot send a content-body with this verb-type.

We can easily eliminate this problem by building a custom pipeline component/pipeline and setting the body part content to null as shown below.

RemoveBodyPipelineComponent

RemoveBodyPipeline

The pipeline then should be configured on the Send Pipeline section of the two-way WCF-WebHttp send port as shown below.

Pipeline config

The source code of pipeline component and pipeline can be found in the code gallery.

Simple GET Request

Let’s take a simple scenario. In this scenario we are just going to hard code a REST end point in our WCF-WebHttp adapter. This is our very basic configuration, the following picture shows the WCF-WebHttp REST adapter configuration. Only couple of things need to be configured in the screen. The address URL and the HTTP Method and URL mapping as shown in the below picture.

WCF-WebHTTP Rest adapter basic settings

<BtsHttpUrlMapping>
<Operation Method=”GET” Url=”?screen_name=saravanamv”/>
</BtsHttpUrlMapping>

Once everything is setup, you can drop the test message and you’ll see the twitter feed of the screen name you specified as an xml file in the destination folder.

Parameterized GET Request

The previous section was great to get you started, but it’s not practical. You are not going to hard code your screen name in your send port, majority of the time you want to get that information from somewhere and pass it. The REST WCF-WebHttp adapter addresses this challenge by a technique call Variable Mapping. The new parameterized configuration looks as shown below

image

 

Twitter-Parameterized.VariableMapping.Get

<BtsVariablePropertyMapping>
<Variable Name=”handle” PropertyName=”handle” PropertyNamespace=”https://BizTalk.RestAdapter.Demo.Schemas.PropertySchema”/>
</BtsVariablePropertyMapping>

Variable mapping technique is very powerful.  You define any custom variable (or place holder) in your Url, in our case {handle} and map that variable to any context property with property name and namespace as shown above.

For our example, I used a simple technique. Defined a simple Xml Schema as shown below and promoted them in Visual Studio.  Then you just need to use XmlDisassembler pipeline in your receive port configuration to get handle get promoted to the context property.

twitter schema

 

GET Request with Authentication

So far in both of our samples the REST end point was open ended without any security. The Url was HTTP (rather than HTTPS) and no user authentication required. But that will not be the case in lot of scenarios. In this example, let’s take a look at using Bing Search API which requires both.

  • Visit https://datamarket.azure.com/dataset/bing/searchweb
  • Navigate to the bottom of the page and sign up for free 5000 transaction/month offer using your LiveID
  • You need to grab you user name and password from “My Account” section. Your user name will be under Account Information\Customer ID and your password will be under Account Keys\Default

Once you have setup your Bing Search account, then configuring your WCF-WebHttp adapter is fairly straight forward as shown in the below picture.

bing configuration 1

You will need to pay attention to back slash characters, make sure you only add it either at end of the address or beginning of the URL mapping, else you’ll see 404 errors.

To configure the security credentials, go to Security tab and make sure you selected “Transport”, since the communication is going to be via HTTPS. For authentication, select “Basic” and provide the username and password as shown below.

bing configuration 2

bing configuration 3

That’s it, I ended up as a very length post, but hope it will help people to kick start their REST journey. Stay tuned until our next post.

You can download the source code and binding files from the code gallery.


Are you using BizTalk Server in your organisation, why not take a look at BizTalk360 and see how it can help you to administer, operate and monitor your environments in an efficient way.

About Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments.

  • TJ Amas

    Thanks for the clear and concise example! I have no doubt it’ll come in useful in the very near future :-)

  • Keen

    Is it possible to consume and send JSON content with the out of the box WebHttp Adapter?

  • preps2

    Awesome !

  • raj

    Excellent.
    BizTalk 2010 R2 ? can you provide the link to download the R2 version please.

  • Venky

    Hi Saravana kumar,

    Am a newbie to Biztalk, I have read through some book to get the overall idea, but that’s not fit for my realtime solutions to my project. my scope is defined below, Kindly advise.

    am crunching data from one db to many db with several logic, every morning using SQL Jobs (SSIS are also included). but am finding a large amount of usage of servers for the specific time. as the source table is same, is biztalk can help me to integrate with other DBs when ever a new data comes into my source table using orchestration?

    Thanks in advance

  • Ari Biswas

    Hi Saravana,

    nice article. I’m involved in integration project that where BizTalk integrates with lots of REST end point. But I’m struggling to find proper error handling. It seems when REST service returns any error the adopter is ignoring the body of the response message. As a result all error derails are getting lost. I was wondering if you have tried any standard REST error handling with Biztalk 2010R2.

    i would love to know your view on error handling in REST and BizTalk integration.

    regards

    Ari

  • Vijay Sharma

    Hi Saravana,

    I have downloaded sample source code and trying to get WCF-WebHttp adapter working but it is ending error.

    As advised created a service account with Bing api and can browse this url directly by providing login credentails.

    This is URI

    https://api.datamarket.azure.com/Bing/SearchWeb

    http Method and URL Mapping

    Error Message:

    The Messaging engine failed to process a message submitted by adapter:WCF-WebHttp Source URL:https://api.datamarket.azure.com/Bing/SearchWeb. Details:The published message could not be routed because no subscribers were found. This error occurs if the subscribing orchestration or send port has not been enlisted, or if some of the message properties necessary for subscription evaluation have not been promoted. Please use the Biztalk Administration console to troubleshoot this failure.

    please advice what is missing piece in this?

  • Gert Vloo

    Hi Saravana,

    I want to create a message-only solution with the WCF-WebHttp. Using promoted properties in the URI. If I do that then the dynamic part does not get executed because I don’t have a BTS.Operation. Do you have a solution for this problem without using a custom pipeline?

    regards
    Gert