Load Balancing Managed Services in Azure

September 22, 2016

azure load balancing

There is always a point in time where your application has reached its peak and you need to scale it, or you are planning for a big release and you want to ensure that on the big day your users are not greeted with an error because your application is not able to cope with the demand. Whatever the reason, it is important to ensure that your application is scalable and you can efficiently process the demand of your users. The first and most obvious step is to increase the power of your machines. However, there are other techniques which might be more cost effective depending on the requirement and deliver the same amount of performance boost, which you are looking for.

With the release of BizTalk360 version 7.1, we have also released another product BizTalk360 Managed Services, which allows consultants to access multiple customer environments from a central location. If you have never heard of managed services, it is because we have never pushed it out to a wider audience. Now we have decided to change it and the first step was to improve the reliability and performance of the product.

You might have seen the recent article from our founder/CTO Saravana Kumar on the uncontrolled cost with moving to cloud, where he described various challenges with controlling the costs of your resources in the cloud and the disasters awaiting if you do not manage them correctly. The improvement process of managed services involved moving the application to a new subscription for better control of running costs. Since we decided to move everything to new subscription and create new VM, we had to do it properly. We wanted to ensure that our customers receive the best service possible, which means that can always access the resources they need and they can do it smoothly without any performance issues. We decided to host Managed Services on two machines in an availability set and attach them to a load balancer.

Therefore, I decided to play around and create some kind of POC to see how the application behaves in a load balanced environment. I did some research on how to load balance in Microsoft Azure and the first thing that came out was the standard Level 4 Azure Load Balancer. The instructions on how to do it were clear and easy to follow, so I was quickly up and running fully balanced solution (You have to love the Cloud!). However, I was asked if we can offload the SSL to the current load balancer and the answer was No.

This time, I decided to spend more time on my research and identify all the possible solutions available in Azure in terms of Load Balancing. The current three solutions available as I write this blog are : Level 4 Azure Load Balancer, Level 7 Application Gateway, DNS Level Traffic Manager and some third party solutions. These options are suited for various needs or they can be used together to create more advanced Load Balancing architecture. The table below represent some of the differences between these three solutions and you can find more in documentation articles on Microsoft website.


Azure Load Balancer

Application Gateway

Traffic Manager

Technology Transport level (level 4) Application level (level 7) DNS level
Application protocols supported Any HTTP and HTTPS Any (An HTTP/S endpoint is required for endpoint monitoring)
Endpoints Azure VMs and Cloud Services role instances Any Azure Internal IP address or public internet IP address Azure VMs, Cloud Services, Azure Web Apps and external endpoints
Vnet support Can be used for both Internet facing and internal (Vnet) applications Can be used for both Internet facing and internal (Vnet) applications Only supports Internet-facing applications
Endpoint Monitoring supported via probes supported via probes supported via HTTP/HTTPS GET

It was clear to me that the best solution for our new architecture is Application Gateway as it is the only Load Balancer with SSL Offloading and can achieve exactly what we require. Perfect, I know what I need to do, now how does it work and how do we set it up. The only available instructions for creating Application Gateway are explained in PowerShell and there are several steps as well. I like PowerShell, but for this kind of task I prefer to have a user friendly experience and unfortunately for me, there are no instructions… anywhere. It took me some time to get it all up and running and with accordance to requirements. Therefore, to save you some time, here are all the steps to ensure you get it right the first time.

Before you begin, I encourage you to create some kind of a diagram that will help you through the process. The diagram below is what we have decided to create in Azure and in the instructions below it, you will find how to create it.

Application Gateway - Architecture Diagram

Step 1

Begin by login in to your Azure Portal and from the left hand side menu choose Application Gateway. If you cannot see it, then go to Browse and search for it. Select Add from the top bar and fill in the details as below:


Ensure to press the star button during browsing to pin it to your navigation bar.

Application Gateway - Configuration Step 1


SKU Size – It is the size of the Application Gateway which defines its performance read more
Instance Count – Defines the maximum number of instances of the backend pool

Step 2

In the next step you need to point the portal to the virtual network and public IP address that you want to use. Then you need to set if you want to use the application gateway for SSL Offloading. If you do, then choose HTTPS and browse your computer for the certificate file in the .pfx format. Next, provide some friendly name and password for the certificate. Make sure that the password you have entered is correct, if you enter a wrong password you will have to start from scratch.


Ensure that you have the certificate that you want to use and not a temporary one. Any changes to the certificate will have to be done through PowerShell as not all features of the Application Gateway are editable from the Portal.

Application Gateway - Configuration Step 2

Step 3

The process takes around 15 min to complete, so we can create something that we can load balance. While looking at the diagram above, we will need to create two Virtual Machines in an availability set connected to the same virtual network. When we put the machines in the availability set, each machine will be placed into its own update domain and fault domain, which will ensure that they do not share common power source and network switch.


Depending on your requirements in might be useful to assign both VMs to one network security group in order to save time on creating duplicate rules to allow Application Gateway backend to connect to the VM.


Application Gateway - Configuration Step 3a


Application Gateway - Configuration Step 3b

Connect to your VMs through RDP and install Web IIS Role on both of your machines. Then go to the default directory on one of the VMs and edit the image that gets displayed on the main page, so that you can test the load balancer once you are done configuring.

Step 4

At this point you should have two VMs and Application Gateway running and ready for the next steps of configuration. Now we will look how do we actually tell our Application Gateway what it should Load Balance. The first step is to copy the public IP address from your VMs to a notepad or any easily accessible place. Then open your application gateway and choose backend pool. Choose the default backend pool and enter the IP addresses of your VMs.


During the initial set-up the portal did not allow to provide names for some components. Unfortunately, they are not editable from the portal, but it can be achieved from PowerShell if you prefer to use custom names.

Application Gateway - Configuration Step 4

Step 5

At this point if you try to access the application gateway IP address or the DNS you will most likely face 502 error saying “Web server received an invalid response while acting as a gateway or proxy server.” or Connection timed out “This site can’t be reached. c3d3a3c3-2681-48e5-91db-883728c45dd9.cloudapp.net took too long to respond.” The 502 error is shown when you access your application gateway through https and the network security group of your VMs does not allow connection from that IP. If you are getting timed out, it is because we set up the application gateway to listen to port 443 only, we need to create new listener that listens for port 80.

To create new Listener choose Listeners from your Application Gateway Settings menu and fill in the details of your new Listener.
Application Gateway - Configuration Step 5a

Next our listener requires a Rule, which redirects user requests to specific backend pool. The default first rule in the portal is round-robin, therefore, all the requests will go to random instance of the backend pool. In the PowerShell we can alter that configuration and apply URL routing rule, which allows to redirect users to a different backend pool based on the URL, which will load balance on different machines. For example, http://contoso.com/user will be redirected to backend pool 1 which load balances between two VMs that host users application. Then http://contoso/admin will redirect the request to backend pool 2 which will load balance between two other VMs that host admin application.

Application Gateway - Configuration Step 5b

Lastly, we need to allow the connection between the Application Gateway and the VMs. First, copy the public IP address of the application gateway and navigate to the resource group. Then select network security group of your VM and choose Inbound Rules, create new rule and fill in the details as below.

Application Gateway - Configuration Step 5c

If you did follow the tip from Step 3 you will only have to do this step once. If you did not, then you must create inbound rule for every VM.

That is it, you should now have a fully working Load Balanced solution. If you are still experiencing 502 you can try to restart the VMs as the new NSG inbound rule might take some time to apply. If you want to directly connect to your VM make sure that the source of your inbound rule is set to Any or choose CIDR and enter public IP address of your machine.

To test if you are getting redirected correctly to a different machine navigate to your application gateway DNS and then open new instance of your browser in incognito mode and enter the same DNS again. You should be redirected to a different machine, if you are not trying to refresh or connect using different browser.

Author: Daniel Szweda

Daniel has joined BizTalk360 UK team as a graduate of computing and games development course. Right now he is our Software Developer responsible for development of new and existing features of BizTalk360 and BizTalk360 Cloud. He is also assisting our customers when they have any questions or issues while using the product.

Get full control of your BizTalk Environments

Over 500 customers across 30+ countries depend on BizTalk360

Start your 14 days Free Trial

Back to Top