Tracking Data Analytics improvements in BizTalk360 v8.3

The Analytics Dashboard is an effective tool provided by BizTalk360 to analyse the performance of your BizTalk environment. Loaded with a bunch of predefined widgets and the ability to customize and create your own tailor made widgets, the Analytics Dashboard provides a unique experience that helps you drill down and identify performance issue of your environments.

Apart from performance data widgets, the Analytics Dashboard also comes loaded with widgets that provide insights into the Tracking Database such as Messaging Performance and Transmission Failure rate widgets.

In this article we describe the challenges some customers faced while collecting analytics from the Tracking database. We addressed these challenges in v8.3 which was released in February 2017.

Analytics Dashboard
Analytics Dashboard

Challenges

Some customer environments that we came across during the support process, had a business requirement which made them retain Tracking data for more than the recommended duration, causing the database to grow exponentially. In some instances, we noticed that not all environments strictly follow the purging policy. In both cases, it’s hard to query the Tracking database for insights.

Initial design of the tracking data widgets was simple; it ran the necessary queries directly against the Tracking database and displayed the results in a graphical form on the widget. A larger database degraded the performance of these queries and it had a direct impact on the user experience of the Analytics dashboard, causing slower widget loading and occasional time outs. Since Analytics widgets offered users the option to view data up to 30 days within the widget, an efficient approach was required to improve the performance and user experience.

Solution

It was after hours of brain storming and multiple Proof of Concept attempts that we concluded that the best way to tackle these challenges, is to create a service that runs in the background. This service will collect the tracking data periodically in small chunks and thereby avoid running expensive queries on the Tracking database. Here only the resulting metrics are collected rather than pulling all the tracking data to BizTalk360 database. This improves the overall performance of the Tracking widgets as they are not querying the Tracking database directly.

One added advantage for this approach is that, say for instance your Tracking database has a purging policy of 5 days, you will be able to view data on Tracking data widgets beyond the set purging period of the Tracking database, as we are collecting these metricises on our database, allowing you to analyse tracked data regardless of the purging policy of the Tracking database.

Advantages

  • Better performance of the Analytics widgets
  • Fine-grained control over the data being collected during tracking data collection
  • Ability to retain Tracked data analytics beyond the purging policy of the Tracking database

Tailoring Tracking data collection to meet your needs

Once we decided to go with the new data collection method, we also saw the potential to provide a customizable interface where user can hand pick the metrics they wanted to collect. A similar user experience to the Performance Data collection in BizTalk360. This allows you to have a fine grained control over the data collection at an environment level.

Tracking Data settings
Tracking Data settings

get biztalk360

Author: Vishnu Balachandran

Vishnu Balachandran is a Software Engineer at BizTalk360 having 2 years experience in Microsoft Technologies. He is passionate about software and hardware aspects of computing and very enthusiastic about learning new technologies. You can reach him on Twitter at @cyberking_vb.

  • Christian

    Don’t you already have a performance issue from tracking? And what about companies that have disabled global tracking, can they still use any analytics? Can they use some BAM functionality perhaps?

    • Hi Christian, tracking is not required for all the capabilities of analytics, we collect data from various sources like performance counters, tracking data and BAM data. If tracking is not enabled then you cannot use certain widgets in BizTalk360 Analytics.

      Performance impact of enabling tracking is scenario based. If your environment is not one of those high throughput environment, but critical data processing happens via BizTalk, then you get so much value out of enabling tracking. You can also fine tune tracking like not enable message body tracking, not enabling orchestration events (which are used for debugging etc) rather than fully turning off tracking.

      BizTalk360 analytics mainly relies on message in/out event in majority of the cases which is pretty much basic tracking.

      What we have done in this release is, if you have a large tracking database say 10GB or more instead of querying the database constantly, we incrementally extract and store only required information in our database using a backend service. So all the Analytics widgets query data from a small dataset and it’s faster.

One Platform Operations, Monitoring and Analytics Software
BizTalk360

microsoft biztalk

Learn more

Over 500 customers across 30+ countries depend on BizTalk360

ServiceBus360

Azure service bus

Learn more

Start managing your Azure Service Bus namespaces in minutes

One Platform - Operations, Monitoring and Analytics Software
BizTalk360

microsoft biztalk

Learn more

Over 500 customers across 30+ countries depend on BizTalk360

One Platform - Operations, Monitoring and Analytics Software
ServiceBus360

Azure service bus

Learn more

Start managing your Azure Service Bus namespaces in minutes

Back to Top