Control and get better insight of your Event Log data using BizTalk360 Advanced Event Viewer

Published on : Aug 21, 2018

Category : BizTalk360 Update

BizTalk360

Author

Are you using the Event Log for diagnosing problems and challenges faced while performing the operation in BizTalk server administration console? Then you might face the difficulty, on having multiple BizTalk / SQL Server users who need to look Event Logs on each server when there is a problem. To access the Event Log, every user should have administration permission to access the physical server, which is not the ideal solution for the business needs.

What is the purpose of the Event Log on a BizTalk Server?

The Event Log holds information related to different aspects of, amongst others, BizTalk Server. The Windows Event Logs are used by the BizTalk admins, mostly for diagnosing problems. When they face any issue, the first step they try to diagnose the problem or faults in the BizTalk Admin console itself. If they could not achieve a solution, then they will view the Event Viewer for detailed information on such cases like errors, messed up environments and failure of message processing.

Filter for log noise reduction during Event Log collection

The BizTalk360 Advanced Event Viewer which collects the events that are related to BizTalk and SQL sources. These sources can be configured in Settings–>Advanced Event Viewer, as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Source configuration screen

The Event Logs are collected based on the Event Log Categories and the administrators can also add categories as per their need. The data collection can be performed based on the environment selection as below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - Server configuration screen

 

Once, after enabling the “Enable Advanced Event Viewer data collection”, that will display all the available BizTalk and SQL server of the specific environment. The BizTalk admins or the people using BizTalk server can select the servers based on their needs. Based on the configurations, the subservice “Eventlog Collection” of the BizTalk360 Monitoring NT service will start to collect Event Log data and store the information in the BizTalk360 database.

This approach will seamlessly solve the problems as we highlighted in this article. The BizTalk administrators no more need physical access to each BizTalk or SQL server to diagnose the issue. This approach will avoid data collision, improves the performance and makes the user interface clear.

Let’s have a look at few scenarios where using the Advanced Event Viewer is beneficial over the Windows Event Viewer.

Scenario 1: If there is any failure of file transaction at an application level, the user can view the relevant event log information in the single user interface. This will avoid context switching and reduces the time to diagnose the issue.

Diagnosing issue made easier than Windows Event Viewer

In real time business transactions using BizTalk can cause any kind of failure.

Scenario 2: When you try to run a BizTalk application after deploying an orchestration with custom components. In case, the BizTalk orchestration engine cannot locate the custom component, that means you will get below error.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - A sample error in windows Event viewer

It is easy to query the Event Log details if BizTalk server is installed in a standalone environment. However, in case of multiple BizTalk servers, the user needs to login to each server to find the Event Log information and it is a tedious process to search the exact issue on ‘n’ number of results. So, it will eat up lots of time on finding the exact result.

In the BizTalk360 Advanced Event Viewer, diagnosing the problem or issue, in a single user interface related to the event “application” of BizTalk and SQL search, is made much easier than with the Windows Event Viewer.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Operation capability

With the Advanced Event Viewer, the BizTalk admins (and other people) can search the results using rich query builder tools, based on the different parameters such as the Event Log, Type, Source, Event, Category, User, Computer, Timestamp, and Message. One or more filters can be added, which will be useful to get an exact result and the user can view the related information the same as in the Event Viewer.

This approach will fasten the search of the Event Log information from the different servers within a stipulated time which obviously saves time.

Archiving distributed events on Centralized Location

BizTalk360 help to get the needed Event Log data from the Event Logs on different servers and shows the information in a simple, intuitive user interface. However, it is practically impossible to manually keep an eye on the Event Log data throughout the day, as they are constantly updated with the latest log data.

In BizTalk360, we solve this problem by downloading the data into Excel Format. The BizTalk admins can download the file by using the Export to Excel feature, that will download the grid data in Excel format and they can store the Event Log Data for further investigation, as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - Download screen

It is possible to monitor Event Log Data!

The Windows Event Logs hold all the important log information from sources like SQL Server, IIS, BizTalk infrastructure and run time problems. As the Windows Event Log contains that much data, it is nearly impossible to monitor the Event Log manually. Luckily, through BizTalk360, the BizTalk admins can easily monitor the BizTalk and SQL server logs by configuring the Event Log monitoring.

Scenario 3: The BizTalk admins can customize the alert in such a way that they must be notified when there has been a sequence of start/stops of the host instances in the past 30 minutes. Without being alerted, you may not know the frequent start/stops of the host instances and this could impact the performance of the environment.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - Advanced data monitoring screen

By mapping the Event Log to the Alarm, the administrator will get notified when the Event meets the condition. The alert can be notified through the required sources like Email and notification Channels.

The administrators can also log this Event Log alert in the Event Viewer itself, this is possible by enabling the Event Log option in alarm creation as below

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- General Monitoring event log flag

This will log the Event log alert in the Event Viewer when the alert exceeds the Host Start/Stop error/warning condition as below

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - Event log data in Operation screen

Create Log rules with an intuitive interface using the Event Log Data Monitoring capability

The primary purpose of auditing Event Log for BizTalk administrator is to discover the events of interest, whether they provide general information or call for further investigation. BizTalk360 will help to get the needed log data from multiple servers and shows the information in a simple, intuitive user interface. However, it is practically impossible to manually keep up the data throughout the day, as they are constantly updated with the latest log data. BizTalk360 also solved this by giving the real-time alert features on Event Log Data Monitoring.

Scenario 4: BizTalk Administrator wants to monitor different Events from multiple servers. For example, they might want to monitor the ESB from BizTalk Server at the same time they want to ensure the performance of the SQL Server & ENTSSO events from a separate SSO server and they also need to monitor the IIS from multiple BizTalk servers. In real-time, there is no option to monitor it with a single tool. It is hard to predict any issues and it is time-consuming for BizTalk admins to constantly monitor the Windows Event Log manually.

BizTalk solves this problem by bringing the heterogeneous data in the single intuitive user interface. On the top of that, it provides a powerful capability of getting real-time notifications through email or SMS when an event occurs on the specific frequency by setting additional threshold conditions. The admins can correlate two or more event calls for the further investigation by setting the conditions as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Advanced Data Monitoring configurations screen

It will be useful to query against different Event Sources of multiple servers of a single BizTalk Environment. The admins can include or exclude the events that pertain to your environment. Also, the admins can get the detailed notification on Event Logs by selecting the ‘Send Event Log in notification’.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Data Monitoring Filter screen

This will send the detailed information on every event source in Email which will look like below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - A sample Email alert

BizTalk Admins can choose the Notification Channel on the alarm creation and they can configure the Event Log as previously mentioned to get notified when any violation happened on a specific frequency. On the Data Monitoring Dashboard, the admins can visualize the triggered alert results based on the day view as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Dashboard View

It also easy to maintain, once after scheduling Event Log Data Monitoring; when you disable AEV for the environment, it will stop collecting Event Log data. So, the admins can enable collecting whenever they need it.

Is there any option to manage the Event Log data?

Managing the Windows Event Log is not simple. Administrators can manage or clean up the data through PowerShell script or by creating a custom log to log application errors. The BizTalk360 Monitoring Service collects the data based on the Event Log and Source configuration and stores in a separate database table in the BizTalk360 database.

In real-time business, BizTalk server will collect the log data for each event which happens during BizTalk operation. This can hugely increase the size of the database. Luckily, the administrator can set the purging for Event Log data in the BizTalk360 database as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Purge settings screen

Once after configuring the Purge Policy for Event Log data, it will clean up the data for every specific days/month/hour.

This approach will save the time and clean the old data easily, which will improve the overall performance of the BizTalk360 database. This also solves the problem of searching the current data with messed up old data.

Can we visualize the performance of Event Log?

 In BizTalk360 the BizTalk Administrators can indeed visualize the performance of Event Log data at a specific time.

Scenario 5: In any business environment, the performance metrics at a specific time will be critically important to take business decisions. To visualize the Event Log data the administrators can add the Event Log Data Count widget in the Reporting Dashboard on BizTalk Analytics.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Analytics Event Log widget screen

The Event Log reporting widget will show the exact count of Event Log data based on error, warning and critical errors for the available sources. The Event Log data widget will collect the data based on the scheduled time.

Leveraging the API Documentation Capability for Event Log

In BizTalk360, Event Log data in the API Documentation provides you with a very powerful way to integrate data and actions into your own tools or programs. Many of the BizTalk Administrators do not come from a development background and perceive an API as something for the exclusive use of BizTalk admins. This is not the case, and a mature and well-designed security product API can be of great value to other people, regardless of their development background.

Within BizTalk360, you can use the Event Log API Documentation to demonstrate broad functionality, primarily in the areas of:

  • user and organization account management
  • sample management
  • sample analysis data collection
  • threat intelligence harvesting

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - API Documentation screen

Of course, an easy way to use an API is to use a product that leverages that API.  Any RESTful API can be operated by anyone who can construct a web URI. the Event Log API documentation is also useful to create a custom widget using the JSON Response as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - A Sample JSON Response screen

In an organization, you could easily tailor that to a list of IPs (or domains, or URLs, file hashes, etc) that had been seen in the analysis of only your own organization’s submitted samples with the help of this API’s.

Conclusion

To collect and process log data in real time, regardless of the volume of log data and the number of devices in the network, organizations need a robust log collection mechanism. BizTalk360 solves this problem by providing the Advanced Event Viewer functionality, which enables BizTalk Administrator to decipher and analyse any log data regard of its source configured. The collected log data is centrally aggregated and presented in a single console for log sources across locations. To summarise here are the Key features of BizTalk360 Event Log

Event Log Maintenance: Maintenance is also very simple when you disable AEV for the environment, it will stop collecting Event Log data. Manages data growth by Providing BizTalk360 purge policy.

Privileged Multiple Server Monitoring: The BizTalk Admins can monitor multiple BizTalk and SQL servers on a single user interface.

Server event trend analysis: It also protects security threats by collecting, analysing, reviewing and archiving event log. Through this, the BizTalk admins can get a comprehensive view of all the actions that is happening in the BizTalk server, SQL server, ENTSSO server etc..,

Real-time alerts: Receive instant alerts on unauthorized access to confidential data or even to the database server itself.

Intuitive dashboard: Drill down to the raw log level and view collected log content to facilitate simplified root cause analysis.

Out-of-the-box reports: Perform audits with pre-packaged reports that provide detailed information on Event Log Data through Event Log Count widgets.