Why did we build FTP/SFTP Monitoring for BizTalk Server?

|  Posted: April 11, 2018  |  Categories: BizTalk360

This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360”.

BizTalk Server FTPS-FTP-SFTP Monitoring

Why do we need this feature?

In the day to day activities of a BizTalk administrator, you might come across integrations where FTP sites are used for receiving and transmitting messages. FTP sites are often used for cross-platform integrations. For example, when you have an SAP system on Unix that has to be integrated, via BizTalk Server, with other systems, you might use FTP for receiving and transmitting of messages.

SFTP & FTPS are just the secured version of FTP with advanced transport encryption mechanisms, so your end-to-end data transmission is secure and safe.

To keep the business process going, it can be of vital importance that the FTP/SFTP sites are online and the messages are being picked up. So, when a BizTalk administrator needs to be constantly aware whether the FTP/SFTP sites are online and working properly, the administrator needs to monitor the sites and the activities which take place on these sites.

What are the current challenges?

BizTalk Server offers no monitoring capabilities, not for Receive Locations / Send Ports and also not for endpoints like FTP, SFTP and FTPS sites. So, using just the out-of-the-box features of BizTalk Server, a BizTalk administrator will have to manually check whether the FTP sites are online and whether all (appropriate) files are being picked up for further processing.

Manual monitoring

This kind of manual monitoring can be quite cumbersome and time-consuming. The administrator will probably use multiple pieces of software to be able to perform these tasks. Think of for example the BizTalk Administration console to check whether the Receive Locations/Send Ports are up and some FTP client to check whether files are being picked up.
It is obvious that this is not a very efficient scenario, which could easily be automated by setting up monitoring.

Maintaining scripts for monitoring FTP sites

To reduce their workload, we experience that BizTalk administrators are creating their own scripts to monitor FTP sites and all kind of other resources. Although this kind of scripts certainly can be of help, we still think this does not fully solve the problem.

For example, often these kinds of scripts need maintenance when FTP sites need to be added, changed or deleted from monitoring. This kind of tasks can be easily forgotten.

Also from a knowledge transfer perspective, it’s easy to forget to update new colleagues about the existence of this kind of scripts, as they will probably be installed on some (monitoring) server.

Another challenge with solving this kind of problems with scripts is that not each administrator is capable to write this kind of scripts, which makes knowledge transfer even harder.

To keep the overview, we think that it is easier to use software, like BizTalk360, to have everything in one easily accessible place, with good visibility of all the features/capabilities, fine-grained security/auditing and without the need to maintain custom scripts etc..

How BizTalk360 solves this problem?

With BizTalk360, we make monitoring of FTP/SFTP/FTPS sites a lot easier. For a very long time, the product offers monitoring of Receive Locations and Send Ports, but for some time now, BizTalk360 also offers to monitor of the physical FTP/SFTP/FTPS endpoints.

We wanted to make setting up this kind of endpoint monitoring as seamless as possible and therefore we simply show all the ports in the current BizTalk group which make use of the FTP/SFTP/FTPS adapter.

In BizTalk360, you can find FTP monitoring under Monitoring => Manage Mapping => File Locations (File, FTP, SFTP).

Next, you can set up monitoring rules based on File Count and Directory Size and have BizTalk360 send Warning or Error notifications through the notification channels which are configured on the associated alarm.

A fully monitored FTP endpoint might look like shown below.

Of course, besides the greater-than-or-equals operator, also other common operators are available.

Conclusion

As a final point, we see from time to time that administration teams maintain a administrators handbook, which contains all the tasks a (BizTalk) administrator should take care of. We think that by using software like BizTalk360, we can reduce the number of pages in such handbooks, as the kind of scripts we mentioned no more have to be described in that kind of books.

This description could be replaced by, for example, a general guideline on how FTP sites should become monitored and the monitoring rules with BizTalk360.

As a result, we hope to make the work of BizTalk administrators a bit easier so the team can focus on the more exciting parts of the job of BizTalk administrators, instead of constantly having to update their handbooks.

So, we think that we make the day to day life of a BizTalk administrator, who needs to monitor the well-being of FTP sites, a little bit easier by bringing this feature.

If you want to read more on FTP/SFTP monitoring in much more detail, you can check the following article:

FTP, FTPS, SFTP Location monitoring

BizTalk360 also offers an advanced monitoring capability called “Data Monitoring” which allows monitoring the traffic/volume of messages going through the ports for a given period, ex: expected 50 PO orders from our partner via FTP/SFTP. Please check out this article. Introducing BizTalk Server Data Monitoring in BizTalk360

Get started with a Free Trial today!

Download and try BizTalk360 on your own environments free for 30 days. Installation will not take more than 5-10 minutes.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Back to Top