Applications and processes often rely on external resources, which can be expensive to retrieve or obtain. Therefore, modern applications often employ cache strategies to save these external resources in the local file system and achieve improved performance. In some other cases, you might have an application that keeps creating random temporary files to facilitate certain system requirements which cannot be achieved otherwise. However, as every coin has two sides, these caches or temporary files, in the long run, can build up in the system and occupy large disk space, which could have adverse effects and result in volume deficiency in the system. Implementing a scheduled cleanup of the cache folder by removing those files that are older than a certain age can become necessary in your ETL process.

SharePoint Custom App (or Add-in app) option is a popular OAuth authentication method to connect to the SharePoint API. In our recent version, we have added support for Custom App in KingswaySoft SharePoint connection manager. In this blog post, we will go over its prerequisites, and how to configure it within our connection manager.

In the general integration practice, it is a very common use case to monitor a folder path for any file changes, and as soon as a file with certain criteria lands in the folder or gets modified, a process should kick off or otherwise, an email notification should be sent via email. With the new Premium File Watcher Task added to our SSIS Productivity Pack in our most recent v23.1 release, this can be easily achievable in an automated fashion. In this blog post, we will demonstrate the design and configurations required to watch a file path on a continuous basis and send an email notification when a specific change happens in it.

In some unique cases, we may receive an XML document that needs to be processed in the ETL process, which might contain general XML content but includes some embedded XML content made of escaped characters. The integration requires the embedded XML content to be fully parsed and processed. In this blog post, we will show you how to properly handle and transform mixed XML content with escaped characters to ensure the successful parsing of API responses and files.

Enumeration fields or optionsets need to be handled well in a migration scenario. And it is easy to not have an option in a target system, which was a legacy option in your Source data set. One such example would be currency options. Currency abbreviations can easily be missed, incomplete, or need to be changed to better reflect the destination system. In this business case, we are working with an old database system and have now been tasked to create the contact information in HubSpot.

While migrating from CRM to CRM, or any other Source system to CRM, in an ETL, records will need to be associated with users. And since the Source and Target Systems are different, there may not be a guarantee that some users may still exist in the organization. And in such cases, such users become either unlicensed, or stub users. These type of users have restrictions when it comes to being set as an owner of records, and in this blog post, we will see how this can be handled efficiently.

In SSIS, the sensitive credentials have to be handled in a special way to ensure that they get preserved and decrypted while the packages run in an unattended way. There are a few means to do this, and in this blog post, we will discuss some popular ways this can be done.

With the recent news of SOAP basic authentication deprecation, it becomes imperative to start using the OAuth authentication for your SharePoint integration process. In response to this deprecation, we have recently added support for OAuth in the latest (v22.1) of our SharePoint toolkit, which offers a paralleled feature set as the SOAP service endpoint. It also includes some enhanced capabilities that were not possible using the SOAP service endpoint. This blog post will demonstrate the process involved in order to start using the OAuth authentication type in your integration.