SSIS Integration Toolkit for Team Foundation Server

Installation

To install and run SSIS Integration Toolkit for Team Foundation Server, your system must have the following components installed.

• A supported SSIS design-time or run-time, which can be one of the following:
  • Azure-SSIS Integration Runtime
  • SSIS 2022
  • SSIS 2019
  • SSIS 2017
  • SSIS 2016
  • SSIS 2014
  • SSIS 2012

  For SSIS run-time, the installation should be done by using the corresponding SQL Server installation media, and you must select the "Integration Services" component during the installation, as shown below:

  

  Note that when using SQL Server 2014, a cumulative update is required (a recent service pack, such as one of the following, is preferred) in order to run our software during runtime.

  • SQL Server 2014 Service Pack 2: https://www.microsoft.com/download/details.aspx?id=53168
  • SQL Server 2014 Service Pack 1: https://www.microsoft.com/download/details.aspx?id=46694

  For SSIS design-time, you should be installing the version of SSDT (SQL Server Data Tools) or BIDS (Business Intelligence Development Studio) that aligns with the SQL Server version that you plan to use for your final deployment (the runtime).

  Target Server	Design Time
  Azure-SSIS IR	Any one of the following: SSDT for Visual Studio 2022 SSDT for Visual Studio 2019 SSDT for Visual Studio 2017
  SSIS 2022	Any one of the following: SSDT for Visual Studio 2022 SSDT for Visual Studio 2019
  SSIS 2019	Any one of the following: SSDT for Visual Studio 2019 SSDT for Visual Studio 2017 SSDT for Visual Studio 2015
  SSIS 2017	Any one of the following: SSDT for Visual Studio 2019 SSDT for Visual Studio 2017 SSDT for Visual Studio 2015
  SSIS 2016	Any one of the following: SSDT for Visual Studio 2019 SSDT for Visual Studio 2017 SSDT for Visual Studio 2015
  SSIS 2014	Any one of the following: SSDT for Visual Studio 2019 Packages created using SSDT 2019 need to have their project's TargetServerVersion setting set to "SQL Server 2014" in order to work with SSIS 2014. SSDT for Visual Studio 2017 Packages created using SSDT 2017 need to have their project's TargetServerVersion setting set to "SQL Server 2014" in order to work with SSIS 2014. SSDT for Visual Studio 2015 Packages created using SSDT 2015 need to have their project's TargetServerVersion setting set to "SQL Server 2014" in order to work with SSIS 2014. SSDT-BI for Visual Studio 2013
  SSIS 2012	Any one of the following: SSDT-BI for Visual Studio 2012 (recommended; most reliable) SSDT for Visual Studio 2019 (not recommended due to potential compatibility issues with ISV solutions) Packages created using SSDT 2019 need to have their project's TargetServerVersion setting set to "SQL Server 2012" in order to work with SSIS 2012. SSDT for Visual Studio 2017 (not recommended due to potential compatibility issues with ISV solutions) Packages created using SSDT 2017 need to have their project's TargetServerVersion setting set to "SQL Server 2012" in order to work with SSIS 2012. SSDT for Visual Studio 2015 Packages created using SSDT 2015 need to have their project's TargetServerVersion setting set to "SQL Server 2012" in order to work with SSIS 2012.

• A Windows Operating System

  Windows operating system requirement largely depends on the version of SSIS runtime or design-time selected. We don't have any additional requirements in terms of the Windows operating system as long as it satisfies the minimum requirements of SSIS runtime or design-time. The general guideline is the newer the operating system is, the better. In summary, our software should work for the following Windows operating systems.

  • For desktop systems (mostly for development workstations)
    • We generally recommend Windows 10, version 1507 or greater.
    • Windows 8.1, 8, or 7 should work fine with our software installation which supports most SSDT versions and SQL Server 2016 or below, but we highly recommend you upgrade to Windows 10 because of their support status with Microsoft.
  • For server systems (mostly for runtime deployments)
    
    • We generally recommend Windows Server 2016 or greater (Including Windows Server 2019 and potentially future Windows Server versions).
    • Windows Server 2012 or Windows Server 2012 R2 should work fine for SSIS versions up to 2016.
      

• A .NET Framework

• • Our software requires the installation of .NET Framework 4.5.2 or above

When you have confirmed that your system satisfies the above prerequisites, you can navigate to the KingswaySoft website at https://www.kingswaysoft.com to download the installation package.

After you have downloaded the package, you can install the software by following the installation wizard.

Working with SSIS Toolbox

SSIS Toolbox is the first place that you will be looking for our components to be added to your ETL process during the design time.

If you are working with SSIS 2012 or later, SSIS Toolbox should be automatically available during the design time once you have an SSIS package opened provided that you have our software installed properly. If you do not see the SSIS Toolbox, click either the SSIS Toolbox menu option under the SSIS menu or the SSIS Toolbox icon in the design window's upper right corner, as shown below.

Note: When working with SSIS Toolbox, you need to make sure that you are in the right view in order to see the right components. For instance, if you are looking for a data flow component, you need to make sure that you are in the Data Flow view, not the Control Flow view. Visual Studio would show different components depending on the design view that you are currently working with.

Once the data flow components are available in the SSIS Toolbox, you can start your ETL development by dragging them from the toolbox to the Visual Studio design surface.

Using the Team Foundation Connection Manager

SSIS Integration Toolkit Team Foundation Connection Manager is an SSIS connection manager that can be used to establish connections with Team Foundation Server.

Team Foundation Server connection manager allows you to specify how you want to connect to your Team Foundation Server.

To add a Team Foundation Server connection to your SSIS package, right-click in the Connection Managers area, and choose "New Connection..." from the context menu.

You will be prompted with the following window, where you can choose "TeamFoundation (KingswaySoft)" to add the connection.

Team Foundation Server Connection Manager contains the following three pages which configure how to connect to your Team Foundation Server.

• General
• Advanced Settings
• More Info

General page

The Connection page of Team Foundation Connection Manager allows you to specify the general settings of the connection.

Connection API (since v20.1)

The Connection API option allows you to specify the service endpoint that is used to connect to Team Foundation Server. There are two options available:

• REST (Azure DevOps, Team Foundation Server 2015 or above)
• SOAP (Azure DevOps, and all major versions of Team Foundation Server as of the release date)

When you choose to use the REST API, please make sure that you add the following code snippet to your DTExec.exe.config, DtsDebugHost.exe.config files in order for the runtime to work properly.

    <dependentAssembly>
        <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-6.0.0.0" newVersion="9.0.0.0" />
        <codeBase version="9.0.0.0" href="C:\Program Files (x86)\KingswaySoft\SSIS Integration Toolkit for Team Foundation Server\Newtonsoft.Json.dll"/>
      </dependentAssembly>

If it happens that the tool has been installed in a different folder, the above file path should be updated - the change is only required when using the REST API. Without the above changes, you will run into an error like the following when you try to execute an SSIS package that has a Team Foundation Source and/or Destination component which uses a REST connection.

System.MissingMethodException: Method not found: 'Newtonsoft.Json.JsonSerializerSettings System.Net.Http.Formatting.BaseJsonMediaTypeFormatter.get_SerializerSettings()'

It is important to note that DTExec.exe.config, DtsDebugHost.exe.config can be in many different folders in your system, make sure that all files are updated in order to avoid the error in case you might be running the SSIS packages against different runtimes.

Authentication Type

The Authentication Type option allows you to specify the deployment type of your Team Foundation Server and what authentication is used. There are two options available:

• Active Directory (On-Premises)
• Visual Studio Team Services by Alternate Credentials

Service URL

The Service URL option allows you to specify the Service URL which you can use to connect to your Team Foundation Server.

User Name

The User Name option allows you to specify the user account used to connect to Team Foundation Server for your integration purposes.

Password

The Password option allows you to specify the personal access token for the above user in order to be able to connect to Team Foundation Server.

Note: The Password is not included in the connection manager's ConnectionString property by default. This is by design for security reasons, however, you can include it in your ConnectionString if you want to parameterize your connection manager. The format would be Password=myPassword; (make sure you have a semicolon as the last character). It can be anywhere in the ConnectionString.

Domain

The Domain option is used to specify the active directory domain of the Team Foundation Server user account. This option is only available if you have chosen Active Directory (On-Premises) as the Authentication Type.

Use Integrated Authentication

The Use Integrated Authentication option specifies whether you want to use Integration Authentication or provide separate login credentials. Note this option is only available if you have chosen Active Directory (On-Premises) as the Authentication Type.

Timeout (secs)

The Timeout (secs) option allows you to specify the maximum number of seconds that the Team Foundation Server connection manager will wait while trying to establish a connection with TFS.

Test Connection

After you have provided all connection information, you may click the "Test Connection" button to test if you can successfully log in to TFS.

Advanced Settings Page

The Advanced Settings page of Team Foundation Connection Manager allows you to specify some advanced options when connecting to TFS.

Misc Settings

Retry on Intermittent Errors

This is an option designed to help recover from possible intermittent outages or disruption of service so the integration does not have to be stopped because of such temporary issues. Enabling this option will allow service calls to be retried upon certain types of failure. A service call may be retried up to 3 times before an exception is fired. Retries occur after 0 seconds, 15 seconds, and 60 seconds.

Warning: We have designed our retry feature carefully so that the retry should only happen when it is deemed to be safe to do so. However, on some occasions, such retry service calls could result in the creation of duplicate data.

Ignore Certificate Errors

This option can be used to ignore SSL certificate errors when connecting to a server, in case your server is using a temporary SSL certificate.

Warning: Enabling the "Ignore Certificate Errors" option is generally NOT recommended, particularly for production instances. Unless there is a strong reason to believe the connection is secure — such as the network communication occurring solely within an internal infrastructure — this option should be left unchecked for best security.

Note: When this option is enabled, it applies to all HTTP-based SSL connections in the same job process; it is not only limited to these specific connections.

Proxy Server Settings

Proxy Mode

The Proxy Mode option allows you to specify how you want to configure the proxy server setting. There are three options available.

• No Proxy
• Auto-detect (Using system-configured proxy)
• Manual

Proxy Server

Using the Proxy Server option, you can provide a proxy server to connect to Team Foundation Server.

Port

The Port option allows you to specify the port number of the proxy server for the connection.

Username (Proxy Server Authentication)

The Username option (under Proxy Server Authentication) allows you to specify the proxy user account.

Password (Proxy Server Authentication)

The Password option (under Proxy Server Authentication) allows you to specify the proxy user's password.

Note: The Proxy Password is not included in the connection manager's ConnectionString property by default. This is by design for security reasons. However, you can include it in your ConnectionString if you want to parameterize your connection manager. The format would be ProxyPassword=myProxyPassword; (make sure you have a semicolon as the last character). It can be anywhere in the ConnectionString.

More Info Page

The More Info page shows some basic information about the toolkit. On this page, you can find the version information of the toolkit.

Using the Team Foundation Source Component

The Team Foundation Source Component is an SSIS data flow pipeline component that can be used to read/retrieve data from your Team Foundation Server.

The Team Foundation Source Component includes the following two pages to configure how you want to read data from TFS:

• General
• Columns

General Page

The General page of the Team Foundation Source Component allows you to specify the general settings of the component.

Connection Manager

The Team Foundation Source Component requires a TFS connection in order to connect to your Team Foundation Server. The Connection Manager option will show all Team Foundation Server connection managers that have been created in the current SSIS package.

Source Type

The Source Type option allows you to specify which type of TFS query you want to use. There are two available options:

• Predefined Query
• Custom Query

When Custom Query is used, you have the flexibility to specify which columns you want to retrieve from Team Foundation Server, for more information on how to build your TFS queries, visit TFS Documentation.

Project

The Project option allows you to specify which TFS project is used.

Output Timezone

The Output Timezone option specifies how datetime values are retrieved. There are 2 options available:

• UTC
• Adjust to Local System Timezone

Query

The Query option is only available Predefined Query has been chosen as the Source Type. When you click the drop-down button of this option, the toolkit will automatically retrieve the list of all available TFS queries, so that you can pick one.

Workitem Detail Types

The Workitem Detail Types option allows you to specify whether to retrieve Attachments,Comments, or/and Links. Once this function is enabled, additional outputs will become available to use for the downstream components.

Refresh Component Button

By clicking this button, the component will retrieve the latest metadata from the Team Foundation Server and update each field. After clicking this button, you will receive the following screen once the refresh is done.

Columns Page

The Columns page of the Team Foundation Source Component shows you all available columns from the query that you have defined on the General page.

On the top left of the grid, you can see a checkbox, which can be used to toggle the selection of all available fields. This is a productive way to check or uncheck all available fields.

The Columns Page grid consists of:

• Field Name: Columns that will be retrieved from Team Foundation Server.
• Data Type: The data type of this field.

Note: As a general best practice, only select the TFS fields that are needed for the downstream pipeline components. This would not only speed up TFS queries but also save resources for the SSIS engine.

Using the Team Foundation Destination Component

The Team Foundation Destination Component is an SSIS data flow pipeline component that can be used to write data to your TFS instance. You may Create, Update, Delete, or Upsert TFS records using the Team Foundation Destination Component.

The Team Foundation Destination Component includes the following three pages to configure how SSIS writes data to TFS.

• General
• Columns
• Error Handling

General Page

The General page allows you to specify the general settings of the destination component.

Connection Manager

Team Foundation Destination Component requires a TFS connection in order to connect to your Team Foundation Server. The Connection Manager option will show all Team Foundation Server connection managers that are available in the current SSIS package.

Action

The Action option specifies how the data should be written to Team Foundation Server. There are 4 action types available.

• Create: Create new record(s) in Team Foundation Server
• Update: Update existing record(s) in Team Foundation Server
• Delete: Delete record(s) from Team Foundation Server
• Upsert: Update an existing record in Team Foundation Server if a matching record can be found. Otherwise, create a new record using the information from upstream pipeline components.

Target Project

The Target Project option specifies which target project to write the data. When the option is clicked, SSIS Integration Toolkit will retrieve a list of all available TFS projects for the selected Team Foundation Server connection.

Destination Entity

The Destination Entity option lists all available work item types for the target project you've specified in the Target Project option.

Refresh Component Button

By clicking this button, the component will retrieve the latest metadata from Team Foundation Server and update each field. After clicking this button, you will receive the following screen once the refresh is done.

Map Unmapped Fields Button

By clicking this button, the component will map any unmapped TFS fields by matching their names with the input columns from upstream components. This is useful when your source component has recently added more columns, in which case you can use this button to automatically establish the association between input columns and unmapped TFS fields.

After clicking this button, you will receive the following message.

Clear All Mappings Button

By clicking this button, the component will reset all your mappings in the destination component.

Columns Page

The Columns page of the Team Foundation Destination Component allows you to map input columns to TFS fields.

On the Columns page, you will see a grid that contains four columns as below.

• Input Column: You can select an input column from the upstream component for the corresponding TFS field.
• Destination Field: The TFS field that you are writing data to.
• Data Type: The target tfsfield's data type.
• Unmap: This column can be used to unmap the field from the upstream input column, or otherwise it can be used to map the field to an upstream input column by matching its name if the field is not currently mapped.

Error Handling Page

The Error Handling page allows you to specify how errors should be handled when they happen.

There are three options available.

1. Fail on error
2. Redirect rows to error output
3. Ignore error

When the Redirect rows to error output option is selected, you can see the following columns in the Error Output:

• ErrorCode: Contains the error code that is reported by Team Foundation Server or the component itself.
• ErrorColumn: Contains the name of the column that is causing the error. Note that this column is not always populated.
• ErrorMessage: Contains the error message that is reported by Team Foundation Server or the component itself.

Note: Use extra caution when selecting the Ignore error option, since the component will remain silent for any errors that have occurred.

On the Error Handling page, there is also an option that can be used to enable or disable the following output fields for the destination component.

• TeamFoundationServer_ID - Contains the newly created TFS record's ID, which you can use to write to log or further process using additional data flow components.

Note: If you don't plan to use any of those fields for any further processing, it is generally recommended to turn them off, so you don't get any warning from SSIS by complaining that those fields are never used, and it should also provide slightly better performance by doing so.

License Manager

SSIS Integration Toolkit comes with a license manager program that helps you manage and activate the product license key to be used for the toolkit. The below information is useful for development workstations and Single Server license management. For Azure-SSIS IR deployments, license management and activation will be handled through the PowerShell script, see Running SSIS Integration Toolkit on the Cloud for further details.

Without a commercial license, SSIS Integration Toolkit will operate under the Developer License which is free to use for development or evaluation purposes. Under the developer license, you can use the software within the development tool (SSDT-BI, BIDS, or Visual Studio).

The only limitation of the free developer license is the inability to run the software outside of the development tool (SSDT-BI, BIDS, or Visual Studio). If you would like to run the software outside the development tool, such as running SSIS packages on a scheduled basis or from a command line, you will need to acquire a license from us.

If you want to test out the functionality by scheduling your SSIS packages, a trial license can be requested. To do so, you can launch the License Manager program, then click the "Change License Key" button, where you can request a free trial license after filling out the necessary Licensee Information.

If you have received a product license key from us after placing an order through our online shopping cart system, you can also click the "Change License Key" button and enter the product license key in order to activate the software to use the fully-featured commercial license.

The Licensee field is where you will specify the company the software is licensed to, you can include your system's machine name for future reference. For example, the Licensee can be "ABC Inc." or "XYZ Corp (SQLSVR-001)." The Contact Email would be the person we reach out to for any license-related notices such as renewal reminders.

To request a free trial license or activate a product license key that you have received, you can use the Web Service option to complete the process by sending the request to our license server directly. An Internet connection is required when the Web Service option is used. This is the option that we recommend.

Alternatively, you can choose the Email option so that the license manager will generate an email for you which you can send to us. The Email option should only be used if your system has no Internet access. It requires manual processing so please expect to wait for 24 to 48 hours before receiving a license file from us. Once you have received the license file from us through email, you can save it to a local file, which you can then install by clicking the "Install License File..." button in License Manager.

If you have acquired a license from us, once the software has been activated, your license manager will be shown as below.

Connection Tier will display the number of distinct connections your license supports per connection type within a 24-hour period at runtime. Multiple connections to the same instance are typically treated as 1 distinct connection (exceptions may apply depending on the nature of the service). Selecting the magnify icon will launch the Runtime Connection Usage Summary window which will display counts on the number of connections made per connection type and when the 24-hour period will reset.

You can see your Maintenance Expiry Date in the License Manager program if you are using a perpetual license. By default, your perpetual license comes with a one-year maintenance and upgrade, which entitles you to use any version of the software released before your Maintenance Expiry Date. You can extend it by entering a new maintenance license key that you have acquired from us.

If your commercial license is a subscription license, you will not see the Maintenance Expiry Date option in the License Manager program, since your subscription license comes with maintenance and upgrade for the entire subscription period. Instead, you will see your license expiry date and a progress bar with the number of days left on your subscription.

Note: You must run the License Manager program under a local administrative account due to the privileges required to write license files to the system.

Contact Us

If you need any further assistance with the toolkit, please don't hesitate to contact us.