Using the Premium Flat File Source Component

The Premium Flat File Source Component is an SSIS data flow pipeline component that can be used to read / retrieve data from a flat file.

The component includes the following two pages to configure how you want to read data.

  • General
  • Columns

General page

The General page of the Premium Flat File Source Component allows you to specify the general settings of the component. 

Premium Flat File Source Editor

Connection Manager

The Premium Flat File Source Component requires a connection in order to connect to the Flat File. The Connection Manager drop-down will show a list of all connection managers that are available to your current SSIS package.

This component will support the following connection managers:

  • Local File
  • FTPS Connection Manager
  • SFTP Connection Manager
  • Amazon S3 Connection Manager(since v6.0)
  • Azure Blob Connection Manager(since v6.0)
  • Azure Data Lake Storage Connection Manager(since v10.0)
  • Box Connection Manager
  • Dropbox Connection Manager
  • Hadoop Connection Manager(since v10.0)
  • OneDrive Connection Manager
  • SharePoint Connection Manager (offered with the SSIS Integration Toolkit for Microsoft SharePoint)
Source File Path

The Source File Path specifies the location of the file that you are trying to read from.

Locale

Specify the Locale of the file.

Code Page

Specify the Code Page of the file.

File Is Compressed

This option allows you to specify if the file you are trying to read is compressed.

Compression Type

This option allows you to specify the type of compression which the selected file is in. Available options are:

  • Gzip.
  • Deflate.
First Row Has Header

Select this option if the first row contains column names.

Nullify Empty Strings

When this option is checked, it will convert empty string column data into NULL values.

Default to DT_WSTR

Use this option to determine whether applying SSIS type DT_WSTR to all string type columns or using DT_STR with corresponding code page for string columns.

File Format (since v6.2)

Use this option to indicate whether the file uses a Delimited or FixedWidth format.

Row Delimiter

Specify a delimiter character to indicate the end of a row.

Column Delimiter

Specify a delimiter character to indicate the end of a column. Available when Delimited File Format is selected.

Text Qualifier

Use the Text Qualifier option to specify the symbol that represents when the field begins and ends. This option is used in cases where the column may contain a character that represents the Column Delimiter. Available when Delimited File Format is selected.

Padding Characters (since v6.2)

Specify the padding character of the fixed width file if the data within the column does not completely use all the characters allotted to it. Available when FixedWidth File Format is selected.

Trim Padding (since v6.2)

Use this option to indicate whether trim the padding characters in each column of the fixed width file. Available when FixedWidth File Format is selected.

Skip First N Rows (since v9.0)

This option specifies how many rows to exclude at the beginning of the file. The Skip Empty Rows option is evaluated before this option. This option is evaluated before the First Row Has Header option.

Only Read N Rows (since v9.0)

This option specifies the max number of rows that can be returned.

Skip Last N Rows (since v9.0)

This option specifies how many rows to exclude at the end of the file. The Skip Empty Rows option is evaluated before this option.

Skip Empty Rows (since v9.0)

This option specifies if empty rows should be excluded from reading. A row that has column delimiters but all columns are empty is NOT considered an empty row.

Reset Columns Button

Clicking the Reset Columns button will bring up a prompt for you to confirm the reset. After clicking "OK", it will remove any existing columns.

Columns page

The Columns page of the Premium Source Component shows you all available attributes from the object that you specified on the General page. 

Premium Flat File Source Editor

On the top left of the grid, you can see a checkbox, which can be used to toggle the selection of all available fields. This is a productive way to check or uncheck all available fields. 

The Columns Page grid consists of:

  • Premium Flat File Field: Column that will be retrieved from Flat File.
  • Data Type: The data type of this field.
  • Misc: Properties window for the field listed. These values are configurable.
    • CodePage: Specify the Code Page of the field.
    • DataType: The data type can be changed according.
    • Length: If the data type specified is a string, the length specified here would be the maximum size. If the data type is not a string, the length will be ignored.
    • Name: Specify the Column name.
    • Precision: Specify the number of digits in a number.
    • Scale: Specify the number of digits to the right of the decimal point in a number.
  • Import External Column: option to import the Columns and its properties from a file.
  • Export External Column: option to save the Columns and its properties to a JSON file later for reuse.
  • +sign: Add field to Flat File.
  • - sign: Remove field from Flat File.
  • Arrows: Move the fields to a desired location in the file.

Note: As a general best practice, you should only select the fields that are needed for the downstream pipeline components.

Using the Local File Connection Manager found in the Premium File Pack components

Within the Control Flow and Data Flow Components found in this toolkit, you will notice a 'Local File' connection manager in the Connection Manager area. Note that the Local File does not have to be configured in order for you to use this. You would simply select the 'Local File' option, and you would be able to access the folder and files within the local machine.