Using the Google Cloud Storage Destination Component

The Google Cloud Storage Destination Component is an SSIS data flow pipeline component that can be used to write data to Google Cloud Storage. There are three pages of configuration:

  • General
  • Columns
  • Error Handling

The General page is used to specify general settings for the Google Cloud Storage Destination Component. The Columns page allows you to map the columns from upstream components to Google Cloud Storage fields in the destination object. The Error Handling page allows you to specify how errors should be handled when they occur. 

General page

The General page allows you to specify general settings for the component. 

SSIS Google Cloud Storage Destination Component

Connection Manager

The Google Cloud Storage Destination Component requires a connection. The Connection Manager drop-down will show a list of connection managers that are available within the package.

Destination Object

The drop-down displays a list of objects including:

  • Bucket
  • Object(File or Folder)
Complex Column Option

This option determine if the Complex fields such as the ObjectResource or BucketResource fields, should be broken down into multiple fields, or accept a single JSON object. Acceptable values:

  • Break down to child level
  • Json Object (ObjectResource or BucketResource)
Action

The Action specifies how you want to write to Google Cloud Storage specified in the selected Connection Manager. There are 7 Actions:

  • Create: Stores a new object and metadata
  • Update: Updates an object's metadata
  • Patch: Updates a data blob's associated metadata
  • Copy: Copies a source object to a destination object
  • Rewrite: Rewrites a source object to a destination object
  • Compose: Concatenates a list of existing objects into a new object in the same bucket
  • Delete: Deletes an object and its metadata
Upload Type

The drop-down displays a list of objects including:

  • Simple: Upload the media only, without any metadata.
  • Multipart: Upload both the media and its metadata, in a single request.
  • Resumable: Upload the file in a resumable fashion
Chunk Size (in MB)

The chunk size to use when sending web requests. Available when Resumable is selected.

Refresh Component Button 

Clicking the Refresh Component button causes the component to retrieve the latest metadata and update each attribute to its most recent metadata.

Map Unmapped Fields Button

By clicking this button, the component will try to map any unmapped Box attributes by matching their names with the input columns from upstream components. This is useful when your source component has recently added more columns, in which case you can use this button to automatically establish the association between input columns and unmapped destination attributes.

Clear All Mappings Button

By clicking this button, the component will reset all your mappings in the destination component.

Expression fx Button 

Clicking the fx button to launch SSIS Expression Editor to enable dynamic update of the property at run time.

Generate Documentation Button 

Clicking the Generate Documentation button to generate a Word document which describes the component's metadata including relevant mapping, and so on.

Columns Page

The Columns page of the Google Cloud Storage Destination Component allows you to map the columns from upstream components to the Google Cloud Storage destination fields. 

In the Columns page, you would see a grid that contains four columns as shown below.

SSIS Google Cloud Storage Destination Component - columns

  • Input Column: You can select an input column from an upstream component for the corresponding Google Cloud Storage field.
  • Destination Field: The Google Cloud Storage field that you are writing data. 
  • Data Type: This column indicates the type of value for the current field.
  • Unmap: This column can be used to unmap the field from the upstream input column, or otherwise it can be used to map the field to an upstream input column by matching its name if the field is not currently mapped.

Error Handling Page

The Error Handling page allows you to specify how errors should be handled when they happen. 

SSIS Google Cloud Storage Destination Component - Error Handling

There are three options available. 

  1. Fail on error
  2. Redirect rows to error output
  3. Ignore error

When the Redirect rows to error output option is selected, rows that failed to write to Google Cloud Storage will be redirected to the 'Error Output' output of the Destination Component. As indicated in the screenshot below, the green output connection represent rows that were successfully written, and the red 'Error Output' connection represents rows that were erroneous. The 'ErrorMessage' output column found in the 'Error Output' may contain the error message that was reported by the server or the component itself.

SSIS Google Cloud Stroage Destination Component - Error Output

Note: Use extra caution when selecting Ignore error option, since the component will remain silent for any errors that have occurred.