Using the HTTP Requester
The HTTP Requester is an SSIS data flow pipeline component that can be used to send a series of HTTP requests and process their responses. A common use case for this component is to download files.
The component includes the following two pages to configure how you handle sending requests and processing responses:
- Error Handling
The General page of the HTTP Requester Transform allows you to specify the general settings of the component.
The HTTP Requester requires a connection in order to connect to the web service. The Connection Manager drop-down will show a list of all connection managers that are available to your current SSIS package.
Note that the Connection Managers offered within this toolkit are the following:
- HTTP Connection Manager
- Relative Path
The Relative Path field allows you to specify an input column that is a string containing the relative location of the URL your request will be sent to. The base URL comes from the connection manager. For example if you want to download a list of files called "file1.txt", "file2.txt", "file3.txt" from http://www.example.com/ you would create a connection manager with the base URL http://www.example.com/ and in the HTTP Requester Transform set the Relative Path to an input column that will have values representing each file name.
The Body field allows you to set an input column that contains the body of the HTTP request that will be sent. If the connection manger is using the the GET method this field would typically be left unmapped.
- Query String Parameters
The Query String Parameters grid allows you to add query string parameters that will be appended the the URL and map them to input columns.
- Content Type Checkbox
Determines if a Content Type output column will be created and populated with the content type of the response.
- Content Length Checkbox
Determines if a Content Length output column will be created and populated with the content length of the response.
- Response Body Output
The Response Body Output field determines what will happen with the response body. The following options are available:
- None - The response body is not used for anything
- Output To Image Column - The response body is output to a column of type DT_IMAGE
- Output To Text Column - The response body is output to a column of type DT_NTEXT
- Output To File - The response body is saved to a file.
- File Path
The File Path field allows you to set an input column that will determine the path of a file where the request response body will be saved. This field will only be enabled if the the response body output field is set to Output To File.
Error Handling page
The Error Handling page allows you to specify how errors should be handled when they happen.
There are three options available.
- Fail on error
- Redirect rows to error output
- Ignore error
When the Redirect rows to error output option is selected, rows that failed to write to the Web Service Destination will be redirected to the 'Error Output' output of the component. As indicated in the screenshot below, the blue output connection represent rows that were successfully written, and the red 'Error Output' connection represents rows that were erroneous. The 'ErrorMessage' output column found in the 'Error Output' may contain the error message that was reported by server or the component itself.