Having worked with numerous application APIs, our team has seen many different API behaviors. One of special scenarios that we have encountered from time to time is the support of multi-part body in some of the API service calls. Essentially a multipart request would consist of different body sections, part of it can be plain text, while the other part can be binary content. This provides ultimate flexibility in processing some complex service calls. At the same time, this particular scenario has created some interesting challenges for ETL developers when SSIS (or any other ETL solutions in general) is used. Without a proper tool, it can be enormously difficult to construct such service calls, it is such a task that is nearly impossible to accomplish unless you roll up your sleeves to write custom C# code for the purpose, which, although possible, can be very challenging.
In this blog post, we will demonstrate how to put together a multipart HTTP request using KingswaySoft SSIS Productivity Pack components without writing a single line of code. Constructing and sending multipart service request can be achieved within only a few mouse clicks when using the HTTP Requester Task and HTTP Requester data flow components.
Multipart Request
HTTP multipart request generally uses a special Content-Type header: multipart/form-data, which indicates to the server that the submitted request consists of different parts in its body that are separated by a specified boundary and each part may contain a body with some (or all) of them having its own headers. A typical multipart HTTP request would look like this.
POST /upload HTTP/1.1 Content-Type: multipart/form-data; boundary=sample_boundary_value Content-Length: [NUMBER_OF_BYTES_IN_ENTIRE_REQUEST_BODY] Accept: application/json --sample_boundary_value Content-Disposition: form-data; name="description" Test upload file. --sample_boundary_value Content-Disposition: form-data; name="file"; filename="file.jpg" Content-Type: image/jpeg [JPEG_DATA_IN_RAW_BINARY_FORMAT] --sample_boundary_value --
Constructing and Sending Multipart Requests in SSIS
With a basic understanding of multipart HTTP request structure, constructing and sending a HTTP multipart request cannot be any easier when using the HTTP Requester Task. The following is such a sample configuration that we want to achieve.
To start with, we first specify POST as the HTTP Method and provide relative path “upload”. The Request Body tab becomes available once the POST HTTP Method is specified. From there, we choose “Form-Data” as the Body Type which is designed for multipart request. In the Body section, we use the + button to add multipart sections, which would bring up the following Add Form Parameters dialog - within this window, we can add different section type. It can be a Text section or a File.
When File Value Type is selected, you can select a local file by clicking the ellipse button beside the Value property. In this example, we are uploading an image file and the corresponding Content Type is “image/jpeg”.
The sample multipart HTTP request also has a description property, we can add it by specifying Text Value Type in the Add Form Parameter dialog.
Once the HTTP Requester Task component has been configured, click Preview button to examine the multipart request the component has generated. As shown in the screenshot below, the HTTP Requester Task component automatically generates a “multipart/form-data” Content-Type header and sets the boundary appropriately with a random string in it that is different every time.
That is all it takes to create a HTTP Requester task which constructs and sends multipart requests to the server. This is done as a control flow task, which is generally designed to work with one service call (or request) at a time. In our next section, we will discuss how to achieve this within SSIS data flow.
Working with Multipart Requests in SSIS Data Flow
In addition to the HTTP Requester Task that we illustrated above, the SSIS Productivity Pack product also provides an HTTP Requester component which can be used within a SSIS data flow task as a transformation component. Let's show you how it works.
First, let's get some sample data prepared by using a Premium Derived Column component. The Premium Derived Column component is an advanced derived column component that supports a total of 250+ functions which support many premium features in our SSIS Productivity Pack.
In the Premium Derived Column component, we use the ReadBinaryContent() function to read the image stored in the file system and get the appropriate binary content data to pass to the HTTP Requester component.
The HTTP Requester component configuration is almost identical as the HTTP Requester Task. Select Relative Path and Body Parameters input columns in HTTP Requester, the HTTP Requester component should be configured like this:
Once the component is configured, it is ready for execution. When executing the data flow task, it is possible to make the service calls repetitively by constructing different request messages each time based on the input column values received from upstream components.
Conclusion
By using the HTTP Requester components offered in SSIS Productivity Pack, you can easily generate multipart HTTP service requests to upload file to server without having to write a single line of code.