The Big Data components from the SSIS Productivity Pack are SSIS components that facilitate integration with Big Data platforms such as Hadoop.
The SSIS Productivity Pack currently supports integration with Hadoop File System (HDFS) and provides a Connection Manager, Source, and Destination component to facilitate the integration. The Hadoop connection manager can also be used with the CDM Source, CDM Destination, EDI Source, EDI Destination, Premium Data File Source, Premium Data File Destination, Premium Flat File Source, Premium Flat File Destination, Premium File Transfer Task, Premium Excel Source, and Premium Excel Destination components.
The following are the SSIS Big Data components available within the SSIS Productivity Pack and their help manuals:
CDM Components:
-
CDM Source
- CDM Source is used to retrieve data from Common Data Model (CDM) Data. The data from the Source objects are read based on schema description files from the CDM folder. It supports files stored locally or in the cloud.
-
CDM Destination
- CDM Destination can be used to write data into Common Data Model (CDM). It can be used to write files to have it stored in the local file system or saved in one of the cloud storage repositories.
EDI Components:
-
EDI Source
- EDI Source is used to retrieve data from EDI files. It supports files stored locally or in the cloud to retrieve EDI data from 1000's of different EDI formats and version types.
-
EDI Destination
- EDI Destination can be used to write data to EDI files. It can be used to write files to have it stored in the local file system or saved in one of the cloud storage repositories. It support 1000's of different EDI format and version types.
- EDI Destination can be used to write data to EDI files. It can be used to write files to have it stored in the local file system or saved in one of the cloud storage repositories. It support 1000's of different EDI format and version types.
Hadoop Components:
-
Hadoop Connection Manager
- Facilitates connecting to Hadoop from within SSIS.
-
HDFS Source Component
- An SSIS data flow component used to retrieve data from HDFS. Includes the option to specify retrieval mode such as retrieving files and folder or only files or choosing whether sub items should be retrieved as well.
- An SSIS data flow component used to retrieve data from HDFS. Includes the option to specify retrieval mode such as retrieving files and folder or only files or choosing whether sub items should be retrieved as well.
-
HDFS Destination Component
- An SSIS data flow component used to facilitate writing data to HDFS. This component supports CreateFolder, Delete, and Upload actions when writing to HDFS.
- An SSIS data flow component used to facilitate writing data to HDFS. This component supports CreateFolder, Delete, and Upload actions when writing to HDFS.
Premium Data File Components:
-
Premium Data File Source Component
- The Premium Data File Source Component is an SSIS data flow pipeline component that can be used to read/retrieve data from an Avro, ORC or Parquet file.
- The Premium Data File Source Component is an SSIS data flow pipeline component that can be used to read/retrieve data from an Avro, ORC or Parquet file.
-
Premium Data File Destination Component
- The Premium Data File Destination Component is an SSIS data flow pipeline component that can be used to write data from the source into an Avro, ORC or a Parquet file.
- The Premium Data File Destination Component is an SSIS data flow pipeline component that can be used to write data from the source into an Avro, ORC or a Parquet file.