Vertica bulk loader

The Vertica Bulk Loader function lets you quickly load a large amount of data into Vertica database table. It’s designed to handle big datasets efficiently, making loading much faster than normal methods.

It also comes with helpful features like:

• Error handling: If some rows have issues, it can catch and save them in a separate table.
• Field mapping: You can match your input data columns with the columns in your database.
• Table management: You can create the table if it doesn’t exist or clear it before loading new data.

This function is ideal when you need to move a huge amount of cleaned or processed data into Vertica for analysis or reporting.

Steps to use Vertica Bulk Loader:

Step 1: Drag and drop the “Vertica Bulk Loader” component and connect it to the source.

Step 2: Click “Select connection” and select the connection where you want to load the data from the drop-down menu.

Step 3: Choose the destination table for loading data. Click “Fetch tables” to see available options in the selected connection.

Step 4: Specify a “Rejected tables” where rejected or failed rows will be saved during the load process. This helps with error tracking.

Step 5: When “Create Table if Not “ is enabled, the system will automatically create the target table if it doesn’t already exist in the selected database.

Step 6: When the “Truncate” is enabled, it clears (deletes all records from) the destination table before loading the new data. This ensures that only fresh data exists post-load

Step 7: “Specify Database Fields” allows manual mapping of fields between the stream and the database.

Step 8: Click “Save” and “Run”

For example:

• Choose the “etl_connection” from the available options.
• Clicked “Fetch Tables” to retrieve the tables and select the desired table.
• The name of the rejected table is also created.
• Enabled “Create table if not exists.”
• Enabled “Truncate”
• Enabled “Specify database fields” and matched the fields by name.
• Save and run.