Snowflake

Snowflake is a cloud-based data warehousing platform that offers high-performance analytics and scalability. The Snowflake connector facilitates seamless integration of applications and tools with Snowflake, enabling data ingestion, querying, and manipulation.

It provides the necessary functionality to connect to Snowflake and enables data exchange between the application and the Snowflake data warehouse.

The Snowflake connector can be used in a variety of scenarios, including:

• Data Integration: The connector allows data integration tools, such as ETL (Extract, Transform, Load) tools or data integration platforms, to connect with Snowflake and transfer data between different systems or data sources.
• Business Intelligence and Reporting: Business intelligence tools, reporting tools, or data visualization platforms can leverage the Snowflake connector to establish a connection with Snowflake and retrieve data for analysis and reporting purposes. This enables users to create interactive dashboards, generate reports, and gain insights from the data stored in Snowflake.

Before connecting to a Snowflake database, you need to whitelist a set of IP addresses on your database server for the specific port you wish to use. To view the complete list of IP addresses, refer to Whitelisting IP Addresses for Database Connectors.

To use the Snowflake connector in Lumenore:

Step 1: Open Lumenore. From home page navigate to Connect Data > Select Snowflake.

Step 2: Click Create New Connection to set up a new connection or select an existing connection to proceed.

Step 3: Provide connection details.

Case 1: By providing all the required details to establish the connection.

Case 2: By using the direct JDBC Connection.

Step 4: After providing the details, click Test Connection. If the connection succeeds, a confirmation message indicates that the “Connection details are valid”; otherwise, an “Invalid connection details” message is displayed.

Step 5: The user can either save the connection or use it without saving.

Step 6: Once saved, the connection is added to the existing connections list and can be reused later.

Actions:

When the user clicks the three dots under the Action column, below mentioned options are displayed:

• Use Connection: Takes you to the next step to select a table.
• Modify: Allows you to edit the connection details.
• Delete: Removes the connection from the existing connections list.

Step 7: To use a connection from the existing connections list, select it and click Next.

Step 8: Select the table you want to export to the Lumenore database. You can also view the data by clicking Preview tab.

The Preview tab allows you to preview the table data, apply filters, change data capture, and perform transformations.

• Preview table data:
  Preview table data lets you quickly view a sample of records from the selected table before proceeding.

• Filter data:
  The Filter data section allows you to restrict the rows loaded from the selected table based on specific conditions.

• Transform data:
  The Transform data tab provides options for cleaning and enriching your data before loading it into the database.

• Change data capture (CDC):
  It is a technique for identifying and capturing only the data that has changed (new, updated, or deleted records) in a source system since the last data load.

Step 9: Click “Save Dataset” to complete data ingestion.

Step 10: Provide the Dataset name and description (optional). Click Ok.

Step 11: A “Dataset saved successfully” message is displayed. After this, the user can cancel and start using the dataset or create a schema by clicking the Create Schema option.