Dataset creation

Dataset

A dataset is a structured collection of data used for reporting and analysis, typically organized into rows and columns, where each column represents a field, and each row represents a record. In Lumenore, datasets are created by importing data from sources such as Excel files, CSV files, databases, and other supported connectors. Once a dataset is created, it serves as the foundation for building schemas which used across various modules, including dashboards, visualizations, Ask Me analyses, and other analytical modules.

Steps for creating a Dataset:

Step 1: Log into your account, then select the “Data” tab from the left panel.
Step 2: Click “Create Dataset” to add your data.

Step 3: On the Lumenore Dataset Connector screen, you can select a connector to connect to your data source, depending on your requirements. For the demonstration, we are selecting an Excel connector.

Select “Browse File” or import the file from cloud storage. Upon successful file upload, a message will appear.

Step 4: Proceed by clicking “Next.” Click the checkbox (es) to Select Data Tables and columns.

Step 5: The Load Strategy allows users to choose how new data should be loaded into an existing dataset or table. This helps control how records are updated, replaced, or added during the data upload process.

The available options are:

• Truncate & Load: Deletes all existing records in the target table and loads the new data from the source file.
  Best case: Full dataset replacement with fresh data.

• Append to Existing: Adds new records to the existing table without deleting current data.
  Best case: Incremental data additions such as monthly or daily uploads.

• Update & Insert: Updates match existing records and insert new records that do not already exist.
  Best case: Maintaining master data or transaction tables with changed and new records.

• Delete & Insert: Deletes matching records first, then inserts the new version of those records.
  Best case: Replacing specific columns while keeping the rest of the data unchanged.

Note:

• By default, Truncate and Load are selected.
• Configuration settings are available for the Update & Insert and Delete & Insert data load strategies.

Step 6: Click Save Dataset, then enter a Dataset Name and Description (optional).

Note: Users can also review and modify the Data Load Strategy from this screen before saving the dataset.