Lumenore dataset

The Lumenore Dataset function helps you import data from a table in your database into the Lumenore platform. Once the data is loaded, you can create dashboards, run analyses, build reports, or generate insights.

It allows you to easily manage datasets, update them, or connect them to other components in your workflow, ensuring that you always work with the most relevant and structured data.

Below are the steps to use the Lumenore Dataset:

Step 1: Attach the “Lumenore dataset” component to the graph at the end.

Step 2: The user will get two options: select existing dataset and create new dataset.

Step 3: Select existing dataset

Step 3.1: Pick from the available datasets using the “Fetch dataset” option.

Step 3.2: Choose the source table from which data will be fetched by clicking “Fetch tables.” 

Step 3.3: Now, select the loading method. There are four types of loading methods available:

1. Truncate & Load: Deletes old data and loads new data into the same schema.
2. Append: Adds new data to the old table, suitable for creating a new table.
3. Update Insert: Inserts new values and updates old values.
4. Delete Insert: Deletes common values and inserts new values.

Step 3.4: To generate a schema, enable the option and specify a name for the schema.

Step 3.5: Enable this option to load data in batches, which can help improve performance when working with large datasets.

Step 3.6: The Map Columns interface aligns (or maps) the fields from the source dataset to the target dataset or table, ensuring the data is transferred correctly between systems. This is a crucial step in data migration or transformation workflows.

Step 3.7:  Click “Save.”

For example:

• Select the dataset “Online-Retail-Sales2024” along with the appropriate source table.
• In the “Select Loading Method” field, choose “Append”.
• Enable the “Generate Schema” option and enter the schema name as “Retail schema”.
• Proceed to “map columns” between the source and target.
• Finally, click “Save” and then “Run” to execute the process.

Step 4: Create new dataset

Step 4.1: Define the “dataset” the user wants to create.

Step 4.2: Enter the name of the “table” within the selected dataset.

Step 4.3: The only available “loading” method is “Append.”

Step 4.4: Enable the encryption option to encrypt the table name. This adds a layer of security, especially useful in sensitive environments.

Step 4.5: Enable the schema option and provide its name to generate a schema.

Step 4.6: Enable Batch Load to insert data in batches, improving performance when working with large datasets.

For example:

• Defined the dataset name “Retail.”
• Entered the table name “Retail_1”
• Loading method is “Append.”
• Save and run.

Notes:

• Ensure the table name consists only of alphanumeric characters and underscores (“_”).
• Avoid starting the table name with a special character or a numeric digit, except for underscores (“_”).
• Additionally, make sure dataset names are always unique.