Source:

The source section reads data from the local file system; you can use pre-created connectors to get data. You can also extract data from the table, database, or cloud connector. 

Under the source section, we will be loading our data from the different import options. 

New Connection:

Under the new connection option, we can import data from various connectors like Athena, Azure Table, Google Spanner, etc. It extends our importing option using different departmental wise like Big Data & No SQL, Marketing, Accounting, etc. The connections section provides the functionality for the user to connect with the database, and users can create their connections to utilize them in any Graph.   

How to use Connections:

1. Just click on the new connection inside the connections section. 
2. Search and select any database, e.g., MySQL, Vertica. 
3. Fill in all the fields to connect with the database. 
4. Verify the connection by the Test connection button. 
5. Click on Add a Connection. 
6. Your connection should get added inside the connections panel. 
7. Users can also edit and delete connections from the tree.

Cloud Storage Connector:

Cloud Storage Connector allows importing the file from the cloud storage. 

How to use Cloud Storage Connector:

1. Click on Cloud Storage Connector. 
2. Enter the Step name. 
3. Select the connection which is built earlier. 
4. Select the file type (xlsx, xls, xlsm). 
5. Click on Add Cloud File, file library window appears; the user has to select a file and click on Download. 
6. Downloaded files will appear in File Names; select the file.

Data Grid:

Here, we import the data from the user as per their requirement.  

1. Upon clicking on the data grid option, it contains the step name where we give a name to our file. 
2. The data type is what we put our data type like long, integer, double, etc., and it acts as a row in an array of our data. 
3. Column Name would let us name the type of columns.

Excel:

It is for importing the Excel file locally. 

1. Enter the Step name, which will appear in the grid. 
2. The user can define the path of the local file through the Source directory. 
3. Use filename column is used to give a name to the column present in the Excel file. 
4. Use sheet name column is used to name the sheet present in the Excel file. 
5. Using Wildcard can be used when the user does not remember the exact file name; all the file name which is similar to that file will appear, which helps the user to select the required file. 
6. To upload files from the local disk, users can use Upload local files. 
   • Click Add Excel file> Browse file 
   • Select file name>Get sheets> Select sheet. 
7. Is header row present: The user can check if the data contains a row header. 
   • Headers start at index: From which row the header is present. 
8. Select all columns: To select all the columns present in the Excel file. 
9. Column start index: The user can customize the columns to be selected (e.g., From 1 to 20) 
10. Click Ok. 

File Reader:

It is used to import CSV files in the Lumenore Data Layer. 

1. Enter the Step name, which will appear in the grid. 
2. The user can define the path of the local file through the Source directory. 
3. Use filename column is used to give a name to the column present in the Excel file. 
4. Delimiter is used for separation (For e.g.,/, @, _, -) 
5. Use Wildcard  
   • Use Regex: can be used when the user does not remember the exact file name; all the file name which is similar to that file will appear, which helps the user to select the required file. 
6. To upload files from the local disk, users can use Upload local files. 
7. Encloser is used for reading the file; it includes double quotes, single quotes, and none. 
8. Is header row present: The user can check if the data contains a row header. 
   • No. of rows to skip: Can be 0, 1, 2, and so on. 
9.  Is Multiline: If a large number of columns are present in the file, then the user can click on the check box. 
10. Click Ok. 

Fixed Width:

It is used for splitting the values present in the file. For example, if Data is written as Vikram 02 36 and the range is set to 1-4, then the data will be shown as Vikr.  

1. Enter the Step name, which will appear in the grid. 
2. The user can define the path of the local file through the Source directory. 
3. To upload files from the local disk, users can use Upload local files. 
   • Click Browse file> Select file>Click ok. 
4. Click Get Files. Select the file from the File Names.  
5. Is Header Row Present: The user can check if the data contains a row header. 
6. Users can customize the limits of the data to be shown through Range. 
7. Click Ok. 

Get File Name:

It is used to view the uploaded files on the server. 

1. Enter the Step name, which will appear in the grid. 
2. Use Regex: can be used when the user does not remember the exact file name; all the file name which is similar to that file will appear, which helps the user to select the required file. Click Get Files. 
3. The user will get all the similar name files from which the required file can be selected through File Names. 
4. Click Ok. 

JSON Reader:

To import and read the JSON file locally. 

1. Enter the Step name, which will appear in the grid. 
2. The user can define the path of the local file through the Source directory. 
3. To upload files from the local disk, users can use Upload local files. 
4. Click Browse file> Select file>Click ok. 
5. Click Ok. 

Table Input:

It is used to read the tables from the database. 

1. Enter the Step name, which will appear in the grid. 
2. Select the connection which is built earlier. 
3. By Table: To get all the tables present in the connection, then select the table from the drop-down.   By Query: For writing the SQL query. 
4. Click Ok.