Load Parquet Data Source based on the Destination

Explanation

This activity is used to Load a Parquet Data Source into a target Data Lake destination (Self-Hosted Data Lake, IFS.ai Platform Data Lake). Apart from the Data Source loading via an Analysis Model or a Workload Job Definition, it can be explicitly loaded as well.

To explicitly trigger a Data Source to load, below two approaches can be followed.

Once all the columns are selected,

1. Click Load and Exit the New Data Source assistant.  
2. Exit New Data Source assistant and Load later - this method will require selecting the created data source(s) from the Parquet Data Source page and then clicking on Load.

Prerequisites

To perform this activity: 

• Data Sources should be created, and an entry should be available on the Parquet Data Source page. 
• During the second step of the New Data Source Assistant, column selection should be completed, and then click Yes in the Load Data Source dialog box.

System Effects

As a result of this activity, the Explicit Load flag will be set to Yes. Once the Loading is completed, the Explicit Load flag will be set to No, and the Load status value will be updated. Refresh History details will be updated in the Parquet Data Source Details page.   

The .parquet file will be created within the respective Data Lake directory.

Pages

Parquet Datasource Details

Activity Diagrams

Load Parquet Data Source