How to Generate Nested Output Format

This lesson shows how to set up a Project Version of a given Project to generate a nested output format for the following file formats:

• XSD (XML)
• JSON
• Avro
• Parquet
• Fixed File

Importing XSD, JSON, Avro, and Parquet Files

A user can generate nested output files with minimal steps by performing an XSD, JSON, Avro, or Parquet file import.

Once the import has finished, users will need to review the assigned Generators and assign new Generators or make Parameter changes where required. Additionally, users may need to alter the assigned Receiver’s Parameters.

How to manually set up a Project to generated Nested Output Format

Before completing the steps in this lesson, users will need to have imported or created the Domains within a selected Project Version of a given Project.

In the example below, three Domains have been added: Department, User, and Address.

Step 1: Assign the SegmentDataCreatorReceiver to each Domain

The SegmentDataCreatorReceiver will need to be assigned to each Domain (e.g., Department, User, and Address).

Click on the New Receiver button within the Domain Dashboard.

Enter the Receiver name within the Filter input field and click on the Plus (+) to select it.

Click the Save button.

Enter a File Name for the fileName parameter:

Step 2: Create a Merge Domain

A Merge Domain will need to be created for merging the individual XML data segments created from each Domain into a nested output file.

Expand the New Domain drop-down menu and select the New Domain option.

Enter Merge as the name and click the Save button.

Step 3: Assign a SegmentMergeReceiver to the Merge Domain

The appropriate SegmentMergeReceiver will need to be assigned to the Merge Domain. The following are the most used to generate nested output format.

• XMLSegmentMergeReceiver
• JSONSegmentMergeReceiver
• AvroSegmentMergeReceiver
• ParquetSegmentMergeReceiver
• FixedDelimitedSgementMergeReceiver

For this example, we will use the JSONSegmentMergeReceiver to generate a nested JSON file.

Click on the New Receiver button within the Domain Dashboard.

Enter the Receiver Name within the filter input field. Then click on the Plus (+) to add it to the Domain.

Click the Save button.

Enter the configuration file name within the configName Parameter.

Step 4: Create Scenarios for each Domain

Scenarios can be created for each Domain in multiple ways within the web platform. For more information about each method, click here.

A Scenario may be created for a single Domain using the Quick Scenario button within the Domain Dashboard.

A batch of Scenarios can be created through the Project Dashboard by selecting the Batch New Scenarios option within the Batch Operation drop-down menu. For more information, click here.

A Scenario should be created for each Domain.

Step 5: Create a Scenario Chain

Next, create a Scenario Chain. The MergeScenario will need to be the last one added to the Scenario Chain.

Click on the New Scenario Chain button within the Scenario Chains Pane.

Enter a Name and Description (optional but recommended). Then click the Save button.

Click on the Plus (+) to add each Scenario to the Scenario Chain. The MergeScenario should be added last.

Step 6: Create the Config File

Within the Project Dashboard, select the Configuration Management Tab in the Management Pane. Click on the New Configuration button.

Select the Configuration Type (e.g., XSD, XML, JSON, Delimited Format) and click the Select button.

Enter a Name for the Configuration File and enter a Config File Name.

Please note the name entered for configFileName Parameter for the SegmentMergeReceiver will match this name.

Configure the Output File Name Format. This is the naming format for the generated output file.

Select an option from the drop-down menu and configure accordingly. Use the Add button to add additional lines.

Select the Segment Files. The Segment Files should be selected in the appropriate order. Use the drop-down menu to choose a
Segment File and use the Add button to add another line for each Segment File.

Click the Save button.

Add Elements by selecting a Domain and entering the Element Name. (Optionally) Choose a different array option for the Element.

Click Save & Next button to add another Element or the Save button to finish.

Click and Drag to the right to arrange the Elements. Click the Close button to finish.

Special Note for nested Fixed File Format

When generating nested Fixed File Format, one more step must be performed for each Domain Attribute.

The Size will need to be defined for each Attribute within each Domain.