How to: Troubleshoot Issues in a Multi-Step Data Workflow

When troubleshooting issues in a multi-step Data Workflow, it’s a best practice to configure Console operators. Doing so lets you inspect the Data Workflow process in the DevTools Console and confirm if issues occurred. That way, you can quickly locate issues in your configuration so they can be corrected. Once the issue is resolved, it’s also a best practice to remove Console operators from your Data Workflow to avoid performance issues.

Below are images of the DevTools Console before and after adding Console operators. The image on the left is a module that does not include Console operators, and the image to the right does. There’s an issue with the Data Workflow in the images. But the issue isn't clear in the image to the left. As you can see in the image to the right, adding Console operators makes it clearer. You can see that Create One, Create Two, and Create Three are functioning properly. After that, the Data Workflow fails. As you'll discover in this how-to guide, an Aggregate operator is the next in line. This operator must be the one with the misconfiguration.

Configuration

You'll begin by creating a Data Workflow with multiple operators. To mimic a real-world issue, you'll incorrectly configure one of your operators. That way, it fails, and you have the opportunity to explore the problem. To find the faulty operator, you'll use Console operators to locate the issue.

These instructions assume you have a new module open, saved with a title.

Configure the Data Table Component

First, configure a Data Table component with sample data that functions as the input of your Data Workflow.

1. In the Module Builder, drag and drop a Data Table component onto your canvas.

2. In the Property ID and Label Text fields, enter dtExample.

3. In the data table, enter the following:

   index	valuea	valueb
   1	A	D
   2	B	E
   3	C	F

   

4. Click Save Component.

Configure the Hidden Component

This Hidden component functions as the output of your Data Workflow.

1. Drag and drop a Hidden component onto your canvas, placing it below the dtExample Data Table component.

2. In the Property ID and Label Text fields, enter outPut.

3. Click Save Component.

Configure the Data Workflow Component

After setting up your Data Workflow, you'll configure all your operators correctly, except for the Aggregate operator. This misconfiguration will break the Data Workflow, so you can troubleshoot.

1. Drag and drop a Data Workflow component onto your canvas, placing it below the dtExample Data Table component.

2. In the Property ID and Canvas Label Text field, enter dwfExample.

Configure the Input Operator

1. Drag and drop an Input operator onto your Data Workflow canvas.

2. Configure the operator's Info window as follows:

   Setting	Value
   Category	Input
   Component	dtExample
   Required	Yes
   Source	Default

Configure the Create Field Operators

1. Drag and drop four Create Field operators onto your Data Workflow canvas.

2. In the Label and Field 1 fields, enter the following for each operator:

   #	Label	Field 1
   1	Create One	createOne = "one"
   2	Create Two	createTwo = "two"
   3	Create Three	createThree = "three"
   4	Create Four	createFour = "four"

3. Connect the output port (right) of the Create One Create Field operator to the input port (left) of the Create Two Create Field operator.

4. Connect the output port (right) of the Create Two Create Field operator to the input port (left) of the Create Three Create Field operator.

   Don't connect any operators to the Create Four Create Field operator at this time.

Configure the Aggregate Operator

1. Drag and drop an Aggregate operator onto your Data Workflow canvas. Remember, this is the operator we're setting up incorrectly.

2. Configure the operator's Info window as follows:

   Setting	Value
   Category	Aggregate By
   Aggregation Type	Max By
   Join Group key?	Yes

3. Leave the Label, Group Key, and Value Key fields empty.

   The Max By setting looks at the Value Key and finds the maximum value based on the column indicated. If the Aggregate operator was set up correctly, the Value Key and Group Key would reference one of your Data Table headers: index, valuea, or valueb. Instead, you'll set up the operator to reference nothing.

4. Connect the output port (right) of the Create Three Create Field operator to the input port (left) of the Aggregate operator.

5. Connect the output port (right) of the Aggregate operator to the input port (left) of the Create Four Create Field operator.

Configure the Output Operator

1. Drag and drop an Output operator onto your Data Workflow canvas.

2. Configure the operator's Info window as follows:

   Setting	Value
   Category	Output
   Component	outPut
   Action	value

3. Connect the output port (right) of the Create Four Create Field operator to the input port (left) of the Output operator.

If you view DevTools Console in Express View, it's unclear where the break lies in your Data Workflow. Let's add some Console operators to locate the issue.

Configure the Console Operators

To locate the issue, add Console operators to your Data Workflow for every step. Doing so lets you view what happens when data passes from one operator to the next.

1. Drag and drop five Console operators onto your Data Workflow canvas.

2. In the Label field for each Console operator, enter the following:

   #	Label
   1	Create One
   2	Create Two
   3	Create Three
   4	Aggregate
   5	Create Four

3. Connect the output port (right) of the Create One Create Field operator to the input port (left) of the Create One Console operator.

4. Connect the output port (right) of the Create Two Create Field operator to the input port (left) of the Create Two Console operator.

5. Connect the output port (right) of the Create Three Create Field operator to the input port (left) of the Create Three Console operator.

6. Connect the output port (right) of the Aggregate operator to the input port (left) of the Aggregate Console operator.

7. Connect the output port (right) of the Create Four Create Field operator to the input port (left) of the Create Four Console operator.

8. Click Save Component.

9. Save your module.

The fully configured Data Workflow component looks like the following:

Here's how your completed example looks in the Module Builder:

In this example, the module is not visible in Express View. There is no dashboard or fields to populate.

View the Console Panel

To open the DevTools Console tab:

1. Preview your module in Express View.

2. Right-click anywhere on your Express View page.

3. Click Inspect. By default, the DevTools panel opens to the right of your page.

   To open the Console in Google Chrome, you can use the Ctrl + Shift + J (Windows/Linux) or Option + Command + J (MacOS) shortcut.

4. At the top of the DevTools panel, click the Console tab.

The Console tab lets you view your Unqork application's submission data. You can see that Create One, Create Two, and Create Three Create Field operators display three arrays each. These arrays indicate that these Create Field operators are working correctly. After Create Three, you can see a circular reference error, preventing data from passing through the Aggregate operator. Using Console operators, you proved the issue lies with the Aggregate operator.

From time to time, the Console panel displays without populating correctly. If this occurs, enter angular.element('.unqorkio-form').scope().submission at the > prompt. Then, press Return or Enter. Refresh your browser and return to the Console panel.