Manipulating and Verifying Data Structures

Overview

The Data Workflow component is an ETL (extract, transform, load) tool. With a Data Workflow component, you can convert objects to tables, tables to objects, arrays to columns in a table, and more. And if any of those words are unfamiliar to you, don't worry. In this article, we'll start by describing the difference between objects, arrays, and tables. Then, you'll learn when you should use a Col2Array, Array2Col, Object2Table, Table2Object, or Sort operator.

What You'll Learn

In this article, you'll learn how to:

Objects, Arrays, and Tables

Let's start by looking at the difference between objects, arrays, and tables. Knowing what data structure you're working with is a valuable skill when using Data Workflow. 

Objects

We'll begin with objects. An object is a type of data structure that represents a single, self-contained entity. The object acts as a container for characteristics about that entity. These characteristics store as key/value pairs in the object. You can think of the keys as questions and the values as answers. For example, say you wanted to create an object called person. You could ask someone 2 questions: "What's your first name?" and "What's your age?" Now, imagine they replied, "My name's Minna, I'm 32." From these answers, you could define 2 characteristics about the object person. Your object would look like this:

person = {"firstName":"Minna", "age":32}

JSON uses curly braces ({...}) to denote objects.

Notice how the keys (firstName and age) correspond to the questions you asked. And the values reflect this specific person's answers. If you asked the same 2 questions to another person, you'd have to create a separate object. That's because each object reflects a single, self-contained person.

Arrays

An array is a data structure used to group related elements. An array can group values, objects, or even other arrays. A key feature of the array data structure is that arrays use indexing. Indexing is a way of assigning an index number to each element in the array. This index gives each element a specific position in the array, for easy reference to specific values.

In arrays, index numbering starts at 0. So, the first element in an array has an index number of 0.

Let’s keep going with our previous example question, "What's your first name?" This time, though, let's say you asked 4 more people. Now you have 5 values for firstName. An array would be a great way to store that data, since you now have 5 related values. Here's what that array could look like:

firstName: ["Minna", "Veronika", "Theola", "Albina", "Ezekiel"]

JSON denotes arrays using square brackets ([...]).

If we expand that array, you can see how the array uses indexing:

  • firstName[0]: "Minna"

  • firstName[1]: "Veronika"

  • firstName[2]: "Theola"

  • firstName[3]: "Albina"

  • firstName[4]: "Ezekiel"

Tables

Tables are a way to visualize data. In Unqork, you can easily create data tables using the Data Table component. Using a Data Table lets you group and visualize data in more than one dimension.

Remember how asking Minna their first name and age gave us the object {"name":"Minna", "age":32}? Let's say we asked the other 4 people from our firstName array their age as well. Now we need a way to track the first names and ages of all 5 people. We could create another array called age, but then it'd be hard to keep track of which age belongs to which name. And if one person didn't give their age, we'd have no easy way to know which name was missing a corresponding age.

Instead, you can use a Data Table. Here's an example of a Data Table called dtInfo. The table's 2 column headers, firstName and age, correspond to our 2 questions. And each row corresponds to a single person's answers. Now you can track each person's information in 2 dimensions. The table layout also makes it easy to visualize the relationships between data.

Now, you just learned that tables are a way to visualize data. Meaning, unlike objects and arrays, tables aren't a data structure. So, if we can't store and send a table as a table, how does JSON store tables? Let's take a look at how the dtInfo Data Table appears in the DevTools Console:

As you can see, tables store and send as an indexed array of objects. The array has 5 objects because the Data Table has 5 rows of data (rows 2-6 in the Data Table). The Data Table's first row, row 1, is what defines the keys in each object of the array. You can see that each object in the array has 2 key/value pairs: firstName and age. And each object corresponds to one row of the Data Table. Or rather, each object corresponds to one person.

Here are other places in Unqork where you'll find data structured as an array of objects:

  • The Advanced DataGrid component

  • Dashboards

  • The response to the Plug-In component's List Submissions for a Dashboard API (application programming interface) call

Verifying Data Structures

Now you know the difference between objects, arrays, and tables. So next, let's take a look at how to tell which data structure you're working with. The Data Workflow component includes many operators designed to help manipulate data structures. For example, you can turn a column in a table into an array or create a table from an object. We'll explain more about these operators in the next section. Here, we'll focus on how to verify your data structure. To use these operators, you first have to understand which one to pick. Which means knowing which data structure you're starting with. The easiest way to figure this out is by inspecting your data in the DevTools Console.

One of the best indicators of what data structure you're looking at is to check the symbols used in each element. JSON denotes objects with curly braces ({...}) and arrays with square brackets ([...]). So an array of 3 values would follow this format: [..., ..., ...]. And an array of 3 objects (a table), would follow this format: [{...}, {...}, {...}].

Let's look at an example. See if you can identify if these 2 elements are objects, arrays, or tables:

If you know that the top element is a table (an array of objects) and the bottom element is an array of values, well done!

Here's one more quick way to verify if the element you're looking at in the Console is an array or an object. Expand the element and look for the _proto_ key. The _proto_ value will show as either Object or Array. If the element is a value, it won't expand further and there won't be a _proto_ key. Let's take a look at this tip in action, with the same 2 elements as above:

Manipulating Data Structures in Data Workflow

Next, let's look at how you can use Data Workflow to manipulate data structures. We'll focus on 5 Data Workflow operators for now:

  • Col2Array

  • Array2Col

  • Object2Table

  • Table2Object

  • Sort

Each of these operators will help you manipulate your data in Data Workflow. Understanding which operator you need to achieve your goal is important. So, before we dive into how to use each operator, let's take a moment to compare them.

Operator

What Does it Do?

What Data Source Does it Use?

What Does it Create?

Col2Array

Converts data stored in a specific column of a table into an array. The data input source must be a table. You'll need to define the specific column's name in the operator's Info window.

A column in a table (an indexed array of objects).

An indexed array of values.

Array2Col

Converts data stored as an array into a column in a table. The source array must be an array of values, not an array of objects. Each value in the array becomes an object.

An indexed array of values.

A column in a table (an indexed array of objects).

Object2Table

Converts data stored in an object into a table. For example, going from an object with 3 key/value pairs, to an indexed array of 3 objects.

An object.

A table (an indexed array of objects).

Table2Object

Converts data stored in a table into an object. While a table is an indexed array of objects, the final object has no indexing.

A table (an indexed array of objects).

An object.

Sort

Sorts a table in either ascending or descending order. The sort operation can apply to a single data column (key) or several.

With the Sort operator, you can connect an Input operator to the Sort operator's argument port. It's this Input operator that contains the information on how to sort the table. For example, using a Dropdown component to select the sort key or sort order. You'd then link the Dropdown component to the Data Workflow using an Input operator. Then, connect that Input operator to the Sort operator's argument port.

A table (an indexed array of objects).

A sorted version of the same array of objects.