Bulk Data Operations: Update

Overview

All BDO endpoints are only accessible to Creator Also known as Unqork Users, or Designer Users; is anyone who is inside the Unqork platform. and Express Administrator Includes the Short Answer component, Long Answer component, Select Boxes, Radio component, Dropdown, Number component and Single Checkbox. users in Unqork.

Bulk Data Operations (BDO) Update endpoint lets you update module, workflow,. and Data Model submission data Also known as Record Data. Submission data consists of information saved by Unqork components. View submission data by using the angular command, or in Record Collections. in the Unqork Designer Platform. You can make an update request using curl, Postman, or any other service that supports a PUT The HTTP POST method is used to send data to a server to create or update a resource, typically including the data in the body of the request. request.

To discover how to set up API calls using a third-party program, view our Testing API Endpoints Using Third-Party Applications article.

You can test the BDO Update endpoint using the Unqork Services page. To access the Unqork Services page, enter the following address into your browser: https://{environment}.unqork.io/fbu/uapi/docs/documentation/#/Bulk%20Operations/update. Replace {environment} with the name of your Unqork environment.

Updating Submission Data Using Bulk Data Operations

The BDO Update method uses the HTTP HTTP (Hypertext Transfer Protocol) is an application-layer protocol used to transmit hypermedia documents like HTML.  PUT The PUT method is an HTTP request method used in REST APIs to update an existing resource or create a new resource if it does not already exist. method and the following endpoint: https://{environment}.unqork.io/fbu/uapi/bulkOperations/update. Replace {environment} with the name of your Unqork environment.

Update jobs are limited to 100 MB.

To make calls using third-party programs, you must add an authorization key/value pair to the header. To learn more, view our Testing API Endpoints Using Third-Party Applications article.

Copy
{
  "createdDateUpdateValue": "string",
  "dataModelOrModuleId": "string",
  "fallbackRecordOwner": "string",
  "fileFormat": "CSV",
  "fileLocation": "string",
  "name": "string",
  "resourceId": "string",
  "storeInData": "true",
  "uniqueKey": "string",
  "upsert": "false"
}

The following parameters are available for the Bulk Data Update schema Schema is a declarative language that defines the structure, content, and constraints of the data.:

Key

Possible Values

Description

Required

createdDateUpdateValue

string (no spaces)

Enter the date to update the data submission(s) to. Enter the date and time in ISO 8601 Format A standard format for representing dates and times. ISO 8601 format is represented in the following order: year, month, day, hour, minutes, seconds, and milliseconds. (UTC Coordinated Universal Time (UTC) is a standard, globally-accepted timezone used to regulate clocks and time around the world.). For example, setting this key's value to 2024-10-17T18:02:55.101Z would export data recorded before October 10th, 2024 at 6:02:55.101PM Zulu time.

 

dataModelOrModuleId

string (no spaces)

Specify the Data Model ID, or Module ID, containing the data you want to update.

To find a Data Model's ID value, view our How to: Query Data Model Records Data article.

Yes

fallbackRecordOwner

string (no spaces)

Enter the userId of the Unqork Service User Unqork Service Users are server-side only credentials used in API calls and authorizations. Unlike regular express users, service users are not tied to a person. Instead, they are managed by the company account. Service users have been replaced by Express Credentials created in the API Access Management administration page. or Express Access Credential that has authorized access to the workspace.

Create new Express Credentials using the API Access Management page.

Yes

fileFormat

CSV

ZIPWITHCSV

JSON

ZIPWITHJSON

ZIPWITHMULTIPLEJSON

Specify the file format you want to extract. File formats include:

The value must be uppercase.

Yes

fileLocation

string

Enter the Unqork Cloud file address. For example, unqorkResource/export/f22047aa-13d3-4070-8def-a8a1a401afff/0.json.gz.

Learn how to upload a file and create an Unqork Cloud file address in our Bulk Operations: Upload-Link article.

Yes

name

string

Enter a value to name the Update operation by. Names can include spaces. This value also displays in the Job Tracker's File Name column.

Yes

resourceId

string (no spaces)

Enter the moduleId, workflowId, or dataModelId of the element to export revisions from.

To find a Data Model's ID value, view our How to: Query Data Model Records Data article.

 

storeInData

Boolean

When set to true, records from the incoming file are stored under the data property in submission record.

When set to false, records from the incoming file are treated as a full submission record, including the full submission data and Unqork metadata.

Unqork recommends setting this property to true.

Yes

uniqueKey

string (no spaces)

To refer to this operation at a later time, enter a unique key.

 

upsert

Boolean

When set to true, updates existing submission data if present, or inserts new submission data if it is doesn't exist.

When set to false, updates only existing submission date.

By default, this value is false.

 

Example Response

Below is an example response of updating records:

Copy
{
    "id": "67169684ec1bf4776fa760a5",
    "created": 1729533572882,
    "modified": 1729533572882,
    "createdBy": "creatorAccount@creatorCompany.com",
    "modifiedBy": "creatorAccount@creatorCompany.com",
    "data": {
        "type": "UPDATE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Update API Test 1",
        "steps": [
            {
                "stepType": "UNZIP",
                "status": "created",
                "options": {
                    "type": "UPDATE",
                    "update": {
                        "upsert": true,
                        "uniqueKey": "",
                        "metaOverride": {
                            "form": "658c3c6b70573db027a0f3a2",
                            "owner": "express-access-user"
                        },
                        "storeInData": false
                    }
                },
                "files": [
                    {
                        "source": {
                            "name": "0.json.gz",
                            "location": "unqorkResource/export/f22047aa-13d3-4070-8def-a8a1a401afff/0.json.gz",
                            "format": "ZIPWITHJSON"
                        },
                        "destination": {
                            "location": "unqorkResource/update/0f8eabed-2926-405c-801d-020fd80ee79c.json",
                            "format": "JSON"
                        }
                    }
                ],
                "collection": "658c3c6b70573db027a0f3a2"
            },
            {
                "stepType": "UPDATE",
                "status": "created",
                "options": {
                    "update": {
                        "headers": [],
                        "delimiter": ",",
                        "uniqueKey": "",
                        "upsert": true,
                        "metaOverride": {
                            "form": "658c3c6b70573db027a0f3a2",
                            "owner": "express-access-user"
                        },
                        "storeInData": false
                    }
                },
                "files": [
                    {
                        "source": {
                            "name": "0f8eabed-2926-405c-801d-020fd80ee79c.json",
                            "location": "unqorkResource/update/0f8eabed-2926-405c-801d-020fd80ee79c.json",
                            "format": "JSON"
                        }
                    }
                ],
                "collection": "658c3c6b70573db027a0f3a2",
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                }
            },
            {
                "stepType": "CLEARDATA",
                "status": "created",
                "files": [
                    {
                        "source": {
                            "location": "unqorkResource/update/0f8eabed-2926-405c-801d-020fd80ee79c.json",
                            "format": "JSON"
                        }
                    }
                ],
                "collection": "658c3c6b70573db027a0f3a2"
            }
        ]
    }
}

Viewing a Bulk Data Update Operation Job's Details

After starting a Bulk Data Update, you can view the job details using the /fbu/uapi/bulk-operations/job/{jobId} endpoint and the GET The HTTP GET method is used to request data from a server, typically retrieving information without modifying any resources. method. Copy the update response's id value and replace the job detail's {jobId} placeholder. For example, https://{environment}.unqork.io/fbu/uapi/bulk-operations/job/67169684ec1bf4776fa760a5.

Example Response

Below is an example of a Job Details endpoint response:

Copy
{
    "id": "6711511ab056c32e103b11e5",
    "created": 1729188122233,
    "modified": 1729188196285,
    "createdBy": "creatorName@creatorCompany.com",
    "modifiedBy": null,
    "data": {
        "type": "UPDATE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Update Example 1",
        "steps": [
            {
                "stepType": "UPDATE",
                "id": "6711512eb056c32e103b11e6",
                "status": "completed",
                "files": [
                    {
                        "source": {
                            "name": "0.csv",
                            "location": "unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv",
                            "format": "CSV"
                        }
                    }
                ],
                "options": {
                    "update": {
                        "headers": [],
                        "delimiter": ",",
                        "uniqueKey": "",
                        "upsert": true,
                        "metaOverride": {
                            "form": "658c3c6b70573db027a0f3a2",
                            "owner": "test-authenticated-user"
                        },
                        "storeInData": true
                    }
                },
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                },
                "collection": "658c3c6b70573db027a0f3a2",
                "details": {
                    "dataLocation": {
                        "database": "unqork",
                        "collection": "submissions",
                        "entityType": "MODULE"
                    },
                    "chunks": [
                        {
                            "status": "completed",
                            "chunkQueueId": "1",
                            "chunkRecord": {
                                "start": 2,
                                "end": 5
                            },
                            "docs": {
                                "total": 4,
                                "processed": 4
                            },
                            "chunkFileKey": "bulkDataOperation/update/unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv-0.csv",
                            "auditFileKey": "unqorkResource/audit/6711512eb056c32e103b11e6/1.json.gz",
                            "startTime": "2024-10-17T18:02:54.728Z",
                            "endTime": "2024-10-17T18:02:55.200Z"
                        }
                    ],
                    !!"docs": {!!
                        !!"total": 4,!!
                        !!"processed": 4!!
                    }
                }
            }
        ],
        !!"status": "completed"!!
    }
}

Using the above example response, the following properties might be important for Creator Also known as Unqork Users, or Designer Users; is anyone who is inside the Unqork platform. processes or pipeline development:

Response Property Description Example

status

The current status of the operation. The overall operation status is displayed at the bottom of the data object. When troubleshooting, review the status property contained in each stepType in the steps array. This value is also reflected in the Job Tracker's Job Status column.

Status values include:

  • inProgress: The updateoperation is still processing the file(s).
  • completed: The update is complete.

  • canceled: The update has been manually stopped by the user.

  • partiallyCompleted: The update is complete, but has reported issues.

  • failed: The update failed. Review each stepType's status value to determine which process failed.

Copy
{
    "data": {
        "type": "UPDATE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {},
        "name": "Example Update 2",
        "steps": [],
        !!"status": "completed"!!
    }
}

docs

Located in the details object, the docs objects displays the following properties:

  • processed: The amount of records successfully processed by the operation.

  • total: The total records processed.

Copy
"details": {
  "dataLocation": {},
  "chunks": [],
  "docs": {
    !!"total": 4,!!
    !!"processed": 4!!
    }
  }
}