Bulk Data Operations: Update

All BDO endpoints are only accessible to Creator and Express Administrator users in Unqork. BDO Update is only available for Data Model records.

Bulk Data Operations (BDO) Update endpoint lets you update module, workflow, and Data Model submission data in the Unqork Designer Platform. You can make an update request using curl, Postman, or any other service that supports a PUT request.

To discover how to set up API calls using a third-party program, view our Testing API Endpoints Using Third-Party Applications article.

You can test the BDO Update endpoint using the Unqork Services page. To access the Unqork Services page, enter the following address into your browser: https://{environment}.unqork.io/fbu/uapi/docs/documentation/#/Bulk%20Operations/update. Replace {environment} with the name of your Unqork environment.

Updating Submission Data Using Bulk Data Operations

The BDO Update method uses the HTTP PUT method and the following endpoint: https://{environment}.unqork.io/fbu/uapi/bulkOperations/update. Replace {environment} with the name of your Unqork environment.

Update jobs are limited to 100 MB.

To make calls using third-party programs, you must add an authorization key/value pair to the header. To learn more, view our Testing API Endpoints Using Third-Party Applications article.

{
"createdDateUpdateValue": "string",
"dataModelOrModuleId": "string",
"fallbackRecordOwner": "string",
"fileFormat": "CSV",
"fileLocation": "string",
"name": "string",
"resourceId": "string",
"storeInData": "true",
"uniqueKey": "string",
"upsert": "false"
}

The following parameters are available for the Bulk Data Update schema:

Key	Possible Values	Description	Required
createdDateUpdateValue	string (no spaces)	Enter the date to update the data submission(s) to. Enter the date and time in ISO 8601 (UTC). For example, setting this key's value to `2024-10-17T18:02:55.101Z` would export data recorded before October 10th, 2024 at 6:02:55.101PM Zulu time.
dataModelOrModuleId	string (no spaces)	Specify the Data Model ID, or Module ID, containing the data you want to update. To find a Data Model's ID value, view our How to: Query Data Model Records Dat article.	Yes
fallbackRecordOwner	string (no spaces)	Enter the userId of the Unqork Service Error or Express Access Credential that has authorized access to the workspace. Create new Express Credentials using the API Access Management page.	Yes
fileFormat	CSV ZIPWITHCSV JSON ZIPWITHJSON ZIPWITHMULTIPLEJSON	Specify the file format you want to extract. File formats include: CSV: Update records from a single CSV file. ZIPWITHCSV: Update records from a ZIP file containing a single CSV file. JSON: Update records from a single JSON Lines file. ZIPWITHJSON: Update records from a ZIP file containing a single JSON Lines file. ZIPWITHMULTIPLEJSON: Update records from a ZIP file containing JSON files where each JSON file is one single record. The value must be uppercase.	Yes
fileLocation	string	Enter the Unqork Cloud file address. For example, `unqorkResource/export/f22047aa-13d3-4070-8def-a8a1a401afff/0.json.gz.` Learn how to upload a file and create an Unqork Cloud file address in our Bulk Operations: Upload-Link article	Yes
name	string	Enter a value to name the Update operation by. Names can include spaces. This value also displays in the Job Tracker's File Name column.	Yes
resourceId	string (no spaces)	Enter the `moduleId`, `workflowId`, or `dataModelId` of the element to export revisions from. To find a Data Model's ID value, view our How to: Query Data Model Records Data article.
storeInData	Boolean	When set to `true`, records from the incoming file are stored under the `data` property in submission record. When set to `false`, records from the incoming file are treated as a full submission record, including the full submission data and Unqork metadata. Unqork recommends setting this property to `true`.	Yes
uniqueKey	string (no spaces)	To refer to this operation at a later time, enter a unique key.
upsert	Boolean	When set to `true`, updates existing submission data if present, or inserts new submission data if it doesn't exist. When set to `false`, it updates only the existing submission date. By default, this value is `false`.

Example Response

Below is an example response of updating records:

{
    "id": "67169684ec1bf4776fa760a5",
    "created": 1729533572882,
    "modified": 1729533572882,
    "createdBy": "[email protected]",
    "modifiedBy": "[email protected]",
    "data": {
        "type": "UPDATE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Update API Test 1",
        "steps": [
            {
                "stepType": "UNZIP",
                "status": "created",
                "options": {
                    "type": "UPDATE",
                    "update": {
                        "upsert": true,
                        "uniqueKey": "",
                        "metaOverride": {
                            "form": "658c3c6b70573db027a0f3a2",
                            "owner": "express-access-user"
                        },
                        "storeInData": false
                    }
                },
                "files": [
                    {
                        "source": {
                            "name": "0.json.gz",
                            "location": "unqorkResource/export/f22047aa-13d3-4070-8def-a8a1a401afff/0.json.gz",
                            "format": "ZIPWITHJSON"
                        },
                        "destination": {
                            "location": "unqorkResource/update/0f8eabed-2926-405c-801d-020fd80ee79c.json",
                            "format": "JSON"
                        }
                    }
                ],
                "collection": "658c3c6b70573db027a0f3a2"
            },
            {
                "stepType": "UPDATE",
                "status": "created",
                "options": {
                    "update": {
                        "headers": [],
                        "delimiter": ",",
                        "uniqueKey": "",
                        "upsert": true,
                        "metaOverride": {
                            "form": "658c3c6b70573db027a0f3a2",
                            "owner": "express-access-user"
                        },
                        "storeInData": false
                    }
                },
                "files": [
                    {
                        "source": {
                            "name": "0f8eabed-2926-405c-801d-020fd80ee79c.json",
                            "location": "unqorkResource/update/0f8eabed-2926-405c-801d-020fd80ee79c.json",
                            "format": "JSON"
                        }
                    }
                ],
                "collection": "658c3c6b70573db027a0f3a2",
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                }
            },
            {
                "stepType": "CLEARDATA",
                "status": "created",
                "files": [
                    {
                        "source": {
                            "location": "unqorkResource/update/0f8eabed-2926-405c-801d-020fd80ee79c.json",
                            "format": "JSON"
                        }
                    }
                ],
                "collection": "658c3c6b70573db027a0f3a2"
            }
        ]
    }}

Viewing a Bulk Data Update Operation Job's Details

After starting a Bulk Data Update, you can view the job details using the /fbu/uapi/bulk-operations/job/{jobId} endpoint and the GET method. Copy the update response's id value and replace the job detail's {jobId} placeholder. For example, https://{environment}.unqork.io/fbu/uapi/bulk-operations/job/67169684ec1bf4776fa760a5.

Example Response

Below is an example of a Job Details endpoint response:

{
    "id": "6711511ab056c32e103b11e5",
    "created": 1729188122233,
    "modified": 1729188196285,
    "createdBy": "[email protected]",
    "modifiedBy": null,
    "data": {
        "type": "UPDATE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Update Example 1",
        "steps": [
            {
                "stepType": "UPDATE",
                "id": "6711512eb056c32e103b11e6",
                "status": "completed",
                "files": [
                    {
                        "source": {
                            "name": "0.csv",
                            "location": "unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv",
                            "format": "CSV"
                        }
                    }
                ],
                "options": {
                    "update": {
                        "headers": [],
                        "delimiter": ",",
                        "uniqueKey": "",
                        "upsert": true,
                        "metaOverride": {
                            "form": "658c3c6b70573db027a0f3a2",
                            "owner": "test-authenticated-user"
                        },
                        "storeInData": true
                    }
                },
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                },
                "collection": "658c3c6b70573db027a0f3a2",
                "details": {
                    "dataLocation": {
                        "database": "unqork",
                        "collection": "submissions",
                        "entityType": "MODULE"
                    },
                    "chunks": [
                        {
                            "status": "completed",
                            "chunkQueueId": "1",
                            "chunkRecord": {
                                "start": 2,
                                "end": 5
                            },
                            "docs": {
                                "total": 4,
                                "processed": 4
                            },
                            "chunkFileKey": "bulkDataOperation/update/unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv-0.csv",
                            "auditFileKey": "unqorkResource/audit/6711512eb056c32e103b11e6/1.json.gz",
                            "startTime": "2024-10-17T18:02:54.728Z",
                            "endTime": "2024-10-17T18:02:55.200Z"
                        }
                    ],
                    "docs": {
                        "total": 4,
                        "processed": 4
                    }
                }
            }
        ],
        "status": "completed"
    }}

Using the above example response, the following properties might be important for Creator processes or pipeline development:

Response Property	Description	Example
status	The current status of the operation. The overall operation status is displayed at the bottom of the `data` object. When troubleshooting, review the `status` property contained in each `stepType` in the `steps` array. This value is also reflected in the Job Tracker's Job Status column. Status values include: `inProgress`: The update operation is still processing the file(s). `completed`: The update is complete. `canceled`: The update has been manually stopped by the user. `partiallyCompleted`: The update is complete, but has reported issues. `failed`: The update failed. Review each `stepType`'s `status` value to determine which process failed.	`{ "data": { "type": "UPDATE", "collection": "658c3c6b70573db027a0f3a2", "dataLocation": {}, "name": "Example Update 2", "steps": [], "status": "completed" }}`
docs	Located in the `details` object, the `docs` object displays the following properties: `processed`: The number of records successfully processed by the operation. `total`: The total records processed.	`"details": {"dataLocation": {},"chunks": [],"docs": { "total": 4, "processed": 4 }}}`

Response Property

Description

Example

status

The current status of the operation. The overall operation status is displayed at the bottom of the data object. When troubleshooting, review the status property contained in each stepType in the steps array. This value is also reflected in the Job Tracker's Job Status column.

Status values include:

inProgress: The update operation is still processing the file(s).
completed: The update is complete.
canceled: The update has been manually stopped by the user.
partiallyCompleted: The update is complete, but has reported issues.
failed: The update failed. Review each stepType's status value to determine which process failed.

{
    "data": {
        "type": "UPDATE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {},
        "name": "Example Update 2",
        "steps": [],
        "status": "completed"
    }}

docs

Located in the details object, the docs object displays the following properties:

processed: The number of records successfully processed by the operation.
total: The total records processed.

"details": {"dataLocation": {},"chunks": [],"docs": {
    "total": 4,
    "processed": 4
    }}}

Bulk Data Operations: Update

Updating Submission Data Using Bulk Data Operations

Example Response

Viewing a Bulk Data Update Operation Job's Details

Example Response

Resources