Bulk Data Operations: Get Job Details

This feature is currently in production early-access. Functionality is subject to change as this feature is prepared for general access.

The Bulk Data Operations (BDO) Get Job Details API endpoint lets you view the details of bulk data operations that are in progress or complete. Job details are available for the following operations: BDO Import, BDO Export, and BDO Delete.

To learn how to set up API calls using a third-party program, view our Testing API Endpoints Using Third-Party Applications article.

You can test the BDO Get Job Details endpoint using the Unqork Services page. To access the Unqork Services page, enter the following address into your browser: https://{environment}.unqork.io/fbu/uapi/docs/documentation/#/Bulk%20Operations/getJobDetails. Replace {environment} with the name of your Unqork environment.

The BDO Get Job Details API uses the HTTP GET method and the following endpoint: https://{environment}.unqork.io/fbu/uapi/bulk-operations/job/{job}. Replace {environment} with the name of your Unqork environment, and replace {job} with an operation's id value. For example: https://training.unqork.io/fbu/uapi/bulk-operations/job/6734cd960b69a01036b3f07a.

Getting Job Details Using Postman

After starting a BDO Import, BDO Export, or BDO Delete operation, use the operation's id property value in the Get Job Details endpoint in Postman.

In the Postman application:

1. Create a GET request using the following endpoint: https://{{environment}}.unqork.io/fbu/uapi/bulk-operations/job/{{job}}. Replace {{environment}} with the name of your Unqork environment, and replace {{job}} with an operation's id value. For example: https://training.unqork.io/fbu/uapi/bulk-operations/job/6734cd960b69a01036b3f07a.

A static image displaying the Postman application. the Get Job Details Request is set to GET and is pointed at the required endpoint.

2. Click Send. The operation job details return in the response body.

The BDO Get Job Details endpoint returns a different payload depending on the operation its targeting.

Click the tabs below to see an example response of each operation type:

In this example, the Get Job Details API on a BDO Import operation displays the following response:

Copy
{
    "id": "6711511ab056c32e103b11e5",
    "created": 1729188122233,
    "modified": 1729188196285,
    "createdBy": "creatorName@creatorCompany.com",
    "modifiedBy": null,
    "data": {
        "type": "UPDATE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Import Example 1",
        "steps": [
            {
                "stepType": "UPDATE",
                "id": "6711512eb056c32e103b11e6",
                "status": "completed",
                "files": [
                    {
                        "source": {
                            "name": "0.csv",
                            "location": "unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv",
                            "format": "CSV"
                        }
                    }
                ],
                "options": {
                    "update": {
                        "headers": [],
                        "delimiter": ",",
                        "uniqueKey": "",
                        "upsert": true,
                        "metaOverride": {
                            "form": "658c3c6b70573db027a0f3a2",
                            "owner": "test-authenticated-user"
                        },
                        "storeInData": true
                    }
                },
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                },
                "collection": "658c3c6b70573db027a0f3a2",
                "details": {
                    "dataLocation": {
                        "database": "unqork",
                        "collection": "submissions",
                        "entityType": "MODULE"
                    },
                    "chunks": [
                        {
                            "status": "completed",
                            "chunkQueueId": "1",
                            "chunkRecord": {
                                "start": 2,
                                "end": 5
                            },
                            "docs": {
                                "total": 4,
                                "processed": 4
                            },
                            "chunkFileKey": "bulkDataOperation/update/unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv-0.csv",
                            "auditFileKey": "unqorkResource/audit/6711512eb056c32e103b11e6/1.json.gz",
                            "startTime": "2024-10-17T18:02:54.728Z",
                            "endTime": "2024-10-17T18:02:55.200Z"
                        }
                    ],
                    "docs": {
                        "total": 4,
                        "processed": 4
                    }
                }
            }
        ],
        "status": "completed"
    }
}

Using the above example response, the following properties might be important for Creator processes or pipeline development:

Response Property Description Example

status

The current status of the operation. The overall operation status is displayed at the bottom of the data object. When troubleshooting, review the status property contained in each stepType in the steps array. This value is also reflected in the Job Tracker's Job Status column.

Status values include:

  • inProgress: The import operation is still processing the file(s).
  • completed: The import is complete.

  • canceled: The import has been manually stopped by the user.

  • partiallyCompleted: The import is complete, but has reported issues.

  • failed: The import failed. Review each stepType's status value to determine which process failed.

Copy
{
    "data": {
        "type": "EXPORT",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {},
        "name": "Example Export 2",
        "steps": [],
        "status": "completed"
    }
}

docs

Located in the details object, the docs object displays the following properties:

  • processed: The amount of records successfully processed by the operation.

  • total: The total records processed.

Copy
"details": {
  "dataLocation": {},
  "chunks": [],
  "docs": {
    "total": 4,
    "processed": 4
    }
  }
}