Bulk Data Operations: Get Job Details

Prev Next

All BDO endpoints are only accessible to Creator and Express Administrator users in Unqork.

The Bulk  Data Operations (BDO) Get Job Details API endpoint lets you view the details of bulk data operations that are in progress or complete. Job details are available for the following operations: BDO Import, BDO Export, and BDO Delete.

To learn how to set up API calls using a third-party program, view our Testing API Endpoints Using Third-Party Applications article.

You can test the BDO Get Job Details endpoint using the Unqork Services page. To access the Unqork Services page, enter the following address into your browser: https://{environment}.unqork.io/fbu/uapi/docs/documentation/#/Bulk%20Operations/getJobDetails. Replace {environment} with the name of your Unqork environment.

Getting Job Details Using Bulk Data Operations

The BDO Get Job Details API uses the HTTP GET method and the following endpoint: https://{environment}.unqork.io/fbu/uapi/bulk-operations/job/{job}. Replace {environment} with the name of your Unqork environment, and replace {job} with an operation's id value. For example: https://training.unqork.io/fbu/uapi/bulk-operations/job/6734cd960b69a01036b3f07a.

Getting Job Details Using Postman

After starting a BDO Import, BDO Export, or BDO Delete operation, use the operation's ID property value in the Get Job Details endpoint in Postman.

In the Postman application:

  1. Create a GET request using the following endpoint: https://{{environment}}.unqork.io/fbu/uapi/bulk-operations/job/{{job}}. Replace {{environment}} with the name of your Unqork environment, and replace {{job}} with an operation's ID value. For example: https://training.unqork.io/fbu/uapi/bulk-operations/job/6734cd960b69a01036b3f07a.

                                               

  2. Click Send. The operation job details return in the response body.

Example Responses

The BDO Get Job Details endpoint returns a different payload depending on the operation its targeting.

Click the tabs below to see an example response of each operation type:

BDO Import - Job Details Response

In this example, the Get Job Details API on a BDO Import operation displays the following response:

{
    "id": "6711511ab056c32e103b11e5",
    "created": 1729188122233,
    "modified": 1729188196285,
    "createdBy": "[email protected]",
    "modifiedBy": null,
    "data": {
        "type": "UPDATE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Import Example 1",
        "steps": [
            {
                "stepType": "UPDATE",
                "id": "6711512eb056c32e103b11e6",
                "status": "completed",
                "files": [
                    {
                        "source": {
                            "name": "0.csv",
                            "location": "unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv",
                            "format": "CSV"
                        }
                    }
                ],
                "options": {
                    "update": {
                        "headers": [],
                        "delimiter": ",",
                        "uniqueKey": "",
                        "upsert": true,
                        "metaOverride": {
                            "form": "658c3c6b70573db027a0f3a2",
                            "owner": "test-authenticated-user"
                        },
                        "storeInData": true
                    }
                },
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                },
                "collection": "658c3c6b70573db027a0f3a2",
                "details": {
                    "dataLocation": {
                        "database": "unqork",
                        "collection": "submissions",
                        "entityType": "MODULE"
                    },
                    "chunks": [
                        {
                            "status": "completed",
                            "chunkQueueId": "1",
                            "chunkRecord": {
                                "start": 2,
                                "end": 5
                            },
                            "docs": {
                                "total": 4,
                                "processed": 4
                            },
                            "chunkFileKey": "bulkDataOperation/update/unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv-0.csv",
                            "auditFileKey": "unqorkResource/audit/6711512eb056c32e103b11e6/1.json.gz",
                            "startTime": "2024-10-17T18:02:54.728Z",
                            "endTime": "2024-10-17T18:02:55.200Z"
                        }
                    ],
                    "docs": {
                        "total": 4,
                        "processed": 4
                    }
                }
            }
        ],
        "status": "completed"
    }}

Using the above example response, the following properties might be important for Creator processes or pipeline development:

Response Property

Description

Example

status

The current status of the operation. The overall operation status is displayed at the bottom of the data object. When troubleshooting, review the status property contained in each stepType in the steps array. This value is also reflected in the Job Tracker's Job Status column.

Status values include:

  • inProgress: The import operation is still processing the file(s).

  • completed: The import is complete.

  • canceled: The import has been manually stopped by the user.

  • partiallyCompleted: The import is complete, but has reported issues.

  • failed: The import failed. Review each stepType's status value to determine which process failed.

                                                                       

{
    "data": {
        "type": "EXPORT",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {},
        "name": "Example Export 2",
        "steps": [],
        "status": "completed"
    }}

docs

Located in the details object, the docs object displays the following properties:

  • processed: The number of records successfully processed by the operation.

  • total: The total records processed.

                                                                       

"details": {"dataLocation": {},"chunks": [],"docs": {
    "total": 4,
    "processed": 4
    }}}

BDO Export - Job Details Response

In this example, the Get Job Details API on a BDO Export operation displays the following response:                                                        

{
    "id": "67114d26340b0b7c0510c680",
    "created": 1729187110966,
    "modified": 1729187221199,
    "createdBy": "[email protected]",
    "modifiedBy": null,
    "data": {
        "type": "EXPORT",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Example Export",
        "steps": [
            {
                "stepType": "EXPORT",
                "id": "67114d50b056c32e103b11e3",
                "status": "completed",
                "files": [
                    {
                        "destination": {
                            "name": "0",
                            "location": "unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv",
                            "format": "CSV"
                        }
                    }
                ],
                "options": {
                    "export": {
                        "chunks": 1,
                        "filter": {
                            "showActive": true,
                            "showArchive": false,
                            "form": "658c3c6b70573db027a0f3a2"
                        },
                        "projection": {
                            "data": false
                        }
                    }
                },
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                },
                "collection": "658c3c6b70573db027a0f3a2",
                "details": {
                    "dataLocation": {
                        "database": "unqork",
                        "collection": "submissions",
                        "entityType": "MODULE"
                    },
                    "chunks": [
                        {
                            "chunkQueueId": "0",
                            "status": "completed",
                            "docs": {
                                "total": 4,
                                "processed": 4,
                                "processedSize": 1723
                            },
                            "startTime": "2024-10-17T17:46:52.716Z",
                            "endTime": "2024-10-17T17:46:52.755Z"
                        }
                    ],
                    "docs": {
                        "total": 4,
                        "processed": 4
                    }
                }
            }
        ],
        "status": "completed"
    }}

Using the above example response, the following properties might be important for Creator processes or pipeline development:

Setting

Description

Example

status

The current status of the operation. The overall operation status is displayed at the bottom of the data object. For troubleshooting, review the status property contained in each stepType of the steps array. This value is also reflected in the Job Tracker's Job Status column.

Status values include:

  • inProgress: The export operation is still processing the file(s).

  • completed: The export is complete.

  • canceled: The export has been manually stopped by the user.

  • failed: The export failed. Review each stepType's value to determine which process failed.

                                                                       

{
    "data": {
        "type": "EXPORT",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {},
        "name": "Example Export 2",
        "steps": [],
        "status": "completed"
    }}

docs

Located in the details object. The docs object displays the following properties:

  • processed: The number of records successfully processed by the operation.

  • total: The total records processed.

                                                                   

"details": {"dataLocation": {},"chunks": [],"docs": {
    "total": 4,
    "processed": 4
    }}}

location

Located in the files array, the location property contains the file key used to create the download file.

                                                                     

"files": [
    {
      "destination": {
      "name": "0",
      "location": "unqorkResource/export/78bdc-df4a-41a61/0.csv",
      "format": "CSV"
        }
    }
 ]

BDO Delete - Job Details Response

In this example, the Get Job Details API on a BDO Delete operation displays the following response:                                                        

{
    "id": "6728e46499610f86e6fac431",
    "created": 1730733156824,
    "modified": 1730733283591,
    "createdBy": "[email protected]",
    "modifiedBy": null,
    "data": {
        "type": "DELETE",
        "collection": "658c3c6270573d1027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Delete Example",
        "steps": [
            {
                "stepType": "DELETE",
                "id": "658c3c6270573d1027a0f3a2",
                "status": "completed",
                "options": {
                    "delete": {
                        "mode": "DELETE",
                        "dataSource": "MODULE",
                        "targetRecords": "ARCHIVED"
                    }
                },
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                },
                "collection": "658c3c6b7057db027a0f3a2",
                "details": {
                    "dataLocation": {
                        "database": "unqork",
                        "collection": "submissions",
                        "entityType": "MODULE"
                    },
                    "chunks": [
                        {
                            "chunkQueueId": "0",
                            "status": "completed",
                            "docs": {
                                "total": 32,
                                "processed": 32
                            },
                            "auditFileKey": "unqorkResource/audit/6728e48799610f86e6fac432/0.json.gz",
                            "startTime": "2024-11-04T15:14:10.970Z",
                            "endTime": "2024-11-04T15:14:11.278Z"
                        }
                    ],
                    "docs": {
                        "total": 32,
                        "processed": 32
                    }
                }
            }
        ],
        "status": "completed"
    }}

Using the above example response, the following properties might be important for Creator processes or pipeline development:

Response Property

Description

Example

status

The current status of the operation. The overall operation status is displayed at the bottom of the data object. When troubleshooting, review the status property contained in each stepType of the steps array. This value is also reflected in the Job Tracker's Job Status column.

Status values include:

  • inProgress: The delete operation is still processing the file(s).

  • completed: The delete is complete.

  • canceled: The delete has been manually stopped by the user.

  • failed: The delete failed. Review each stepType's status value to determine which process failed.

                                                                     

{
    "data": {
        "type": "DELETE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {},
        "name": "Example Delete",
        "steps": [],
        "status": "completed"
    }}

docs

Located in the details object, the docsobject displays the following properties:

  • processed: The number of records successfully processed by the operation.

  • total: The total records processed.

                                                                     

"details": {"dataLocation": {},"chunks": [],"docs": {
    "total": 4,
    "processed": 4
    }}}

Resources