Bulk Data Operations: Get Job Details

This feature is currently in production early-access. Functionality is subject to change as this feature is prepared for general access.

Overview

The Bulk Data Operations (BDO) Get Job Details API APIs (application programming interfaces) are a set of protocols and definitions developers use to build and integrate application software. APIs act as the connective tissue between products and services. endpoint lets you view the details of bulk data operations that are in progress or complete. Job details are available for the following operations: BDO Import, BDO Export, and BDO Delete.

To learn how to set up API calls using a third-party program, view our Testing API Endpoints Using Third-Party Applications article.

You can test the BDO Get Job Details endpoint using the Unqork Services page. To access the Unqork Services page, enter the following address into your browser: https://{environment}.unqork.io/fbu/uapi/docs/documentation/#/Bulk%20Operations/getJobDetails. Replace {environment} with the name of your Unqork environment.

Getting Job Details Using Bulk Data Operations

The BDO Get Job Details API uses the HTTP HTTP (Hypertext Transfer Protocol) is an application-layer protocol used to transmit hypermedia documents like HTML. GET The HTTP GET method is used to request data from a server, typically retrieving information without modifying any resources. method and the following endpoint: https://{environment}.unqork.io/fbu/uapi/bulk-operations/job/{job}. Replace {environment} with the name of your Unqork environment, and replace {job} with an operation's id value. For example: https://training.unqork.io/fbu/uapi/bulk-operations/job/6734cd960b69a01036b3f07a.

Getting Job Details Using Postman

After starting a BDO Import, BDO Export, or BDO Delete operation, use the operation's id property value in the Get Job Details endpoint in Postman.

In the Postman application:

1. Create a GET The HTTP GET method is used to request data from a server, typically retrieving information without modifying any resources. request using the following endpoint: https://{{environment}}.unqork.io/fbu/uapi/bulk-operations/job/{{job}}. Replace {{environment}} with the name of your Unqork environment, and replace {{job}} with an operation's id value. For example: https://training.unqork.io/fbu/uapi/bulk-operations/job/6734cd960b69a01036b3f07a.

A static image displaying the Postman application. the Get Job Details Request is set to GET and is pointed at the required endpoint.

2. Click Send. The operation job details return in the response body.

Example Responses

The BDO Get Job Details endpoint returns a different payload depending on the operation its targeting.

Click the tabs below to see an example response of each operation type:

BDO Import - Job Details Response

In this example, the Get Job Details API on a BDO Import operation displays the following response:

Copy 
{
    "id": "6711511ab056c32e103b11e5",
    "created": 1729188122233,
    "modified": 1729188196285,
    "createdBy": "creatorName@creatorCompany.com",
    "modifiedBy": null,
    "data": {
        "type": "UPDATE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Import Example 1",
        "steps": [
            {
                "stepType": "UPDATE",
                "id": "6711512eb056c32e103b11e6",
                "status": "completed",
                "files": [
                    {
                        "source": {
                            "name": "0.csv",
                            "location": "unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv",
                            "format": "CSV"
                        }
                    }
                ],
                "options": {
                    "update": {
                        "headers": [],
                        "delimiter": ",",
                        "uniqueKey": "",
                        "upsert": true,
                        "metaOverride": {
                            "form": "658c3c6b70573db027a0f3a2",
                            "owner": "test-authenticated-user"
                        },
                        "storeInData": true
                    }
                },
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                },
                "collection": "658c3c6b70573db027a0f3a2",
                "details": {
                    "dataLocation": {
                        "database": "unqork",
                        "collection": "submissions",
                        "entityType": "MODULE"
                    },
                    "chunks": [
                        {
                            "status": "completed",
                            "chunkQueueId": "1",
                            "chunkRecord": {
                                "start": 2,
                                "end": 5
                            },
                            "docs": {
                                "total": 4,
                                "processed": 4
                            },
                            "chunkFileKey": "bulkDataOperation/update/unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv-0.csv",
                            "auditFileKey": "unqorkResource/audit/6711512eb056c32e103b11e6/1.json.gz",
                            "startTime": "2024-10-17T18:02:54.728Z",
                            "endTime": "2024-10-17T18:02:55.200Z"
                        }
                    ],
                    !!"docs": {!!
                        !!"total": 4,!!
                        !!"processed": 4!!
                    }
                }
            }
        ],
        !!"status": "completed"!!
    }
}

Using the above example response, the following properties might be important for Creator Also known as Unqork Users, or Designer Users; is anyone who is inside the Unqork platform. processes or pipeline development:

Response Property Description Example

status

The current status of the operation. The overall operation status is displayed at the bottom of the data object. When troubleshooting, review the status property contained in each stepType in the steps array. This value is also reflected in the Job Tracker's Job Status column.

Status values include:

  • inProgress: The import operation is still processing the file(s).

  • completed: The import is complete.

  • canceled: The import has been manually stopped by the user.

  • partiallyCompleted: The import is complete, but has reported issues.

  • failed: The import failed. Review each stepType's status value to determine which process failed.
Copy 
{
    "data": {
        "type": "EXPORT",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {},
        "name": "Example Export 2",
        "steps": [],
        !!"status": "completed"!!
    }
}

docs

Located in the details object, the docs object displays the following properties:

  • processed: The amount of records successfully processed by the operation.

  • total: The total records processed.
Copy 
"details": {
  "dataLocation": {},
  "chunks": [],
  "docs": {
    !!"total": 4,!!
    !!"processed": 4!!
    }
  }
}

BDO Export - Job Details Response

In this example, the Get Job Details API on a BDO Export operation displays the following response:

Copy 
{
    "id": "67114d26340b0b7c0510c680",
    "created": 1729187110966,
    "modified": 1729187221199,
    "createdBy": "creatorEmail@creatorCompany.com",
    "modifiedBy": null,
    "data": {
        "type": "EXPORT",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Example Export",
        "steps": [
            {
                "stepType": "EXPORT",
                "id": "67114d50b056c32e103b11e3",
                "status": "completed",
                "files": [
                    {
                        "destination": {
                            "name": "0",
                            !!"location": "unqorkResource/export/78b9abdc-df4a-42e0-8b02-1ce0377a1a61/0.csv",!!
                            "format": "CSV"
                        }
                    }
                ],
                "options": {
                    "export": {
                        "chunks": 1,
                        "filter": {
                            "showActive": true,
                            "showArchive": false,
                            "form": "658c3c6b70573db027a0f3a2"
                        },
                        "projection": {
                            "data": false
                        }
                    }
                },
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                },
                "collection": "658c3c6b70573db027a0f3a2",
                "details": {
                    "dataLocation": {
                        "database": "unqork",
                        "collection": "submissions",
                        "entityType": "MODULE"
                    },
                    "chunks": [
                        {
                            "chunkQueueId": "0",
                            "status": "completed",
                            "docs": {
                                "total": 4,
                                "processed": 4,
                                "processedSize": 1723
                            },
                            "startTime": "2024-10-17T17:46:52.716Z",
                            "endTime": "2024-10-17T17:46:52.755Z"
                        }
                    ],
                    !!"docs": {
                        "total": 4,
                        "processed": 4
                    }!!
                }
            }
        ],
        !!"status": "completed"!!
    }
}

Using the above example response, the following properties might be important for Creator Also known as Unqork Users, or Designer Users; is anyone who is inside the Unqork platform. processes or pipeline development:

Setting Description Example

status

The current status of the operation. The overall operation status is displayed at the bottom of the data object. For troubleshooting, review the status property contained in each stepType in the steps array. This value is also reflected in the Job Tracker's Job Status column.

Status values include:

  • inProgress: The export operation is still processing the file(s).

  • completed: The export is complete.

  • canceled: The export has been manually stopped by the user.

  • failed: The export failed. Review each stepType's value to determine which process failed.
Copy 
{
    "data": {
        "type": "EXPORT",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {},
        "name": "Example Export 2",
        "steps": [],
        !!"status": "completed"!!
    }
}

docs

Located in the details object, the docs object displays the following properties:

  • processed: The amount of records successfully processed by the operation.

  • total: The total records processed.
Copy 
"details": {
  "dataLocation": {},
  "chunks": [],
  "docs": {
    !!"total": 4,!!
    !!"processed": 4!!
    }
  }
}

location

Located in the files array, the location property contains the file key used to create the download file.
Copy 
"files": [
    {
      "destination": {
      "name": "0",
      !!"location": "unqorkResource/export/78bdc-df4a-41a61/0.csv",!!
      "format": "CSV"
        }
    }
 ]

BDO Delete - Job Details Response

In this example, the Get Job Details API on a BDO Delete operation displays the following response:

Copy 
{
    "id": "6728e46499610f86e6fac431",
    "created": 1730733156824,
    "modified": 1730733283591,
    "createdBy": "creatorName@companyName.com",
    "modifiedBy": null,
    "data": {
        "type": "DELETE",
        "collection": "658c3c6270573d1027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Delete Example",
        "steps": [
            {
                "stepType": "DELETE",
                "id": "658c3c6270573d1027a0f3a2",
                "status": "completed",
                "options": {
                    "delete": {
                        "mode": "DELETE",
                        "dataSource": "MODULE",
                        "targetRecords": "ARCHIVED"
                    }
                },
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                },
                "collection": "658c3c6b7057db027a0f3a2",
                "details": {
                    "dataLocation": {
                        "database": "unqork",
                        "collection": "submissions",
                        "entityType": "MODULE"
                    },
                    "chunks": [
                        {
                            "chunkQueueId": "0",
                            "status": "completed",
                            "docs": {
                                "total": 32,
                                "processed": 32
                            },
                            "auditFileKey": "unqorkResource/audit/6728e48799610f86e6fac432/0.json.gz",
                            "startTime": "2024-11-04T15:14:10.970Z",
                            "endTime": "2024-11-04T15:14:11.278Z"
                        }
                    ],
                    "docs": {
                        "total": 32,
                        "processed": 32
                    }
                }
            }
        ],
        "status": "completed"
    }
}

Using the above example response, the following properties might be important for Creator Also known as Unqork Users, or Designer Users; is anyone who is inside the Unqork platform. processes or pipeline development:

Response Property Description Example

status

The current status of the operation. The overall operation status is displayed at the bottom of the data object. When troubleshooting, review the status property contained in each stepType of the steps array. This value is also reflected in the Job Tracker's Job Status column.

Status values include:

  • inProgress: The delete operation is still processing the file(s).

  • completed: The delete is complete.

  • canceled: The delete has been manually stopped by the user.

  • failed: The delete failed. Review each stepType's status value to determine which process failed.
Copy 
{
    "data": {
        "type": "DELETE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {},
        "name": "Example Delete",
        "steps": [],
        !!"status": "completed"!!
    }
}

docs

Located in the details object, the docs object displays the following properties:

  • processed: The amount of records successfully processed by the operation.

  • total: The total records processed.
Copy 
"details": {
  "dataLocation": {},
  "chunks": [],
  "docs": {
    !!"total": 4,!!
    !!"processed": 4!!
    }
  }
}

Resources