Bulk Data Operations: Delete

Overview

The Bulk Data Operations (BDO) Delete API APIs (application programming interfaces) are a set of protocols and definitions developers use to build and integrate application software. APIs act as the connective tissue between products and services. endpoint lets you soft delete A soft delete is a data persistence strategy that retains deleted records in a database instead of permanently removing them. Soft deletes can help protect data from accidental or malicious deletion, and they can also improve performance. Also known as archiving. or hard delete Hard Delete, also known as "Permanent Delete" or "Destroy", irreversibly erases a file or submission record. There is no option to recover hard deleted files.  record collections containing submission data Also known as Record Data. Submission data consists of information saved by Unqork components. View submission data by using the angular command, or in Record Collections. stored in the Unqork platform.  You can make a delete request using curl, Postman, or any other service that supports a POST The HTTP POST method is used to send data to a server to create or update a resource, typically including the data in the body of the request. request.

To discover how to set up API calls using a third-party program, view our Testing API Endpoints Using Third-Party Applications article.

You can test the BDO Delete endpoint using the Unqork Services page. To access the Unqork Services page, enter the following address into your browser: https://{environment}.unqork.io/fbu/uapi/docs/documentation/#/Bulk%20Operations/delete. Replace {environment} with the name of your Unqork environment.

Understanding Soft Delete and Hard Delete

The BDO Delete operation contains two stages of functionality: soft delete and hard delete. The first time you run the BDO Delete Operations on a set of records, it soft deletes A soft delete is a data persistence strategy that retains deleted records in a database instead of permanently removing them. Soft deletes can help protect data from accidental or malicious deletion, and they can also improve performance. Also known as archiving. (archives) them. Running the operation again on the same set of records using the destroy query parameter hard deletes Hard Delete, also known as "Permanent Delete" or "Destroy", irreversibly erases a file or submission record. There is no option to recover hard deleted files. all archived records, including records that were archived at an earlier time.

Soft Delete

The first time the BDO Delete operation runs on one or more records, it soft deletes (archives) them. Soft delete removes records from the application's Record Collections, but retains them in the database. This functionality allows you to recover the records at a later time.

While this feature is in early access, Unqork recommends exporting a copy of the record collection before deleting it. Please reach out to your Unqork representative if a record collection has been soft deleted and needs to be recovered.

Hard Delete

Running the BDO Delete operation a second time on the same set of records using the destroy: trueparameter permanently and irreversibly deletes the records. Doing so is useful for records containingPII Personal Identifiable Information (PII) is any representation of information that permits the identity of an individual to whom the information applies to be reasonably inferred by either direct or indirect means. or other sensitive information that should not be retained for regulatory or safety reasons.

Deleting Submission Data Using Bulk Data Operations

The BDO Delete operation uses the HTTP HTTP (Hypertext Transfer Protocol) is an application-layer protocol used to transmit hypermedia documents like HTML. POST The HTTP POST method is used to send data to a server to create or update a resource, typically including the data in the body of the request. method and the following endpoint: https://{environment}.unqork.io/fbu/uapi/bulk-operations/delete. Replace {environment} with the name of your Unqork environment.

BDO Delete removes a maximum of 500,000 records per operation. Collections larger than 500k require the operation to be run multiple times.

To make calls using third-party programs, you must add an authorization key/value pair to the header. To learn more, view our Testing API Endpoints Using Third-Party Applications article.

Query String Parameters

{
   "destroy": Boolean
}

The following parameter is available for the Bulk Data Operation Delete endpoint:

Body Parameters

{
  "dataModelOrModuleId": "string",
  "name": "string"
  "dateFieldToFilterOn": "created",
  "dateFieldStartToFilterOn": "string",
  "dateFieldEndToFilterOn": "string",
}

The following fields are available for the Bulk Data Operation Delete schema Schema is a declarative language that defines the structure, content, and constraints of the data.:

Example Response

Below is an example response when deleting records:

{
  "id": "6728e46499610f86e6fac431",
  "created": 1730733156824,
  "modified": 1730733156824,
  "createdBy": "creatorName@creatorCompany.com",
  "modifiedBy": "creatorName@creatorCompany.com",
  "data": {
    "type": "DELETE",
    "collection": "658c3c6b70573db027a0f3a2",
    "dataLocation": {
      "database": "unqork",
      "collection": "submissions",
      "entityType": "MODULE"
    },
    "name": "Delete Example",
    "steps": [
      {
        "stepType": "DELETE",
        "status": "created",
        "options": {
          "delete": {
            "mode": "DELETE",
            "dataSource": "MODULE",
            "targetRecords": "ARCHIVED"
          }
        },
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {
          "database": "unqork",
          "collection": "submissions",
          "entityType": "MODULE"
        }
      }
    ],
  }
}

Viewing a Bulk Data Delete Operation Job's Details

After starting a Bulk Data Delete, you can view the job details using the /fbu/uapi/bulk-operations/job/{jobId} endpoint and the GET The HTTP GET method is used to request data from a server, typically retrieving information without modifying any resources. method. Copy the BDO Delete response's id value and replace the job detail's {jobId}placeholder with it. For example, https://{environment}.unqork.io/fbu/uapi/bulk-operations/job/67169684ec1bf4776fa760a5.

Example Job Details Response

Below is an example of a Job Details endpoint response:

{
    "id": "6728e46499610f86e6fac431",
    "created": 1730733156824,
    "modified": 1730733283591,
    "createdBy": "creatorName@companyName.com",
    "modifiedBy": null,
    "data": {
        "type": "DELETE",
        "collection": "658c3c6270573d1027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Delete Example",
        "steps": [
            {
                "stepType": "DELETE",
                "id": "658c3c6270573d1027a0f3a2",
                "status": "completed",
                "options": {
                    "delete": {
                        "mode": "DELETE",
                        "dataSource": "MODULE",
                        "targetRecords": "ARCHIVED"
                    }
                },
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                },
                "collection": "658c3c6b7057db027a0f3a2",
                "details": {
                    "dataLocation": {
                        "database": "unqork",
                        "collection": "submissions",
                        "entityType": "MODULE"
                    },
                    "chunks": [
                        {
                            "chunkQueueId": "0",
                            "status": "completed",
                            "docs": {
                                "total": 32,
                                "processed": 32
                            },
                            "auditFileKey": "unqorkResource/audit/6728e48799610f86e6fac432/0.json.gz",
                            "startTime": "2024-11-04T15:14:10.970Z",
                            "endTime": "2024-11-04T15:14:11.278Z"
                        }
                    ],
                    "docs": {
                        "total": 32,
                        "processed": 32
                    }
                }
            }
        ],
        "status": "completed"
    }
}

Using the above example response, the following properties might be important for Creator Also known as Unqork Users, or Designer Users; is anyone who is inside the Unqork platform. processes or pipeline development:

{
    "data": {
        "type": "DELETE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {},
        "name": "Example Delete",
        "steps": [],
        "status": "completed"
    }
}

"details": {
  "dataLocation": {},
  "chunks": [],
  "docs": {
    "total": 4,
    "processed": 4
    }
  }
}

Resources

• Bulk Data Operations Landing Page

• Import Submissions using Bulk Data Operations

• Export Submissions using Bulk Data Operations

• Update Submissions using Bulk Data Operations

• Create a Download Link for Data Files

• Create an Upload Link for Data Files

• Call API Endpoints Using a Third-Party Service