Bulk Data Operations: Delete

This feature is currently in production early-access. Functionality is subject to change as this feature is prepared for general access.

Overview

The Bulk Data Operations (BDO) Delete API APIs (application programming interfaces) are a set of protocols and definitions developers use to build and integrate application software. APIs act as the connective tissue between products and services. Endpoint lets you soft delete A soft delete is a data persistence strategy that retains deleted records in a database instead of permanently removing them. Soft deletes can help protect data from accidental or malicious deletion, and they can also improve performance. Also known as archiving. or hard delete Hard Delete, also known as "Permanent Delete" or "Destroy", irreversibly erases a file or submission record. There is no option to recover hard deleted files.  record collections containing submission data Also known as Record Data. Submission data consists of information saved by Unqork components. View submission data by using the angular command, or in Record Collections. stored in the Unqork platform.  You can make a delete request using curl, Postman, or any other service that supports a POST The HTTP POST method is used to send data to a server to create or update a resource, typically including the data in the body of the request. request.

To discover how to set up API calls using a third-party program, view our Testing API Endpoints Using Third-Party Applications article.

You can test the BDO Delete endpoint using the Unqork Services page. To access the Unqork Services page, enter the following address into your browser: https://{environment}.unqork.io/fbu/uapi/docs/documentation/#/Bulk%20Operations/delete. Replace {environment} with the name of your Unqork environment.

Understanding Soft Delete and Hard Delete

The BDO Delete operation contains two stages of functionality: soft delete and hard delete. The first time you run the BDO Delete Operations on a set of records, it soft deletes A soft delete is a data persistence strategy that retains deleted records in a database instead of permanently removing them. Soft deletes can help protect data from accidental or malicious deletion, and they can also improve performance. Also known as archiving. (archives) them. Running the operation again on the same set of records using the destroy query parameter hard deletes Hard Delete, also known as "Permanent Delete" or "Destroy", irreversibly erases a file or submission record. There is no option to recover hard deleted files. all archived records, including records that were archived at an earlier time.

Soft Delete

The first time the BDO Delete operation runs on one or more records, it soft deletes (archives) them. Soft delete removes records from the application's Record Collections, but retains them in the database. This functionality allows you to recover the records at a later time.

While this feature is in early access, Unqork recommends exporting a copy of the record collection before deleting it. Please reach out to your Unqork representative if a record collection has been soft deleted and needs to be recovered.

Hard Delete

Running the BDO Delete operation a second time on the same set of records using the destroy: trueparameter permanently and irreversibly deletes the records. Doing so is useful for records containingPII Personal Identifiable Information (PII) is any representation of information that permits the identity of an individual to whom the information applies to be reasonably inferred by either direct or indirect means. or other sensitive information that should not be retained for regulatory or safety reasons.

Deleting Submission Data Using Bulk Data Operations

The BDO Delete operation uses the HTTP HTTP (Hypertext Transfer Protocol) is an application-layer protocol used to transmit hypermedia documents like HTML. POST The HTTP POST method is used to send data to a server to create or update a resource, typically including the data in the body of the request. method and the following endpoint: https://{environment}.unqork.io/fbu/uapi/bulk-operations/delete. Replace {environment} with the name of your Unqork environment.

BDO Delete removes a maximum of 500,000 records per operation. Collections larger than 500k require the operation to be run multiple times.

To make calls using third-party programs, you must add an authorization key/value pair to the header. To learn more, view our Testing API Endpoints Using Third-Party Applications article.

Query String Parameters

Copy 
{
   "destroy": boolean
}

The following parameter is available for the Bulk Data Operation Delete endpoint:

Parameter

Data Type

Description

 Required

destroy

boolean

After soft deleting a set of records, permanently delete them by adding the destroy=truequery to the BDO endpoint. Then, run the operation again. For examplehttps://envName.unqork.io/fbu/uapi/bulk-operations/delete?destroy=true.

This setting has no effect if records have not been soft deleted A soft delete is a data persistence strategy that retains deleted records in a database instead of permanently removing them. Soft deletes can help protect data from accidental or malicious deletion, and they can also improve performance. Also known as archiving. first.

No

Body Parameters

Copy 
{
  "dataModelOrModuleId": "string",
  "name": "string"
}

The following fields are available for the Bulk Data Operation Delete schema Schema is a declarative language that defines the structure, content, and constraints of the data.:

Key

Possible Values

Description

 Required

dataModelOrModuleId

string (no spaces)

Specify the Data Model ID, or Module ID, containing the data you want to import.

To find a Data Model's ID value, view our How to: Query Data Model Records Data article.

Yes

name

string

Enter a value to name the import operation by. Names can include spaces. This value also displays in the Job Tracker's File Name column.

Yes

Example Response

Below is an example response when deleting records:

Copy 
{
  "id": "6728e46499610f86e6fac431",
  "created": 1730733156824,
  "modified": 1730733156824,
  "createdBy": "creatorName@creatorCompany.com",
  "modifiedBy": "creatorName@creatorCompany.com",
  "data": {
    "type": "DELETE",
    "collection": "658c3c6b70573db027a0f3a2",
    "dataLocation": {
      "database": "unqork",
      "collection": "submissions",
      "entityType": "MODULE"
    },
    "name": "Delete Example",
    "steps": [
      {
        "stepType": "DELETE",
        "status": "created",
        "options": {
          "delete": {
            "mode": "DELETE",
            "dataSource": "MODULE",
            "targetRecords": "ARCHIVED"
          }
        },
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {
          "database": "unqork",
          "collection": "submissions",
          "entityType": "MODULE"
        }
      }
    ],
    "directGqlCall": false
  }
}

Viewing a Bulk Data Delete Operation Job's Details

After starting a Bulk Data Delete, you can view the job details using the /fbu/uapi/bulk-operations/job/{jobId} endpoint and the GET The HTTP GET method is used to request data from a server, typically retrieving information without modifying any resources. method. Copy the BDO Delete response's id value and replace the job detail's {jobId}placeholder with it. For example,https://{environment}.unqork.io/fbu/uapi/bulk-operations/job/67169684ec1bf4776fa760a5.

Example Job Details Response

Below is an example of a Job Details endpoint response:

Copy 
{
    "id": "6728e46499610f86e6fac431",
    "created": 1730733156824,
    "modified": 1730733283591,
    "createdBy": "creatorName@companyName.com",
    "modifiedBy": null,
    "data": {
        "type": "DELETE",
        "collection": "658c3c6270573d1027a0f3a2",
        "dataLocation": {
            "database": "unqork",
            "collection": "submissions",
            "entityType": "MODULE"
        },
        "name": "Delete Example",
        "steps": [
            {
                "stepType": "DELETE",
                "id": "658c3c6270573d1027a0f3a2",
                "status": "completed",
                "options": {
                    "delete": {
                        "mode": "DELETE",
                        "dataSource": "MODULE",
                        "targetRecords": "ARCHIVED"
                    }
                },
                "dataLocation": {
                    "database": "unqork",
                    "collection": "submissions",
                    "entityType": "MODULE"
                },
                "collection": "658c3c6b7057db027a0f3a2",
                "details": {
                    "dataLocation": {
                        "database": "unqork",
                        "collection": "submissions",
                        "entityType": "MODULE"
                    },
                    "chunks": [
                        {
                            "chunkQueueId": "0",
                            "status": "completed",
                            "docs": {
                                "total": 32,
                                "processed": 32
                            },
                            "auditFileKey": "unqorkResource/audit/6728e48799610f86e6fac432/0.json.gz",
                            "startTime": "2024-11-04T15:14:10.970Z",
                            "endTime": "2024-11-04T15:14:11.278Z"
                        }
                    ],
                    "docs": {
                        "total": 32,
                        "processed": 32
                    }
                }
            }
        ],
        "status": "completed"
    }
}

Using the above example response, the following properties might be important for Creator Also known as Unqork Users, or Designer Users; is anyone who is inside the Unqork platform. processes or pipeline development:

Response Property Description Example

status

The current status of the operation. The overall operation status is displayed at the bottom of the data object. When troubleshooting, review the status property contained in each stepTypeof thesteps array. This value is also reflected in the Job Tracker's Job Status column.

Status values include:

  • inProgress: The delete operation is still processing the file(s).

  • completed: The delete is complete.

  • canceled: The delete has been manually stopped by the user.

  • failed: The delete failed. Review each stepType's status value to determine which process failed.
Copy 
{
    "data": {
        "type": "DELETE",
        "collection": "658c3c6b70573db027a0f3a2",
        "dataLocation": {},
        "name": "Example Delete",
        "steps": [],
        !!"status": "completed"!!
    }
}

docs

Located in the details object, the docsobject displays the following properties:

  • processed: The amount of records successfully processed by the operation.

  • total: The total records processed.
Copy 
"details": {
  "dataLocation": {},
  "chunks": [],
  "docs": {
    !!"total": 4,!!
    !!"processed": 4!!
    }
  }
}

Resources