Export Live Data

Data Refinery Workbench provides export APIs to download Live Data objects and their Attributes. The data can be exported to a standardized JSONL or Excel file. The export is a 2 step asynchronous process: create a new export request and download the exported file when it is ready.

Table of contents

Create a New Export

The Create Export POST API submits a new request to export Live Data from Workbench. The Entity Type ID, Data Object ID, Export File Type, and up to 3 Attribute filter criteria can be provided when creating the export. The export job is queued to extract the requested data from Workbench, format it to the appropriate structure, and then write it to the desired file type. The response contains the ID of the export, which is used to check the status, and download the exported data once it is ready. Refer to the DR Workbench Create Export API reference for more information.

API Parameters

The Create Export API parameters specify the data to be exported and the type of export. Data related to a single Entity Type can be exported to the same export file. This ensures a uniform structure and format of the exported file. Additional filter criteria to select data by specific Attributes can be added. Refer to the Managing Entity Types documentation for more information about Entity Types and Attributes.

  • EntityTypeID - Required. Unique ID of the Entity Type definition. The sample request specifies the entityTypeID as 1 which must be the unique ID of an existing Entity Type definition in DR Workbench.
  • AttributeTypeID and AttributeValue pairs - Optional. If provided, the Data Objects that match the provided Attribute ID and its value will be included in the export. Up to 3 pairs can be specified - attributeTypeID1 and attributeValue1, attributeTypeID2 and attributeValue2, and attributeTypeID3 and attributeValue3. The sample request specifies one pair of an Attribute filter to select the Data Objects with Attribute Type ID 3 having “United States” as value.
  • DataObjectID - Optional. If provided, the single Data Object that matches the ID will be exported.
  • FileType - Optional. JSONL or XLSX, default is JSONL.

Note. The export process estimates the size of data being requested to be in the export. The maximum export file size is 50GiB for JSONL, and 2GiB for Excel. If the estimate exceeds the maximum file size, then the create export request will fail with HTTP 422 Unprocessable Content error, and the export file will still be created with only a partial export. Refine the data selection by adjusting the Attribute and value filter criteria.

Create an Export Sample Request 

curl -X 'POST' \
  'https://example.workbench.kingland-data-refinery.com/api/export' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
      "entityTypeID": 1,
      "attributeTypeID1": 3,
      "attributeValue1": "United States"
    }'

Create an Export Sample Response

The ID in the response is the ID required for checking the status and downloading the exported file once available. The create export response is similar to the response from GET /export API for getting the export status. Refer to the Get Export Status section for more information on all of the fields. Some fields are omitted for brevity.

{
  "ID": 1,
  "startTime": "2024-08-05T22:31:30.152539Z",
  "completionTime": "0001-01-01T00:00:00Z",
  "status": 1,
  "exportedFileLocation": null,
  "lastHeartbeatTime": "2024-08-05T22:31:30.736121Z",
  "exportJobID": "9ab861da-947f-4da5-afb3-180a4c2e3e15",
  "stringStatus": "IN_PROGRESS"
}

Get Export Status

Once the export is created, the export job is queued for processing. If there is no queue, the export job will start quickly to fetch the required data and write to a JSONL or Excel file. Depending on the size of the data being exported, it might take some time for the data export file to be ready for download. The export status API can be used to check the progress of the export process. Refer to the DR Workbench Get Export API for more information.

Get Export Status Sample Request

The ID query parameter is the ID of the export job to check the status. This is not the exportJobID string in the create export response, but instead it’s the numeric ID value.

curl -X 'GET' \
  'https://dev.workbench.kingland-data-refinery-dev.com/api/export?ID=1' \
  -H 'accept: application/json

Get Export Status Sample Response

The export job will periodically update the status and heartbeat time to notify that the job is still processing the Data Objects and writing to export the file.

  • StringStatus - IN_PROGRESS, SUCCESSFUL or FAILED. This is the status to look for.
  • Status - numeric code corresponding to StringStatus field. 1 - IN_PROGRESS, 2 - SUCCESSFUL and 3 - FAILED.
  • ExportFileLocation - Internal export file that was created. The download API will return this file. This field will be populated if the export job succeeds.
  • ExportJobID - Internal ID of the export job processing the request.
  • StartTime - Time the export job started.
  • CompletionTime - Populated with the timestamp of when the export job completed.
  • LastHeartbeatTime - This is the latest notification timestamp from the export job. Ideally this should be less than a minute ago if the export job is still in progress. The heartbeats, or periodical notifications, will stop once the job completes (either successfully or failure).

Some fields are omitted for brevity.

[
  {
    "ID": 1,
    "startTime": "2024-08-05T22:31:30.152539Z",
    "completionTime": "2024-08-05T22:31:53.292938Z",
    "status": 2,
    "exportedFileLocation": "1/exported.jsonl",
    "lastHeartbeatTime": "2024-08-05T22:31:53.292938Z",
    "exportJobID": "9ab861da-947f-4da5-afb3-180a4c2e3e15",
    "stringStatus": "SUCCESSFUL"
  }
]

Download the Exported Data

Once the export has completed successfully, the exported data file is ready to be downloaded. The GET /export/{ID}/data API can be used to download the generated file. Refer to the DR Workbench Download Export File API for more information.

Download Export File Sample Request

The path parameter in the URL is the numeric ID of the export job created. This is not the exportJobID string in the create export response, but instead it’s the numeric ID value returned by the CREATE or GET APIs.

curl -X 'GET' \
  'https://dev.workbench.kingland-data-refinery-dev.com/api/export/1/data' \
  -H 'accept: text/plain'

Download Export File Sample Response

The generated export file data will be streamed directly in the response body. The file name will be in the format <Export ID>.<jsonl xlsx>. Here is a sample header of the response. 
content-disposition: attachment; filename="1.jsonl"
content-length: 167
content-type: text/plain; charset=utf8
date: Mon,05 Aug 2024 22:47:19 GMT
strict-transport-security: max-age=315360000; includeSubdomains; preload
x-content-type-options: nosniff

Here is an example of 2 Live Data Objects in a DR Workbench Live Data Store that are to be converted to export JSON format and written to the exported JSONL file - one line for each object. Some fields are omitted for brevity.

Sample Data Objects:

[
  {
    "EntityTypeID": 1,
    "EntityType": { "Name": "Legal Entity" },
    "Attributes": [
      {
        "AttributeTypeID": 1,
        "AttributeType": { "Name": "Legal Name" },
        "Value": "Kingland Corporation"
      },
      {
        "AttributeTypeID": 2,
        "AttributeType": { "Name": "Address Line 1" },
        "Value": "1401 6th Ave South"
      },
      {
        "AttributeTypeID": 3,
        "AttributeType": { "Name": "Country" },
        "Value": "United States"
      }
    ]
  },
  {
    "EntityTypeID": 1,
    "EntityType": { "Name": "Legal Entity" },
    "Attributes": [
      {
        "AttributeTypeID": 1,
        "AttributeType": { "Name": "Legal Name" },
        "Value": "XYZ Corporation"
      },
      {
        "AttributeTypeID": 2,
        "AttributeType": { "Name": "Address Line 1" },
        "Value": "123 Main Street"
      },
      {
        "AttributeTypeID": 3,
        "AttributeType": { "Name": "Country" },
        "Value": "United States"
      }
    ]
  }
]

Expected export file format:

{"Legal Name": "Kingland Corporation", "Address Line 1":"1401 6th Ave South", "Country":"United States"}
{"Legal Name": "XYZ Corporation", "Address Line 1":"123 Main Street", "Country":"United States"}

Copyright © 2025 Kingland Systems LLC