Attribute Validation

The Attribute Validation functionality within Workbench allows administrators to enforce data quality checks on Data Object Attributes by customizing what values are allowed at an Entity Type Attribute level. Attribute Validation rules are configured via the Entity Type Attributes using Regular Expressions (RegEx). Entity Types also allow users to define whether a Data Object’s Attribute values should be validated upon creation.

To manage Entity Types, a user must work with Data Refinery Workbench APIs. This page will explain how to use Data Refinery Workbench APIs to validate Data Object Attribute values.

To configure Entity Types, a user must have the DEFINITION_ADMIN permission.

To Create or Update Live Data without a Workflow, a user must have the WORKFLOW_ADMIN permission.

Table of contents

How Validation Works
Validation Failures
Commonly Used Regular Expressions for Validations
How to Create an Entity Type For Validation
How to Create a Data Object

How Validation Works

Validation is performed at an attribute level using RegEx. When an attribute is updated to a non-empty value, the value will be validated against the regular expression that’s configured in the Attribute Type. An error will be returned if the value fails validation. This validation can occur in multiple areas of Workbench:

When creating a new Data Object, validation will be performed if and only if the Entity Type used in the new Data Object is configured to validate on creation.
When updating an existing Data Object, validation will always be performed.
When creating a new Workflow, validation will never be performed.
When updating an existing Workflow, validation will be performed as a warning.
When transitioning a Workflow to the next step, validation will be performed if and only if the transition being performed is configured to validate on transition.
When completing a Workflow and updating the Live Data Object, validation will always be performed.

Validation Failures

When an object encounters an issue with validation, errors are returned to the end user. If a ValidationMessage has been configured in the Entity Type, the configured message will be returned. If no ValidationMessage has been configured, a default message will be returned instead. The default message is: Attribute doesn't match the following pattern: '<RegEx>'.. This message can be difficult for end users to understand, so it’s a best practice to configure a ValidationMessage in the Entity Type.

Commonly Used Regular Expressions for Validations

The following is a list of commonly used RegEx patterns and is not a comprehensive list of all options. These can be used in the Entity Type Attribute RegEx field to perform common validation actions for Data Object Attribute values.

Warning. When RegEx is input into the API, there may be special characters that need to be escaped. Characters such as quotes or slashes are examples of this. This will depend on what tool is being used to interact with the API. Contact Kingland Support for additional assistance.

Character Patterns

Characters or a set of characters together make a pattern. When a character or pattern is present, it must match in its entirety:

pattern	description
`[xyz]`	Field must contain x, y or z
`[^xyz]`	Field must not contain x, y or z
`[[:alpha:]]`	Field must contain at least one ASCII character
`[[:^alpha:]]`	Field must contain at least one that is not an ASCII characters

Composites

The | symbol may be used to OR multiple patterns together:

pattern	description
`xy`	Field must contain x AND y AND x must be followed by y
`x`\|`y`	Field must contain x OR y

Character Classes

Digits, Words, and Spaces have special character classes that match all values:

pattern	description
`[\d]`	Field must contain at least one digit
`[^\d]`	Field must not contain only digits
`[\D]`	Field must not contain only digits
`[^\D]`	Field must contain at least one digit
`[\w]`	Field must contain at least one word character
`[^\w]`	Field must not contain only word characters
`[\W]`	Field must not contain only word characters
`[^\W]`	Field must contain at least one word character
`[\s]`	Field must contain at least one whitespace
`[^\s]`	Field must not contain only whitespace
`[\S]`	Field must not contain only whitespace
`[^\S]`	Field must contain at least one whitespace
`[.]`	Any character may match this

Negation

The ^ symbol may be used to negate a pattern:

pattern	description
`[^[\d]]`	Field must not only contain digits.

Match Multiple Times

{} blocks may be used to specify how many times a pattern must match, and may include a range:

pattern	description
`\d{3}`	Field must contain exactly 3 digits
`[a-z]{1,3}`	Field must contain between 1 and 3 lowercase characters between a and z
`\d{1}-\d{4}`	Field must contain exactly 1 digit, followed by exactly 1 dash, followed by exactly 4 digits

Examples of Data Validation

The above rules can be combined together to form complicated validation rules, allowing precise control of validating Data Object Attributes:

pattern	description
`^\d{3}-\d{2}-\d{4}$`	Simple Social Security Number Validation, with no validation on whether individual parts are valid
`(?!666`\|`000`\|`9\\d{2})\\d{3}-(?!00)\\d{2}-(?!0{4})\\d{4}`	Social Security Number Validation, validating that individual parts are valid
`^\d{5}(-\d{4}){0,1}$`	Simple US postal code 5 or 9 digit validation
`^\d{4}\/\d{2}\/\d{2}$`	YYYY/MM/DD date format, validating digits in each part

How to Create an Entity Type For Validation

The Entity Type POST API allows users to create an Entity Type. An Entity Type must have at least 1, but not more than 5 key Attributes and can contain no more than 100 Attributes total. The RegEx applied to the Entity’s Attribute Type goes in the Validation field. The message returned, if the data fails validation, will be in the ValidationMessage field. The ValidateOnCreate field, which determines whether to validate a Data Object’s Attributes at creation, defaults to true. Regardless of this value, a Data Object’s Attributes will always be validated during its update.

Note. The Entity Type Name is globally unique and Attribute Type Names are unique within an Entity.

To create an Entity Type, use the following curl command below.

curl -X 'POST' \
  'https://{host}/api/entitytypes' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "legal entity",
    "validateOnCreate": true,
    "attributeTypes": [
        {
            "name": "DoingBusinessAs",
            "isRequired": true,
            "isKey": true
        },
        {
            "name": "Address",
            "isRequired": true,
            "isKey": false
        },
        {
            "name": "email",
            "isRequired": true,
            "isKey": false,
            "validation": "RegEx Here",
            "validationMessage": "Invalid email, does not contain @."
        }
    ]
}'

The response will look similar to this:

Create Entity Type API Sample Response

 
// Some fields are omitted for brevity
[
  {
    "ID": 1,
    "name": "legal entity",
    "validateOnCreate": true,
    "attributeTypes": [
      {
        "ID": 1,
        "name": "DoingBusinessAs",
        "isRequired": true,
        "isKey": true,
        "entityTypeID": 1
      },
      {
        "ID": 2,
        "name": "Address",
        "isRequired": true,
        "isKey": false,
        "entityTypeID": 1
      },
      {
        "ID": 3,
        "name": "email",
        "isRequired": false,
        "isKey": false,
        "entityTypeID": 1,
        "validation": "[@]"
        "validationMessage": "Invalid email, does not contain @."
      }
    ]
  }
]

To view sample API responses for creating new Entity Types, refer to the DR Workbench API Entity Type Reference.

How to Create a Data Object

The Data Object POST API allows users to create a new Data Object if they have an Entity Type ID and its corresponding Attribute Type IDs. All Attributes in a Data Object should correspond to the Attribute Type IDs of the Data Object’s Entity Type. To create the Data Object, the user must give an EntityTypeID and Attributes, which is the array of Data Attributes, each containing an AttributeTypeID and a Value.

To create a Data Object, use the following curl command below.

  curl -X 'POST' \
  'https://{host}/api/data' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '[{
    "entityTypeID": 1,
    "attributes": [
        {
            "attributeTypeID": 1, 
            "value": "Nike"
        },
        {
            "attributeTypeID": 2, 
            "value": "123 1st Street"
        },
        {
            "attributeTypeID": 3,
            "value": "nike@gmail.com"
        }
    ]
}]'

Warning. Data Attribute values are validated upon creation/update. If validation fails the Data Object will not be created. Based on the Entity Type created earlier, if the third attribute did not have the @ symbol in it, then validation would fail. ValidateOnCreate can only make validation optional during creation. Validation is not optional during update, though it is possible to update a field with an empty value (that will not be validated), if it is not required.

To view sample API responses for creating a Data Object, refer to the DR Workbench API Data Object Reference.