Audit Logging
This document will walk through the audit logging component within Data Refinery.
Table of contents
Overview
Data Refinery has an audit logging system designed to enable governance, compliance, and auditing. The audit logging system can store and query for audit events, which are changes made to the system by a user. Specifically, the events track changes to Sources over time.
The audit logging feature is enabled by default and cannot be disabled. Events are stored automatically and are immutable.
Data Security
The audit log system does not store or return any sensitive information that a user cannot access via other APIs. This includes user password hash, salt, and SSO tokens. In addition, any data records within a Source will not be stored in the audit log, even if they are accessed via the Query APIs.
Permissions
Permissions governing the audit log component can be split into 2 halves: creating logs and querying logs.
Creating Audit Logs
Due to the nature of the audit log, any user that can cause changes to DR Designer can also indirectly create audit logs via their primary actions.
There is no exposed endpoint to directly create audit logs because users cannot insert arbitrary logs without actually performing the prescribed changes.
Querying Audit Logs
The AUDIT_READALL permission grants users the ability to view all logs. This is a system-level permission. Users with the USER_ADMIN permission can assign the AUDIT_READALL permission to any active user.
Users without the AUDIT_READALL permission can only view events related to the Projects and Sources they otherwise have view access to. If a user can view a Source or Project via the User Interface (UI) or API, then they can also view the audit events that are related to those resources. An exception to this rule would be users with the PROJECT_CREATOR permission, who can only see audit events for Sources they have access to via UPRs.
Audit Events
An audit event contains the following information:
| Field | Data Type | Description |
|---|---|---|
| ID | int | unique numerical ID of the event |
| Event Name | string | indicates the type of change; options are described below |
| Event Time | timestamp | when the event took place |
| Requester | string (UUID) | ID of the user that made the change |
| Source ID | int | ID of the Source, if applicable |
| Project ID | int | ID of the Project, if applicable |
| Event record | string | additional information about the event |
The types of events tracked by the audit log system include:
| Event Name | Description | Event record |
|---|---|---|
| CreateProject | Creating a new Project | JSON of the created Project |
| UpdateProject | Updating a Project’s metadata | JSON of the updated Project |
| DeleteProject | Deleting a Project | N/A |
| AssociateSource | Associating a Source to a Project | SourceID that was associated to the Project |
| DisassociateSource | Disassociating a Source from a Project | SourceID that was disassociated from the Project |
| AssignUPR | Assigning UPR to a Project | UPR Object including ID, UserID, and RoleID |
| RemoveUPR | Removing UPR from a Project | UPR Object including ID, UserID, and RoleID |
Note. Usually one API request results in one audit event. However, some APIs calls, such as creating a Project with multiple Owner roles, may result in multiple audit events. These events will be logged separately with no guarantee of order.
Querying Audit Events
The DR Designer API provides REST endpoints to query for audit events. The following query parameters are available to use:
| Parameter | Type and Restrictions | Description |
|---|---|---|
| eventName | string, case insensitive | Get all events of the given name |
| sourceID | int, min=1 | Get all events for the given sourceID |
| projectID | int, min=1 | Get all events for the given projectID |
| requester | UUID string, case insensitive | Get all events made by a specified user |
| before | timestamp | Get all events on or before (<=) the timestamp |
| after | timestamp | Get all events on or after (>=) the timestamp |
Expected functionality of the query API:
- When multiple parameters are provided, they are compounded with “AND” logic.
- String values only match when they are strictly equal, i.e. no fuzzy search.
- The endpoint will always sort the results by “EventTime” descending (most recent event displays first).
- If more than 1000 records are available, only the first 1000 records will be returned.
Example
For example, to find all “AssignUPR” events for a given Project, the following query can be used:
curl -X GET "https://example.kingland-data-refinery.com/api/events?eventName=assignupr&projectID=12" \
-H "Authorization: Bearer $JWT_TOKEN"
Refer to the DR Designer API Documentation for more details on querying audit events.