Skip to main content

Data Retention Enhancements Announcement

Data retention policies within the platform are being enhanced by JupiterOne, and as valued JupiterOne customers we wanted to update you on these improvements. Historically, clarity in our application's data retention policies has not been emphasized enough; there are numerous instances where historical data has been retained, and JupiterOne aims to standardize this process. The objective is to ensure that correct and clear expectations regarding data retention are maintained for all customers, and that data is expunged in a timely manner.

In summary, increased data retention and a clearer understanding of where that retention applies and how it operates will be experienced by all customers:

  • Graph data: Unlimited retention for all customers
  • Soft-deleted data: 7 days for existing customers
  • Alert aggregate metrics: Unlimited retention for all customers
  • Alert raw results: 180 days retention for existing customers

Data under Consideration​

When data retention is discussed by JupiterOne, it pertains to customer data in the platform, specifically the information captured in JupiterOne through integrations, data sent over the API, or manual data entry. This is referred to as the "data plane" in JupiterOne, specifically:

  • The data that is stored and accessible in the graph, i.e., the data accessed via running a J1QL query. Ref: Data Retention in the Graph
  • Historical data returned by alerts, including both raw results from the queries and the high-level metrics for generating the visualizations. Ref: Alert Results and Metrics

Some data in JupiterOne is not affected by this data retention update and is considered "control plane" or audit data. This includes information about customer user accounts, audit logs, and logs from integration runs.

Graph Data Retention​

Moving forward, data retention in the graph is intended to be for an "unlimited" period. As long as a customer contract is in place, any data sent to the graph (and not deleted) will be retained indefinitely.

This implies that there is no requirement to resend data after a period of time or in any other way "refresh" it unless the data requires an update.

NOTE: If your own data is sent to JupiterOne using a custom integration or directly via the API, it is recommended that you ensure you are able to resend that data should it be necessary.

Two notable ways in which data can be routinely deleted from the graph are:

  1. If an asset has been removed from the source system (e.g., an AWS EC2 instance is terminated), JupiterOne will mark that data as "deleted," and the data will be permanently removed from the graph in the future. Ref: Soft Deleted Data
  2. An integration instance is deleted by an administrator. When this occurs, JupiterOne will delete all assets and relationships that are created by that integration and clean up the graph.

Soft Deleted Data​

When data is "deleted" from the graph, JupiterOne does not immediately remove that data; it is retained for a short period of time so that queries can capture details of entities and relationships being removed (e.g., "how many EC2 machines were removed in the last 24 hours").

Soft-deleted data is excluded from query results by default and can be identified with the property _deleted=true, and the data will be permanently deleted from the graph 7 days after it is soft deleted. This allows any rules, which can have up to a 7-day polling interval, to capture details on deleted assets.

Alert Results and Metrics​

Recognition by JupiterOne that once controls and queries that are critical are identified, it is important to retain the results of those evaluations. Two outputs from alerts that are relevant for data retention are metrics (i.e., total number of results, and if triggers fired), and the raw results of the queries.

The aggregate metric from the alerts is intended to be retained indefinitely as long as the alert is in place. Alert raw results are intended to be retained for 180 days; this allows customers to go back up to 6 months and re-fetch the raw results from an alert run.

NOTE: Alert actions are designed to facilitate the continuous and near real-time export of alert results. This should be the primary mode of long-term alert retention for customers.

Timeline​

When are these changes being made? As a customer, significant changes in the way JupiterOne is used will not be seen. Work will be conducted through December 2023 and January 2024 to ensure adherence to these clearer data retention policies.

If any questions or concerns arise, please reach out to support@jupiterone.com.

Contents