Skip to main content

Changes to JupiterOne Query API

Summary​

Being able to retrieve all results of a query reliably is a critical capability for users of JupiterOne. There have been various improvements made to the JupiterOne query API over the years; JupiterOne is now proactively monitoring customers using legacy and unsafe techniques for retrieving query results.

From the 1st September 2024 two features will be deprecated from the JupiterOne query language and query API. Customers using these features will be proactively contacted through August 2024 to adapt their usage.

  • SKIP will not longer be accepted in the JupiterOne Query Language (J1QL)
  • VariableResultsSize will be forced to true for all Query API requests regardless of the flag set by the caller

Why are we making this change now?​

As part of the graph upgrade program JupiterOne has improved pagination performance, but has also identified that the legacy pagination cursor is now more unstable than before. Customers should be getting reliable results from API queries, we want to accelerate the adoption of the improved pagination and deprecate the legacy approach as soon as possible.

What changes might you need to make?​

Using JupiterOne's GraphQL API it is possible to submit a J1QL query to the service and retrieve results. There are various flags that can be passed to this query interface that impact the behavior and are relevant to this issue.

For API calls using SKIP and LIMIT pagination: use VariableResultSize: true and cursor for requesting addition pages.

For API calls already using cursor: use VariableResultSize: true.

info

JupiterOne has observed examples where API consumers have assumed that receiving less than 250 results for a page (including zero results) indicates that the results pages have been exhausted. This is not a safe assumption, only a NULL cursor being returned indicates that the result pagination is complete.

SKIP and LIMIT versus cursor pagination​

Using SKIP and LIMIT for pagination is not reliable as the order of results isn't guaranteed, specifically the results that you "skip" may not have been returned on a previous call (missed results), or results may be returned on multiple pages (duplicated results).

When using the cursor approach it is important to continue to call the cursor until a NULL cursor is returned, even if a page has no results.

Fixed versus Variable Page Size​

VariableResultSize: true means that a returned page may include any number of results, including 0 (zero) results. For some queries (e.g. FIND UNIQUE ...) it is necessary for the API to respond with ALL results on the first page. Any API consumer should be prepared to deal with many results per page.

Clients and SDKs​

For customers using the client libraries and SDKs there are some details that are relevant to this work

Python Client​

The Python Client was originally built by Auth0. JupiterOne is currently working with Auth0 to adopt the PIP package. The canonical implementation of the python package is available here: https://github.com/JupiterOne/jupiterone-api-client-python/.

Go Client​

The Go client is already compatible with these requirements but defaults to VariableResultSize: false, users should be sure to set the flag to true. This client is pending updates, and is available here: https://github.com/JupiterOne/jupiterone-client-go/.

Node Client​

The node client has been updated to use the correct configuration, and no longer uses SKIP and LIMIT for pagination. This client is pending a release at this time. The client is available here: https://github.com/JupiterOne/jupiterone-client-nodejs/.

Contents