Entity property normalization

note

The following documentation is for a feature that is not yet completely enabled. You may see a preview of some of these normalizations already.

Normalization

JupiterOne normalizes some entity properties on specific classes to make querying easier and to make query results more consistent. In the event that we normalize properties into a property provided in a raw upload, or from an integration, the original raw value is persisted in raw_<property> to prevent any data loss.

Devices

Collect and normalize serial numbers

_class	matching property name (one of)	normalizes into	matching property value (one of)
Device	serial OR serialNumber	serials	/^((([^0-9a-f]*[0-9a-f]){7}[0-9a-z\W]*)$\|(([^0-9a-z]*[0-9a-z]){12}[0-9a-z\W]*))$/i

Example Normalizations

Collect IP(v4) addresses

_class	matching property name (one of)	normalizes into	matching property value (one of)
Device	ipAddresses OR ipAddress OR publicIpAddress OR privateIpAddress OR lastIpAddress OR network.ipAddress OR lastExternalIpAddress OR IP_Address	ipAddress	/^(((25[0-5]\|2[0-4][0-9]\|[01]?[0-9][0-9]?)\.){3}(25[0-5]\|2[0-4][0-9]\|[01]?[0-9][0-9]?))$/

Example normalizations

Collect associated email addresses

https://html.spec.whatwg.org/multipage/input.html#valid-e-mail-address

_class	matching property name (one of)	collects into	matching property value (one of)
Device	email OR primaryEmail OR recoveryEmail OR contactEmails OR userEmail OR userEmails OR serviceAccountEmails OR serviceAccountEmail OR publisherEmail OR maintainerEmails OR homeEmail OR verifiedEmail OR verifiedEmails OR userEmails OR emailAddress OR workEmail OR Email OR general.assignedUserEmail OR techLeadEmail OR managerEmail OR cooEmail OR ceoEmail OR ctoEmail OR techContractEmail OR registrantContactEmail OR adminContactEmail OR abuseContactEmail	associatedEmailAddresses	/^([a-zA-Z0-9.!#$%&'*+/=?^_{\|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)\*$)/i

Example normalizations

Normalize make

_class	matching property name (one of)	collects into	matching property value (one of)
Device	make OR Manufacturer OR hardwareManufacturer OR hardwareVendor	make	/^(.+)$/

Further refine make based on MAC address

Using IEEE OUI lookups based on the registration authority, we can determine device make from its MAC Address in over thirty thousand MAC address blocks. Below are a few examples:

_class	matching property name (one of)	normalizes into
Device	macAddress	make

Example normalizations

Normalize OS versions

_class	matching property name (one of)	normalizes into	matching property value (one of)
Device	osVersion OR os_version OR operatingSystemVersion OR os OR OS OR Operating System OR operatingSystem OR operatingsystem	osVersion	/^(?:.*?)(?:\b\|\()(\d+(\.\d+)*)(?:\b\|\))(?:.*)$/i

Example normalizations

Normalize OS types

_class	matching property name (one of)	normalizes into	matching property value (one of)
Device	operatingSystem OR operatingsystem OR os OR OS OR osFamily OR os_family OR osName OR osname OR os_name OR Operating System OR os_version OR @osVersion OR osVersion	osType	/(^.*(ios [\d]\|iphone).*$)/i OR /(^.*ipad.*$)/i OR /(^(Cupcake\|Donut\|Eclair\|Froyo\|Gingerbread\|Honeycomb\|Ice Cream Sandwich\|Jelly Bean\|KitKat\|Lollipop\|Marshmallow\|Nougat\|Oreo\|Pie\|Android).*$)/ OR /(^.*ipad.*$)/i OR /(^.*(OS\s?x\|macos).*$)/i OR /((^.*(windows\|microsoft).*$)\|(^.*\s+(win\|ms)(\s\|\d\|-)+.*$)\|(^(win\|ms)(\s\|\d\|-)+.*$))/i OR /^(Linux\|Debian\|Ubuntu\|Fedora\|Red Hat\|CentOS\|Arch(\s?Linux)?\|openSUSE\|Gentoo\|Slackware\|Mint\|Kali(\s?Linux)?\|Zorin(\s?OS)?\|Manjaro\|Mageia).*$/i

Example normalizations

Normalize and collect MAC addresses

_class	matching property name (one of)	normalizes into	matching property value (one of)
Device	macAddress OR macAddresses OR MacAddress OR MACADDRESS OR MACAddress OR MAC_Address OR mac_address OR mac-address OR altMacAddress OR wifiMacAddress OR wifiMacAddresses OR wirelessDeviceMac OR recentDeviceMac OR wifiMac OR mac OR network.macAddress OR bluetoothMac	macAddress	/^([0-9a-f]{1,2})([.:-])?([0-9a-f]{1,2})([.:-])?([0-9a-f]{1,2})([.:-])?([0-9a-f]{1,2})([.:-])?([0-9a-f]{1,2})([.:-])?([0-9a-f]{1,2})$/i

Example normalizations

Normalize encryption status

_class	matching property name (one of)	normalizes into	matching property value (one of)
Device	encrypted OR encryptionStatus OR isEncryptionEnabled OR encryptionState	encryptionStatus	[true] OR [false] OR /^(encrypted\|true\|yes)/i OR /^(not encrypted\|false\|no\|not_encrypted\|notencrypted\|unencrypted)$/i OR /^(unknown)?/i

Example normalizations

Findings

Normalize severity

_class	_type (one of)	matching property name (one of)	matching property value (one of)
Finding	aws_accessanalyzer_finding OR aws_guardduty_finding OR aws_inspector_finding OR aws_inspectorv2_finding OR aws_macie_finding OR azure_advisor_recommendation OR bugcrowd_submission OR cbdefense_alert OR checkmarx_finding OR cisco_amp_finding OR cisco_tenable_finding OR cisco_wiz_finding OR cobalt_finding OR crowdstrike_vulnerability OR cycognito_issue OR detectify_finding OR github_code_scanning_finding OR github_finding OR github_repo_finding OR gitlab_finding OR gitleaks_finding OR hacker_one_report OR hackerone_report OR microsoft_defender_vulnerability OR netskope_compliance_finding OR nowsecure_finding OR nuclei_finding OR orca_finding OR orca_finding_alert OR probely_finding OR sast_scan_finding OR snyk_finding OR sysdig_finding OR tenable_container_finding OR tenable_vulnerability_finding OR trivy_finding OR veracode_finding OR qualys_finding OR qualys_host_finding OR wiz_vunerability_finding	severity OR Severity	/(^(info(rmational)?\|(very low)\|none\|negligible))/i OR /(^low)/i OR /(^(medium\|moderate))/i OR /(^high)/i OR /(^(critical\|(very high)\|hazardous\|urgent))/i OR /(^unknown)/i

Example normalizations