Lookups in App

While most lookups in the app Splunk Security Essentials contain sample data, but some CSV lookups and kvstore collections are key to app functionality. Below is a list of the lookups used for app functionality.

Table of Contents

AppDependencies

Type

CSV file

File Name

AppDependencies.csv

Description

This lookup contains links to the app dependencies for the Analytics Advisor.

bookmark_custom_lookup

Type

kvstore collection

Collection Name

bookmark_custom

Fields in kvstore

_key, _time, showcase_name, status, user, datasource, description, journey

Description

This lookup is deprecated. It enabled a limited custom content capability prior to SSE 2.5. Today, it’s there so that when users ugrade we can grab and update to the latest verison.

bookmark_lookup

Type

kvstore collection

Collection Name

bookmark

Fields in kvstore

_key, _time, showcase_name, status, user, notes

Description

One of the most important SSE lookups, bookmark_lookup is a kvstore collection that stores the bookmark status (including whether content is marked as implemented!) and bookmark notes. The Correlation Search Introspection feature dumps into this lookup.

ColorScheme

Type

CSV file

File Name

ColorScheme.csv

Description

This lookup contains the colours used in the Analytics Advisor. You can change the colour scheme to something else by modifying this lookup.

custom_content_lookup

Type

kvstore collection

Collection Name

custom_content

Fields in kvstore

_key, _time, showcaseId, channel, local_json, json, user

Description

In SSE, we have the ability to create custom content. That content is stored in the custom_content_lookup. For simplicity, we store all the real detail in the json field, and we will (upon page load) grab all of that content and insert into the ShowcaseInfo (via the showcaseId) that is the backbone of the app. Notably, there is a json and a local_json field – it was the intent to leave open the ability to provide overrides for content (either Splunk OOTB content or partner-provided content) by adding a local_json. That capability has yet to be implemented though, so that field just sits there as a quiet reminder of things that could have been.

data_inventory_eventtypes_lookup

Type

kvstore collection

Collection Name

data_inventory_eventtypes

Fields in kvstore

_key, created_time, updated_time, eventtypeId, status, basesearch, search_result, search_status, coverage_level

Description

Pair to data_inventory_products, this kvstore collection stores the status for each data source category (DSC). Most fields aren’t used anymore, it’s primarily there for a simplified Yes/No status indicator. Important note for those digging in here: the DSC is stored in a field called eventtypeId – we renamed this in the UI because while DSCs are thematically similar to eventtypes in Splunk, they’re functionally different and we didn’t want to confuse the matter. Unfortunately, it was too complicated to change the underlying field.

data_inventory_products_lookup

Type

kvstore collection

Collection Name

data_inventory_products

Fields in kvstore

_key, created_time, updated_time, productId, productName, vendorName, eventtypeId, status, basesearch, stage, metadata_json, cim_detail, eventsize, cim_compliant_fields, daily_event_volume, daily_host_volume, desired_sampling_ratio, termsearch, coverage_level, jsonStatus

Description

This kvstore collection contains the list of all of the products configured for data availability. You will see an entry for each product (e.g., “Palo Alto Networks”) with associated metadata (e.g., daily_event_volume, cim_compliance_fields, etc), the location of the data (basesearch for full SPL, termsearch that will work in a | tstats where clause), and most importantly the data source categories (aka DSCs) that this product is mapped to. Important note for those digging in here: the mapped DSCs are stored in a field called eventtypeIds – we renamed this in the UI because while DSCs are thematically similar to eventtypes in Splunk, they’re functionally different and we didn’t want to confuse the matter. Unfortunately, it was too complicated to change the underlying field.

data_source_check_lookup

Type

kvstore collection

Collection Name

data_source_check

Fields in kvstore

_key, _time, showcaseId, showcaseName, searchId, searchName, status

Description

Used only the data source check dashboard, this kvstore collection persists the most recent result of the data source check results. Running the app check every time you load that dashboard can take a long time (like 10 min!) which feels obnoxious when the data probably doesn’t change much. Admittedly, this feature was built to make it less arduous to develop the Auto Dashboarding feature, but it’s useful for everyone! This lookup is paired with data_source_check_outputs_lookup which contains the summarized data_source_check_outputs_lookup, which is no longer actively used.

data_source_check_outputs_lookup

Type

kvstore collection

Collection Name

data_source_check_outputs

Fields in kvstore

_key, _time, elementId, elementName, status

Description

This lookup is paired with data_source_check_lookup, and stores the summaried “per example” status. Prior to the Data Availability framework, this lookup was used to add a filter on the main Security Content page, but it is no longer actively used (though the code does keep it up to date and it cannot be removed – it consumes almost no space)

deleted_custom_content_lookup

Type

kvstore collection

Collection Name

deleted_custom_content

Fields in kvstore

_key, _time, showcaseId, channel, local_json, json, user

Description

In the Custom Content dashboard, you can delete content but then recover it via the recycling bin. This lookup is that recycling bin.

external_content_lookup

Type

kvstore collection

Collection Name

external_content

Fields in kvstore

_key, first_checked, last_checked, last_updated, channel, build

Description

SSE has an extensible collection of external content sources that can be updated. This includes automatically grabbing the latest data from ESCU (or UBA) and grabbing the latest available MITRE ATT&CK and Pre-ATT&CK. Partners also have the option to add create content channels.

kill_chain_phases

Type

CSV file

File Name

kill_chain_phases.csv

Description

This lookup contains the Kill Chain phases and the order they appear in

lightweight_cim_regex_reference_only

Type

CSV file

File Name

lightweight_cim_regex_reference_only.csv

Description

This CSV probably (definitely) should be added into sseidenrichment search command instead if for no other reason than localization, but it’s already here so /shrug. When we package SSE, we will automatically update this CSV with the contents from appserver/static/components/localization/data_inventory.json.

local_search_mappings_lookup

Type

kvstore collection

Collection Name

local_search_mappings

Fields in kvstore

_key, _time, showcaseId, search_title, user

Description

For users who go through the correlation search introspection process, the app retains a connection of enabled correlation searches to MITRE details. This lookup stores a key element of that, making the association of a saved search name (search_title) to the internal showcaseId.

mitre_environment_count

Type

CSV file

File Name

mitre_environment_count.csv

Description

This lookup contains the count of content associated with each Mitre ATT&CK Technique. It is automatically maintained by the app.

mitre_matrix_list

Type

CSV file

File Name

mitre_matrix_list.csv

Description

This lookup contains a list view of the current Mitre ATT&CK Framework. It is automatically maintained by the app.

SSE-data-availability-products-categorization

Type

CSV file

File Name

SSE-data-availability-products-categorization.csv

Description

On the back end of the data availability functionality, we track whether a particular product is high volume (aka, we should do high volume monitoring on it) or not. This lookup enables that. Important: Today, this differentiation isn’t supported out of the box, because everything is run by the lag. But there was a desire to filter for high volume products and track those for event volume as well.

SSE-data-inventory-config

Type

CSV file

File Name

SSE-data-inventory-config.csv

Description

This lookup contains custom configurations you can apply to the data inventory configuration. If you would like to add in data source owner, or SLA, or something like that, you can add it to this lookup.

SSE-data_availability_latency_status

Type

CSV file

File Name

SSE-data_availability_latency_status.csv

Description

This lookup is updated every time the search generates the data availability MLTK model, and stores status on when it was last updated, how mnay products were there, etc. It is used by a variety of health checks on the Data Availability dashboard.

SSE-default-data-inventory-products

Type

CSV file

File Name

SSE-default-data-inventory-products.csv

Description

When we start the data availability functionality, we look for typical source / sourcetypes in your data. Those typical source / sourcetypes come from this CSV file.

sse_app_config_lookup

Type

kvstore collection

Collection Name

sse_app_config

Fields in kvstore

_key, _time, user, param, value

Description

A barely-used location for general system configuration parameters. As of this doc writing, it is only used to store a demo-mode flag, but no reason it couldn’t include other details that we don’t want to make public by default.

sse_bookmark_backup

Type

CSV file

File Name

sse_bookmark_backup.csv

Description

All configuration snapshosts are stored in this CSV file (see? How easy it is to back up everything!)

sse_json_doc_storage_lookup

Type

kvstore collection

Collection Name

sse_json_doc_storage

Fields in kvstore

_key, _time, version, description, json

Description

SSE has an extensible collection of external content sources that can be updated. While most of that content will go into custom_content_lookup, sometimes there are just updated documents. At the time of this doc creation, it is only MITRE ATT&CK and MITRE Pre-ATT&CK that get stored here, but it could be used for any other sources. When the user’s browser grabs the latest MITRE ATT&CK JSON from the MITRE GitHub, it will add it to this kvstore collection. Why a kvstore collection rather than just a custom search command that updates the file on the filesystem? SHC compatibility.

use_cases

Type

CSV file

File Name

use_cases.csv

Description

This lookup contains the high level use cases for Splunk security.