Monitor Data Ingest
Table of Contents
There are many methods build for admins to help monitor data availability, but often fewer for users. This guide will help you with the configuration required and then point you to a dashboard built just for security users.
In the app configuration, you can include / exclude different sources of content, allowing you to filter out Splunk solutions you might not own, or avoid seeing the free content from Splunk Security Essentials. Most users will leave this at the default settings.
Find the Configuration menu in the navigation.
Under Enabled Sources you can turn on or off different apps. This will apply globally across the app.
Data Source Categories use standardized searches to find data configured with the tags that are used in Splunk’s Common Information Model. You can also add custom products that either don’t match the Common Information Model, or mark that you have products you expect to add in the future.
The Data Inventory dashboard allows you to configure what products you have in your environment. Products have a variety of metadata (sourcetypes, event volume, CIM compliance) and are connected with data source categories, allowing the app to show you what content can be turned on with your present data.
Here’s an example of several data source categories, under the EDR data source. DSCs are detailed categories that have been proven out through thousands of professional services engagements.
When you first open this page, it will prompt you to use the automated scans. If you install SSE on your production search head, most of the work from this page is automated!
There are four automated introspection steps that pulls a variety of data.
For any sources or sourcetypes that are uncommon, you can tell the app what product it is.
If you have a product that wasn’t detected, or you aren’t installing this app on your production search head, you can always manually add products by clicking Add Product. If you don’t have data for a DSC, you can say No Data Present.
Correlation Search Introspection and Mapping
Tracking what content you have active is key to so much Splunk Security Essentials functionality (enriching the MITRE ATT&CK Matrix, guiding you to the right content, integrations with Splunk Enterprise Security, Risk-based Alerting, the Data Availability Dashboard). This can be accomplished through bookmarking (set status Implemented), but it’s often easier to configure via Correlation Search Introspection on the Bookmarked Content dashboard.
Splunk Security Essentials uses bookmarking to track what content is active in your environment, or to just help you remember what content you want to deploy.
To make the process of recording your active content easier if you’ve installed this app on your production search head, it contains a Correlation Search Introspection feature which walks you through marking active content.
This introspection will pull a list of all of your enabled local scheduled searches that have an action associated with them. It will also automatically enable any directly enabled ES, ESCU, or SSE content.
For most content, the introspection will provide you with the option to indicate that a search is not a security detection, search through all of the out-of-the-box content contained in the app, or create new custom content in the app. Any of these will help you accurately map data source, MITRE, and other metadata for your content.
Splunk Security Essentials includes a search engine to help you search the app and map any detection search to all of the out-of-the-box content.
For content that doesn’t exist in Splunk out-of-the-box, you can create custom content. Custom content shows everywhere throughout the app, just like normal Splunk content.
You can even define all of the same metadata content (such as MITRE ATT&CK, Kill Chain, data source categories, etc.). You can also add all the normal descriptive fields (how to respond, known false positives, etc.).
If you don’t have Splunk Security Essentials on your production environment, you can always individually mark content as installed, or bookmarked.
Splunk Security Essentials includes a machine-learning driven dashboard that tracks the typical data ingest latency of the products configured in SSE (effectively: how slow is typical for the logs). When a log source slows down, it will color code it, and you can click on it to see what detections are at risk for issue.
The Data Availability dashboard shows you the products in your environment, and the most recent latency seen from each of them.
If you click on a product, it will tell you what detections depend on it along with the expected latency.
The dashboard will also throw a variety of errors in case you have any configuration issues.