There are a couple of technical processes that have standard troubleshooting approaches. Those are documented below.
Table of Contents
Correlation Search Introspection
This overall process is very reliable, and pretty simple. I’ve yet to get it to fail, but here are the four steps that it takes.
First, check if there’s data if you run the search:
| inputlookup sse_content_exported_lookup
(Make sure to refresh the bookmarks page after the introspection — generally not a problem, but if someone is testing they will sometimes not do it)
- This Tests: whether the lookup is generated or not.
- Resolution If Failed: we’ve yet to see a case where the root cause for this was anything other than not configuring the correlation search introspection. It’s possible there is an environment specific bug not yet found though!
If you should be able to search index=notable, you should see mitre_technique fields.
index=notable OR index=risk | stats count as num_total count(eval(isnotnull(mitre_technique))) as num_with_mitre_technique
- This Tests: the automatic lookup configuration in props.conf
- Resolution If Failed: See why the auto lookup isn’t working. This is a core Splunk configuration shipped in props.conf via SSE. We’ve also had no reported failures of this.
If that works, try the same search from within the ES app.
- This Tests: ES Permissions. Prior to ES 5.3, changes had to be made to a regex that was applied to the default or local.meta.
- Resolution If Failed: Run the ES Integration in the SSE Setup. If that fails for whatever reason (also never seen), you can manually configure ES, or upgrade to 5.3+
Open Incident Review and see if the fields show up.
- This Tests: that the custom fields are added to log_review.conf
- Resolution If Failed: Run the ES Integration in the SSE Setup. If that fails for whatever reason (also never seen), you can manually configure the fields in ES under Configure Incident Review Settings, and add the fields you see in the lookup at the top.
Data Inventory Introspection
First, make sure that you are running at least SSE 3.0.3 (Release Date: Dec 11, 2019.) Most of the issues that have been seen with the Data Introspection have been intermittent and go away after resetting and running the configuration.
If you are running into issues, run through the following troubleshooting steps to reset the system and allow it to start from scratch.
Refresh the data inventory page. If you just opened it, you can proceed past this step – the concern here is that while the system tries to kill all the running configurations, in corner cases it is possible that after we clear the config some activity on the page could re-start it.
Open the status dialog. This will either say “XX Remaining” or “Completed” and is in the upper right hand side.
Click Reset Configurations. It will process for one moment and then ask you to refresh the page.
Wait for the prompt to run data introspection. If this doesn’t appear, the configuration wasn’t fully complete reset and you should continue from step two.
Once the prompt appears, click Run Data Introspection.
Wait for the introspection to complete.
Go through all “Review” configurations and define what product they belong to, (as documented)[/features/datainventory/].