SSE Schema
Splunk Security Essentials has a standard format expected for content in the app. This format is spread across two types of files:
- Main: ShowcaseInfo.json
- For Native SSE Content: The Search Builder files (showcase_simple_search.json, showcase_first_seen_demo.json, showcase_standard_deviation.json)
For most users, the only file needed is ShowcaseInfo.json.
For partners, your configurations will be automatically merged into ShowcaseInfo.json (and you don’t need to worry about the Search Builder files). Find more detail on the Partner Integration page.
Below find detailed descriptions of the schemas, along with examples from several pieces of content in the app.
Table of Contents
ShowcaseInfo.json
Schema
Field Descriptions
For a guide on what each field means, view the Add Custom Content display, where detail is provided (and much of the configuration is automated). Here is a point-in-time reference:
JSON Field Name | Descriptive Field | Relevance | Description | Examples |
---|---|---|---|---|
name | Name | All | The name of the example, e.g. “New Local Admin Account.” You should have a maximum of 150 characters, and avoid crazy punctuation. | Basic Brute Force Detection |
description | Description | All | This is the key description of your custom content. Should generally be under 250-300 characters. | |
highlight | Featured | All | Should this show up as a featured example on the main page. When we build content, we target having approximately 15% of the content be featured, just the most prominent, highest value content. | No |
advancedtags | Advanced Tags | All | You can optionally add advanced tags here, which will show up in the Advanced filter. | Cool Search |
alertvolume | Alert Volume | All | Is this a high volume search that will create lots of noise, or a low volume / high fidelity search that a human could handle the results of? | Medium |
bookmark_status | Bookmarked Status | All | Bookmark Status is how SSE tracks content, either simply “bookmarked” or even tracking implementation status, e.g. “Waiting On Data” or “Successfully Implemented.” Internal names only (see common_data_objects.js) | bookmarked |
category | Category | All | The category is a user-specifable field (though it is generally advisable to choose from the standard choices provided via SSE) | IAM Analytics |
data_source_categories | Data Source Category or Categories | All | The data source category ID that maps to the types in Data Inventory and data_inventory.json | DS003Authentication-ET02Failure |
domain | Security Domain | All | The security domain for this (not heavily leveraged, but assumed to be present). Valid options: Access, Network, Endpoint, Threat, Other. | Access |
hasSearch | Has Search | All | Does this detection include the search string | Yes |
icon | Icon | All | The icon that shows up on the main Security Content page (or wherever the tile exists). Logic implemented via BuildTile.js | |
journey | Journey | All | The stage of the journey that this content will appear in. | Stage_1 |
killchain | Kill Chain Phase | All | The phase of the kill chain | Actions on Objectives |
mitre_tactic | MITRE ATT&CK and Pre-ATT&CK Tactics | All | The pipe-delimited list of the MITRE ATT&CK Tactic IDs, e.g., TA0001 | TA0006 |
mitre_technique | MITRE ATT&CK and Pre-ATT&CK Techniques | All | The pipe-delimited list of the MITRE ATT&CK Technique IDs, e.g., T1008 | T1110 |
released | Released | All | When was this content released? | 3.0.0 |
searchkeywords | Search Keywords | All | The in-browser search keywords automatically indexes the description, title, category, use case, how to respond, how to implement, known false positives, and help. But if you want to add some highly-weighted custom words (e.g., “AWS cloudtrail amazon web services”) then you can add them here, space separated. | login log in logon log on sign |
severity | Severity | All | Impact indicates the severity of this event when it fires. It is not directly surfaced in the UI today, but is available as an enrichment field via the sseanalytics command. | Medium |
SPLEase | SPL Ease | All | How easy is this SPL to understand (for those looking at SSE just as a learning tool). | Basic |
usecase | Use Case | All | The high level marketing-friendly use case for the use case. | Security Monitoring |
additional_context | Array of Custom Panels | All | Optional: Can provide custom panels | |
additional_context.detail | Custom Panel: Detail | All | Supports markdown. An optional text block. | |
additional_context.link | Custom Panel: Link | All | An optional URL for users to learn more. If defined, a button will be added with the text “Learn More…” | |
additional_context.open_panel | Open Custom Panel | All | Whether the custom panel should be open by default or not. | |
additional_context.search | Custom Panel: Code Block Contents | All | Can present SPL or any other raw code. Code highlighting implemented automatically. | |
additional_context.search_label | Custom Panel: Code Block Title | All | Text-only, provides the label for the search, directly before the pre. Default if blank: “Search” | Example |
additional_context.search_lang | Custom Panel: Code Block Language | All | Optional definition of which language the below code is in. Can leave empty for normal SPL, but could be provided as conf if you are printing config file options. In the event you really needed to print options such as python, the default languages for highlight.js are supported. (Valid options: properties, python, rust, less, perl, diff, scss, bash, shell, makefile, json, ini, http, coffeescript, css, objectivec, ruby, yaml, java, sql, apache, kotlin, xml, markdown, swift, plaintext, typescript, nginx, go, javascript, php, cs, lua, cpp) | conf |
additional_context.title | Custom Panel: Title | All | Text-only, provides the name of the custom panel. Default if blank: “Additional Context” | Config Settings |
help | Help | All | The value for the help field (less prominently displayed, generally less important). Supports markdown. | |
howToImplement | How to Implement | All | Optional text describing how to implement the detection. Supports markdown. | |
knownFP | Known False Positives | All | Optional text describing the known false positives that will be created for this search. Supports markdown. | |
operationalize | How to Respond | All | Optional text describing the how to respond when this search fires. Supports markdown. | |
printable_image | Printable Image URL | All | Optional field: When creating a PDF export, we will include a screenshot showing the demo results. | |
relevance | Security Impact | All | (Recommended) Text describing in lamens terms why this content is important. Supports markdown. | |
open_search_panel | Open Search Panel | All | Should the search panel open by default. If a search string is provided, it will default to true unless this option specifies false. | Text String: “False” |
search | Search String | All | Optional, the search itself. | |
search_name | Saved Search Name | All | If there is a standard saved search name that could be found via Correlation Search Introspection, put that search name here. This will map to the stanza name in savedsearches.conf, not to any pretty correlation search names in ES. | ESCU - Abnormally High AWS Instances Terminated by User - Rule |
company_description | Company Description | Partners | If you would like to present a description of your company when a user views this content, insert it here. Supports markdown. | Your Short Marketing Description |
company_link | Company Link | Partners | If you would like a “Learn More” link to appear after the description when a user views this content, provide the URL here. | https://…/ |
company_logo | Company Logo | Partners | If you would like to present a logo for your organization when a user views this content, provide the URL and dimensions here. The height may not be more than 250 px, and the width may not be more than 500 pixels. For a good user experience, it is recommended not going more than 400x150px. | https://…/yourlogo.png |
company_logo_height | Company Logo: Height | Partners | Specify the height of your logo. (Your logo will be proportionately reduced if the height is > 250px) | 200 |
company_logo_width | Company Logo: Width | Partners | Specify the width of your logo. (Your logo will be proportionately reduced if the width is > 500px) | 400 |
company_name | Company Name | Partners | If you would like to present the name for your organization when a user views this content, insert the name here. | Your Org |
create_data_inventory | Create Data Inventory | Partners | When present and set to True, it will create an entry in the Data Inventory and overwrite the existing data_source_categories for this detection. For more, see: Partner Integration: Populating Data Inventory | Obj: True |
inSplunk | Solved in Splunk | Partners | If you would like to track functionality that exists in the environment, but exists outside the realm of Splunk (for example, Carbon Black detection rules tied to specific MITRE ATT&CK tactics), you can mark it as outside of Splunk here. | Yes |
escu_cis | ESCU - CIS Mapping | ESCU Authors | ES Content Update Specific: The CIS Mapping | CIS 13 |
escu_data_source | ESCU - Data Source | ESCU Authors | ES Content Update Specific: The Data Sources | AWS CloudTrail logs |
escu_nist | ESCU - NIST Mapping | ESCU Authors | ES Content Update Specific: The NIST Mapping | DE.DP |
escu_providing_technologies | ESCU - Providing Technologies | ESCU Authors | ES Content Update Specific: The technologies that enable a detection | AWS |
story | ESCU - Story ID | ESCU Authors | ES Content Update Specific: Story ID | 2e8948a5-5239-406b-b56b-6c50f1268af3 |
examples | Array of Examples for Detection | SSE Authors | For each variety of the search (Live Data, Demo Data, Accelerated Data, etc.) another object is added to the examples array. | |
examples.label | Name for Example | SSE Authors | The name for the example (typically “Live Data” or similarly short) | Live Data |
examples.name | ID in Search Builder JSON | SSE Authors | Confusingly, the “name” attribute is more the ID that will exist in the Search Builder JSON (e.g., showcase_simple_search.json). E.g., “Basic Brute Force - Live”. This field only appears in the browser URL, it’s never otherwise shown to users. | |
app | App | SSE Authors | Provides the name of the content (no spaces, special characters). For any custom content, overwritten with the channel definition. | Splunk_Security_Essentials |
dashboard | Dashboard | SSE Authors | If you want to have users go to a dashboard when they click on the name, provide that dashboard name here. Usually a link including any fields appended | showcase_simple_search?ml_toolkit.dataset=Basic Brute Force - Demo |
displayapp | Display App | SSE Authors | Provides the name of the content (user-readable version). For any custom content, overwritten with the channel definition. | Splunk Security Essentials |
visualizations | Array of Visualization Objects | SSE Authors | SSE has the option for visualizations. Those visualizations can be either native Splunk Viz or Screenshots. | |
visualizations.basesearch | Base Search | SSE Authors | If defined, the search builder will initiate a base search with this value, and then a post-process with the value in visualizations.search. Use when you have multiple panels powered by the same dataset so that you don’t re-run the same search multiple times. | |
visualizations.dashboard | Dashboard Name | SSE Authors | When using the automatic dashboarding, this viz will be placed under this dashboard name. You may have 10-30 dashboard panels per dashboard. | Essential Network Security |
visualizations.description | Panel Description | SSE Authors | For automatic dashboarding, you can optionally add a paragraph-form description to the bottom of the panel. | Provides a running count of identified DNS connections over time |
visualizations.header | Dashboard Header | SSE Authors | When using the automatic dashboarding, this viz will be placed under this header, on the dashboard. You’ll generally want a 1-5 panels for a header. | DNS Traffic |
visualizations.hideInSearchBuilder | Hide in Search Builder | SSE Authors | If you do not want to show a particular viz in the actual search builder but want it to be available in the automatic dashboarding feature, set this to true. | Obj: True |
visualizations.panel | Panel ID | SSE Authors | When rendered on the search builder, this refers to which panel the viz is rendered in. There are three columns and three rows (two above the main output, one below) | row1cell1 |
visualizations.path | Image URL | SSE Authors | For visualizations.type = image, the path to that viz. | |
visualizations.recommended | Is Base Search | SSE Authors | For panels in the automatic dashboarding feature, you have the option to mark them as recommended (enabled by default) or not. | Obj: True |
visualizations.search | Search String | SSE Authors | The search string for the viz | |
visualizations.title | Panel Title | SSE Authors | The title for this viz | DNS Traffic Over Time |
visualizations.type | Image or Visualization | SSE Authors | A flag for whether it is an image or a Splunk Viz | image |
visualizations.vizParameters | Parameters For Native Viz | SSE Authors | When you initialize Splunk Viz via SplunkJS you pass an object with parameters (e.g., charting.chart: pie, etc.). Provide those parameters here. | |
visualizations.vizType | Visualization Type | SSE Authors | If a splunk viz, a flag for what type it is. Valid options include: ChartElement, SingleElement, MapElement, TableElement | TableElement |
anomalies | UBA: Array of ShowcaseIDs for UBA Anomalies | UBA Authors | UBA Specific: A list of the SSE ids for any UBA Anomalies that the Threat looks at. Each should resolve to a threat (generally, ID matches regex AT\d*) | |
contributes_to_threats | UBA: Array of ShowcaseIDs for UBA Threats | UBA Authors | UBA Specific: A list of the SSE ids for any UBA Threats that the Anomaly contributes to. Each should resolve to a threat (generally, ID matches regex TT\d*) | TT01 |
detections | UBA: Array of Detection Objects | UBA Authors | UBA Specific: Anomaly Types in UBA are powered by multiple detections. This array has an object for each of those detections. | |
detections.data_source | UBA: Array of DSC IDs | UBA Authors | UBA Specific: Each DSC ID should resolve to the data_inventory.json, just like the data_source_categories field. Unlike that field, this is a proper JSON array rather than a pipe-delimited string. | DS009EndPointIntel-ET01ProcessLaunch |
detections.description | UBA: Description for the Detection | UBA Authors | UBA Specific: Description of the Detection that powers an Anomaly Type. | |
detections.id | UBA: ID for the Detection | UBA Authors | Not currently used | |
detections.internal_id | UBA: Internal ID for the Detection | UBA Authors | Not currently used | |
detections.name | UBA: Detection Name | UBA Authors | UBA Specific: Name of the Detection that powers an Anomaly Type. | |
internal_id | UBA: Internal ID | UBA Authors | Not currently used | |
is_custom | UBA: Is a Custom Threat | UBA Authors | UBA Specific: Not currently used, but displays whether the threat is a rule-based custom threat, rather than an ML threat. | Yes |
long_description | UBA: Long Description | UBA Authors | UBA Specific: A description field that is not subject to the character limits applied to the normal description field. This shows up when someone clicks into the UBA Anomaly or Threat |
Except where noted above (UBA fields and arrays of objects), multi-value fields are populated with pipes separating each element. E.g., content that maps to Security Monitoring and Advanced Threat Detection would have:
{
"usecase": "Security Monitoring|Advanced Threat Detection"
}
For a sense of valid values, consider looking at an example search:
| sseanalytics | fieldsummary | table field *count values
Examples
Custom or Partner Content
The app expects to download a JSON file. Each top-level key should be the ID of the showcase. See the following example:
{
"buttercuplabs_detect_bad_ponies": {
"name": "Detect Bad Ponies",
"inSplunk": "yes",
"journey": "Stage_1",
"usecase": "Security Monitoring",
"highlight": "Yes",
"id": "buttercuplabs_detect_bad_ponies",
"channel": "ButtercupLabs",
"alertvolume": "Low",
"severity": "Very High",
"category": "Account Compromise",
"description": "This detection uses advanced analytics to determine which ponies are not good.",
"domain": "",
"killchain": null,
"SPLEase": "None",
"searchkeywords": "",
"advancedtags": "",
"printable_image": "",
"icon": "https://static.mylogocloud.com/shop/store/20180213730/assets/items/largeimages/SPK0153.jpg",
"company_logo": "https://image.slidesharecdn.com/splunklivesfhowtoalignyourdailysplunkactivitiesbreakoutsession-160317192319/95/how-to-align-your-daily-splunk-activities-breakout-session-23-638.jpg?cb=1458242654",
"company_logo_width": "444",
"company_logo_height": "250",
"company_name": "Buttercup Labs",
"company_description": "Buttercup Labs is the premier distributor of Pony-related security analytics. We have been protecting organizations from bad ponies for over ten years now.\n\\n\\n\nEnjoy our freely available content for detecting bad ponies in your environment, and reach out to us for a demo or trial license of our premium Pony Detection app!\n\\n\\n\nHave you successfully found bad ponies in your own environment? Buttercup Labs is hiring! We are a wholly owns subsidiary of Buttercup Games, and are a proudly equal opportunity employer. We welcome ponies of all heights, colors, hoof styles, sexual orientations, and neuro-diversities.",
"company_link": "http://buttercupgames.com/",
"dashboard": "showcase_custom?showcaseId=buttercuplabs_detect_bad_ponies",
"relevance": "Placeholder Text",
"help": "",
"howToImplement": "Placeholder Text",
"knownFP": "Placeholder Text",
"operationalize": "Placeholder Text",
"search": "index=buttercup sourcetype=content",
"data_source_categories": "DS003Authentication-ET02Failure|DS003Authentication-ET01Success",
"mitre_technique": "T1098|T1003",
"mitre_tactic": "TA0006"
}
}
Simple Content
While you can configure a large volume of content in SSE, you don’t necessarily need to.
{
"phantom_ec2_instance_isolation": {
"alertvolume": "Very Low",
"app": "Splunk_Phantom",
"category": "Account Compromise|IAM Analytics|Account Sharing|SaaS|Insider Threat",
"dashboard": "showcase_phantom?showcaseId=phantom_ec2_instance_isolation",
"data_source_categories": "VendorSpecific-aws-cloudtrail",
"description": "Isolate an EC2 instance by changing its security group in order to protect it from malicious traffic. This playbook can be started alone or used from another playbook after doing investigation and notification.",
"displayapp": "Splunk Phantom",
"domain": "Access",
"hasSearch": "No",
"help": "Simply deploy Phantom and work with your technical team to deploy this.",
"highlight": "Yes",
"howToImplement": "This playbook can be triggered off of several example searches available in the Security Essentials app to take immediate action to quarantine an instance when suspicious behavior has been detected.",
"icon": "phantom_logo.png",
"includeSSE": "Yes",
"journey": "Stage_5",
"name": "EC2 Instance Isolation",
"operationalize": "Depending on the use case, this playbook can be modified using several of the available Phantom apps to increase the scope of the actions taken in this playbook. For example using the AWS WAF or AWS IAM app, additional actions can be added based on the type of alert triggered.",
"printable_image": "https://raw.githubusercontent.com/phantomcyber/playbooks/4.5/ec2_instance_isolation.png",
"released": "3.0.0",
"relevance": "Compromised AWS credentials can allow a malicious actor access to currently running instances and configurations as well as the ability to start new instances and services. By detecting suspicious behavior early this playbook allows for a security team to react quickly and further investigate any suspicious behavior.",
"searchKeywords": "",
"usecase": "Advanced Threat Detection|Insider Threat|SOC Automation",
"visualizations": [
{
"panel": "row1cell1",
"path": "https://raw.githubusercontent.com/phantomcyber/playbooks/4.5/ec2_instance_isolation.png",
"title": "EC2 Instance Isolation",
"type": "image"
}
]
}
}
Full-fledged Native Content
There are a variety of additional fields exposed to the native SSE content.
{
"new_cloud_provider": {
"SPLEase": "Medium",
"alertvolume": "High",
"app": "Splunk_Security_Essentials",
"category": "Data Exfiltration|Insider Threat|Shadow IT",
"dashboard": "showcase_first_seen_demo?ml_toolkit.dataset=New Cloud Provider for User - Demo",
"data_source_categories": "DS005WebProxyRequest-ET01RequestedWebAppAware",
"description": "<p>Detect a user who is accessing a cloud storage provider they've never used before.</p>",
"displayapp": "Splunk Security Essentials",
"domain": "Network",
"examples": [
{
"label": "Demo Data",
"name": "New Cloud Provider for User - Demo"
},
{
"label": "Live Data",
"name": "New Cloud Provider for User - Live"
},
{
"label": "Accelerated Data",
"name": "New Cloud Provider for User - Accelerated"
}
],
"hasSearch": "Yes",
"help": "<p>This example leverages the Detect New Values search assistant. Our dataset is an anonymized collection of Palo Alto Networks events. For this analysis, we are effectively grouping by username and app name after filtering for the category, which will give us a row for each username+appname combination. We check if the first time that has occurred was in the last day.</p>",
"highlight": "Yes",
"howToImplement": "<p>Implementation of this example (or any of the First Time Seen examples) is generally very simple. <ul><li>Validate that you have the right data onboarded, and that the fields you want to monitor are properly extracted.</li><li>Save the search.</li></ul></p><p>For most environments, these searches can be run once a day, often overnight, without worrying too much about a slow search. If you wish to run this search more frequently, or if this search is too slow for your environment, we recommend leveraging a lookup cache. For more on this, see the lookup cache dropdown below and select the sample item. A window will pop up telling you more about this feature.</p>",
"icon": "Core_Use_Case.png",
"includeSSE": "Yes",
"journey": "Stage_2",
"killchain": "Actions on Objectives",
"knownFP": "<p class=\"disclaimer\">This is a strictly behavioral search, so we define \"false positive\" slightly differently. Every time this fires, it will accurately reflect the first occurrence in the time period you're searching over (or for the lookup cache feature, the first occurrence over whatever time period you built the lookup). But while there are really no \"false positives\" in a traditional sense, there is definitely lots of noise.</p><p>You should not review these alerts directly (except for access to extremely sensitive system), but instead use them for context, or to aggregate risk (as mentioned under How To Respond).</p> ",
"mitre_tactic": "TA0010|TA0011",
"mitre_technique": "|T1048|T1102",
"name": "New Cloud Provider for User",
"operationalize": "<p>When this search returns values, validate whether the usage of this cloud provider is permitted by your policy, and investigate to see what data is being stored there. Common allowable scenarios can be uploading into a box folder provided by a vendor for secure support file upload, which might be allowable, versus the backup of data to a personal Google drive account. Ultimately this search will generate many shades of gray, so it's prudent to understand supporting information such as the amount of data transmitted before reaching out to the employee or their manager to determine next steps.</p>",
"phantomPlaybooks": [
"phantom_malicious_insider_containment",
"phantom_prompt_and_block_domain"
],
"printable_image": "/static/app/Splunk_Security_Essentials/images/printable_demo_images/new_cloud_provider.png",
"released": "2.2.0",
"relevance": "Data exfiltration techniques vary across the world, but certainly a very common approach taken in 2018 is to upload data to a non-corporate file storage solution. Tracking new file storage solutions end up in your environment is a key capability to track where data flows in your organization along with the adoption of Shadow IT.",
"searchKeywords": "",
"usecase": "Insider Threat|Security Monitoring"
}
}
Product-Specific Varieties
Splunk’s premium products have a few product-specific configurations.
ES
{
"search_name": "Change - Abnormally High Number of Endpoint Changes By User - Rule",
}
ESCU
{
"escu_cis": "CIS 13",
"escu_data_source": "AWS CloudTrail logs",
"escu_nist": "DE.DP|DE.AE",
"escu_providing_technologies": "AWS",
"search_name": "ESCU - Abnormally High AWS Instances Terminated by User - Rule",
"story": "2e8948a5-5239-406b-b56b-6c50f1268af3"
}
The ESCU Story ID defined in the “story” field is a reference to the story object in the ShowcaseInfo.json file. This is a lower-level field, hosted at ShowcaseInfo[‘escu_stories’]. Here is the story detail referenced above:
{
"2e8948a5-5239-406b-b56b-6c50f1268af3": {
"detections": [
"detection_abnormally_high_instance_termination",
"detection_ec2_instance_created_by_previously_unseen_user",
"detection_aws_activity_in_new_region",
"detection_abnormally_high_instances_launched"
],
"escu_category": [
"Cloud Security"
],
"investigations": [
"AWS Investigate User Activities By ARN",
"Get EC2 Instance Details by instanceId",
"Get Notable History",
"Get Notable Info",
"Get User Information from Identity Table",
"Investigate AWS activities via region name",
"Get EC2 Launch Details"
],
"modification_date": "2018-02-09",
"name": "Suspicious AWS EC2 Activities",
"narrative": "AWS CloudTrail is an AWS service that helps you enable governance, compliance, and risk auditing within your AWS account. Actions taken by a user, role, or an AWS service are recorded as events in CloudTrail. It is crucial for a company to monitor events and actions taken in the AWS Console, AWS command-line interface, and AWS SDKs and APIs to ensure that your EC2 instances are not vulnerable to attacks. This Analytic Story identifies suspicious activities in your AWS EC2 instances and helps you respond and investigate those activities.",
"references": [
"https://d0.awsstatic.com/whitepapers/aws-security-best-practices.pdf"
],
"responses": [],
"story_id": "2e8948a5-5239-406b-b56b-6c50f1268af3",
"support": [
"Previously Seen EC2 Launches By User",
"Previously Seen AWS Regions"
]
}
}
UBA Anomalies
{
"contributes_to_threats": [
"TT01",
"TT12",
"TT13",
"TT14",
"TT15",
"TT17",
"TT18",
"TT26",
"TT28",
"TT30",
"TT32",
"TT33",
"TT34"
],
"detections": [
{
"data_source": [
"VendorSpecific-winsec",
"DS009EndPointIntel-ET01ProcessLaunch"
],
"description": "Checks application launches in Windows Event Log against a predefined list of known bad applications. ",
"id": "DM07",
"internal_id": "Blacklisted_Application",
"name": "Blacklisted Application"
}
],
"internal_id": "Blacklisted_Application",
"long_description": "Triggered when an application is executed on an endpoint or server. Blacklisted applications are pre-defined in UBA and updated on a regular basis.",
}
UBA Threats
{
"anomalies": [
"AT11",
"AT06"
],
"internal_id": "Potential_Flight_Risk_Exfiltration",
"long_description": "This threat brings to light users who are possible flight risks. However, in addition to being a flight risk, this user has also performed actions that signal that it may be possible he will exfiltrate data. Actions that lead to exfiltration of data in this instance are events such as: suspicious data movement, upload of information to a cloud file system, or a DLP external alarm. ",
"is_custom": "Yes",
}
Visualizations
SSE’s native search builders support custom visualizations.
Screenshots
{
"visualizations": [
{
"panel": "row1cell1",
"path": "https://raw.githubusercontent.com/phantomcyber/playbooks/4.5/ec2_instance_isolation.png",
"title": "EC2 Instance Isolation",
"type": "image"
}
]
}
Splunk Dashboard Panels
You can provide a panel similar to the below, but because the search will generally be different depending on search being used (e.g., live data vs demo data) you should usually define this in the Search Builder JSON file (below). See more detail on this framework in that section.
{
"visualizations": [
{
"dashboard": "General Windows and Linux Posture",
"header": "AV Data",
"panel": "row1cell1",
"search": "| `Load_Sample_Log_Data(Symantec Endpoint Protection Operations)` |stats max(eval(if(like(Event_Description, \"%LiveUpdate session ran successfully%\") , _time, null))) as LatestUpdate max(_time) as LatestMessage max(eval(if(tag=\"error\", _time, null))) as LatestError by Host_Name | eval Up_To_Date = if( LatestUpdate < relative_time(LatestMessage, \"-3d\") OR LatestError > LatestUpdate , \"No\", \"Yes\") | lookup gdpr_system_category host as Host_Name| search category=* | stats count by Up_To_Date",
"title": "Percentage of In-Scope Hosts with Up To Date AV",
"type": "viz",
"vizParameters": {
"charting.chart": "pie",
"link.exportResults.visible": "false",
"link.inspectSearch.visible": "false",
"link.openPivot.visible": "false",
"link.openSearch.visible": "false",
"link.visible": "false",
"refresh.link.visible": "false",
"resizable": true
},
"vizType": "ChartElement"
},
{
"dashboard": "General Windows and Linux Posture",
"header": "AV Data",
"panel": "row1cell2",
"search": "| `Load_Sample_Log_Data(Symantec Endpoint Protection Operations)` |stats max(eval(if(like(Event_Description, \"%LiveUpdate session ran successfully%\") , _time, null))) as LatestUpdate max(_time) as LatestMessage max(eval(if(tag=\"error\", _time, null))) as LatestError by Host_Name | eval Up_To_Date = if( LatestUpdate < relative_time(LatestMessage, \"-3d\") OR LatestError > LatestUpdate , \"No\", \"Yes\") | search Up_To_Date = Yes | lookup gdpr_system_category host as Host_Name| search category=*| convert ctime(LatestUpdate) ctime(LatestMessage) ctime(LatestError) ",
"title": "Hosts with Up To Date AV",
"type": "viz",
"vizParameters": {
"link.exportResults.visible": "false",
"link.inspectSearch.visible": "false",
"link.openPivot.visible": "false",
"link.openSearch.visible": "false",
"refresh.link.visible": "false",
"resizable": true
},
"vizType": "TableElement"
}
]
}
Search Builder Content
The search builder files are those actually used to show off a file. There are five search builders in SSE:
Dashboard | Dashboard Name | JSON File | Notes |
---|---|---|---|
showcase_simple_search | Simple Search | showcase_simple_search.json | This is for the majority of the native SSE content – it simply runs a search string without any additional SPL added. |
showcase_first_seen_demo | First Time Seen | showcase_first_seen_demo.json | The first time seen search builder takes a base dataset, two fields that we can split by, and automatically tacks on the logic for alerting if this happens for the first time in the last day. It has high scale versions, allowing users to cache the result to a lookup. It also has a peer group option, allowing the search to find values that are new not just for the entity in question, but also for their department (or peer group, or geo, or etc.) |
showcase_standard_deviation | Time Series Spike | showcase_standard_deviation.json | The time series spike search builder takes a base dataset that is already aggregated per day, along with a field representing the numerical measure, and a field representing the entity being analyzed. It then does a standard deviation analysis to find unusually high values. |
showcase_phantom | Splunk Phantom | N/A (Uses main ShowcaseInfo) | This is used for Phantom content. |
showcase_custom | Custom Content | N/A (Uses main ShowcaseInfo) | This is used for custom content, and Partner content. |
Schema
JSON Field Name | Descriptive Field | Search Builder | Description | Examples |
---|---|---|---|---|
label | Name | All | This matches the examples.name from ShowcaseInfo.json and must match the ID (referenced above) | Emails With Lookalike Domains - Demo |
value | Search | All | The search string is contained here. Line breaks in the line-by-line SPL are implemented via \n and the # of lines should match the length of the description array. | index=abc \n |
description | Line-by-Line SPL Documentation | All | An array tha contains the line-by-line SPL docs. The # of elements in the array must be equal to the # of ‘\n’s + 1 in the search (in effect: doing a split by ‘\n’ on the search will result in the same number). | “Base Data”, “Count them”, “Filter” |
prereqs | Array of Pre-Requisites | All | An array that contains all of the pre-req objects. | |
prereqs.field | Pre-Req Field | All | The field that is analyzed to determine whether the pre-req is met. (Typically count) | count |
prereqs.greaterorequalto | Pre-Req Min Value | All | The value of the field in prereqs.field must be >= this value (Typically 1) | 1 |
prereqs.name | Pre-Req Name | All | The label provided to the user for this pre-req | “Must have data” |
prereqs.resolution | Pre-Req Resolution | All | How the user can remedy a failed pre-req. Typically “install this add-on” or “acquire this data.” | |
prereqs.test | SPL for Pre-Req | All | The search that is run for pre-reqs | |
actions_createNotable | Create Notable | All | Whether a notable event should be created. (Typically 0, but could be 1 for high confidence detections.) Will trigger ES workflow. | 1 |
actions_createRisk | Create Risk | All | Whether to create a risk object for this search. (Typically 1, but could be 0 if it doesn’t apply to the situation). | 1 |
actions_riskObject | Risk Object | All | The risk_object (aligning to ES Risk Framework) | Computer_Name |
actions_riskObjectScore | Risk Score | All | The risk_score (aligning to ES Risk Framework) | 60 |
actions_riskObjectType | Risk Object Type | All | The risk_object_type (aligning to ES Risk Framework) | system |
actions_UBASeverity | UBA Severity | All | An indication that the event should be sent to UBA, and the severity provided (1-9) | 5 |
visualizations | Array of Visualization Objects | All | SSE has the option for visualizations. Those visualizations can be either native Splunk Viz or Screenshots. | |
visualizations.basesearch | Base Search | All | If defined, the search builder will initiate a base search with this value, and then a post-process with the value in visualizations.search. Use when you have multiple panels powered by the same dataset so that you don’t re-run the same search multiple times. | |
visualizations.dashboard | Dashboard Name | All | When using the automatic dashboarding, this viz will be placed under this dashboard name. You may have 10-30 dashboard panels per dashboard. | Essential Network Security |
visualizations.description | Panel Description | All | For automatic dashboarding, you can optionally add a paragraph-form description to the bottom of the panel. | Provides a running count of identified DNS connections over time |
visualizations.header | Dashboard Header | All | When using the automatic dashboarding, this viz will be placed under this header, on the dashboard. You’ll generally want a 1-5 panels for a header. | DNS Traffic |
visualizations.hideInSearchBuilder | Hide in Search Builder | All | If you do not want to show a particular viz in the actual search builder but want it to be available in the automatic dashboarding feature, set this to true. | Obj: True |
visualizations.panel | Panel ID | All | When rendered on the search builder, this refers to which panel the viz is rendered in. There are three columns and three rows (two above the main output, one below) | row1cell1 |
visualizations.path | Image URL | All | For visualizations.type = image, the path to that viz. | |
visualizations.recommended | Is Base Search | All | For panels in the automatic dashboarding feature, you have the option to mark them as recommended (enabled by default) or not. | Obj: True |
visualizations.search | Search String | All | The search string for the viz | |
visualizations.title | Panel Title | All | The title for this viz | DNS Traffic Over Time |
visualizations.type | Image or Visualization | All | A flag for whether it is an image or a Splunk Viz | image |
visualizations.vizParameters | Parameters For Native Viz | All | When you initialize Splunk Viz via SplunkJS you pass an object with parameters (e.g., charting.chart: pie, etc.). Provide those parameters here. | |
visualizations.vizType | Visualization Type | All | If a splunk viz, a flag for what type it is. Valid options include: ChartElement, SingleElement, MapElement, TableElement | TableElement |
outlierPeerGroup | Peer Group | First Time Seen | First Time Seen can optionally do peer group analysis. This is disabled by default (though the user can always configure it!) but it is available to be specified here. | |
outlierValueTracked1 | Value #1 | First Time Seen | First Time Seen splits by two values. Typically the first value is the subject (e.g., user, dest, src, etc.) | user |
outlierValueTracked2 | Value #2 | First Time Seen | First Time Seen splits by two values. Typically the second value is the unusual attribute (e.g., geo, API, etc.) | eventName |
cardinalityTest | Cardinality Test | Time Series Spike | SPL that tests to see if this split-by will result in an excessive number of results, overwhelming Splunk. Generally it is the core search, but instead of stats … by user it would be stats dc(user) as count | |
outlierSearchType | Search Type | Time Series Spike | Reserved for future use, must be “Avg” today | Avg |
outlierVariable | Numeric Measure | Time Series Spike | The value containing the “count” that we are measuring. Often “count” but could be any Splunk field. | count |
outlierVariableSubject | Subject | Time Series Spike | The subject (or entity) we are monitoring. Typically user, src, or dest, but could be any Splunk field. | user |
scaleFactor | # of StDevs | Time Series Spike | How many Standard Deviations away from avg is abnormal (aka z-score). Typically 3-10 in real world usage – 3 for anomaly detection, 10 for high confidence. | 6 |
windowSize | windowSize | Time Series Spike | Not used, must be 0. | 0 |
Examples
Simple Search
{
"DynDNS - Live": {
"actions_UBASeverity": 7,
"actions_createRisk": 1,
"actions_riskObject": "user",
"actions_riskObjectScore": 30,
"actions_riskObjectType": "user",
"description": [
"First we bring in our dataset of proxy logs.",
"Because we are looking for dynamic dns providers, we're going to need to separate out subdomains from the registered domain. URL Toolbox is just the tool for this job!",
"Next we can use our lookup of ddns domains (see How to Implement). This will add a field called inlist with the value \"true\" for any matches.",
"And finally we can look for those records that are matches.",
"With our dataset complete, we just need to arrange the fields to be useful."
],
"label": "DynDNS - Live",
"prereqs": [{
"field": "count",
"greaterorequalto": 1,
"name": "Must have Proxy data",
"resolution": "Proxy data can come in many forms, including from Palo Alto Networks and other NGFWs, dedicated proxies like BlueCoat, or network monitoring tools like Splunk Stream or bro.",
"test": "| metasearch earliest=-2h latest=now index=* sourcetype=pan:threat OR (sourcetype=opsec URL Filtering) OR sourcetype=bluecoat:proxysg* OR sourcetype=websense* | head 100 | stats count "
},
{
"field": "count",
"greaterorequalto": 1,
"name": "Must have URL Toolbox Installed",
"resolution": "The URL Toolbox app, written by Cedric Le Roux, provides effective URL Parsing. Download <a href=\"https://splunkbase.splunk.com/app/2734/\">here</a>.",
"test": "| rest /services/apps/local | search disabled=0 label=\"URL Toolbox\" | stats count"
}
],
"value": "index=* sourcetype=pan:threat OR (tag=web tag=proxy) earliest=-20m@m earliest=-5m@m \n| eval list=\"mozilla\" | `ut_parse_extended(url,list)`\n| lookup dynamic_dns_lookup domain as ut_domain OUTPUT inlist\n| search inlist=true \n| table _time ut_domain inlist bytes* uri",
"visualizations": [{
"panel": "row1cell1",
"header": "DNS Traffic",
"dashboard": "Essential Network Security",
"description": "Provides a running count of identified DNS connections over time",
"search": "| `Load_Sample_Log_Data(Sample Firewall Data)` | search dest_port=53 OR app=dns | timechart count",
"title": "DNS Traffic Over Time",
"type": "viz",
"vizParameters": {
"charting.chart": "column"
},
"vizType": "ChartElement"
},
{
"panel": "row1cell2",
"header": "DNS Traffic",
"dashboard": "Essential Network Security",
"recommended": false,
"description": "Shows the top destinations for DNS traffic. These should be your DNS servers, or a standard upstream DNS server.",
"basesearch": "index=* sourcetype=pan:threat OR (tag=web tag=proxy) (dest_port=53 OR app=dns)",
"search": "| `Load_Sample_Log_Data(Sample Firewall Data)` | search dest_port=53 OR app=dns | stats count by dest_ip | sort - count",
"title": "Likely Resolvers",
"type": "viz",
"vizParameters": {},
"vizType": "TableElement"
}
]
}
}
First Time Seen
{
"AWS New API Call Per Peer Group - Live": {
"actions_UBASeverity": 7,
"actions_createRisk": 1,
"actions_riskObject": "user",
"actions_riskObjectScore": 30,
"actions_riskObjectType": "user",
"description": [
"First we bring in our basic dataset. In this case, AWS CloudTrail logs, filtered for individual APIs that we want to pay close attention to."
],
"label": "AWS New API Call Per Peer Group - Live",
"outlierPeerGroup": "peer_group_for_git_use_case.csv",
"outlierValueTracked1": "user",
"outlierValueTracked2": "eventName",
"prereqs": [{
"field": "count",
"greaterorequalto": 1,
"name": "Must have AWS CloudTrail data",
"resolution": "In order to run this search, you must have AWS CloudTrail data onboard. Visit the <a href=\"/app/Splunk_Security_Essentials/data_source?technology=AWS%20CloudTrail\">data onboarding guide for AWS CloudTrail in this app</a>, or browse to <a href=\"https://splunkbase.splunk.com/app/1876/\">apps.splunk.com</a> for more information.",
"test": "| tstats count where earliest=-2h latest=now index=* sourcetype=aws:cloudtrail"
}],
"value": "index=* sourcetype=aws:cloudtrail eventType=* NOT errorMessage=* NOT eventName=Describe* NOT eventName=Get* NOT eventName=List*"
}
}
Standard Deviation
{
"GCP APIs Called More Often Than Usual Per Account - Live": {
"actions_UBASeverity": 2,
"actions_createRisk": 1,
"actions_riskObject": "data.protoPayload.authenticationInfo.principalEmail",
"actions_riskObjectScore": 10,
"actions_riskObjectType": "user",
"cardinalityTest": "index=* sourcetype=google:gcp:pubsub:message \n| bucket _time span=1d | stats dc(data.protoPayload.authenticationInfo.principalEmail) as count by _time",
"description": [
"First we bring in our basic GCP Audit logs, filtering out list activity.",
"Bucket (aliased to bin) allows us to group events based on _time, effectively flattening the actual _time value to the same day.",
"Next we use stats to summarize the number of events per user per day."
],
"label": "GCP APIs Called More Often Than Usual Per Account - Live",
"outlierSearchType": "Avg",
"outlierVariable": "count",
"outlierVariableSubject": "data.protoPayload.authenticationInfo.principalEmail",
"prereqs": [{
"field": "count",
"greaterorequalto": 1,
"name": "Must have GCP Audit data",
"resolution": "In order to run this search, you must have GCP Audit data onboard. Browse to <a href=\"https://splunkbase.splunk.com/app/3088/\">apps.splunk.com</a> for more information.",
"test": "| tstats count where earliest=-2h latest=now index=\"*\" sourcetype=google:gcp:pubsub:message"
}],
"scaleFactor": 2,
"value": "index=* sourcetype=google:gcp:pubsub:message NOT data.protoPayload.methodName=v1*.list* NOT data.protoPayload.methodName=storage*.list* \n| bucket _time span=1d \n| stats count by data.protoPayload.authenticationInfo.principalEmail _time",
"windowSize": 0
}
}