Hosts with Varied and Future Timestamps

Description

One technique for foiling correlation searches is to alter the system time. This search will detect this scenario.

(and should be fixed)


Use Case

Advanced Threat Detection

Category

Endpoint Compromise

Security Impact

Splunk uses the timestamps in the logs generated on an endpoint in order to log an accurate time when an event actually happened. This is why having all of the systems reporting into Splunk leverage an authoritative time source, like NTP, is so important. Attackers may realize that your correlation rules are based on time boundaries, e.g. 'search the last five minutes of data' but if a system is logging time 'in the future’ then your correlation rules may not trip when suspicious behavior is logged. It is important to find the systems with inaccurate system times and fix them.

Alert Volume

Low (?)

SPL Difficulty

Advanced

Journey

Stage 1

MITRE ATT&CK Tactics

Defense Evasion

MITRE ATT&CK Techniques

Timestomp

MITRE Threat Groups

APT28
APT32
Lazarus Group
TEMP.Veles

Kill Chain Phases

Actions on Objectives

Data Sources

Any Splunk Logs

   How to Implement

This search should work universally on all Splunk environments, since it uses Splunk internal fields.

   Known False Positives

This should not fire often -- the idea is it should only fire when the system time changes, which should not happen often. The only known false positives are hosts that are sending data with multiple timestamps simultaneously, which is usually an instance of an incorrect _time extraction (which should be fixed) or a data source with known lag (which should be filtered out).

   How To Respond

When timestamps are detected as being incorrect, it should be noted which system's timestamps are off and when it started. Notification of the system owner needs to be made to correct this condition, but additional investigation may be required to determine if this was an accidental or malicious configuration change and by whom.

   Help

Hosts with Varied and Future Timestamps Help

This example leverages the Simple Search assistant. Here we are looking through all Splunk logs for hosts that begin sending logs at very different timestamps. The goal is to detect an attacker who suddenly sets the year to 2020 or some point in the future to evade detection. We use the tstats command here, because we are only looking at indexed field, and it is super fast for those use cases. If we find a broad range of time for any host, the search surfaces it. We use 21 years into the future for the time range, because that is the maximum time range you can put into a Splunk search. We're looking for a range of more than one hour, just to make sure we're above chance.

SPL for Hosts with Varied and Future Timestamps

Live Data (Auto Accelerated)

First we use tstats to grab events that were indexed in the last half hour, with timestamps ranging from the last half hour to the distant future.
Next we look for the time ranges from a single host. This is a quick and dirty baseline, because if we're constantly running this search and an attacker changes the system time from now to a year from now, there will be some period when there is a massive range of timestamps coming from that host.
Finally we filter for those large ranges.