Pull List of Privileged Users

Description

To focus detection or response on privileged users, you must first build a list of accounts that have elevated rights or access to privileged information.


Use Case

Insider Threat

Category

Insider Threat

Security Impact

Every organization should seek to inform their detections with the relative risk of the users and systems involved. If two users send out a large volume of data but one of those users is a database administrator, you should probably respond to that alert first. Security organizations serious about insider threat should be able to look at different dimensions of privilege and then inform both their detection methods and their response prioritization accordingly. The two easiest of those dimensions are "users with elevated rights on systems" (e.g., sysadmins, database admins, etc.) and "employees who have access to privileged information" (e.g., executives, employees involved in mergers and acquisitions, etc.). It's entirely possible that you will build out a series of different lists for different scenarios (for example, a list of users read in on a particularly acquisition project, etc.) but it's easiest to start with just accounts that have elevated rights and those that have privileged information.

Alert Volume

Very Low (?)

SPL Difficulty

Hard

Journey

Stage 4

Kill Chain Phases

Actions on Objectives

Data Sources

Windows Security

   How to Implement

There are multiple approaches for pulling lists of Privileged Users, and multiple definitions of what privileged really means for a given scenario.

Pulling Users. In this example we show three different methods for pulling lists of users: querying LDAP, querying a database, or looking for users who take the actions that a privileged user would take. Let's walk through these examples.

  • Pulling from LDAP - there are three different approaches for pulling from LDAP. We generally recommend that most users start by using the Supporting Add-on for Active Directory (aka SA-ldapsearch) which provides the ldapsearch command. This command is native in Splunk, and works great for on premises installations in organizations with fewer than 50,000 users (app link, docs link). You can also pull data using AD Monitor, which is the Splunk Universal Forwarder's ability to monitor Active Directory for changes -- recommended for large organizations, and for Cloud environments (doc link). Finally, it is sometimes simpler to use Powershell for this effort (link).
  • Organizations with formal systems for managing privileged users that don't sync to Active Directory will most frequently need to pull a list of users by querying a database (though sometimes an API is available). This frequently occurs when you have an on-prem software application backed by a database that stores a list of privileged users. The process you will walk through to pull that data into Splunk is to first discover the username and password to log into the database along with the query that will return your desired set of users, and then configure Splunk DB Connect (link, doc link) to pull that data. Learning the credentials and the required query is sometimes very easy, and sometimes very difficult, but often you will be able to find help from Splunk Professional Services or on Splunk Answers.
  • Finally, we can always infer a list of privileged users by finding users who take privileged actions. There are many ways to approach this process, but the simplest is by searching for the use of tag=privileged on authentication events, as defined by the Splunk Common Information Model.

Now we've discussed how to ingest the data in the first place -- but we haven't discussed what data we actually should be ingesting. There are two primary categories of privileged users that we might want to monitor: those who have administrative rights, and those that are privileged.

  • Those who have administrative rights is a very simple question to answer in a smaller organization, but exponentially more difficult with the complexity of the environment. It's best to start with standard privileged AD groups, such as Domain Admins, and then expand into environment-specific groups, such as the custom help desk admin group that had local admin rights on all workstations (every company has at least one of these). It's often easiest to find these types of groups by hunting through LDAP for key terms such as "*admin*" or "*operator*" etc. Finally, wrap up by searching for users with specific admin rights, such as privileged service accounts and see what groups (if any) they belong to. It's generally advisable to look for known administrative users across the organization (either specific names, or by naming scheme such as jsmith-a) and seeing which groups these users belong to.
  • Generally speaking, understanding VIP users in an organization can be dramatically easier than finding all admin accounts. Title will frequently get you all you need, though you may also find that tracking the # of steps away from the CEO based on reporting structure can be very effective. It's important to not forget employees that are associated with highly visible people but aren't themselves as visible, particularly the executive assistants for all your important users. They generally have full access to an exec's calendar, mailboxes, and more often than we would like they will also have a text file somewhere on their computer with the executive's password (or similar).

   Known False Positives

The concept of a false positive versus false negative is slightly different for this example as ultimately we are working to build out a list of users, but both are extremely common. Mapping the organizational structure (and permissions structures) of an organization is inherently very complicated, and will require lots of testing and tuning to get a meaningful list. The benefit though is that since this search does not directly create alarms, some amount of inaccuracy is acceptable.

   How To Respond

This search does not directly create alarms, but is only used to generate lists of users for use in other searches. See Related Content.

   Help

Pull List of Privileged Users Help

This example leverages the Simple Search Assistant. There are several ways to pull this data from live environments -- see the different options for more information.

SPL for Pull List of Privileged Users

Demo Data

First we bring in our basic demo dataset. In this case, anonymized LDAPSearch Output. We're using a macro called Load_Sample_Log_Data to wrap around | inputlookup, just so it is cleaner for the demo data.
Next we filter to look for interesting titles, or interesting groups
Finally, put just the fields we want in a table.

SysAdmins

Okay, this is pretty tricky. Because ldapsearch (at least in my testing) doesn't allow us to look for users who are members of particular groups (which would be so much simpler, much like the VIPs from LDAP example), we have to first pull the list of privileged groups and then look for the information of every user in those groups. We will start by pulling the list of privileged groups. As noted in the How To Implement, there are many different approaches to ingesting this data -- ldapsearch is generally the simplest to get started with.
Next we filter for security groups, as we don't care about mailing lists. Any tuning you need to do to this query will likely occur at this point.
Now we just want the list of group members -- there are many ways to do this (dedup, etc.) but I personally prefer | stats count by member.
Because we're going to be turning this into a new LDAP Search in a moment, we have to convert this to the LDAP search syntax
Smaller organizations don't need to worry as much about this particular section. While the LDAP RFC doesn't list the max size of a query, AD limits queries to 10 MB each. We are setting a limit slightly lower (10,000,000 bytes) and breaking into different LDAP queries. This way, we'll never generate a query too big for LDAP to handle. You could tune this as much as you want, as well.
With this stats query, we will transform a long list of distinguishedNames into an actual LDAP query that can be run. First we group them together into a big multi-value field, then we use eval to combine them all into one field.
Now that we have everything into a single field, we can use the map search command. map will run the same query as many times as you ask it to, which is really useful for two reasons. For one, ldapsearch is a generating command and must be the first command in a search -- without map, we would have to output to a lookup, then start an entirely new search to use ldapsearch (what a hassle!). With map, we are effectively telling Splunk "Okay, I've coaxed my data into the right form, let's start a brand new search from here" and any earlier fields and results (not used by the $tokens$ in map) will be thrown away. The second reason, is that if we have a small to medium organization we will have just a single query (less than 10 MB in size). A big organization might have several queries, but we don't have to deal with any complexity to make that work.
Finally, put just the fields we want in a table.
And we can outputlookup to a CSV file that we can use in other searches. That wasn't so hard, was it?

VIP Employees

First we bring in our dataset from LDAPSearch. As noted in the How To Implement, there are many different approaches to ingesting this data -- ldapsearch is generally the simplest to get started with. For performance reasons, rather than bringing ALL the data into Splunk and then filtering in Splunk, we're using the LDAP search syntax to filter at the data source.
Next we use the table command to restrict to just the fields we actually want to look at.
Now we rename sAMAccountName to user because user is the standard field in the Common Information Model (and let's be honest, getting the punctuation right in sAMAccountName is probably the source of bugs in at least 20% of the queries that use that field).
Finally we output the results into a lookup that we can use in future searches.

Privileged Actios

First we bring in our privileged authentication data from the accelerated data model. (If you're wondering, yes you could do this without acceleration, but this is a relatively easy data model to accelerate and it's way faster).
Next we rename Authentication.user to just user because shorter is better than longer.
Next we append an existing lookup. The idea here is that we want to cache this data so that we don't have to run over long periods of time every time we run this search -- acceleration makes it fast, but we still like efficiency.
Now we have two sources of data in the same stream of results -- the first are the times that we've observed in the last day of users taking accelerated actions. The second is the results of the lookup that has our historical information. We now use | stats to combine those two data sources and get the "true" earliest and latest time for each user. Notably, you could filter for accounts older than 3 months here if you wanted to (or keep that stuff around for other searches!).
Finally we output the results to a lookup.

Calculate Risk Scores

First we bring in our dataset from LDAPSearch. As noted in the How To Implement, there are many different approaches to ingesting this data -- ldapsearch is generally the simplest to get started with.
First we initialize our risk score with information about the title of the employee. We're using the case function in eval because it allows us to try out several different titles and assign them different scores, but not risk doubling up. For example, if someone is an Executive Vice President, it would match on the second clause and assign a score or 40, but then stop before moving on to *Vice President*.
Next, we look for users who are members of privileged groups. You could use case here as well, but I opt'd to instead be additive by using different if statements (such that if someone was a member of Enterprise Admins and Domain Admins, they'd end up with a much higher score. In some organizations with many different domains, you might also wish to calculate how many different domains a user has domain admin rights in.
Next, we use table to include just the fields that we really want (excluding _raw and generally excluding memberOf as it can be very large).
Finally we output the results into a lookup that we can use in future searches.

Demo Data (Related Accounts)

First we bring in our basic demo dataset. In this case, anonymized LDAPSearch Output. We're using a macro called Load_Sample_Log_Data to wrap around | inputlookup, just so it is cleaner for the demo data.
Next we determine what users are privileged. There are faster ways to do this, but we use eval's searchmatch just for the sake of simplicity. This will assign a value of 1 for any users where we see the specified titles or groups.
Now we use eventstats to distribute the title and the privilege level across all the accounts from the same employee. The theory here is that if we have an employee named Chuck who has an admin account and a normal account, then we see some suspicious actions on his normal account, we want to know that this account belongs to a privileged employee even though this account itself isn't. Often you will also want to distribute other fields such as phone numbers, managers, etc.
Finally we can filter for just the accounts belonging to privileged employees.
Finally, put just the fields we want in a table.

Related Accounts

First we bring in our dataset from LDAPSearch. As noted in the How To Implement, there are many different approaches to ingesting this data -- ldapsearch is generally the simplest to get started with.
Next we determine what users are privileged. There are faster ways to do this, but we use eval's searchmatch just for the sake of simplicity. This will assign a value of 1 for any users where we see the specified titles or groups.
Now we use eventstats to distribute the title and the privilege level across all the accounts from the same employee. The theory here is that if we have an employee named Chuck who has an admin account and a normal account, then we see some suspicious actions on his normal account, we want to know that this account belongs to a privileged employee even though this account itself isn't. Often you will also want to distribute other fields such as phone numbers, managers, etc.
Finally we can filter for just the accounts belonging to privileged employees.

Screenshot of Demo Data