Before You Write a Custom Rule

This topic has information about writing custom Cloud SIEM rules.

tip

Before you create a custom rule, check to see if there is a built-in rule that meets or comes close to meeting your need. You can easily tailor built-in rules using rule tuning expressions.

By tuning and using a built-in rule, you avoid the effort of writing a rule, and get the benefit of on-going improvements when we update core rule logic. Added bonus: Signals and Insights from built-in rules leverage crowd-sourced machine learning that custom rules cannot.

Related topics

The following topics provide information that’s relevant to the process of writing a custom rule:

• Record Processing Pipeline. This topic describes how Cloud SIEM creates Records for incoming messages. It provides facts about how message fields are mapped to Cloud SIEM schema attributes; about the attributes Cloud SIEM adds to Records to enrich and provide context about IP address, URLs, and domains; “list” features, like Match Lists and Suppress Lists that allow you to include or exclude Records based on indentiers found in Records; how to leverage threat intel data and more.
• Schema Attributes. This topic defines the Record attributes you can reference in rules.
• Cloud SIEM Rules Syntax. This topic describes rules language functions and syntax, which you’ll use in writing rule expressions.
• Searching for Cloud SIEM Records in Sumo Logic. This topic explains how to search Cloud SIEM Records in the Sumo Logic platform. Typically, you’ll build and refine your rule expressions in Sumo Logic. Once you’re happy with the results, you’ll copy the query into the rule expression field in the Rules Editor.

Step 1: Perform use case analysis and select rule type

The first step is determining your use case. In part, this involves deciding what behavior you want the rule to detect, and which of your data sources will provide evidence of that behavior. 

In addition to what you're looking for, and where you can find it, you’ll decide on what sort of logic to apply when the rule encounters the target behavior. For example, is detecting one Record that matches your rule expression sufficient to fire a Signal, or should multiple matching Records be a condition for firing? Perhaps you need to look for multiple different types of events related by a common entity. The answers to these questions will determine what type of rule is appropriate for your use case. 

Review the standard rule types to determine if any of them can address your use case.

Step 2: Review the log mapping for your source

Before you write a rule, you’ll want to verify what attributes are available in the Records created from the target data source. You can do this by reviewing the log mapping for the data source.  

Let’s say you’re going to write a rule that fires every time a successful Windows login occurs from a user account that doesn’t match your standard account naming convention. You know, maybe because you’ve checked Microsoft documentation, that the Windows event that records successful logins is Security Log Event ID 4624. So, you’ll take a look at the Cloud SIEM log mapping for that event, assuming there is one.

To find and review a log mapping:

1. Classic UI. In the top menu select Configuration, and then under Incoming Data select Log Mappings.
   New UI. In the top menu click Configuration, and then under Cloud SIEM Integrations select Log Mappings. You can also click the Go To... menu at the top of the screen and select Log Mappings.
2. You can use the filter area at the top of the Log Mappings page to search for a mapping by various options. The screenshot below shows the results when we enter the filter Name matches wildcard pattern *4624. A mapping matches. For the mapping, you can see how many times it’s been used in the last 24 hrs and also over the last 7 days. Select the mapping.
   
3. Once you’ve opened the mapping, you’ll see the top of the page shows the Vendor, Product, and Event ID that is written to the Records produced by the mapping.
   
4. The Fields section of the page shows how raw message fields are mapped to Cloud SIEM schema attributes. In this mapping, EventData.LogonProcessName is mapped to application, EventData.WorkstationName is mapped to device_hostname, and so on. 
   

Now that we understand the mapping in Cloud SIEM, we can see we will want to be looking for logs where the metadata_vendor is “Microsoft”, metadata_product is “Windows”, and metadata_deviceEventId is “Security-4624”, and we will also want to use the user_username field to find users that don’t match our naming convention.

Step 3: Create the query in Sumo Logic

In this step, we’ll create the query that will serve as the rule expression when we create the rule.

Using the attributes we discovered from looking at the log mapping, we’ll run the following query, which returns the usernames that have successfully logged on over the last week, counted by user_username.

_index=sec_record_*
| where metadata_vendor = "Microsoft" and metadata_product = "Windows" and metadata_deviceEventId = "Security-4624"
| count by user_username

The results show two of our standard username patterns: 

• The username for regular user accounts are a plain string, with no special characters, like specops and jask.
• Machine usernames are a string, followed by a dash character, followed by a string, followed by a dollar sign, like win10-admin$ and win10-client$.

Now, we can refine our search to return usernames that do not comply with either of our standard patterns.

_index=sec_record_*
| where metadata_vendor = "Microsoft" and metadata_product = "Windows" and metadata_deviceEventId = "Security-4624" and !(user_username matches /^[a-zA-Z]*$/ or user_username matches "*-*$")
| count by user_username

Usernames returned include “anonymous logon”. A little research indicates that this is typically no cause for alarm, so we’ll refine our search again to exclude “anonymous logon”.

_index=sec_record_*
| where metadata_vendor = "Microsoft" and metadata_product = "Windows" and metadata_deviceEventId = "Security-4624" and !(user_username matches /^[a-zA-Z]*$/ or user_username matches "*-*$") and user_username != "anonymous logon"

Now that we’ve sorted out the usernames formats and values we want to exclude, we’ve removed | count by user_username from the query.

Let’s say there is a field of interest in our raw messages—EventData.ProcessName—that isn’t mapped to a Cloud SIEM schema attribute. We want to parse that field out of the message so we can use it in our logic as well. We only want our rule to fire if a user with an anomalous logon ran an .exe process after successfully logging in. You can see all of the fields in the raw message in the Messages tab of your search results.

We update the query to parse out EventData.ProcessName, naming it process_name, and filtering to only fire on .exe files. 

_index=sec_record_*
| json field=_raw "$['EventData.ProcessName']" as process_name
| where metadata_vendor = "Microsoft" and metadata_product = "Windows" and metadata_deviceEventId = "Security-4624" and !(user_username matches /^[a-zA-Z]*$/ or user_username matches "*-*$") and user_username != "anonymous logon" and process_name matches "*.exe"

Now we have a query we can use as the rule expression for our rule. Note that when you paste it into the Rules Editor you should remove the first portion of the query, which is only necessary when you are querying Records in Sumo Logic:

_index=sec_record_*

You can use an expression like this example in any rule type. Here is an example Match rule with the expression, shown in the Rules Editor.

Degraded rules

A degraded rule is one that has had a portion of the rule shut off to prevent it from exceeding a processing limit. If you write a custom rule that becomes degraded, you must tune the rule to correct the problem.

For example, rules have a limit on the number of records per second they can evaluate. If there is a value used in the "group by" field that causes the rule to exceed that threshold, the particular value will be ignored, but the rest of the rule is still be used. In this case, Cloud SIEM might display a message like this:

The aggregation on the group key 'admin@company.com' has a record volume exceeding the supported limit, and has been disabled. Consider tuning the rule to exclude records producing this group key.

To resolve a degraded rule issue, create a rule tuning expression to address the portion of the rule causing the rule degradation.