Gmail Trace Logs Source

note

This source was originally developed for Gmail logs in BigQuery, which has been replaced by Google Workspace logs and reports in BigQuery.

You'll need to use our Google BigQuery source instead (see Example 3: Query Gmail Logs).

The Gmail Trace Logs integration pulls the Gmail log from the BigQuery using BigQuery Library APIs and ingests them into the Sumo Logic to store, analyze, and alert.

Data sources

The Gmail Trace Logs integration will query a single data source (BigQuery). The Gmail Trace Logs Source ingests the following:

• Set up Gmail logs in BigQuery
• Schema for Gmail logs in BigQuery

Metadata fields

Metadata fields will be set, if the integration is configured with the SIEM forward option. See Metadata Fields table below:

Fields	Value
_siemparser	/Parsers/System/Google/GCP BigQuery Gmail

Setup and Configuration

Follow the below steps to get the Service Account's Credential JSON file to to complete the authorization process.

1. Open IAM & Admin under Google Cloud Console.
2. Select the Service Account tab.
   
3. From the project dropdown button, select the project where you will run the BigQuery jobs.
   
4. Click on Create a Service Account and follow the instructions in Create service accounts google cloud docs.
   
5. Click on the email address provisioned during the creation and then click the KEYS tab.
   
6. Click ADD KEY and choose Create new key.
   
7. Select key type as JSON.
   
8. Click Create. A JSON key file is downloaded to your computer.

States

Gmail Trace Logs source is a fully managed enterprise data warehouse that helps you to manage and analyze your data. When you create an Gmail Trace Logs source, it goes through the following stages:

1. Pending. Once the Source is submitted, it is validated, stored, and placed in a Pending state.
2. Started. A collection task is created on the Hosted Collector.
3. Initialized. The task configuration is complete in Sumo Logic.
4. Authenticated. The Source successfully authenticated with Gmail Trace Logs.
5. Collecting. The Source is actively collecting data from Gmail Trace Logs.

If the source has any issues during any one of these states, it is placed in an Error state.

When you delete the source, it is placed in a Stopping state. When it has successfully stopped, it is deleted from your Hosted Collector. On the Collection page, the Health and Status for Sources is displayed. You can click the text in the Health column, such as Error, to open the issue in Health Events to investigate.

Create Gmail Trace Logs Source

When you create an Gmail Trace Logs Source, you add it to a Hosted Collector. Before creating the Source, identify the Hosted Collector you want to use or create a new Hosted Collector. For instructions, see Configure a Hosted Collector.

To configure Gmail Trace Logs Source:

1. In Sumo Logic, select Manage Data > Collection > Collection. 
2. On the Collection page, click Add Source next to a Hosted Collector.
3. Search for and select Gmail Trace Logs.
   
4. Enter a Name for the Source. The description is optional.
   
5. (Optional) For Source Category, enter any string to tag the output collected from the Source. Category metadata is stored in a searchable field called _sourceCategory.
6. (Optional) Fields. Click the +Add button to define the fields you want to associate. Each field needs a name (key) and value.
   •  A green circle with a check mark is shown when the field exists in the Fields table schema.
   •  An orange triangle with an exclamation point is shown when the field doesn't exist in the Fields table schema. In this case, an option to automatically add the nonexistent fields to the Fields table schema is provided. If a field is sent to Sumo Logic that does not exist in the Fields schema it is ignored, known as dropped.
7. Project ID. Enter the unique identifier number. You can find this from the Google Cloud Console.
8. Dataset ID. Enter the ID. The Dataset ID is the project-wise unique identifier for your dataset.
9. Data Location. Enter the location of DataSet which is set while creating Dataset in BigQuery.
10. Private Key. Enter the private key of Service Account JSON. This is a security key which is required for authentication. You can find this from the Google Cloud Console.
11. Client Email. Enter the user email collected from the Google Cloud Console.
12. Token URI. Enter the token URI used for generating the token. You can find this from the Google Cloud Console.
13. The Collection should begin is set to 24 Hours ago by default. You can adjust it based on your needs.
    note

    If you set Collection should begin to a collection time that overlaps with data that was previously ingested on a source, it may result in duplicated data to be ingested into Sumo Logic.

14. (Optional) Processing Rules for Logs. Configure any desired filters, such as allowlist, denylist, hash, or mask, as described in Create a Processing Rule.
15. When you are finished configuring the Source, click Save.

Error Types

When Sumo Logic detects an issue, it is tracked by Health Events. The following table shows the three possible error types, the reason for the error, if the source attempts to retry, and the name of the event log in the Health Event Index.

Type	Reason	Retries	Retry Behavior	Health Event Name
ThirdPartyConfig	Normally due to an invalid configuration. You'll need to review your Source configuration and make an update.	No retries are attempted until the Source is updated.	Not applicable	ThirdPartyConfigError
ThirdPartyGeneric	Normally due to an error communicating with the third-party service APIs.	Yes	The Source will retry indefinitely.	ThirdPartyGenericError
FirstPartyGeneric	Normally due to an error communicating with the internal Sumo Logic APIs.	Yes	The Source will retry indefinitely.	FirstPartyGenericError

Restarting your Source

If your Source encounters ThirdPartyConfig errors, you can restart it from either the Sumo Logic UI or Sumo Logic API.

UI

To restart your source in the Sumo Logic platform, follow the steps below:

1. Open the Collection page, and go to Manage Data > Collection > Collection.
2. Select the source and click the information icon on the right side of the row.
3. The API usage information popup is displayed. Click the Restart Source button on the bottom left.
   
4. Click Confirm to send the restart request.
   
5. The bottom left of the platform will provide a notification informing you the request was successful.
   

API

To restart your source using the Sumo Management API, follow the instructions below:

• Method: POST
• Example endpoint:

  https://api.sumologic.com/api/v1/collectors/{collector_id}/sources/{source_id}/action/restart

Sumo Logic endpoints like api.sumologic.com are different in deployments outside us1. For example, an API endpoint in Europe would begin api.eu.sumologic.com. A service endpoint in us2 (Western U.S.) would begin service.us2.sumologic.com. For more information, see Sumo Logic Endpoints.

JSON configuration

Sources can be configured using UTF-8 encoded JSON files with the Collector Management API. See how to use JSON to configure Sources for details. 

Parameter	Type	Required	Description	Access
config	JSON Object	Yes	Contains the configuration-parameters of the Source.	n/a
schemaRef	JSON Object	Yes	Use {"type":"Gmail Trace Logs"} for Gmail Trace Logs Source.	not modifiable
sourceType	String	Yes	Use Universal for Gmail Trace Logs.	not modifiable

Config parameters

Parameter	Type	Required	Description	Access
name	String	Yes	Type the desired name of the Source and it must be unique per Collector. This value is assigned to the metadata field _source.	modifiable
description	String	No	Type the description of the Source.	modifiable
category	String	No	Type the category of the source. This value is assigned to the metadata field _sourceCategory.	modifiable
fields	JSON Object	No	JSON map of key-value fields (metadata) to apply to the Collector or Source. Use the boolean field _siemForward to enable forwarding to SIEM.	modifiable
projectId	String	Yes	The project ID is the globally unique identifier for your project. For example, pelagic-quanta-364805.	modifiable
datasetId	String	Yes	The Dataset ID is the project-wise unique identifier for your dataset. For example, gmaillogsbigquery.	modifiable
privateKey	String	Yes	The Private Key is part of Service Account JSON, it is a security key which is required for authentication.	modifiable
clientEmail	String	No	User email collected from the Google Cloud Console.	modifiable
tokenUri	String	Yes	The Token URI is part of Service Account JSON, it is used for generating the token.	modifiable
startTime	String	Yes	This sets how many hours the Source checks for new data. The default is 24 hours.	modifiable
dataLocation	String	Yes	Dataset ID is the project-wise unique identifier for your dataset.	modifiable

JSON example

{
   "api.version":"v1",
   "source":{
        "config":{
            "name":"Gmail Trace Log",
            "category":"gmail",
            "projectId":"Product123",
            "datasetId":"Product123",
            "privateKey":"*****************",
            "tokenURI":"dshjfgbkjlafdhbdhfvhjksdg",
            "clientEmail":"product123@gmail.com",
            "dataLocation":"US",
            "startTime":"24 Hours ago"
        },
        "schemaRef":{
            "type":"Gmail Trace Logs"
        },
        "sourceType":"Universal"
   }
}