Azure Event Hubs

Azure Event Hubs is a modern big data streaming platform and event ingestion service that can seamlessly integrate with other Azure and Microsoft services, such as Stream Analytics, Power BI, and Event Grid, along with outside services like Apache Spark. This integration helps in monitoring data plane access operations (such as send or receive events), and tracking performance metrics like consumer lag, consumer and publisher throughput, and active connections in your Event Hub.

Log and metric types

For Azure Event Hubs, you can collect the following logs and metrics:

• Resource logs. To learn more about the different resource log category types and schemas collected for Azure Event Hubs, refer to Azure documentation.

info

Some log types are only available in premium and dedicated tiers.

• Platform Metrics for Azure Event Hubs. These metrics are available in the namespaces below:
  • Microsoft.EventHub/clusters
  • Microsoft.EventHub/namespaces

For more information on supported metrics, refer to Azure documentation.

Setup

Azure service sends monitoring data to Azure Monitor, which can then stream data to Eventhub. Sumo Logic supports:

• Logs collection from Azure Monitor using our Azure Event Hubs source.
• Metrics collection using our HTTP Logs and Metrics source via Azure Functions deployed using the ARM template.

You must explicitly enable diagnostic settings for each Event Hub Namespace you want to monitor. You can forward logs to the same event hub provided they satisfy the limitations and permissions as described here.

When you configure the event hubs source or HTTP source, plan your source category to ease the querying process. A hierarchical approach allows you to make use of wildcards. For example: Azure/EventHub/Logs, Azure/EventHub/Metrics.

Configure metrics collection

In this section, you will configure a pipeline for shipping metrics from Azure Monitor to an Event Hub, onto an Azure Function, and finally to an HTTP Source on a hosted collector in Sumo Logic.

1. Create a hosted collector and tag the tenant_name field. You can get the tenant name using the instructions here.
   
2. Configure an HTTP Source.
3. Configure and deploy the ARM Template.
4. Export metrics to Event Hub. Perform the steps below for each Event Hub Namespaces that you want to monitor.
   • Choose Stream to an event hub as the destination.
   • Select AllMetrics.
   • Use the Event hub namespace created by the ARM template in Step 2 above. You can create a new Event hub or use the one created by the ARM template. You can use the default policy RootManageSharedAccessKey as the policy name.

Configure logs collection

In this section, you will configure a pipeline for shipping diagnostic logs from Azure Monitor to an Event Hub.

1. To set up the Azure Event Hubs source in Sumo Logic, refer to Azure Event Hubs Source for Logs.
2. To create the Diagnostic settings in the Azure portal, refer to the Azure documentation. Perform the steps below for each Event Hub Namespaces that you want to monitor.
   • Choose Stream to an event hub as the destination.
   • Select allLogs.
   • Use the Event hub namespace and Event hub name configured in the previous step in the destination details section. You can use the default policy RootManageSharedAccessKey as the policy name.

Troubleshooting

HTTP Logs and Metrics Source used by Azure Functions

To troubleshoot metrics collection, follow the instructions in Collect Metrics from Azure Monitor > Troubleshooting metrics collection.