How to enable Datadog performance monitoring

GeniePy comes with built-in support for tracking application performance using Datadog.

Description

Datadog is an excellent service for measuring application performance metrics of different kinds.

In this how-to, we'll focus on enabling two kinds of metrics: Application Performance Monitoring (also known as Tracing), and Dogstatsd metrics.

Setup

Register for a Datadog account if you do not already have one.

After you have an account, generate an API key and an application key for sending application data over to Datadog. Datadog has good documentation on what those two keys mean and how to generate them here: https://docs.datadoghq.com/account_management/api-app-keys/.

In short, API keys are unique to your organization while Application (or App) keys are required for programmatic access to the Datadog API.

  1. Generate an API key here: https://datadoghq.com/organization-settings/api-keys
  2. Generate an Application key here: https://datadoghq.com/organization-settings/application-keys

Configuration

Before enabling tracing or dogstatsd metrics, set the following two environment variables:

  1. DATADOG_API_KEY: this is the API key generated in the previous step
  2. DATADOG_APP_KEY: this is the application key generated in the previous step

Application Performance Monitoring (Tracing)

Set the DDTRACE_ENABLED environment variable to true.

This will automatically patch Starlette and SQLAlchemy to start sending application level traces to Datadog. These traces will start appearing in your APM dashboard.

Dogstatsd

Set the DOGSTATSD_ENABLED environment variable to true.

This will start a thread in the main application process that sends over custom metrics to the Datadog agent container, which in turn sends them over to Datadog. All these metrics will start appearing in your Metrics Explorer.

Development

In local development, we've switched both DDTRACE_ENABLED and DOGSTATSD_ENABLED variables to false.

In case you'd like to enable them locally to debug performance issues, simply set them both (or either one of them) to true and restart the main application process.