Analytics

Big Query

BigQuery is a Google data warehouse that a lot of our data calls home. This includes all our analytics data and some (not all) production data that might be of interest when joins are necessary for answering richer, more complex questions. From sentry/getsentry, our data goes through reload, our ETL for BigQuery.

As a general rule, do not add an analytic event if it has the potential to be triggered over 1,000,000 times/day. If this threshold is exceeded, it may cause the analytics service to crash. If you are unsure of how frequently the analytic event will be triggered, add it as a Datadog metric first.

Amplitude

Amplitude is a product analytics tool that we use to track user behavior. We use Amplitude to track events such as when a user views a page, clicks a button, or performs a search.

Backend Analytics

Step 1: Create your Analytics Events!

Conventionally, the analytics events are stored in a file named analytics within the folder of the code it tracks. i.e.: sentry/integrations/analytics.py for the Integration analytic events and sentry/api/analytics.py for the API analytic events.

The Event classes look like this:

from __future__ import absolute_import, print_function

from sentry import analytics

class ExampleTutorialCreatedEvent(analytics.Event):
  type = 'example_tutorial.created'

   attributes = (
      analytics.Attribute('id'),
      analytics.Attribute('user_id'),
  )

class ExampleTutorialDeletedEvent(analytics.Event):
  type = 'example_tutorial.deleted'

   attributes = (
      analytics.Attribute('id'),
      analytics.Attribute('user_id'),
  )

analytics.register(ExampleTutorialCreatedEvent)
analytics.register(ExampleTutorialDeletedEvent)

Your event classes will inherit from analytics.Event as shown above. All events have a type and attributes.

• type: Describes what the Event is, and this name should be unique across all analytics event classes.

• attributes: Parameters what you would like to track, for example the user_id of the user performing the action. All attributes must be an Attribute object as shown above. Note that you cannot create an attribute named 'type'.

Finally, register your event classes so that the analytics event_manager will pick them up.

If you are creating the analytics.py file for the first time:

If you are creating a new analytics file for the first time, you will need to add an import to the package's __init__.py.

If the Event classes are defined in a file named: sentry/examples/analytics, then the class below would be defined at sentry/examples/__init__.py:

from __future__ import absolute_import

from .analytics import *  # NOQA

Here, you have your usual absolute_import but in addition you will import every class in your analytics.py add # NOQA to avoid linter complaints.

Step 2: Add it to the code you want to track

You'll be adding code in some user-facing area like an API endpoint.

from sentry import analytics

class ExampleEndpoint(Endpoint):

  def post(self, request):
    example = Example.objects.create(...)
    analytics.record(
        'example_tutorial.created',
        id=example.id,
        user_id=request.user.id,
    )
    return Response(serialize(example, request.user))

Do whatever you would normally with the endpoint, then use the analytics.record method to gather the information you want. Note that it takes input of the form:

analytics.record(
	'event_type_as_string',
	<attribute_name_0>=<value>,
		....
	<attribute_name_n>=<value>,
)

Run the tests that touch the endpoint to ensure everything is Gucci.

Step 3:

By default, a new event type is aggregated and sent to Amplitude as long as there is a user_id sent along with the event. If you would like to send events unaggregated, refer to our Amplitude aggregation docs

Route-Based Frontend Analytics

All in-app page loads will generate analytics events by default (both Reload and Amplitude). All analytics related to page load should be handled within the route-based analytics system instead of manually instrumenting front-end events. The default names are based on the parameterized path and are prefixed with Page View. Here is an example event name:

Page View: :OrgId :ProjectId Releases

Any field that has a : at the front is a parameterized field. The value of the field will be the value of the parameter. For example, the :OrgId field will be the value of the orgId parameter (really the slug).

Customization

Route-based analytics can be customized to have different event names and parameters. For functional components, the following hooks can be used:

• useDisableRouteAnalytics: Disables route-based analytics for the component.
• useRouteAnalyticsEventNames: Customizes the event name. First argument is the Reload eventKey and second argument is the Amplitude eventName.
• useRouteAnalyticsParams: Customizes the parameters.

Example:

export default function SpanDetailsContentWrapper(props: Props) {
  const {location, project,} = props;

  // customize the route analytics event we send
  useRouteAnalyticsEventNames(
    'performance_views.span_summary.view',
    'Performance Views: Span Summary page viewed'
  );
  useRouteAnalyticsParams({
    project_platforms: project ? getSelectedProjectPlatforms(location, [project]) : '',
  });

The main purpose of customizing the event names is to create more readable event names that won't change if the underlying route changes.

For class based components, the following functions can be used:

• setDisableRouteAnalytics: Disables route-based analytics for the component.
• setEventNames: Customizes the event name. First argument is the Reload eventKey and second argument is the Amplitude eventName.
• setRouteAnalyticsParams: Customizes the parameters.

You will need to import both withRouteAnalytics and WithRouteAnalyticsProps. WithRouteAnalyticsProps needs to be added to the Props of the class. The functions can then accessed through the Props. withRouteAnalytics is called while exporting the class.

Example:

class Monitors extends AsyncView<Props, State> {
  componentDidMount() {
    this.props.setEventNames('monitors.page_viewed', 'Monitors: Page Viewed');
  }

Button Click Analytics

All in-app button clicks will generate analytics events by default for Reload. As of now, they are not automatically being sent to Amplitude, but you can set it up so that the events are sent there.

Out of the box, each click on a button will track the text (aria-label), priority, href, and parameterized_path of the button. You might encounter an issue where some of the data will be an empty string which is a known bug. The event name will be based on the parameterized path , similar to route-based analytics (see above for more information) but will be prefixed with button_click.

Customization

There are a few ways to customize the button click analytics:

• analyticsEventName - Allows the event to be sent to Amplitude with the given event name. If this is not set, then the event will not be sent to Amplitude
• analyticsEventKey - Overrides the default parameterized path event key and replaces it with the given event key. Unless analyticsEventName is also set, this will continue to only send events to Reload, but using the new event key. The parameterized path will still be sent as part of the event, but will not be the event key.
• analyticsParams - Adds additional parameters that will be tracked with each button click

Example:

<RestartButton
  analyticsEventName="Growth: Guided Tour Restart"
  analyticsEventKey="growth.guided_tour_restart"
  analyticsParams={{ tour: "issues" }}
/>;

In this example, clicking this button will now send an event to amplitude with the analyticsEventName , send an event to reload with the new analyticsEventKey, and add on tour as a parameter sent with each event.

For Frontend events

For analytics events that happen in Sentry, the function trackAnalytics is what should be used. For Getsentry, trackGetsentryAnalytics is what to use.

Step 1: Add the Typescript Definition

First, add the Typescript definition of the event to an analytics event file inside the analytics directory like issueAnalyticsEvents.tsx. There are two parts of this:

1. Define the event parameters that get exported

export type ExampleTutorialEventParameters = {
  "example_tutorial.created": {
    source: string;
  };
  "example_tutorial.viewed": {
    source: string;
  };
};

2. Define the Reload to Amplitude name mapping. If the value is null, then the event will not be sent to Amplitude. This is usually only done for very high volume events.

export const exampleTutorialEventMap: Record<
  keyof ExampleTutorialEventParameters,
  string | null
> = {
  "example_tutorial.created": "Example Tutorial Created",
  "example_tutorial.viewed": null, // don't send to Amplitude
};

Step 2: Add the Event in Code

Now, you can use the event in code using trackAnalytics or trackGetsentryAnalytics. Both functions take the following arguments:

• eventKey: This describes the key used in Sentry's own analytics system (Reload). It will map to an Amplitude name as determined in step 1.

• analyticsParams: This object will hold all the information you're interested in tracking. Generally, you always pass in an Organization object into it unless the event is not tied to a specific organization. In getsentry, you should pass the Subscription as well. Certain fields like the role and plan will be pulled out of those entities and added to the event payloads.

• options: This field allows passing the following optional fields:

  • mapValuesFn: An arbitrary function to map the parameters to new parameters
  • startSession: If true, starts an analytics session. This session can be used to construct funnels. The start of the funnel should have startSession set to true.
  • time: Optional unix timestamp.

Typing and Mapping

All events should be typed which specifies what the payload should be. We also define a mapping from the Reload event name to the Amplitude event name.

Naming Convention

Our current naming convention for Reload events is descriptor.action e.g. what we have above with example_tutorial.created and example_tutorial.deleted . You want these to be specific enough to capture the action of interest but not too specific that you have a million distinctly named events with information that could be captured in the data object. For example, if you can create your example tutorial from multiple places, fight the urge to have the source as part of your descriptor i.e. example_tutorial_onboarding.created and example_tutorial_settings.created. Your future self running analytics will thank you. Amplitude event names should be similar to the Reload event name except you should capitalize the words and use spaces instead of underscores.

Testing your Analytics Event

It's important to test analytic events to ensure the data you are looking at is accurate. Any additional analytic event should be tested before merging to make sure that the events are firing correctly (with the correct values at the right times). A common issue we see when testing gets overlooked is events firing multiple times when they should only fire once.

When developing locally, analytics events will not be sent to Reload or Amplitude. To test to see if your event is sending as expected and with the correct data, you can set "DEBUG_ANALYTICS" to "1" in local storage on your browser. Then it will log the analytics event data to your console, whenever it would've sent an analytics event, allowing to check your analytics locally.

getsentry

import React from "react";

import { trackAnalytics } from "getsentry/utils/analytics";

class ExampleComponent extends React.Component {
  componentDidMount() {
    trackAnalytics("example_tutorial.created", {
      organization,
      subscription,
      source: "wakanda",
    });
  }

  render() {
    return <h1> HI! </h1>;
  }
}

sentry

All you'll actually need is to import analytics from utils and call it wherever you need. Keep in mind the effect of React lifecycles on your data. In this case, we only want to send one event when the component mounts so we place it in componentDidMount .

import React from "react";

import { trackAnalytics } from "getsentry/utils/analytics";

class ExampleComponent extends React.Component {
  componentDidMount() {
    trackAnalytics("example_tutorial.deleted", {
      organization,
      source: "wakanda",
    });
  }
  render() {
    return <h1> HI! </h1>;
  }
}

After deploying your changes, you can open the Dev Tools in your browser and in the "Network" tab, search for the event/ request. This will show the events being sent to Reload and Amplitude.

Debugging

If your analytics aren't showing up after you added it, you can't find an event you expect to be there, or something else goes wrong, there are a few troubleshooting steps you can try:

• Follow the steps above to confirm that your analytics event is sending correctly, with the correct parameters.
• Check Amplitude for blocked events: In Amplitude, go to the "Data" section in the sidebar. From there, navigate to "Events" and search for your event name. It will show up with status "Blocked" if blocked, which means events won't show up. Some events may be blocked in favor of automatic route or button analytics.
• For route analytics, confirm that the analytic event isn't being blocked with useDisableRouteAnalytics. Some components already had an analytic event so the route analytics were disabled.
• Check the types of the data you are sending. Arrays aren't recommended data types to send (they can be hard to query and produce some unexpected behavior). Try to remove those if you are using them.
• Remember there will always be some discrepency. Ad-blockers, for example, can block events from being sent. This could be a cause of why some numbers aren't adding up.

Metrics

Track aggregrate stats with Metrics. For example, this can be used to track aggregate response codes for an endpoint.

Import the metrics library and use the metrics.inc function. The key needs to be unique.

from sentry.utils import metrics

metrics.incr(
    "codeowners.create.http_response", # needs to be unique
    sample_rate=1.0,
    tags={"status": status},
)

If you don't put a sample rate, you get 1 in 10 events. If the service is expected to have low traffic, we can start with a sample rate of 1.