Setting Thresholds in Canary

Canary by Pagos is the ideal tool for keeping on top of any surprise deviations in your key payment information. It expertly monitors your payments performance—thereby removing any manual monitoring responsibility from you and your team—and notifies you of data anomalies in real time. To learn more about the Canary product in whole, visit our blog post Announcing Canary.

But how exactly does Canary monitor your data? With threshold triggers!

Any Canary user can create a trigger by configuring the following fields:

1. The metric you want to monitor

2. The frequency you want the metric checked

3. The type of threshold: simple or relative

4. Any groupings or data subdivisions you want

Essentially, Canary tells you when the value of the metric you choose crosses the established threshold, given the frequency and grouping parameters you’ve set. Together we will explore the types of thresholds you can choose in Canary and how they work.

Threshold Types

Canary has two types of thresholds: 

• Simple - A completely customizable threshold value; you enter any value you want, and specify whether you want Canary to notify you if the your data falls above or below that value

• Relative - A threshold value that is created relative to your historical data; Canary will notify you when your data falls above or below the established norm

Let's dig in further to each type of threshold with an example.

Simple Thresholds

A simple threshold is basically a blank space. As the user, you get to set your own expectations for individual metrics and instruct Canary exactly when you want to receive a notification about changes to your processing data.

Say you’re on a payments operation team that has an established KPI for Approval Rate of 90%. Every month, you present to your leaders on whether or not you succeeded in this KPI and report any incidents where the KPI was missed. Because you have a specific desired value for Approval Rate, you'll want to set up a Canary simple threshold that alerts you every time Approval Rates fall below 90%.  To do so, you log into your Canary account and add a new trigger:

From there, you add in the configuration details you want, add a title and description, and choose the Approval Rate metric. Then, complete the metric configuration details:

• Where it says Threshold Type, choose Simple from the drop-down menu

• Since this is an overall KPI, skip the optional Group and Filter parameters

• For Direction, choose Below

• Beside Threshold, type 90

In the Canary dashboard, there’s a chart that shows a day’s worth of historical data with the threshold overlaid as a dotted line. This should give you a sense about how frequently your data crosses your proposed simple threshold. If it looks agreeable to you, save the metric. It’s that simple! Now any time your approval rate drops below 90%, Canary will alert you, according to your set notification preferences. You can be notified by web application, mobile application, email, and webhook—or any combination therein.

Relative Thresholds

Relative thresholds are automatically configured by the Canary application for your specific data. When you select Relative for the Threshold Type, the Canary application calculates the value where 95% of your historical data sits below the threshold (or above, depending on how you configure the trigger). You don’t have to know what this value is—Canary calculates it for you relative to your actual data.

If your company doesn’t have a specific KPI you want to hit for Approval Rate, for example, you can add a trigger with a relative threshold to keep you updated when this metric deviates unexpectedly from your historical Approval Rates. You’ll configure this relative threshold in the same way as the simple threshold example above—the only difference is that you will not enter a threshold value yourself!

How Do You Choose a Threshold Type?

Ultimately, it depends on how you and your organization think about your data. The simple threshold is a great option if you already know what good performance looks like. Maybe, like in our simple threshold example above, you already have established KPIs. Maybe you would like to configure multiple simple threshold triggers to determine what your KPI should be. Either way, you have a hypothesis about what is “normal” for your business and you just want to track that.

Relative thresholds are great if you don’t want a noisy trigger from Canary. With a relative threshold, you’ll only receive Canary notifications about 5% of the time. It is important to note, however, that this is based on historical data. If there is a major change to operations or your payment stack that creates durable change in your metrics, you will likely need to create a new threshold. 

You may need both types of thresholds; you are not limited to just one! Maybe you have a well-established KPI for Approval Rate. In that case, use a simple threshold that matches your KPI. However, you may be just starting to explore monitoring Average Order Value (AOV). Then, you may create a relative threshold for AOV. The combinations and possibilities are near-endless, and perfectly customizable to your business needs.

Where Do We Go From Here?

Monitoring your payments performance has never been easier. 

No-code beta integration is now open to businesses processing with Braintree, Stripe, or Adyen, and our API is available for use with other processors. Register to join the Canary Beta today or contact us for more information!