Baseline conditions and seasonality

Lesson 5 · Extension track · the feature built for lesson 2's problem, which CCAP does not use

What a baseline condition is

A STATIC condition compares each data point to a number you chose. A BASELINE condition compares each data point to a number New Relic predicted from the signal's own history, and the threshold you set is a count of standard deviations from that prediction.

That single change moves the tuning problem. You stop asking "what error rate is too high for this carrier?" and start asking "how unusual must this be before I care?"

How much Shippo uses it

Two readings. First, baseline is a 5.6% minority. Second, only 10 Terraform blocks account for 56 live baseline conditions, so most of them were made in the UI, the same click-ops gap measured in lesson 4.

baseline_direction

A prediction has two sides. This argument says which side you care about.

The pattern is legible. Error signals use upper_only: too many errors is bad, unusually few is a gift. Signals where both directions are suspicious, such as login volume and IT alert flow, use upper_and_lower. Nothing at Shippo uses lower_only, because a drop in successes is normally caught with a static BELOW threshold instead, which is easier to reason about.

signal_seasonality, the answer to lesson 2

Lesson 2 established that Shippo carrier traffic swings 1.84× between its 09:00 trough and its 14:00 peak. Seasonality is the New Relic feature built precisely for that: it tells the baseline model that the signal has a repeating time-of-day or day-of-week shape, so 3 a.m. is compared against other 3 a.m.s rather than against a flat average.

Shippo sets it in exactly three places, always to the same value.

new_relic_calculation lets New Relic detect the period itself. The provider also accepts explicit periods; check the provider documentation for the current accepted set rather than assuming, because the Shippo module applies no validation:

variable "signal_seasonality" {
  type        = string
  description = "The seasonality of the signal."
  default     = null
}

shippo-tf-services/modules/newrelic-alert/vars.tf:158

No CCAP condition sets it. Not one.

Why CCAP chose ratios instead, and when that is wrong

There are two ways to make a threshold survive diurnal volume.

That last row is the decision rule. A ratio needs a denominator that means something. Error rate has one, which is total calls. A raw count of registrations does not. That is why [CCAP] Carrier MCA Registration Count | Anomaly Detection, which counts registrations per carrier, was written as a baseline condition rather than a ratio: there is no sensible denominator for "how many registrations should have happened".

When to reach for baseline

1. The signal is a count with no denominator, and the count varies with time of day.
2. The signal has months of stable history to learn from.
3. You cannot name a correct static threshold, and neither can anyone else on the team.
4. A false negative is more acceptable than a permanently mistuned static bar.

If you do use it, add signal_seasonality = "new_relic_calculation" whenever the signal shows the daily shape from lesson 2. Three Shippo services already do. Carrier signals have exactly that shape and none of them do.

Check yourself

Read this next

Primary source: New Relic docs, Alert conditions, section on baseline thresholds.

Provider reference for the accepted signal_seasonality and baseline_direction values: Terraform newrelic_nrql_alert_condition.