Skip to content

Watching performance for changes

Standing watches over the figures platforms report, and the anomalies and viral opportunities they find. Every number shows the window and denominator behind it.

Updated

A performance watch is a standing question about one figure: tell me when it stops behaving the way it normally does, and show me why you say so.

It reads figures platforms have reported into your performance records over time, and reports two kinds of finding. An anomaly is a figure that departed from its own recent history by more than the threshold you set. A viral opportunity is one piece of content responding at a far higher rate than its siblings.

Every finding arrives with the whole basis behind it: the days it was compared against, how many of them were used, the middle value and spread of that baseline, the score, the threshold, and - for anything expressed as a rate - the exact numerator and denominator and the period they cover. Nothing here is worked out by a language model. If an explanation is ever attached to a finding, it explains figures that were already calculated; it never produces one.

BEFORE YOU START

  • Performance figures recorded for the workspace. Without them there is no series to watch, and the page says so rather than showing an empty table.
  • Enough history. A watch needs a stated minimum of settled days before it will compute anything, so that ordinary weekly ups and downs are not read as changes.
  • The "edit" permission to create, change or run a watch. "View" is enough to read watches and findings.
  • Nothing else, and no credits.

STEPS

  1. Open Engage and choose "Performance watch", or go to /app/performance-watch.
  2. Read the watches table. Each row shows the figure being watched and the whole rule behind it, so a finding can always be checked against it.
  3. Check the "Last run" column before reading anything else. See below - this is the column that matters most.
  4. Read the findings underneath. Each one states the day, the comparison window, how many days or peers it was measured against, and the sentence that makes it checkable.
  5. Where you have the "edit" permission, choose "Run now" on a watch to assess it immediately rather than waiting for the overnight run.

WHAT YOU SHOULD SEE

The "Last run" column says one of three things, and they mean different things:

  • "Never run" - the watch has been created and not yet evaluated.
  • "Assessed" - the figures were read and the arithmetic ran. Only this one means an empty findings list is reassuring.
  • "Could not be assessed" - the watch could not read what it needed, with the reason beside it. This is NOT a clean bill of health, and the reason names what is missing.

Where nothing has been ingested at all, the page says that plainly and lists what a watch would need, instead of showing an empty findings table that would read as "nothing happened".

WHAT THIS WILL NOT DO

  • It does not predict anything. Every finding is about days that have already been measured.
  • It does not state a confidence percentage. The score behind a finding is a robust distance, not a significance test, and turning it into a percentage would require assuming the figures are distributed in a way they are not. What a finding does say is how many of the baseline days it exceeded, which assumes nothing.
  • It does not fill in a missing day. A day nobody reported is left absent, and a watch refuses rather than reading the gap as a collapse.
  • It does not compare two platforms with each other. Platforms count days against boundaries they choose and some will not say which, so figures from two of them are never combined.
  • It does not email, notify or push. Findings are recorded and read here.

IF IT DOES NOT WORK

  • "No platform has reported a figure for this organisation" is the standing state where nothing has been ingested. It lists the three things a watch needs, and it is a statement that nothing was measured rather than that performance was flat.
  • "Not enough history to say what normal is" means the baseline is shorter than the minimum the watch states. It resolves itself as more days are recorded.
  • "The baseline is too patchy to say what normal is" names the exact dates with no reported figure. Those days point at the ingestion that stopped.
  • "No spread to measure against" means the figure has been identical on more than half the baseline days. There is no scale to measure a change against, so no score is given, and the constant is named instead.
  • "No figure was reported for that day, so there is nothing to compare" means the day being assessed is missing. It is not a fall.
  • "The figure is below its baseline, but the provider is still counting that window" means the day has not settled. It will rise, so no fall is reported.
  • "The evaluation could not be reached" means it is not known whether the run happened. It is not a report that nothing was found.

COMMON QUESTIONS

Why does a finding say "Modelled" or "Estimated" instead of "Observed"?

Because of how the underlying figure was arrived at. An observed figure was counted by the platform on its own property. A modelled one was produced by the platform's model rather than counted, and an estimated one came from a sample. Those can move because a platform changed its method, with nothing having changed in your audience - so a finding resting on one is a weaker claim, and it says so. Where a figure is worked out from several others, it carries the weakest of them.

Why is a post with a very high engagement rate not reported as a viral opportunity?

Most often because too few people saw it. A rate over a handful of impressions is noise, so a watch states a minimum below which it will not judge an item. It may also be that the platform withheld some of the figures, in which case the denominator is smaller than reality by an unknown amount and no rate is computed at all.

Why did running a watch twice not produce two findings?

Because a finding identifies a fact - this watch, this series, this day - rather than a run. Re-running over the same day updates the same finding instead of reporting it again.

  • Reading your reporting

    What the workspace report totals, why a figure can say "Not reported" instead of zero, and the two ways a figure reaches it.

  • Performance recommendations

    Reading the performance recommendations, the sample size every claim must clear, and the list of questions the recorded data cannot answer.