Alert sources and levels
An alert source is where an alert goes, and every alert source is a URL. Anything that can make an HTTP request can send to it: a probe or a vital of ours, Grafana, a CI job, Home Assistant, a line of shell. The alert source decides how loud a message is allowed to be, who it reaches, and when it should keep quiet.
What a sender can put in a message is on Sending alerts. Who a page goes to on a team is on On call.
Making one
Make an alert source under Alert sources in the web app, from the Alert sources tab in Mast Pager, over the API, or from a mast.toml with ribo mast apply. Whichever you use, it asks for three things:
- name: letters, digits, dot, underscore, hyphen and spaces, up to 32 characters, starting with a letter or digit.
- kind: an Alert source receives messages when something happens; a Vital expects a heartbeat, and silence is the alert. A new vital starts at every hour with ten minutes of grace.
- default priority: quiet, normal, loud or page. This also sets the alert source's highest level, so only an alert source created at Page can carry a page.
Creating it hands you the key card: the send address as text and as a QR code, with Copy and Share, and a curl line to paste. For a vital the line is curl -fsS on the end of a crontab entry. Send a test page sends at the alert source's highest level so you hear what the loudest thing this alert source can do sounds like; on a vital the test doesn't count as a beat.
The address is the secret
The URL is the whole credential for one alert source. Anything holding it can send to that alert source, and nothing else; it can't read the feed, make alert sources or reach any other alert source. Keep it wherever you keep other secrets. If it leaks, someone can fill your feed, but they can't make the phone ring louder than the highest level you set, and Rotate key on the alert source's page replaces it in one tap. The old key keeps working for seven days, so there's time to update whatever sends with it, unless you're at your plan's key cap, when it stops at once. If you think someone else has the address, This isn't mine on a message cuts the old key off immediately. Delivery has that one.
An alert source's settings
Open an alert source and every setting is a row. The web app and the app show the same ones:
| Row | Choices | Notes |
|---|---|---|
| Default priority | quiet / normal / loud / page | What a send gets when it names no level |
| Highest level | quiet / normal / loud / page | A send asking for more gets this. Only the phone can allow page, because it needs permission to ring through Do Not Disturb. Lowering it below the default lowers the default too |
| Sound | default, bleep, chirp, rising, pager, klaxon | Tap a tone to hear it. A send that names its own tone wins |
| Mute hours | off, or from / until | Inside the window this alert source arrives silently. Pages are the exception |
| During a pause | quiet / still sounds | Whether a phone-wide pause holds this alert source's pages |
| Escalation | none, or a policy | Web app only. Which people a page goes to, in what order. On call |
| Escalation partner | never, or after 2 / 5 / 10 / 30 min | Where an unacknowledged page goes next. A second phone |
| Mute | not muted, 15 min, 1 h, 4 h, until 8:00 tomorrow | Silence in a hurry. Pages are the exception |
| Repeat | once, or every 1 / 2 / 5 min | How often a page comes back while nobody has acknowledged it |
| Expire | 15 min, 1 h, 4 h, 1 day, never | When a page stops repeating and stays in the feed as one nobody answered |
| Dedupe window | off, 1 / 5 / 15 min, 1 h | Merges an identical message into the one already there and counts it. Alert sources only |
| Grouping threshold | never, or 10 / 20 / 50 in five minutes | An alert source that floods collapses into one row with a count. Alert sources only |
A vital has three rows of its own instead of dedupe and grouping: Heartbeat period (5 min, 15 min, an hour, 6 hours, a day), Grace, and When it stops, which is the level a missed heartbeat pages at. Crontab schedules and run limits are set in the web app or over the API, and the phone shows them read-only. Vitals has them.
At the foot of every alert source: Send a test, Rotate key, Delete alert source. Deleting stops the address at once; messages already in the feed stay until they age out of retention.
Testing it end to end
Send a test sends one message and stops there. It doesn't ask for an acknowledgement and doesn't repeat, so it tells you the phone can be reached and nothing about who gets woken at 3am.
To try the whole thing, open the test's options in the web app and pick Run the escalation policy. That opens a real incident, labelled as a test, and runs the alert source's policy on it: step 1's people are paged, and if nobody acknowledges, step 2's are, on the waits you set. Acknowledge it to stop it. It's still held to the alert source's highest level, so on an alert source that tops out at loud the test arrives at loud and says so. Test incidents are marked in Incidents and don't count toward uptime. Over the API it's {"escalate": true} on the test route.
To check one device on its own, Send a test next to it under Devices sends a single loud push to that device and no other. A device Apple or Google has stopped accepting pushes for is marked there as not reachable, with the date it was turned down.
Neither a dedupe window nor a grouping threshold ever collapses a page. A pager that swallowed the second page because it looked like the first has lied to you once, and you wouldn't know which time.
The list on the phone
The Alert sources tab is two sections, vitals and alert sources, each row with a one-line summary: pages, escalates to Sam, mute hours 22–07, muted until 14:10, 3 today. Swipe right on a row to Mute 1h (or Unmute); swipe left to Delete, behind a confirmation. Neither fires on a full swipe. Pull the row and the button waits to be tapped.
Levels
Every message carries one of four levels. A level isn't a score for how bad something is; it decides what the phone does and who gets interrupted.
| Level | In one line | The detail |
|---|---|---|
| quiet | No sound, shows in the list | Into the feed and the notification list. No banner, no lit screen |
| normal | Sound and banner, Do Not Disturb silences it | Also held by the ringer switch, by a Focus and by the alert source's mute hours |
| loud | Breaks through Focus, Do Not Disturb still silences it | Time Sensitive. Once, unless the send asks for every |
| page | Rings through silent mode and Do Not Disturb, and repeats until someone answers | Needs Critical Alerts on, on iPhone. Re-alerts every two minutes until somebody acknowledges it, then moves to the next person or a second phone |
Three questions pick a level, in order, stopping at the first no. Does a person have to do something? If not, quiet or normal. Does it have to be today? If it can wait for morning, normal, and mute hours will hold it there. Would you rather be woken than find it at breakfast? Yes is a page. No is loud.
The cost of over-paging isn't the noise. It's that the next real page arrives at a phone you've already taught yourself to ignore.
The highest level
A sender can ask for any level it likes. Asking for more than the alert source's highest level gets the highest level, and the feed shows both: asked for page · sent loud. An alert source made in the app can be given any highest level, page included, because the person choosing it is holding the phone that will ring. An alert source made with an API token starts at loud and the token can't raise it. Deciding to be woken up is not something a credential should do on your behalf.
What to put on each one
quiet is for things you want on the record and never want to be told about: a deploy that landed, a backup that finished with the byte count in the body, a certificate that renewed early. You read these when you go looking.
normal is for something a person should see today, on their own time: a payment that failed and will retry, a webhook that gave up, a signup flagged for review. Most alert sources never need to be louder.
loud is for something that turns into a page if it's left alone: storage down to two replicas of three, a queue growing for an hour, a disk at 92%. It gets through a Focus once and stops, unless the send asks for every.
page is for when somebody has to act and it can't wait until they next pick up the phone: the database is unreachable, the backup hasn't checked in for 25 hours, a probe says the site is down from six places. It assumes somebody will be found, so use it where that's true.
Mute hours, muting, and pausing pages
Mute hours are a window on the alert source. Inside it, that alert source's messages arrive without a sound. Mute is the same idea in a hurry: an hour of silence from a swipe on either list, from the notification you're looking at, or longer from the alert source's own page.
A page ignores both, and there is no setting to change that. An alert source that can wake you is an alert source you asked to be woken by. If you have an alert source that should never wake anyone, the setting you want is its highest level: hold it at loud and mute hours hold everything it sends.
There is one thing a page does obey. Pause pages, in the app's settings, runs for half an hour, an hour or two hours. Pages that land inside it go into the feed without a sound, and the moment it closes, the ones nobody answered go out again at full volume. Nothing is dropped. While a pause runs the feed carries a banner counting it down, End now is one tap, and you can move the end. An escalation partner is still paged on time.
An alert source can be exempted under During a pause in its own settings. Keep that for the one thing worth standing up and walking out of the meeting for.