13.10Prometheus metrics

Your own Prometheus can scrape obsrvly as one more target. One GET to /api/gw/result/v1/metrics returns every monitor in your team in the Prometheus text format. It works on every plan.

Use a read-only API key (13.2). It can read the metrics but cannot change anything, so a leaked Prometheus config cannot pause or delete your monitors. A full-access key works too. A signed-in session works, but its token expires in a minute.

To make one, select new read-only key in the scrape with prometheus box on the api keys page. The new key form opens with the name prometheus and read only ticked.

prometheus.yml

yaml

scrape_configs:
  - job_name: obsrvly
    scheme: https
    metrics_path: /api/gw/result/v1/metrics
    scrape_interval: 30s
    authorization:
      type: Bearer
      credentials_file: /etc/prometheus/obsrvly_key # holds a read-only obs_ key
    static_configs:
      - targets: [www.obsrvly.com]

The api keys page shows the same snippet with this site's address filled in.

Try it with curl

bash

curl -s https://www.obsrvly.com/api/gw/result/v1/metrics \
  -H "Authorization: Bearer $OBSRVLY_KEY"
# obsrvly_monitor_up{monitor_id="0b5e3c1a-...",type="url"} 1

The metrics#

Every sample has two labels: monitor_id and type (url, tcp, dns, api, browser, heartbeat, cert, agent, udp, domain, ping). The monitor's name is not a label yet. ListMonitors maps an id to its name. All metrics are gauges.

Metric	Value
`obsrvly_monitor_up`	1 when the last check passed, 0 when it failed or was degraded
`obsrvly_monitor_active`	0 when the monitor is paused or the team is suspended
`obsrvly_monitor_overdue`	1 when no result has arrived for longer than the interval allows
`obsrvly_monitor_last_check_timestamp_seconds`	Unix time of the last check
`obsrvly_monitor_response_time_seconds`	The last check's response time
`obsrvly_monitor_response_time_p95_24h_seconds`	95th percentile response time over the last 24 hours
`obsrvly_monitor_uptime_24h_ratio`	Share of the last 24 hours' checks that passed, 0 to 1, counted as the dashboard counts uptime
`obsrvly_monitors_truncated`	1 when your team has more than 1,000 monitors. It has no labels.

A sample with nothing true to say is left out, not sent as 0. A paused, overdue or never-run monitor has no obsrvly_monitor_up. A monitor with nothing measured in 24 hours has no uptime.
A certificate close to expiry counts as up, as it does in uptime.
Metric and label names stay the same. New ones may be added.

Limits#

One scrape lists at most 1,000 monitors, the first by monitor_id.
The answer is cached for 15 seconds per team. Scraping more often than that shows nothing newer.
Each team may scrape 30 times a minute, with bursts of 10. Past that the answer is 429 with a Retry-After header (13.4).