Minimal Homelab Monitoring with Monit, Simple Scripts, and Pushover Alerts

Most small homelabs do not need an observability platform. They need a host-local way to complain before something quietly rots. That sounds less glamorous, which is part of the appeal.

Monit fits that job: it runs simple checks on a schedule and can alert or execute a command when one of them fails. If you have one or two hosts, a few services, and a backup path you still expect to work next month, start by monitoring outcomes rather than building dashboards. That is the sort of setup you still maintain when normal life intrudes.

This guide stays deliberately small: one NixOS Monit setup, one loopback HTTP check, one filesystem threshold, one storage-health script, one backup-freshness check, and one Pushover alert path.

Monitor Outcomes First

A process check is better than nothing, but it is the weaker signal. A service answering HTTP on loopback tells you more than a PID existing, a backup freshness file that stopped moving tells you more than a timer having existed once, and a storage-health check exiting non-zero tells you more than a shell wrapper technically still running.

That is the general rule for the small setup:

  • check the outcome directly when you can
  • fall back to a process check when the outcome is awkward to express
  • use a helper script only when Monit’s built-in primitives are too blunt

What A Small Monit Setup Should Actually Check

For a modest homelab, these check categories catch most of the failures worth hearing about first:

CheckWhat it catchesWhy it mattersBuilt-in or scriptFalse-positive risk
Root filesystem usageDisk filling upBasic host survivalBuilt-inMedium if the threshold is too tight
Loopback HTTP responseService dead, stuck, or misroutedProcess alone is weaker evidenceBuilt-inLow if the endpoint is simple
zpool healthStorage degraded or unhealthyStorage trouble is worth hearing about earlyScriptMedium if the script is sloppy
Backup stamp freshnessBackups quietly stopped runningSilent backup failure is the bad oneBuilt-inLow if the backup job always updates the stamp

If you are not using ZFS, swap that storage check for the signal you actually care about. The point is the category, not the command.

That is already enough to catch several boring but real problems:

  • the host is running out of room
  • the service is no longer responding locally
  • the storage layer is unhappy
  • the backup job stopped succeeding even though nothing dramatic crashed

What The Minimum NixOS Monit Config Looks Like

The NixOS Monit module mostly stays out of the way, which is an advantage here. This setup wants one readable config block, not another layer to debug next month.

This example assumes one NixOS host, one local web service, a ZFS pool for the storage-health example, and a backup job that updates /var/lib/backup/status/last-success only after a successful run. This is a pattern, not a copy-paste-complete module. You still need to place two executable scripts at the shown paths and provide a root-only env file containing PUSHOVER_APP_TOKEN and PUSHOVER_USER_KEY.

{
  services.monit = {
    enable = true;
    config = ''
      set daemon 60
      set logfile syslog

      check filesystem rootfs with path "/"
        if space usage > 85% then exec "/etc/monit/scripts/alert-pushover.sh rootfs_space"

      check host local-web-service with address 127.0.0.1
        if failed port 9090 protocol http request "/" then exec "/etc/monit/scripts/alert-pushover.sh local_web_service_http"

      check program zpool-health with path "/etc/monit/scripts/check-zpool-health.sh"
        every 5 cycles
        if status != 0 then exec "/etc/monit/scripts/alert-pushover.sh zpool_health"

      check file backup-stamp with path "/var/lib/backup/status/last-success"
        if timestamp > 26 hours then exec "/etc/monit/scripts/alert-pushover.sh backup_freshness"
    '';
  };
}

Every check above uses the same alert wrapper on purpose: one failure path, one place to test, and no quiet split between mail, syslog, and wishful thinking.

  • the filesystem check is blunt but useful
  • the HTTP check asks whether the service still works locally
  • the zpool script exists because storage health is more specific than a generic process test
  • backup-stamp means a file whose modification time changes only after a successful backup

That is the difference between “small” and “careless.” The config is short because it is focused, not because it ignores failure modes.

Which Checks Need Scripts

Only two pieces need shell glue here: the ZFS health check and the alert wrapper.

The storage-health script should stay short enough that you can still trust it:

#!/bin/sh
set -eu

if zpool status -x 2>/dev/null | grep -q '^all pools are healthy'; then
  exit 0
fi

printf '%s\n' "zpool health check failed" >&2
exit 1

The point is not to cram the whole storage policy into a shell file. The point is to expose one narrow truth to Monit: the pool looks healthy, or it does not.

The alert wrapper can stay equally small:

#!/bin/sh
set -eu

check_name="${1:-monit_check}"
. /run/secrets/monit-pushover.env

curl -fsS https://api.pushover.net/1/messages.json \
  --form-string "token=${PUSHOVER_APP_TOKEN}" \
  --form-string "user=${PUSHOVER_USER_KEY}" \
  --form-string "title=$(hostname): Monit alert" \
  --form-string "message=Check failed: ${check_name}" \
  >/dev/null

The public example keeps the secrets out of the script and out of the Monit config. Keep that environment file root-only and manage it through the secret system you already use.

Why This Example Uses Pushover

Pushover is here for one reason: it gets failures from Monit to one operator without adding an SMTP side project.

  • one shell wrapper
  • one root-only credentials file
  • one HTTPS call

If mail alerts already work on your host, keep them. If not, do not build them just to make the setup look proper.

Test It Like You Mean It

The monitoring setup is not real because the config looks tidy. It is real after you make it fail on purpose and the alert arrives.

At minimum:

  1. syntax-check the config with monit -t
  2. break the local HTTP check temporarily by pointing it at a dead port
  3. make the backup stamp stale
  4. force the storage-health script to exit non-zero

You are done when monit -t passes, monit status shows the declared checks, and each forced failure produces the expected Pushover notification.

Do that one failure at a time, then return it to normal.

If you skip the failure test, what you built is not monitoring. It is decorative optimism with indentation.

When Monit Stops Being Enough

Monit is not a metrics warehouse, a dashboard engine, or a pleasant place to explore long-term trends.

You should move to a bigger stack when you actually need:

  • multi-host visibility in one place
  • long-retention metrics
  • trend and capacity analysis
  • richer service graphs
  • team workflows instead of one operator being annoyed in private

If your setup is one host, a few services, and a backup path you care about, this is enough. Watch outcomes, keep the scripts short, and prove the alert path with deliberate failures. Move to a larger stack when you need shared visibility or long-term trends, not because the internet made you feel under-instrumented.

Source Notes