Systems that repair themselves, and protect themselves

The difference between a demo and a system you live with is what happens on the bad days. A surprising amount of home automation breakage is transient (a wedged handshake, a route that vanished, a token that expired) and can be repaired automatically if something is willing to check and try again.

Most breakage in a house like this is transient: a library that could not reach its cloud for a minute, a radio that missed a heartbeat, a container that came back a second late. Fixing these by hand is tedious; fixing them automatically is easy; knowing which ones to fix automatically and which ones to shout about is the whole craft.

Repair before you notify

The shape that works, over and over:

detect a bad state
  → attempt the repair
  → wait
  → re-check
      recovered → stay silent (or log quietly)
      still bad → now escalate, and say what was tried

The final clause matters. An alert that says "this is broken and the automatic recovery did not fix it" is far more actionable than one that just says "this is broken", because it has already ruled out the easy explanation.

Things worth healing automatically

Wedged audio handshakes. Some consumer AV links get stuck in a state where everything reports connected but no sound flows. A watchdog that notices the impossible combination (panel on, amplifier awake on the right input, but not actually playing) and re-runs the wake sequence turns a recurring annoyance into nothing.
Half-dead local connections. A device WebSocket can stop delivering updates mid-cycle without dropping. Detect staleness, reload just that integration.
Routes and firewall rules on other devices. Some routers forget hand-made state on every reboot and on ordinary config changes. Rather than fight it, re-assert the desired state on a schedule and treat drift as normal. The check is read-only when nothing is wrong, so running it often is free.
Sockets that must never be off. If the router lives on a smart plug, an interrupted script can leave the home without internet and no way to fix it remotely. A rule that switches it back on after a couple of minutes is trivial and has saved the situation. Two provisos: the plug must not depend on the router it feeds (Zigbee or Thread, never Wi-Fi or anything cloud), and give the re-on rule a retry bound, a plug that keeps failing has a reason, and it deserves attention rather than an infinite argument.
Frozen polled sensors. They keep their last value forever. Reload the integration; asking politely for an update does nothing.
Escalating internet recovery. Restart the link; if that fails, power-cycle the router; if that fails, retry on a long cycle and flag that the fault is probably upstream. Each step is more disruptive than the last, so try them in order.

Every self-healing loop needs a stuck detector. Recovery mode itself can get wedged, retrying forever, achieving nothing, telling nobody, like a very patient employee who has not noticed the building is empty. Watch for "we have been in recovery for more than N hours" and escalate that as its own fault.

Self-protection: the system defending itself from itself

Automation can also damage its own host. Three mechanisms worth having.

A circuit breaker for runaway logging

An automation that errors in a loop can generate hundreds of thousands of log events an hour, inflate the database, and fill the disk. By the time you notice, the machine is already struggling.

Measure errors-per-minute per source. Above a threshold, automatically disable the offending automation and say so. This is prevention rather than notification: it stops the damage while you are asleep.

Two details make it safe. Keep an explicit protect list of automations that must never be auto-disabled, anything touching alarms, leaks, fire or heating. And key that list on structural metadata, an explicit entity list or, better, a registry label, never on name substrings: a fuzzy pattern once protected a hallway dimmer while leaving a water heater unprotected, purely by accident of naming.

A storage watchdog that acts, not just warns

A recorder database that grows ten-fold overnight is a symptom whose consequence is a full disk hours later. Watch the database size, and above a threshold run a shortened purge automatically. Warning is not enough when the failure mode is unattended and time-boxed.

Check free space before compacting. Reclaiming space usually needs a temporary copy roughly the size of the database. On a nearly-full disk that either fails or finishes the job the runaway started. Purge without compaction is safe; compaction is not.

Absolute failsafes on anything that heats

For a high-power heating load, have a last-resort cut-off independent of the clever logic, and make it restart-proof, since the clever logic is exactly what tends to be reloaded during troubleshooting. Then verify the cut actually happened: wait, re-read the state, and if the appliance is still drawing power, escalate loudly instead of reporting success. A safety mechanism that assumes its own command worked is not a safety mechanism. It is a prayer with a log entry.

Latch what the API forgets

A small trick with wide application. Many appliance integrations clear useful fields the moment a cycle ends, the programme name becomes unknown at exactly the moment you want to say "your cottons finished". Latch the value into your own storage while the cycle runs, and read the latch afterwards. The same applies to meter readings you want to difference across a cycle: capture the baseline at the start, because afterwards it is too late.

A system that heals everything silently is not resilient. It is a system with a secret. We try to keep ours to the harmless kind.

Ask of every recurring annoyance: could something have noticed this and tried the obvious fix before I did? Surprisingly often the answer is yes, and the fix is three lines and a re-check.