A description of how the whole thing fits together, and why each protocol sits where it does. The shape matters more than the brand names: almost every long-lived home automation system converges on these layers, and most of the pain comes from choices made in the bottom two. None of it was designed, exactly. It was arrived at, which is how all real systems are built and almost none are described.
Every architecture diagram is a lie of omission. Ours omits, for instance, the fact that three of the boxes are in a different country, that one of the arrows is infrared and does not come back, and that the whole thing runs on a computer with the memory of a modest phone. With those caveats, here is the lie.
The layers
| Layer | What lives there |
|---|---|
| Radio / bus | A Zigbee mesh, a Thread mesh, Wi-Fi, infrared, a serial dongle or two |
| Transport | An MQTT broker for the Zigbee side; native IP for everything else |
| Integration | Per-vendor adapters turning devices into entities |
| State | Entity registry, a short-retention recorder database, forever-retention statistics |
| Logic | Automations, template sensors, small scripts for anything awkward |
| Presentation | Dashboards, push, a second alert channel, static pages |
| Observation | Watchdogs, auditors, and one heartbeat that lives outside the box |
Choosing a radio
The single decision that shapes everything afterwards. Rough guidance from living with all of them:
| Protocol | Good for | The catch |
|---|---|---|
| Zigbee | Battery sensors, buttons, plugs. Cheap, meshed, genuinely local. | The coordinator is a single point of failure. Mains devices must be spread out or the mesh has holes. |
| Thread / Matter | Newer sensors, low latency, IP-native | The border router adds a failure mode. Matter over Wi-Fi across network segments is painful, see below. |
| Wi-Fi | Mains devices, higher bandwidth | Many are cloud-dependent by default. Check for a local API before buying. |
| Infrared | Older air conditioners, hi-fi | One-way. The system never knows the real state, only what it last sent. |
| Cloud API | Appliances with no local option | Breaks on their schedule, not yours. Always pair with a local signal. |
Local first, and what to do when you cannot be
The rule: anything that matters (safety, heating, alarm, lighting) must work with the internet unplugged. Cloud is acceptable only for things already useless offline.
Where cloud is unavoidable, pair it with a local signal. An appliance's vendor cloud gives programme names and phases; a power-monitoring plug on the same appliance gives a reliable “it is running” from the current draw. When the cloud fails mid-cycle, and it will, the automations keep working and only the cosmetic detail is lost. That pairing is the difference between an appliance automation that lasts years and one that lasts until the next API change.
Infrared, and the honesty problem
An IR-controlled air conditioner cannot be queried. The system knows what it sent, not what happened, someone used the handset, the batteries died, the beam was blocked. Treat the tracked state as an assumption, and where it matters, verify with an independent sensor: if the unit is supposedly cooling, room temperature should be falling. Trust the thermometer, not your own last command.
Network segmentation, and the multicast wall
Putting IoT devices on their own segment is standard advice and worth following. What the advice usually omits: discovery protocols do not cross segments. Casting, printer discovery and Matter over Wi-Fi all rely on multicast DNS, which stops at the boundary.
- You will need an mDNS reflector between segments, and many consumer routers do not have one. Running it as a small container on the automation host works.
- Prefer Thread over Wi-Fi for Matter devices where you have the choice, the border router terminates the mesh, so only the border router itself needs mDNS visibility to the controller, instead of every bulb.
- The automation host ends up with a presence on every segment, which makes it the most network-privileged machine in the building. Treat it accordingly.
Reaching a second location
A VPN tunnel between sites is the easy part. The parts that are not:
- Some devices only answer their own subnet. A local protocol may ignore requests whose source address is on the tunnel. The fix is a narrow source-NAT rule on the remote router so the traffic appears local, scoped to that one device, not the whole subnet.
- Consumer routers forget hand-made rules on reboot and on ordinary config changes. Re-assert them on a schedule rather than fighting it, see self-healing.
- Tunnels inside containers. If the VPN runs in its own container, its routes exist only there; the host has no path to the remote network unless you add one, and you will re-add it after updates.
Data: three tiers with different lifetimes
Storage is where long-running systems quietly fail, so decide the tiers deliberately:
| Tier | Retention | For |
|---|---|---|
| Recorder database | Days | Recent detail, graphs, debugging. Keep it short, this is what fills disks. |
| Long-term statistics | Forever | Hourly aggregates. Free and automatic for anything with a state class; enough for most trends. |
| Plain files | Forever, by hand | Anything that must outlive a rebuild: ledgers, event records, incident notes. |
total with no last_reset (the recorder's source
is one line: _sum += new_state - old_state, sign included); declared
total_increasing, a drop below 90 % of the previous value is read as a meter
reset and no negative is booked, verified in the source rather than recalled, at a review
panel's insistence that somebody eventually read the code. One footnote from the same review:
the band between 90 and 100 % of the previous value is the blind spot, a genuinely
decreasing total_increasing sensor there is warned about but not reset, and books
nothing; if your quantity can truly decrease, it was never total_increasing.
Choose deliberately. Long-term statistics are permanent, a mistake there is far harder to undo
than a bad graph.Write files atomically: temporary file in the same directory, flush, rename. A truncated ledger after a power cut is an unpleasant way to learn this.
Containers and the supervisor
Running the broker, the Zigbee bridge, the VPN and assorted helpers as supervised containers is convenient and adds one specific failure mode: a container can report running while the service inside it is dead. Health checks pass, restart policies do nothing, the supervisor's own watchdog is blind. Probe the port or the process, not the container. More in the field guide.
Where the seams are
Draw the whole thing out and the fragile points are always the same, always at boundaries: the radio coordinator, the broker, the segment crossings, the cloud APIs, and the disk. Five things. You will meet all five. Everything on the monitoring pages exists because of one of those five.
If this looks over-engineered for a flat, that is because it is. It is also under-engineered for what it does, which is the normal condition of everything that works.