Human Middleware: The Missing Operating System for the Smart Home

I have not yet installed solar panels at home. The reason is not scepticism about the technology but that the moment I think through what I actually want, the component list quickly outruns any coherent idea of how to manage it.

The culture war has colonised almost every energy choice. It produces systematically wrong answers by insisting on single-system solutions when the best answer is hybrid. Increasingly the best engineering solution is not one technology replacing another but different technologies doing the jobs they are best suited to. Hybrid systems are underdeployed, not because nobody has thought of them but because they are genuinely difficult to manage without software that does not exist.

Once you accept that the shopping list starts to write itself.

Solar PV in the garden, rather than the roof, because mature trees make the roof a poor place to harvest sunshine. The obvious engineering solution immediately becomes a complex planning regulation problem.

A battery, a smart charger for the EV. It sounds simple enough but that does not optimise for all situations. The correct answer to “EV or diesel” is really “both used appropriately.” A large diesel for 300 mile drives to Cornwall and long family runs abroad alongside the small EV that handles the school run and the village.

The same applies to home heating. A heat pump running in hybrid alongside your existing gas boiler, rather than replacing it, because a heat pump and a gas boiler when it is minus five outside are not the same appliance. The answer to “heat pump or boiler” is ‘both’.

Thermal solar panels for hot water, rather than running 20% efficiency PV then using the electricity for an immersion heater.

A battery that is not a Powerwall because Fogstar currently delivers more storage per pound. I want the freedom to upgrade when a better chemistry arrives in the future without being told I must also replace the inverter.

A borehole for water supply pumping to a header tank and feeding the irrigation system whose demand swings from continuous in August to irrelevant in November.

It runs on. A generator in the shed bought originally for emergencies when it became clear that net zero policy was going to make grid reliability an interesting question but which turns out to be another variable in the optimisation. If generating a kilowatt-hour on diesel is cheaper than buying one from the grid, the operating system ought to know that. Whether exporting it back ever makes economic sense is simply another calculation. An Octopus Agile tariff. Two car diaries because whether Thursday is a trip to Scotland or a school run changes what the battery needs to have done by Wednesday night.

Then there is the whole question of Vehicle to Home power connectivity. Why buy an expensive battery system when you can buy a second hand Nissan Leaf with a 40kWh battery you could plug into your house.

Every one of those components exists. Every one has an app. Nothing manages the lot.

Somewhere during this exercise I realised I was not designing an energy system at all. I was designing a distributed computer whose controllers happen to live inside batteries, inverters, cars, generators, weather stations and heating systems.

Each one is perfectly capable in isolation. But as they are you find yourself acting as scheduler systems integrator and exception handler. There should be a box under the stairs silently handling it all.

The components for that software have been available for years. What is missing is the product.

Home Assistant is the most impressive piece of software most people writing about smart homes have never used. It is free open-source, runs locally, requires no cloud and supports over two thousand integrations. It can read Octopus Agile prices, coordinate a battery, monitor solar, track both EVs schedule, the heat pump and log the borehole pump’s run hours simultaneously. Engineers who have built automations on it have produced extraordinary things.

But getting it to do the simplest coordination task, charging a battery during an Octopus cheap slot, currently requires installing the platform, configuring a messaging broker called Mosquitto, adding an addon called GivTCP, installing the Octopus integration from a third-party store called HACS and importing a YAML automation blueprint written by a hobbyist in Alberta hosted on GitHub last maintained several months ago. When this breaks, when the GivTCP updates in March and silently changes the topic structure, you go back to the forum and find the new blueprint.

When a friend hears you want to do any of this and cheerfully says it is easy, you just need to watch a few YouTube videos, what they are actually saying is dedicate a weekend to acquire a passing familiarity with Linux and plan to repeat the exercise in three months time. For the person who simply wants the thing to work, the YouTube videos are the seventh level of Hell.

The engineers who built Home Assistant built it for people like themselves. The platform they created is like a kit plane, technically extraordinary and endlessly configurable, but most people simply want to catch the Easyjet flight.

Octopus Energy’s Labs platform is the most interesting partial solution in the UK. An energy retailer, a company whose core job is reading meters and sending invoices, has assembled better multi-device home energy coordination than Apple, Google or Amazon. It coordinates GivEnergy batteries, SolarEdge inverters, Nibe heat pumps and Tado thermostats, against Agile half-hourly tariffs and does it automatically. For homeowners on Octopus tariffs with Octopus-approved hardware it is close to functional. Switch supplier and the coordination layer goes with the bill. If an electricity supplier has come closest to building the operating system for the smart home, what exactly has the consumer technology industry been building instead?

AI is now clever enough to write legal briefs and pass medical exams but it cannot manage a house.

The models are not the problem. An AI with access to real-time tariff prices, a seven-day weather forecast, the state of charge of every battery in the system, including both cars, the household diary, the borehole pump schedule, the irrigation controller’s soil moisture readings, the current diesel price and the generator’s fuel level, could produce a continuously updated optimal energy schedule and explain its reasoning in plain English if asked. Running the heat pump between nine and eleven when outdoor temperature peaks and Agile prices are lowest, charging the EV overnight on the cheap rate, deciding whether this half-hour slot is cheap enough to buy from the grid or whether generating locally makes more sense - none of that taxes current AI. It is an integration problem not an intelligence problem.

AI cannot optimise what it cannot see, which is most of it. The thermal solar controller with no API', the generator nobody has written a driver for, the borehole pump reporting only to a cloud service that went offline last Tuesday. All invisible and unoptimisable. The optimisation layer cannot exist until that changes.

The product I want is not complicated to describe.

A black box in the cupboard under the stairs. Every energy device connects to it. It reads the tariff. It reads the weather. It takes one input from the homeowner, which is your travel diary. That is all the homeowner should have to do.

The box handles everything else. It is not Home Assistant which is extraordinary for people who enjoy spending a Sunday in the YAML editor. I mean that without a trace of irony because some people do and the results are impressive. It is the television not the oscilloscope. You switch it on, the picture appears and somewhere seventeen layers deep in the settings menu the pixel calibration controls wait patiently for the one person in ten thousand who wants them. The other nine thousand nine hundred and ninety-nine watch television.

I am hoping a reader of this post will now give me a link to just what I need and the phone number of a man to plug it in. I would be very grateful. Though I think it unlikely.