The House Keeps Its Counsel

Who the assistant works for

The voice assistant on the kitchen bench is very helpful. It is just not working for you.

It works for the company that sold it. It suggests things to buy. It sends what you say to the company’s computers to be understood, and the recordings can be kept. In 2025, Amazon removed the option for some Echo devices to process requests locally, so that every request now goes to the cloud. When the company’s servers have an outage, the lights in your house can stop responding to your voice.

The other devices are no better. A smart television can report what is on its screen to its maker. A robot vacuum can upload a map of your rooms. Printers, cameras and doorbells all talk to servers you have never heard of, and there is no easy way to see what they say.

I do not think any of this is necessary any more. A useful language model now fits on a single chip. So consider an assistant that does everything these devices do, but lives entirely in the house and answers only to the people in it.

The idea

A small box in the hall. Call it the steward, after the old word for someone who runs a house on the owner’s behalf.

• Its mind is in a chip. The AI model’s weights are built into the silicon. It needs no internet connection to understand you, answer you or decide what to do.
• Its voice and ears are in every room. Small speakers with microphones, one per room, connect to the steward over the house’s own network. It can use the speakers you already have.
• It runs the house. It talks to the lights, heating, locks, cameras, printer and television directly, over local standards, without going through anyone else’s servers.
• It answers only to the household. There is no account to sign up for, no subscription, and no advertising. The owner holds the keys.

The steward is also the house agent that Folio IV builds on. This folio describes the agent itself; that one describes what it does for the street.

The gate

The steward is also the house’s router, and it stands between every device and the internet.

• Stopped by default. No device in the house can reach the internet unless the owner allows it. The television can still show films, the printer can still print, and the cameras can still record, all inside the house.
• A ledger in plain words. The steward keeps a record of every attempt to leave: “The television tried to report what you watched, 212 times today. Stopped.” You can ask it out loud what went out and why.
• One exception at a time. The owner can allow a specific thing, such as a streaming service for the television, the weather forecast, or a firmware download for the printer, and see it in the ledger afterwards.

This is the part I find most useful. Even if you never speak to the steward, it gives the house back its privacy.

A mind on a cartridge

A model built into silicon cannot be changed remotely. That is the point, but it also means it will age. So the chip sits on a sealed cartridge.

• To upgrade, swap it by hand. Buy a newer cartridge, pull out the old one and push in the new one. Nothing is ever pushed to the house over the internet.
• Keep the old one. If you prefer how the old model behaves, put it back.
• Nobody can change it from outside. No company can quietly make your assistant more persuasive, add advertising, or remove a feature you rely on.
• An open cartridge standard. The connector and the way the steward talks to a cartridge are published, so anyone can make one: a large company, a university, a community.

The box’s ordinary software still needs security fixes. Those are separate from the model, small, signed, and installed only when the owner approves them.

Who owns it

Open source and open hardware are welcome, but they are not the point. Control is.

• The owner holds the keys to the box, not the maker.
• It works fully with no account and no internet.
• It can be repaired, and its parts can be replaced.
• It keeps working if the company that made it goes out of business.

A sketch of the numbers

Rough figures, to show the scale.

Quantity	Value
Model on the cartridge	≈ 8 billion parameters
Weights, at 4 bits each	≈ 4 GB
Power, listening	≈ 5 W
Power, thinking	≈ 20 W
Time to start answering	≈ half a second
Distance a question travels, cloud assistant	hundreds of km or more
Distance a question travels, here	a few metres
Connected devices in a typical home	≈ 20
Attempts by those devices to reach the internet	often hundreds a day
Price of the box	a guess: like a good router
Price of a cartridge	a guess: like a game cartridge

A model of this size will not match the largest cloud models at writing essays. But running a house is a narrower job: understand a request, know the house, and control the devices in it. For that, I think a small model that knows your house will do better than a large one that does not.

The hard parts

Weaker than the cloud. The best cloud models are much larger. Some questions will be beyond the steward. For those, it could ask a cloud model, but only if the owner allows it, and the ledger would show exactly what was sent.

Stale knowledge. A model on a chip knows the world only up to the day it was made. News, opening hours and the weather need the internet. Each of these is an exception at the gate, chosen by the owner.

Devices that refuse. Many devices stop working if they cannot reach their maker. The steward would need to emulate what they expect, or the household would need to choose devices that work locally. Device makers may resist.

One box, one target. Everything in the house goes through the steward. It must be very hard to break into, and the house must still work, more simply, if it fails.

Who can give orders. A child, a guest, or a voice from the television. The steward needs to know who is speaking and what each person may ask for.

Cost. A chip with a model built in, plus a speaker in every room, costs more than a cheap smart speaker that is paid for with your data.

Where I would start

• The gate first. A home router with a ledger in plain words, and devices stopped by default. It is useful on its own, with no AI at all.
• Then local voice. Existing open home-automation software, a small computer with a graphics chip, and an open model, to prove that a steward can run a house without the cloud.
• Then the cartridge. A first cartridge with a model built into the chip, and the cartridge standard published from the start.

Others who have tried parts of this

• Home Assistant, an open-source home automation project, already runs houses locally and has added its own voice assistant.
• Mycroft built an open voice assistant, but the company wound down in 2023. It shows how hard the hardware business is.
• Network-wide blockers such as Pi-hole stop many devices from reaching advertising and tracking servers. The gate takes the same idea further.
• The Matter standard lets smart devices from different makers be controlled locally, without each maker’s cloud.
• Phones now run small AI models on the device for some tasks, which shows the chips are ready.
• Chip makers have started building a model’s weights directly into silicon, which makes the cartridge plausible.

What I add is the combination: a mind that cannot be changed from outside, a gate that stops the house from talking behind its owner’s back, and a household that owns both.

A good steward keeps the house running, and keeps its counsel.