The main dashboard, rebuilt for you to click.
This is a faithful recreation of the real home view, in native Hebrew RTL with the same glass cards, coloured top rails and layout. Toggle lights, move shutters, or fire a simulated missile alert and watch the house react.
אורות
תריסיםZigbee · כיול אוטומטי
חדרים
מצלמת כניסה · תיאור AIGemini
מנעול חכם
מזגן סלון
מזג אוויר
חשמל היוםמונה חכם
בריאות המערכת
תחזית סוללותמגמת 30 יום
Faithful recreation with demo data only. Room names, people and devices are generic, and nothing on this page talks to a real home.
The same dashboard, captured from the running system.
These are captured card by card from the live Home Assistant. Before capture, people's names, IP addresses and bill amounts were replaced in the page, and the real floor plan was swapped for an illustration. The background photos are the MIT-licensed Hemma demo image and an original illustration.
A doorbell camera that knows which way you're walking.
A single snapshot can't tell you whether someone is coming home or leaving, and that's where most "person detected" alerts go wrong. So I don't send the model one frame. I stream four frames across four seconds to Gemini, along with a written map of the hallway and a strict JSON schema. Direction comes from how the person's size changes across the frames. The answer then drives a four-way notification router.
1 · Pick what happens in the hallway
2 · Structured answer from the model
3 · Where the alert goes
Twelve systems, one nervous system.
Front-door vision AI
A 4-frame / 4-second stream analysis with Gemini, a written hallway map and an enum-constrained JSON schema. It tells a resident coming home apart from a neighbour passing by or a courier at the door, and ignores permanent objects in the corridor.
llmvision.stream_analyzer · Reolink · FrigateHome Front Command choreography
When a siren alert hits the area, the house turns on the shelter-path lights, closes all six shutters, switches the AC off, sends critical pushes that bypass silent mode, and counts down out loud in Hebrew on the living-room speaker.
oref_alert · mode: restart · he-IL TTSTelegram command center
Commands like /salon or /entrance, or just typing "מה קורה בסלון?", grab a live frame and get a one-line description from Gemini. Inline keyboards turn off lights, close shutters or stop the AC. Everything is locked to an allow-list of two user IDs.
Leaving-home summary
Three minutes after a phone leaves the home zone, it gets one message listing which lights are on, which shutters are open and at what %, whether the AC, lock and water heater are on, and one-tap buttons to fix each of them.
person · for: 3 min · Jinja inventorymmWave presence fan
The bathroom fan runs on PIR + mmWave radar. It switches on only after a full minute of continuous presence that the radar confirms as body movement. It has three independent ways to switch off, so a stuck sensor can never leave it running.
Zigbee radar · timer safety netBattery death forecast
Every sensor and the smart lock get a 30-day battery-drain trend. The house predicts the day each battery will die and warns 45 days ahead. There's also a shopping reminder, and a vacation mode for the lock battery.
statistics · 30d change · namespace()Self-healing Zigbee
If the Zigbee bridge is down for 5 minutes, the add-on restarts itself and checks again after 2 minutes. It either reports "fixed" or escalates to a critical push that points at the USB dongle. A monthly report counts the disconnects.
hassio.addon_restart · escalateSun, heat & rain advisor
When there's low western sun and it's over 27°C outside, the house asks (it doesn't act) whether to close the west shutters, by voice and with a Telegram button. If rain is forecast while shutters are open, it names each open shutter.
sun azimuth · ask-don't-actEnergy watchdog
It reads the national utility's smart meter directly. At 20:00 it compares today's consumption with yesterday's and flags a jump of more than 40%. A cost view breaks electricity spend down per device.
IEC meter · power-cost-cardBackup & uptime guard
It confirms every backup, alerts on failures and cloud-upload problems, and escalates if no backup has run for 3+ days. Uptime probes watch the websites I run, with "down" and "back up" alerts.
backup · UptimeRobotAI daily & weekly digests
Every vision event goes onto a timeline calendar. At 20:00 the house sends a short story of the day, and on Friday a weekly summary. On Sunday it sends a stats report with temperature records, door openings, alert count, network speed and the last backup.
llmvision.get_events · countersHebrew-native glass dashboards
12 views built by hand with mushroom cards, card-mod and layout-card. Coloured top rails give each domain its own colour, Jinja makes the cards react to state (the alert strip turns red and pulses), and a bottom nav bar keeps it one-thumb friendly on a phone.
mushroom · card-mod · navbar-cardLocal first. Cloud only where it earns its place.
Home Assistant OS runs the core. Zigbee devices never touch a vendor cloud, and cameras are decoded and analysed on the LAN with Frigate. Cloud is used deliberately: a few frames go to Gemini for vision, notifications with snapshots go out through Telegram and push, speech uses cloud TTS, and a few appliances are cloud integrations.
Seven decisions worth reading about.
Direction comes from change over time, never from one frame
The first version described a single snapshot and got "leaving vs arriving" wrong. Now the model compares size across four frames: growing means arriving, shrinking means leaving, and the same size crossing the frame means passing by. A pet or shopping bags suggest a resident, but they don't decide direction. If it's unclear, the answer is unknown, never a guess.
The model returns a schema, not prose
Location, subject and alert level are closed enums, and the temperature is 0.2. Routing reads fields, not free text. The one free-text field (the Hebrew description) is only shown to humans, never branched on.
Let other sensors veto a false alarm
If the smart lock changed state in the last 150 s, someone at the door is almost certainly a resident. That single variable turns what used to be a nightly false "stranger at the door" into a normal note.
Every AI path has a dumb fallback
If Gemini times out, the automation still sends the photo with "couldn't analyse, take a look", and it still speaks. continue_on_error is set on the AI step so a cloud hiccup never swallows an alert.
Ask, don't act, when you might be wrong
Shutters close automatically only in an emergency. For heat, sun and rain the house proposes and a person taps a button. Automation that fights the people living in the house gets turned off, and then it helps nobody.
One notification script, one change point
Every alert goes through a single script with audience, critical and keyboard fields. Swapping a phone means editing one place instead of 60 automations.
Assume sensors get stuck and radios drop
The fan has three independent ways to switch off (no presence, no body motion, a hard timer). Zigbee restarts itself and escalates with a pointer to the physical dongle if that doesn't help. Backups are audited by a separate automation, so "the backup job exited 0" is never the only proof.
A showcase, not a floor plan.
!secret references.