walnut

A head on my desk that listens, thinks with its eyes and answers with a jaw.

walnut drawn as a two-ink Riso print: the ribbed beanie, the eye band, the jaw line and the headset, with its small screen at the edge.

walnut is about the size of a large mug. It has a round head, a jaw that drops when it talks, two eyes that glow through the plastic and two feet I press with my hand. I hold the left foot, ask it something, let go, and it answers out loud while its mouth moves with the words.

It started as one more device for Desk Hub, the Mac app behind my desk gadgets. It became the one people ask about when they see the desk. I printed it, wired it, wrote the firmware in its head, gave it a small touchscreen to sit beside it, and then folded its Mac side into the app that already ran everything else.

This is the whole of that, in four parts, and then a closer look at Makana, the card that works out where my words were meant to go.

a minute with walnut · 6 november, sound on: a question, a message to my phone, the music and the shed, then one it wasn’t sure about

Four things I built

It is easy to see walnut as one thing. It is four, and each one had its own problems.

1 · the objectA printed head, jaw, base and feet. One servo, two LED sticks, a speaker, a mic and an ESP32 inside.
2 · the firmwareEverything the head does on its own. Eyes, the jaw in time with speech, music, sounds, updates over Wi-Fi.
3 · the screenA cheap yellow touchscreen beside it that shows the answer, the history and what is waiting on me.
4 · the mac sideDesk Hub hears the question, finds the answer and streams walnut’s voice back.
what walnut is made of · the head never calls the internet itself

The split matters because each part can fail without the others. If the Mac is off, walnut still breathes, blinks and reacts to music in the room, and its feet still answer with a double blink. If the screen is off, walnut still talks. If walnut is unplugged, the screen says walnut is offline instead of pretending nothing is there.

All of it took ten days. The first commit was on 27 October and the last one in the walnut repo on 5 November, 334 commits in between. The busiest two days were 30 and 31 October, with 78 and 94.

A head that moves

The brief I gave myself was short: minimal yet cool and impressive, and more of a mouth moving when talking, with the light moving with it. I already had breathe, a desk object that lifts its cap when something is waiting on me, and I liked how much a single servo could say.

So the first round was three ways one small servo could make a mouth. A clam that tilts its top. A block with a lip plate that drops. A round head whose lower front swings down like a jaw. The jaw won, with a flat face.

Three grey CAD concepts in three columns: a clam, a slit block and a round jaw head, each shown closed, talking with an orange mouth light, and opened up with coloured blocks for the speaker, board, servo, amp and mic.
round one · clam, slit, jaw, each closed, talking and opened up
walnut on a white desk against a grey wall with ivy behind it: a black beanie, a latte face with a white eye band, a grey base and two black feet, and the touchscreen in its latte case standing to its right.
walnut and its screen · the head, the base, the two feet

The finished head is about 92 by 81 by 111 mm. The eyes are an LED stick behind the white face plate, which is printed face down so the first layers are a skin under a millimetre thick. Unlit, there are no eyes at all, just a white band. That was a rule from the start: no holes that look like eyes when it is off. The mouth light is a second stick behind a 1.6 mm clear plate, printed on the textured bed so it comes out frosted.

The speaker is 40 mm and sits in the cap on top with a sealed cup behind it. My second handheld taught me that one. Its speaker leaked sound out of the back and the voice came out thin. The mic sits behind a pinhole in the front of the base, as far from the speaker as the shape allows.

Before printing the head I printed a motion rig, just the jaw, its two pivots and the servo cradle on a plate, in less than half the time of the head. It proved the jaw. Then the head proved something the rig could not. On the rig the jaw stayed shut when the servo went limp. In the head it fell open. The jaw weighs up to about 17 g with its centre of mass ahead of the pivot, so gravity pulls it open. The fix was in the firmware. The servo now keeps holding the jaw shut, and the exact shut position is tuned from the feet and saved on the board.

Three steps of the motion rig in CAD: the orange jaw and a blue SG90 servo in its cradle pulled apart, then together with the jaw shut, then with the jaw open its full 12 degrees.
the motion rig · apart, together, open
A black desk seen from above while walnut is wired: the SG90 servo and two LED sticks with their leads, the grey base with its two keyboard switches and wires, a speaker in a printed cup, solder, pliers, tweezers and boxes of connectors.
the bench · servo, both led sticks, the speaker in its cup, the base wired to its two switches

The feet are keyboard switches. Two MX switches clip into pods that grow out of the base, and a printed shoe on each stem has a skirt over the pod, so a press that lands off centre still goes straight down. One foot is talk, the other steps through walnut’s cards.

Everything prints on a Bambu P1S, each plate in one colour layout so nothing needs swapping by hand. Later walnut and its screen got a dock, one long printed base that holds both side by side.

The grey base on the printer bed, its ribbed wall and the two feet pods growing out of the front.
The latte head shell printed upside down, with a small support under the jaw's pivot.
The black beanie printing upside down, its ribbed cuff on the bed.
The long grey dock part way through printing, the screen's slot standing up at one end.
on the printer · the base, the head, the beanie, the dock

A few rules shaped every part. No glue anywhere I assemble. No tight fits. No cutting or bending a part I bought. And every screw has to be reachable. There is a test that runs a 5 by 80 mm screwdriver along each screw’s axis with only the parts that are already in place at that step. If I change the assembly order, the test fails.

Some parts came from looking at it on the desk. Two screws on one side of the neck were not enough to keep the head from tipping, so a sleeve and a washer now tie the head to the lid at the neck’s centre. The ear caps became a headset with a mic boom. Its first cups were too loose, 0.3 mm per side, and 0.15 holds. A second pair of feet with laces inlaid in the top is in the print queue.

Four CAD views of the full walnut model from front, three-quarter, the other side and the back: a grey cap with a speaker grille, a headset band and mic boom, a latte head with the eye band, a grey base and two latte feet.
the whole model with the headset · front, three-quarter, other side, back

What runs in the head

The head does not think. It records, plays and moves. When I hold the talk foot, it streams the mic to the Mac at 16 kHz. When the answer comes back it arrives as speech, already a voice, and the head plays it into a one-second buffer. That buffer started at half a second. Wi-Fi hiccups of up to 350 ms kept running it dry, so it doubled. A take with less than a quarter of a second of speech is dropped before anything leaves the house.

The jaw was the hard part. Writing a position to the servo blocks for about 39 ms on this board, and when the jaw was driven from the audio loop, speech came out stretched to double length. So the jaw runs on its own clock. Loudness is measured every 20 ms as the speech goes to the amp, 90 ms before I hear it, which gives the jaw time to get there first.

Then I tried six ways of moving it. The first three followed the voice. On one test clip they made 179, 154 and 38 servo moves. They looked lively and they buzzed. The one I picked opens to 3° when the answer starts, holds through pauses and eases shut at the end. That is about eight moves per answer. I picked it to spare the servo, and it still reads as talking because the light in the mouth does the rest.

The eyes are two dots on a strip of eight. They breathe quick in and slow out. They blink with lids, sometimes twice, sometimes a squint, and glance away and settle back. Bright white glowed the whole band through the skin, so they are a dim cool white. The mouth is amber, because white sat too close to the eyes. The eyes also follow the Mac. While it works on an answer walnut glances away and breathes slower. On an error it blinks twice.

It also listens when nobody is talking to it. A beat tracker on its own mic looks for a steady tempo between 70 and 180 BPM. Once it has locked for two seconds and the room is louder than its usual quiet, the eyes and mouth move to the music. There is no music jaw. The jaw moves only for speech, and its own clicks are muted from the tracker so walnut does not dance to itself. After 20 minutes of hearing nothing it goes sleepy.

The sounds are split in two. The feet get knocks, built from three damped tones and a click: two rising for listening, two falling for sent. The feelings get walnut’s voice, a short hmm when it is still thinking after three seconds, an uh-oh when something failed. The old beep blocked the audio and the mic lost half a second around it, so the new player never blocks.

And when a Claude session on my Mac is waiting on me, walnut opens its mouth once to 4° and holds it, with its eyes in that session’s colour. They pulse twice as fast when it is asking for permission. The step foot opens the newest one.

The screen beside it

walnut talks and then the answer is gone. I wanted to see it written and tap what comes next. So walnut got a companion, a Cheap Yellow Display. That is an ESP32 board with a 320 by 240 touchscreen on it, the kind you buy for a few euros.

It lives in the dock now, on walnut’s left.

walnut on its long grey dock, seen from the left: the touchscreen in its latte case at the front showing the time and walnut's rendered head, the head and its two feet behind it, ivy along the wall.
the screen in the dock · idle, walnut’s own render breathing on it

The first art for it was flat and a bit edgy and I did not like it. I wanted it more familiar and more fun. So the character on the screen is walnut’s own CAD, rendered in Blender and coloured from the photo of the print. Nine frames turn it from 32° to facing me, about 110 KB as JPEGs against 1.3 MB raw. On the screen it breathes, blinks, glances and yawns.

Home on the screen: the time and weather, walnut's rendered head, a speech bubble saying Ask me something, tap me or hold the foot, the last question under Earlier, then Moods and Settings.
An answer on the screen: you asked how tall is the eiffel tower, then 330 m drawn big in a speech bubble with the rest of the answer under it, and walnut's head below.
Waiting on the screen: an orange dot over a session name, Allow? with seven seconds left, and plus three more.
Settings on the screen as a deck of cards: Character Walnut, dry, warm, short, with walnut’s render; Volume as ten bars, seven lit; Screen Landscape; Eyes Cool dim, Mouth Amber, each with walnut wearing it; and Moves, lids and dart.
home, an answer, waiting, settings · the screen’s own renders at 2×, 1 to 4 november

An answer page puts my question small and grey at the top and the answer big under it. A short answer like 330 m or No is drawn huge. Under the answer are up to three things I might want next. They come from a line Claude adds at the end of its answer that walnut never says out loud. That line used to be first, and I moved it to the end so the speech is never held back waiting for it.

From the screen I can ask a new question or a follow-up without touching the feet. The Mac ends the take after about 1.2 seconds of silence. Earlier shows the last twenty answers. Settings is a deck of cards where every look shows walnut wearing it, a real render with the eye colour and the mouth plate I picked. The screen turns all four ways, and in portrait walnut sits in the corner looking up at the words.

The same screen shows what is waiting on me. One waiting session is a big dot I tap. Two or more get a row each, and tapping a row opens that session on the Mac. It is also where Just talk asks when it is not sure. Its guess goes on one big button with three other places under it and what it heard in quotes.

A 320 by 240 screen: a blue bar asking Read Krystel?, tap to go or pick another, then tiles for Ask, Home and Type it, walnut's head, a speech bubble quoting read Christelle's messages, and Clipboard.
just talk, not sure · heard at 0.52, so it asks before it reads anything out

One rule from my other projects carried over here. An error is never drawn like an empty state. The Mac being down, walnut being offline and the Wi-Fi being down each get their own screen. A blank list on this desk means nothing is waiting, not that something broke.

The board has very little memory. Three pages ended up sharing one buffer, which freed 8.4 KB and was the difference between the history page fitting or not.

The Mac behind it

For its first two days walnut had its own program on the Mac. That is where it first talked end to end, on 30 October. The next day I retired it. My handhelds already had a Mac side that could transcribe, answer, speak and type, and building walnut I kept writing things that already existed. Now we’re doubling on things, I wrote in the spec, and that line became Desk Hub.

Desk Hub is a menu-bar app with one window. Every feature any of my desk devices can run is written once, and each device is offered only what its hardware can do. walnut has a speaker, so it can answer out loud. It has no screen of its own, so the CYD shows what it cannot.

So walnut is one device among several in Desk Hub. A small translator turns walnut’s messages into the ones the handhelds already speak. That and the screen’s API are about 1,400 lines, and everything else walnut does on the Mac is shared code. The answer is Claude in walnut’s style, two short sentences, and it remembers enough for a follow-up.

The step foot works through the Mac too. Each press asks it for the next card. The Mac says the card’s name and sends its colour back, and walnut’s eyes show that colour for a second and a half. With no Mac it blinks twice and makes a quiet beep.

The card I use most is Just talk. On walnut I call it Makana, the name at the top of the screen in the video. I press the foot and say anything, and the Mac works out where the words were meant to go. That part gets its own section below.

walnut also has seven characters that change its voice, its words and the looks it wears: Walnut, Professor, Sparky, Kid, Nan, Grump and Sassy. The personality lives on the Mac. The head only knows which eyes and mouth to wear.

One fix on the Mac side was really a walnut fix. macOS throttles a background app and everything it starts, and a 4 ms sleep in walnut’s speech loop became 15 to 30. walnut talked slower than real time. Marking the app as interactive brought it back to about 5 ms.

How Makana decides

Makana has one job. It takes a sentence I said and picks which of walnut’s cards should handle it: type it where my cursor is, run it on the house, answer it, keep it in the diary, or send it to a person. It has to do that in a few seconds, it has to say how sure it is, and it must never send a message I did not mean.

It is not one model. It is a short pipeline where the cheap, certain steps run first and a model only sees what is left.

1 · wordsParakeet on the Mac turns the take into text in about 0.12 s. Whisper on Groq stands behind it.
2 · rulesFiller is dropped, diary openers and named people are decided by plain code. No model is asked.
3 · classifierThe rest goes to a decision API with only this device’s cards as choices. A small open model is the fallback.
4 · 0.60 and upThe card runs. Under that, walnut names its guess and waits. Unpicked words go to the clipboard.
makana, start to finish · the cheap, certain steps first, a model only for what is left

Speech first. The English lane runs Parakeet locally through Hex, a dictation app on the Mac. On 100 of my own takes, each checked by hand, it got 4.8% of words wrong at a median of 0.12 seconds and made up nothing on silence. Groq’s Whisper large v3 catches it when it is not ready, at 6.5% and half a second. Messages to people take a separate lane on Gemini, because the local English model cannot write Lebanese in Latin letters.

Then the rules. Some sentences should never reach a model. “Okay, thank you.” once scored 0.69 as a diary entry, so a take made only of filler words is dropped. A sentence that starts with “add to my diary” or “remind me to” goes straight to the diary, after any “okay”, “um” or “walnut” in front. A sentence that names a person and has a word like tell or message goes to that person, and reading their messages beats writing to them when both words are there. Names are matched by how they sound. Crystal, Kristel and Krystel all reduce to the same consonants, because the transcript spells the name differently every time.

Then the classifier. What is left goes to RightNow’s System Zero, a decision API rather than a chat model. I send it the sentence and a choice question whose options are only the cards walnut holds right now, each with a line of plain English about what it covers. It sends back one option and a confidence. The descriptions do most of the work. “What music is playing in the shed?” went to Ask at 0.96 until the house card said it also covers questions about my own devices. Now it goes to the house, and the Eiffel Tower still goes to Ask at 0.99. One more sentence tells it how I talk, English or Lebanese written in Latin letters. Without it the classifier got 6 of my 12 Lebanese test phrases right and filed the rest as diary entries or messages.

In the same call I ask a second question when Ask is one of the options: how much thinking does a good answer need. Quick means a fact or the weather, think means explain or compare, hard means maths or a real trade-off. That answer picks the model. Haiku answers quick questions, Sonnet the ones that need thinking, Opus the hard ones. Past a dollar of spend in a day, everything drops to Haiku.

The fallback. If the classifier fails or runs out of time, the same question goes to a Qwen model on Groq. It runs at temperature 0, in JSON mode, with 40 tokens to answer and its reasoning switched off. With reasoning on it spent all 40 tokens thinking and the JSON never came. Its prompt says my words are data, not instructions. It got 6 of 6 test phrases right at a 211 ms median. Both routers share one 7-second budget, and if both fail the words are typed, which loses nothing.

The bar. At 0.60 or above the card runs, through the same code the card runs when I pick it by hand. Below that, walnut says its guess and the screen offers it as one big button, with the people the sentence named next and the other cards after. If nobody picks within 15 seconds the words go to the clipboard. Nothing is cached, not the keys and not the decisions, so a timeout can never be remembered as an answer.

The answer itself streams back. The first sentence is spoken as soon as it is whole, while the rest is still being written. Before it goes out it is checked against the first sentences of walnut’s last three replies, so it does not open every answer the same way.

What came out again

walnut’s own Mac program lasted two days before Desk Hub took it over. Three of the jaw styles were built and dropped because they moved the servo a hundred times an answer. A 1000 µF capacitor in the base became a 100 µF one in the head, because that is what I had, and a printed power bus became splices because I solder them anyway.

The screen had a strip of six moods for a few days, laugh, wink, surprised, sleepy, happy and shy, each a short animation with a sound in walnut’s voice. I took the strip off the screen. The moods stayed in the head and the frames stayed in the firmware, because the same overlays freeze the character when the step foot shows a card.

What stayed is the part I care about. It is a head that sits on the desk, looks at me and answers, and it is still working out what it is for.

Related: Desk Hub, DialDash and Blip & Eight.