Entry #3: Giving It Tools

Neural Cube can now act on what it decides. Before tools, the robot could reason its way to the right thing to do and still have no way to do it consistently. The tool-calling layer fundamentally changes what the robot is by allowing it to perform actions. This is the difference between a mind that narrates and a mind that's actually able to interact with the world it's reasoning about, laying the foundation for experience.

⌬The model

Decides what needs doing and which tool does it. It chooses the tool and what to ask of it to initiate an action.

one call shape

≡Tool-calling layer

Every tool call follows the same pattern

Anything the robot does is one of these

Which ones it used, and when, is visible in the dashboard

Physical tools

Virtual tools

⌁Tools that act on the world

MoveDrive to a place, roam and explore, or dance

LookAsk a question about the current view, or save a photo

SoundPlay music or sing

Tools that act internally

RoutineSet up a behaviour to run on a schedule or a trigger

BrowseLook up something current it does not already know

SimulateStart a simulation off the robot’s critical path

for anything that takes a while

⧖Running in the background

Driving somewhere or running a long job runs in the background, so the robot can keep talking to you while it happens. You can interrupt or redirect it mid-way.

The toolbox, as it stands (ongoing updates)

Move is how the robot traverses the space it's in, whether that's navigating to a goal, following someone, or dancing because it wants to. Dancing goes straight to the motors rather than through the navigation system, because it isn't trying to get anywhere. A person can still speak mid-movement to steer or stop it.

Look exists for moments when a first glance wasn't enough, and the robot needs to actually inspect something more closely to understand it. It can also take a photograph, but only when asked. Whatever it chooses to hold onto needs to always be a choice. Either way, nothing leaves the machine, so photos stay on the robot itself.

Sound lets the robot play sounds or music. This can be music it's been asked for or music it chooses on its own. It composes with the other tools, so a single moment can involve movement and sound together, the way a body's expressions naturally do.

Routine is where an instruction becomes a habit the robot carries forward on its own, triggered by time or by something it notices, like a doorbell. Routines can be named, referred to later, edited, paused, or removed.

Something to keep in mind (since this often gets confused and Neural Cube can use both): a routine is not a skill. A routine is a when, written in advance and waiting for its trigger. A skill is a how, something the robot has actually learned, and can call on regardless of when it's asked.

◷Routine

When to do it

Triggered byA time, or something the robot notices

Example“Every weekday at 7am, come to the kitchen and say good morning”

You control itName it, pause it, edit it, remove it

vs

⚙︎Skill

How to do it

Triggered byA request that needs it, whenever it comes up

Example“How to find an object someone asked about”

It writes themWorked out from experience, not scheduled by you

Browse is how the robot searches beyond what it already knows, for things that are true right now rather than generally, like the weather or a price that changes by the hour. When it already knows the answer, it's meant to just answer.

Simulate is a sandbox where the robot can try something out and fail safely before being executed in the real world. It's still experimental but allows new features or capabilities to be tested out before actually committing to them.