Scrappy Singh

The one-line version: Scrappy Singh is a sovereign AI operator that talks like a person, remembers like a partner, works like an engineer, and grows like a system — while its owner keeps a hand on the gate.

Scrappy Singh is a personal AI operator built by Karnveer Singh — not a chatbot. It holds a live voice conversation, carries durable memory that compounds the longer it is used, runs real work on its owner's machine through Claude Code and shell agents, and can extend its own capabilities behind a human-controlled approval gate.

The core bet is a retention thesis: most assistants are stateless, so the relationship resets to zero every session. Scrappy is designed to get measurably better the longer it knows you — remembering goals, learning workflows, and turning repeated tasks into reusable automation.

It is single-owner by design. The same codebase runs in the cloud for always-on availability and entirely offline on the owner's hardware, so the person using it — not a platform — owns the intelligence.

A layered split between the brain (a portable, testable core), the surfaces (web console, terminal CLI and a real-time voice loop, streamed over SSE) and the hands (Claude Code, autonomous terminal agents and Mac control).

One deliberately generic multi-turn agent tool loop powers everything. It knows nothing about voice, Macs or connectors — it streams tokens, executes server-side tools, feeds results back and forwards client-side actions through a small ToolRouter interface that every surface and sub-agent reuses.

Memory is two-layered: episodic storage of every session and message, plus a semantic vector store of typed memories — facts, reflections and procedural playbooks — retrieved by relevance into each turn's context. Nightly consolidation distils raw history into durable knowledge, and a procedural miner turns recurring tool sequences into reusable playbooks.

The voice loop eliminates walkie-talkie dead air: phrase streaming peels complete sentences off to text-to-speech while later sentences are still generating, voice-activity detection allows barge-in mid-sentence, and live narration says what a running tool is doing instead of going silent.

A worker bridge is the hinge: Scrappy reasons in the cloud while its shell commands execute on the owner's Mac, where the browser, toolchain, files and the claude CLI actually live.

Capabilities are modular connectors — memory, coder, terminal, self, web, email, calendar — each exposing typed tools, with expert sub-agents (Memory Keeper, Strategist, Researcher, Engineer, Analyst) delegated to only when a task genuinely needs a specialist.

Self-modification runs describe → propose → apply → roll back: it can read its own implementation, draft a concrete change with rollback, apply it and restart behind a health probe, and request third-party secrets whose values are never shown to the model.

Persistence works across SQLite locally and PostgreSQL in the cloud, with the two backends kept in deliberate lockstep and verified by parity tests.

Long builds blocked silently for minutes — solved with an end-to-end streaming progress channel: Claude Code runs in streaming JSON mode and each meaningful step is distilled into a short spoken milestone pushed all the way to the owner's speakers.

A self-modifying AI could approve its own changes — solved with a server-enforced approval chokepoint. The model structurally cannot self-approve; a self-modifying tool is refused unless the human authorised that action on that request, backed by a global freeze/unfreeze kill switch and default-deny protection on critical files.

One rate-limited API key breaking everything — solved with a rotating key pool with rate-limit awareness, and calm spoken messaging instead of a dumped stack trace when constrained.

Speech-to-text mishearing 'Claude Code' as 'cloud code' — the system prompt teaches every variant as the same intent.

Two databases drifting apart — SQLite and PostgreSQL reads kept in lockstep and verified by backend-parity tests.

Keeping memory useful instead of noisy — retrieval is scoped and decayed rather than dumped wholesale into the prompt.

Honesty as a system property — a capability-honesty rule forbids claiming a task is done without verifiable proof: a real tool result, command output or a created file. Tasks needing a human web step are named as such rather than faked.

Scrappy Singh is the working laboratory behind Karnveer's research into AI memory, pattern recognition and human-AI collaboration.

Design principles: leverage over effort; a human hand on every irreversible action; honesty beats confidence; sovereignty — it must run entirely on the owner's hardware; and test the core, disclose the edges.

Roadmap: retention analytics that quantify whether the system gets better the longer it knows you, memory and analytics layers pooled beyond a single owner, and a widening skill surface extended increasingly by Scrappy itself through its gated self-modification loop.