Documentation

Quick start

git clone https://github.com/fornevercollective/GrokYtalkY.git
cd GrokYtalkY
go build -o bin/gy .
./bin/gy                      # companion dock (auto-update on launch)
./bin/gy serve                # hub + rooms (?room= · /api/rooms)
./bin/gy burst                # dual Glyph Matrix
./bin/gy watch clip.mp4
./bin/gy venue --sink st2110 --rtp-b rtp://…   # 2110-20 + 2022-7
./bin/gy doctor st2110
./bin/gy install                  # → ~/.local/bin/gy
./bin/gy update | upgrade
./bin/gy install dependencies     # go · ffmpeg · yt-dlp
./bin/gy clean install
./bin/gy uninstall

Install: gy install or make install → ~/.local/bin/gy. Update: gy update / gy upgrade (TUI also auto-updates). Pages: this site deploys from site/ on main · GrokGlyph PWA.

Install · update · uninstall CLI

All package-lifecycle commands on the terminal binary (proper spellings):

Also: make install · make uninstall · scripts/install.sh.

GrokGlyph PWA

Progressive web app for multi-user 25×25 luminance Glyph Matrix previews. Glyphs stack side-by-side with a 1px gutter and fill the page under a slim header; each matrix can cast live video as 25×25 luminance (cam or file) over the gy hub.

• Video cast — cam samples face → 25×25; cast ships formal gyst kind=hexlum (lane hex) + walkie vburst-frame
• Screen cast — screen / p opens full-resolution player (glyph-cast.html) for TV / second display / Presentation API; integer LED scale for 13·25·37·49 multi-feed matrices
• Live hexlum lane — hub promotes any vburst-frame.glyph → type:gyst for SFU / agent / venue (skip when hex_lane)
• Full-page grid — 1px column/row between matrices; cell size from device scale
• Header — cam · cast · screen · hub · + · mesh (main needs)
• Footer drawer — hub URL · video file · compass · roster · NFC · links
• Clear room — header / menu deletes all peers (keeps you)
• Regions — preset tabs Regions · States · Towns · World → mesh room id
• Geo mesh — GPS nearest + gyro heading facing triangulation → auto-join room
• Resolution — 13 / 25 / 37 / 49 · styles matrix · blocks · braille · ascii
• Roster search — filter chips · sort · focus; stage follows search
• Keys: c cam · v cast · p screen · h hub · / search

# local: gy serve  then open
# http://127.0.0.1:9876/grokglyph.html
# Pages: …/GrokYtalkY/grokglyph.html

TUI help pages

In the terminal: press ? or F1. While help is open, tab / shift+tab cycle pages:

esc or ? closes help. Prompt bar modes remain chat · live · grok · watch.

Companion dock

Default UI is a small Charm dock (alt-screen, width-clamped lines). Camera is off until you press c or run /watch. This avoids taking over the Grok terminal.

• --full — larger dashboard
• F — toggle full ↔ companion
• serve — no TUI; mesh only (Colossus / multi-machine)

Modes

Cycle with tab / shift+tab. Enter is mode-dependent.

Camera & video

On this site: multi-feed vwall (below). Terminal strip on the left mirrors the active feed.

In the TUI:

• c — small camera strip (Half truecolor)
• /watch file.mp4|mkv|mov or URL
• /vstop — stop pipe

# terminal pipeline
ffmpeg -re -i SRC -an -vf scale=W:H,format=rgb24 -f rawvideo pipe:1
# → clamped truecolor ▀ half-blocks
ffplay -vn SRC   # audio parallel

Layout clamps every line to terminal width so resize does not spool half-blocks.

Open streaming hero on home →

VWall simulcast

Home page multi-feed wall, modeled on overview VideoFeedsLab / vwall. Up to 6 simultaneous streams; settings persist in localStorage (gy-vwall).

• + Sim feed — procedural motion (used when camera is blocked)
• + Camera — getUserMedia; falls back to sim on deny
• + Video file — local mp4/webm/mov (looped, muted)
• Click a tile to set it active (mirrors into the terminal strip)
• × on a tile removes that feed

Open VWall on home →

Video burst / Glyph Matrix

Dual circular Glyph Matrices (you | peer) for short face + voice walkie bursts. Layout matches the official Nothing Glyph Matrix Developer Kit LED disk (not a filled square). Full truecolor in the terminal; mesh ships device-N brightness for hardware.

gy burst                 # dual ◎ circles
# space = PTT · [ ] scale · g res · b exit
# 80×24 → full 13×13 dual (auto-fit)
# ~54×31+ → full 25×25 dual

• Web burst orb — hold the circle · Spaces stage · listeners grid
• Live News — major streams as 25×25 Glyph tiles (collapsible regions)
• Android scaffold in glyph/ — Glyph Button down/up
• Protocol: vburst-start · vburst-frame · audio · vburst-end
• Each ◎ is centered in its half of the terminal frame

Full write-up: docs/burst.md in the repo.

Forge multi-pcap + dual Glyph

Cursor-Grok Forge stamps multi-pcap lab tiles with NFT-style cgf: IDs and a 4×4 corner lattice on hexlum. Dual Glyph: left rotates local slots; right holds peer RX.

/forge examples/dojo.pcap examples/dojo.pcap
/forge status | next | hold | rotate | stop
# hub fans forge-mark meta + stamped hexlum
# peer: forge → dual Glyph auto-open

• Marks: cgf:<16hex> · content commitment per slot
• Lattice pass-through on hub / SFU bridge / venue — never re-stamped
• Cap handshake: GY_CAP=term-lean|glyph-iot|bridge on join
• Thin edge: gy agent → JSON glyph + forge-mark lines

Conductor · program bus

On-air control plane for jam + venue sinks. Mesh type program; hub remembers last bus for late joiners.

/conductor claim
/preview 2
/caption HELLO DOJO
/caption lang=en role=lower speaker=dir TAKE NEXT
/caption en: Hello world
/take            # or /take 1
/hold · /black
/program         # PGM/PVW + venue adapter hint

Modes: live · hold · black. ANC: mark · tally (preview/caption flags) · bus · preview · caption (DID 0x5F). Caption UDW: plain UTF-8 or rich 0xC1+JSON (lang · role · speaker · source); bus carries caption_meta when rich.

Venue · NDI · ST 2110 · 2022-7

gy venue consumes the program bus + on-air hexlum/glyph and drives pluggable sinks. Lattice is nearest-neighbor upscaled for HD egress — identity preserved.

gy venue --sink log
gy venue --sink ndi --ndi-name GrokYtalkY-PGM
gy venue --sink st2110 --profile 2110-20 --tp 2110TPN \
  --fps-exact 30000/1001 --depth 10
gy venue --rtp rtp://239.100.1.10:5004 \
  --rtp-b rtp://239.100.2.10:5004          # ST 2022-7 dual-dest tee
gy venue --audio-rtp rtp://239.100.1.10:5006  # ST 2110-30 L24
gy venue --anc-rtp rtp://239.100.1.10:5008    # ST 2110-40 mark/tally/bus ANC
gy doctor st2110 · sync · cameras

ANC capture point: conductor /take · /hold · /black (hub type:program). Honest limits: free-run PTP until facility GM; no NMOS/GM; CEA-708/full VANC = facility. Spec: docs/st2110-sync-cameras.md · capacity: docs/streams-capacity.md.

Simulcast scale · Cloudflare vs custom SFU

Today’s mesh is a lean WebSocket hub (hexcast-style frames + JPEG thumbs + PCM + glyph grids) — perfect for DOJO terminals and Glyph LEDs, not full HD fan-out to “every user on the internet.”

JAX/FFmpeg fit processing (depth, style, resample) off the hot path; delivery should stay WebRTC SFU or Cloudflare media. Terminal/HexCast clients keep the low-res vibe — don’t push 1080p into ▀ cells.

In-repo scaffold: sfu/ (make sfu → gy-sfu on :9880) — room registry, WS signaling, glyph/hex lane fan-out. Full write-up: docs/streams-capacity.md.

make sfu            # signaling + WS glyph/hex/chat
make sfu-media      # + webrtc-rs track fan-out + DataChannels
./sfu/target/release/gy-sfu --bind 127.0.0.1:9880 --token secret
curl -s http://127.0.0.1:9880/health   # "media": true|false
# WS: ws://127.0.0.1:9880/ws?room=dojo&nick=qbit&token=secret
# offer without "to" → SFU answers; DataChannel labels: glyph|hex|chat
gy sfu-bridge --sfu 'ws://127.0.0.1:9880/ws?room=dojo&nick=bridge&token=secret'
# hub vburst + gyst hexlum (lattice) → SFU glyph|hex

Space / Creator Studio chat

Text is a separate plane from media (X Spaces–style). Public 1k+ chat → Cloudflare Workers + Durable Objects. DOJO 16–32 → existing gy serve type:chat or SFU chat messages.

Docs: docs/chat.md · scaffold: chat/ · schema: chat/protocol.json · demo: chat.html

gy serve
cd chat/worker && npx wrangler dev
gy chat-bridge --hosts YOUR_NICK
gy chat-bridge --hosts YOUR_NICK --program-caption   # host chat → rich on-air caption/ANC
# open chat.html → Public Space · stage lines from gy ↗ bridge

Patterns (Strudel-lite)

s("bd sd hh cp")
s("bd*4, [~ sd]*2")
setcps(0.6)
stack(s("bd*4"), note("c2 g2"))
bpm(124)

Live mode + p play/stop. Local synth (afplay/ffplay) + MIDI hits.

MIDI

Buffered per-device outs (signls/sektron lineage). Virtual port name: GrokYtalkY.

• Pattern hits → GM drums / notes
• PTT → note + clock
• Levels → CC

Point signls, sektron, or a softsynth at the virtual port for external sound.

Grok prompt

export XAI_API_KEY=xai-...
# or GROK_API_KEY · loads ~/.grok/env when present
export GROK_MODEL=grok-3-mini
export GROK_CLI_URL=http://127.0.0.1:3000   # optional notes backend

Tab to grok or press g. Slash: /grok make a house beat.

If a reply contains s("…") / stack(…), the dock can auto-eval into live mode.

Server / mesh

./bin/grokytalky serve --port 9876 --bind 0.0.0.0
./bin/grokytalky join HOST:9876

Headless WebSocket hub for multi-peer chat, audio, frames, patterns. No TUI takeover.

CLI map

Platform integration (FFmpeg · Grok · streaming)

Partner handoff for a Grok streaming platform built on supervised FFmpeg, multimodal vision takes, Aito side channels, SFU glyph DCs, and Live News mesh consumers.

gy doctor platform
gy platform export -o handoff.json
gy platform contract   # integrations/grok-stream-platform.json

• v1.70–1.77 wins — FFmpeg control plane · Aito · retarget · theme-vision · SFU token+DC · Live News vision badges · platform export · Cast screen (main → glyph-cast)
• Pipeline — capture→encode→infer→apply(+ffmpeg)→mesh→Live News / glyph-cast
• Not in-process — TensorFlow / full gsplat (Aito only)
• Not auto-all-cast — pin main mosaic → Cast screen / Cast TV

Docs: docs/platform-integration.md · contract: integrations/grok-stream-platform.json · powerhouse: integrations/powerhouse-protocol.json.

Powerhouse stack

Clean route across overview (research tools), blank (StageForge multi-stage), grok-cli (notes + multi-stage), and Qbpm (jam graph) — with GrokYtalkY as mesh / forge / program-bus / venue authority.

# hub
gy serve
# or: ~/Projects/GrokYtalkY/scripts/powerhouse-up.sh

# blank StageForge (+ optional gy)
STAGEFORGE_GY=1 ~/dev/blank/Launch-StageForge.command

# grok-cli multi-stage (notes → hub)
~/dev/grok-cli-main/scripts/powerhouse-stage.sh

# Qbpm: auto-connects window.qbpmGy when hub is up
# override: window.QBPM_GY_HUB = 'ws://127.0.0.1:9876/'

Contract: docs/powerhouse-stack.md · integrations/powerhouse-protocol.json. Authority for on-air stays on gy conductor (/take · ANC · venue).

Environment

See .env.example in the repo. Flag --no-update skips launch auto-update once.

Keys

Stack & lineage

• Bubble Tea v2 · Lip Gloss v2
• cliamp — Charm player reference
• signls · sektron — MIDI
• strudel.cc — live coding
• grok-repo-template — Colossus/Dojo
• charm.land — design inspiration for this site

Module: github.com/fornevercollective/grokytalky · Apache-2.0